Comparison of Performance and Movement Kinematics of Virtual Reality Practice with Real-World Practice in the Dart-Throwing Skill

1. Background

A virtual reality (VR) environment has the ability of training and assessment of different sports skills. Scientists use it for training motor skills, technical skills, military aims, aerospace, astronauts, medicine, neural activities (EMG, EEG, fMRI), rehabilitation, and recreational activities as a simulator to ease performance (1-4). Virtual reality has been used to contribute to many training tasks through the implementation of pedagogical processes, too (5-9). Previous research suggests VR can improve benefits in motor skills. Besides, motor skill structures can be learned in VR and transferred to real-world situations (6-8).

In the VR system, the main goal is the transfer of cognitive and motor skill tasks to the real world (RW) (10). Although instructors' main goal is not the similarity of the RW and VR tasks, they care about learning something that can be transferred and generalized to another task with different conditions. However, the two different tasks may have some similar elements. These elements can include fundamental movement patterns, perceptual elements, and strategic conceptual similarities. These similar elements cause positive transfer (11-13).

Fundamental movement patterning is the most significant element in the simulation. It means that tasks with different relative timing characteristics have different movement patterns and lead to the production of different generalized motor programs (GMPs). Therefore, the similarity of GMPs is the most important element for transfer between VR and RW situations. However, researchers should pay attention to the target context and environmental situation similarities that affect transfer (14).

Most research, on the other hand, suggests that the most application of transfer theory occurs when an individual is in the first step of learning (11). In the first step of learning, beginners look at two purposes. First, they attempt to acquire movement patterns. Secondly, they attempt to discriminate between the characteristics of the environment of performance that influence the goal achievement and the characteristics that have no influence or only have an indirect influence (15).

Literature suggests that RW and VR practice produce comparable improvements in novice golfers (6). VR increases the benefits of motor skills such as sitting, standing, walking, running, and jumping (7). The skills of the beginners improve in complex motor tasks through the VR system, and its effect on performance is positive (9, 16). In a comparison of video clips versus VR training in handball goalkeepers, visual perception of the ball resulted in better performance in the analysis of visual information in virtual-based training (17). Although slightly less effective than RW training, a study concluded that VR balance training is an acceptable method for improving balance performance (18).

Other studies, on the other hand, have yielded conflicting results. Review studies have revealed that weaker studies showed larger effects in favor of VR training than RW. VR refers to a narrow range of constructivist pedagogy in the design of participants' experiences (18, 19). Embodied VR can be used for learning RW tasks in a highly controlled environment, which enables applying visual manipulations to an RW full-body task. VR is gaining popularity, but the transfer to the RW is presumably limited due to the lack of embodiment (20).

Examining RW and VR teamwork training showed that RW training resulted in more motivation, the perceived value of the training, and knowledge after the training session than VR training. But about the learning transfer measured by the behavior in a real and complex situation, VR training was as good as RW training (21). A comparison of upper limb movement kinematics in children showed that real table tennis players had smaller wrist angle forehand and slower average speed forehand. The movement patterns of RW and VR were different in this study (22). Investigation of the transfer of sequential motor learning indicated that the sequential reaching task in VR environments was facilitated after the sequential finger task in RW environments (23). Participants who train in the RW environment have an improvement in motor performance when they transfer to the VR environment. In contrast, participants who train in the VR environment show a decrease in skill level when transferring to the RW environment (24).

Despite the diverse findings in research, In the VR environment, individuals can keep their cognitive clarity during injury times. Because they experience the current data in the RW environment, the application of VR work can retain ready athletes' perceptual processes in the competition (25). They can train or compete with competitors with low-cost instruments in places other than the gym, in any weather conditions, at any time, in a COVID-19 situation, and controlling environment features in an unreal position. Therefore, VR is an educational window of opportunity for healthy and unhealthy of different age and gender people. It can suggest new perspectives to users and could lead to more understanding via games.

Achieving the transfer of performance outcomes and movement patterns from VR to the RW will help to apply VR systems as an effective educational tool in motor skills. In this study, we concentrate on motor immersion as a nearly unique method. To some extent, the difference between movement patterns of VR and the RW returns to non-modifiable fidelity constraints of the VR system, such as throwing without darts (22). We controlled modifiable aspects such as a special kind of input control device (Kinect) and task constraint (avatar as an opposing player).

2. Objectives

This study aimed to determine (1) whether the learning environment (i.e., VR vs. RW) influenced performance (i.e., accuracy and consistency variables) and movement kinematics (i.e., angular velocity) in a dart-throwing task. (2) It was also determined whether the time of the tests (i.e., pre-test, post-test, and retention) influenced performance and movement kinematics.

3. Methods

3.1. Participants

This study was semi-experimental. Participants were 24 healthy male students selected from a junior high school in the city of Qazvin in the year 2017. The mean age of the participants was 13.66 years (± 0.48 years, mean height was 163.91 m (± 7.94 m), and mean weight was 60.50 kg (± 7.11 kg). The demographic characteristics of the participants are represented in Table 1. Twenty-two of the participants were right-handed, and two were left-handed (identified by Edinburgh Handedness Inventory). All the right-handed participants threw with their right hand, and the two left-handed participants threw with their left hand. Participants were divided into two homogeneous groups based on the performance of five training throws on the real dartboard. We applied the following as the first inclusion criteria: We applied the following as the first inclusion criteria: The age range was (13 to 14 years). Participants should have no previous experience in RW and VR dart-throwing. They should have normal vision and hearing based on the school health test. After qualifying the initial condition, as the second inclusion criterion, we excluded participants who had a developmental, musculoskeletal, neurological disorder, and poor motor coordination (below the 16th percentile based on movement assessment battery for children-second edition (MABC-2) (26). All participants had the conditions for participation in this study. Participants voluntarily withdrew from the study whenever they wished. All of the participants completed the study. We obtained written informed consent from the participants' legal guardians. The Ethics Review Board of Research Ethics Committee of Sport Sciences Research Institute approved the present study with the following number; IR.SSRI.REC.1397.297

3.2. Variables

In this study, the independent variables were dart-throwing: VR dart throwing (the first group with 12 participants) and RW dart throwing (the second group with 12 participants). Performance measures (mean radial error, bivariate variable error) and kinematics measures (elbow maximum flexion angle, release time angle, and angular velocity of the elbow) were applied as the dependent variables. The mean radial error was the average deviation of the dart from the bull’s eye; the lower the score, the better. The bivariate variable error was computed through the standard deviation of each throw from the mean of throws (15). In each throw, the dart landing point coordinates in the horizontal and vertical axes were recorded in terms of distance from the center in centimeters (27). Considering the high volume of data, to calculate the mean radial error and bivariate variable error, the following formula was written in MATLAB software, and data were extracted from this software.

We set the last frame of moving the forearm backward, before the first frame of moving forward as maximum flexion of the elbow. Also, the last frame before releasing the dart is set as an elbow extension angle. Angular velocity (degrees per second) is calculated by the difference of degree between elbow maximum flexion and release time divided by throwing time (28). To perform angular calculations in MATLAB software, first of all, data was smoothed via the Butterworth low pass filter with a cut-off frequency = of 10. Then the unreal time of 240 frames per second was initially converted to real-time by coding in MATLAB. Secondly, we achieved the slope of the lines AB and BC at maximum elbow flexion and the slope of the lines A'B 'and B'C' at elbow release time via X and Y coordinates. Then using the formula arc tan, we obtained the angle between the lines for both variable maximum elbow flexion (α) and release time angle (β). Also, the average angular velocity of each throw was calculated using the same code and placed in SPSS statistical software.

3.3. Procedures

Standard (unicorn ELIPSE HD) dartboard and darts (PUMA/23 gm) with official height (1.73 m off the ground) and distance (2.37 m from the throwing line) were fixed on the wall so that the center was at eye level for each subject. The Xbox 360 that was used in this study is a device that encodes the player's movement via a camera-based Kinect sensor that figures out individuals’ moving facing it (29). To start, players stand in front of Kinect to calibrate the game. Then move their upper and lower limbs as the controller of the system (30). We placed it in the front of the participants on a small table with a 70 cm height on the ground, below the TV screen that was attached to the wall. Participants played the dart game from Kinect sports season two (Microsoft, USA) on the TV screen (LCD, 48 Samsung). Official height and distance set as mentioned for the standard dartboard.

We used a Casio high-speed camera (EX-ZR1000, China) sampling at 240 Hz to capture dominant upper limb motion during dart-throwing play. The examiner fixed the camera perpendicular to the line of the throw in the sagittal plane to track the position of two-dimensional light reflector markers on the tripod. Light reflector markers were attached to the acromion process, lateral epicondyle, and styloid process of the dominant hand to record throwing time, elbow flexion, and extension on the sagittal plane (28). The examiner trained the participants to take the darts, stand at the start line, and throw them accurately in the bull’s eye. For this purpose, participants had five throws on the real dartboard. Participants were trained to throw with the least time spent between throws as possible. Also, the examiner asked them to throw darts in the sagittal plane only by flexion and extension of the dominant arm and wrist (examiner-trained participants do not switch sides in the throw) as previously has been done (28). Researchers used blocked and constant motor learning training plans for throwing darts and compared them in two groups. Each group performed ten blocks of six trails in the acquisition time, considering 2 minutes' rest between blocks. The RW group practiced with the real standard dartboard. However, the examiner set up the Xbox360 system and the dart game for the VR group. The examiner trained the VR group on how to deal with and immerse in the Kinect in dart play. They were asked to have three throws to ensure doing properly. The last block of training was computed as the acquisition test. Participants of the two groups had the retention test the same as the performance test 48 hours after acquisition. Every six trials of acquisition and retention tests were recorded through the camera on the exterior side of the dominant hand of the students. We used Kinovea-0.8.26 software to analyze video data, Figure 1.

Sample analysis of dart-throwing motion in Kinovea software

Figure 1.

Sample analysis of dart-throwing motion in Kinovea software

3.4. Statistical Analysis

The Shapiro-Wilk test of normality showed that distribution is normal. To analyze performance and kinematic parameters, 2 training methods (VR vs. RW) × 3 Times (pretest, post-test, retention test), One-way MANOVA was performed. SPSS version 25.0 with critical alpha (0.05) was used to analyze data. We used MATLAB R2015b to compute performance measures kinematics before exporting data to SPSS.

4. Results

4.1. Performance Measures

Descriptive analysis of mean radial error (MRE), bivariate variable error (BVE), and the significance level of each variable between the two groups in each phase of the test is presented separately in Table 2. Multivariate tests indicated that there was no significant difference in MRE of the dart-throwing pre-test, acquisition, and retention test between groups, F (3, 20) = 1.05, P = 0.392; Wilk's Λ = 0.864, partial η2= 0.136. Multivariate tests indicated that there was a significant difference in BVE of dart-throwing pre-test, acquisition, and retention between groups, F (3, 20) = 3.94, P = 0.023; Wilk's Λ = 0.628, partial η2 = 0.372. Test of between-subject effects showed that the significant difference is in pre-test F = 5.83, P = 0.024; partial η2 = 0.210. There were no significant differences in acquisition and retention between groups (F = 3.55, 3.58, P = 0.073, 072; partial η2 = 0.139, 140).

4.2. Kinematic Measures

Descriptive analysis of elbow maximum flexion angle, release time angle, angular velocity, and the significance level of each variable between the two groups in each phase of the test is presented separately in Table 3. Multivariate test indicated that elbow maximum flexion angle variable had no significant difference between groups, F (3, 20) = 0.633, P = 0.585; Wilk's Λ = 0.910, partial η2 = 0.090. Multivariate test indicated that release time angle variable had no significant difference between groups, F (3, 20) = 0.980, P = 0.422; Wilk's Λ = 0.872, partial η2 = 0.128. Multivariate test indicated that angular velocity variable had no significant difference between groups, F (3, 20) = 0.808, P = 0.504; Wilk's Λ = 0.892, partial η2 = 0.108. The comparison of the angular velocity variable between the groups is shown as an example in Figure 2.

Angular velocity of virtual reality (VR) and real-world (RW) groups, acquisition and retention

Figure 2.

Angular velocity of virtual reality (VR) and real-world (RW) groups, acquisition and retention

5. Discussion

This research compared the performance and kinematic variables of VR and RW dart throw in high school students. The results revealed that there is no significant difference between the test stages of performance and the kinematic variables of VR throws and the RW except for the pre-test of the BVE variable. This finding resulted in that different learning environments (i.e., VR vs. RW), do not lead to different results in performance and movement kinematics in a dart-throwing task.

5.1. Performance Outcomes

Considering that previous literature has indicated that VR can improve and increase the benefits of motor skills as RW (6, 7). Among the various simulators, VR performs better. A comparison of a handball goalkeeper’s performance revealed that handball goalkeepers were more accurate (radial error) when facing a VR handball thrower than when facing the video clip (17). Research has suggested that RW and VR practices promote accuracy in novice golfers (6). Free throw shooting in basketball and its transfer to dart-throwing revealed that nearly elite basketball participants had a positive transfer from basketball to dart-throwing in the accuracy component (31). A behavioral study with two scenarios about the judgment of trainers and helicopter crew on the police personnel in a team approach with the virtual training found that the VR training group performs the same as the RW group (21). Research about the presence effect of others during a rowing exercise in a VR environment indicated no significant difference in physiological and performance factors between the VR and the non-VR groups (32). Although there are differences between the tasks of the above research and the present study, this study is consistent with the research in comparing VR and RW training.

A study surveyed a simulated game of clay pigeon shooting in novice and expert participants, with an average age of 35 years in the VR environment. This study showed skill improvement is still evident in the participants, and there is no difference between beginners and experts (9). A research study investigated simulated golf games in a VR environment on 18 male and female expert participants, with a mean age of 29 years. Results showed that there was no significant difference in the radial error rate in RW and VR training. The same study in the second experiment on 40 beginner golf participants, 21 of whom were female, showed a significant improvement in radial error but no significant difference between the VR and RW groups (6). These two studies' results are consistent in confirming the positive impact and transferability of VR to the RW. On the other hand, the present study shows that the VR training method has a positive effect on dart-throwing skills in beginners, and there is no difference between the results of VR and RW groups. In this respect, the present study is in line with the research. However, the participants' age differences, skill levels, and complexity of their skills should be considered in the studies.

Researchers put VR hardware devices into two groups: Input devices that enable you to gain information about the performance of the players, and output devices that enable you to immerse in the VR environment (29). Typically, the athletes in the RW have full control over their whole bodies. All senses of their body are aroused. Their performance has an immediate and anticipated behavior. The input device of the VR environment determines the accuracy of the athletes' performance and constrains it. Another constraint is that the athletes understand what is in the output devices. According to this, VR participants should have poorer performance than RW ones. However, as it turned out, these constraints did not have an effect, and VR training and RW training groups in our research have been the same as those in another research.

Systematic reviews express that weaker research showed larger effects in favor of VR training than RW (18), and less VR allows participants to gain experience in learning the structure of skills (19). Participants who trained sequential motor tasks in the RW environment have an improvement when they transferred to the VR environment (23, 24). In contrast, participants who train in the VR environment show a decrease in skill level when transferring to the RW environment (24). The present study is not consistent with this research. It should be noted this inconsistency is probably due to embodiment applied in the VR environments’ research. Therefore, the transfer to the RW is presumably limited due to the lack of embodiment (20).

5.2. Kinematic Outcomes

Fundamental movement kinematics is a crucial element which is the similarity between two tasks leading to positive transfer (12). In this case, it can be claimed that the tasks have the same GMPs. Thus, the similarity of motor sequencing and relative timing between two tasks create a positive transfer (12). A study compared forehand and backhand in real table tennis to VR table tennis. There was no significant difference at the maximum angle of elbow between groups (real, Wii, move, Kinect) for forehand and backhand (22). Other research evaluated the transfer of procedural learning from VR to the RW. Both VR and RW groups performed maintenance training on a tank. Results indicated that both groups had a similar performance in timing and success (10). The finding of our study is in line with this literature. Our results indicated that VR and RW training groups did not have significant differences in dart-throwing movement patterns. The most amount of transfer occurs when the individual is in the first step of the learning. On the other hand, the discrete skills of short duration, such as dart-throwing components, interact strongly (11). Considering that our participants are beginners, and they are in the first step of learning and dart-throwing is a discrete skill with rapid, simple movement patterns, Therefore, VR training has not created different kinematics relevant to RW training.

A comparison of RW and VR table tennis forehand and backhand revealed that in the Kinect group, the minimum angle of the elbow in forehand was significantly smaller than the Wii and RW groups. RW table tennis significantly had a larger minimum angle of the elbow in the backhand than VR groups, the Kinect group was the worst of all. RW table tennis significantly had a slower average speed than VR groups. The Kinect group was faster than the others (22). Our kinematic variables were not consistent with these results. Bufton et al. (22) proposed that RW tasks have smaller and slower hand paths. On the other hand, Kinect without any bat, dart, and racket brings a larger and faster hand path and acts as a constraint. However, the application of skill tools is very important. Although, it seems that skill type, complexity, and tools should not be ignored. The task that we used has a more limited range of movement than table tennis forehand and backhand. It is less complicated than table tennis forehand and backhand, and no darts in throwing. Researchers express participants put in less effort when they are performing in a VR environment, and this is not appropriate; for example, performing dart-throwing without any darts or running in a place instead of RW running, this problem can make it different from RW performance (33). Although, throwing without a dart can be a constraint and should be investigated. Nevertheless, the shape, size, and weight of the racket and the dart differ from each other.

5.3. Conclusions

This study examined the influence of the learning environment, and the time of tests on performance and movement kinematics in the dart-throwing skill. There were not differences in performance and movement kinematics between VR and RW environments. Future research should examine whether VR and RW dart-throwing share enough similar constraint elements to allow for some transfer of learning. If VR skills are transferable to RW skills, they can be promoted to a broad population of children as a viable means to enhance the development of motor skills. It is also suggested that future researchers examine the effects of weight and force on throwing skills, given that often a step forward is taken before performing throwing skills, causing weight transfer.

5.4. Research Limitations

Inability to control the subjects’ nutrition, inability to control the subjects' sleep, as well as inability to control the amount of other physical activity outside the test environment, caused physical fatigue and mental fitness of the participants during the present study. Impossibility to use motion analysis due to restrictions on taking participants to a laboratory in another city.