Logo

EEG-Based Evaluation of Mental Workload in a Simulated Industrial Human-Robot Interaction Task

Author(s):
Babak FazliBabak FazliBabak Fazli ORCID1, Seyed Saman SajadiSeyed Saman SajadiSeyed Saman Sajadi ORCID2, Amir Homayoun JafariAmir Homayoun JafariAmir Homayoun Jafari ORCID2, Ehsan GarosiEhsan GarosiEhsan Garosi ORCID3, Soheila HosseinzadehSoheila HosseinzadehSoheila Hosseinzadeh ORCID4, Seyed Abolfazl ZakerianSeyed Abolfazl ZakerianSeyed Abolfazl Zakerian ORCID1,*, Kamal AzamKamal AzamKamal Azam ORCID5
1Department of Occupational Health Engineering, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran
2Department of Medical Physics and Biomedical Engineering, School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
3Department of Ergonomics, School of Public Health, Iran University of Medical Sciences, Tehran, Iran
4Department of Neuroscience and Addiction Studies, School of Advanced Technologies in Medicine, Tehran University of Medical Sciences, Tehran, Iran
5Department of Epidemiology and Biostatistics, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran


Health Scope:Vol. 14, issue 1; e158096
Published online:Feb 19, 2025
Article type:Research Article
Received:Nov 25, 2024
Accepted:Jan 25, 2025
How to Cite:Babak FazliSeyed Saman SajadiAmir Homayoun JafariEhsan GarosiSoheila HosseinzadehSeyed Abolfazl ZakerianKamal Azamet al.EEG-Based Evaluation of Mental Workload in a Simulated Industrial Human-Robot Interaction Task.Health Scope.14(1):e158096.https://doi.org/10.5812/healthscope-158096.

Abstract

Background:

The rapid advancement of robotics and artificial intelligence is poised to revolutionize industrial settings through widespread automation. This study investigates the impact of robotic assistance on human operator mental workload (MWL) within a simulated industrial environment. Utilizing electroencephalography (EEG) to measure changes in alpha and theta band power, we aim to identify the cognitive challenges associated with human-robot collaboration (HRC) and inform the design of safer and more efficient collaborative systems.

Objectives:

The main objective of the current study was to assess the MWL in a simulated industrial human-robot interaction (HRI) task.

Methods:

The EEG data were collected from 17 participants (aged 25 - 35 years) using a 64-channel system while they engaged in an ecologically valid robotic task that induced three distinct levels of cognitive load: Low, medium, and high. Subsequent analysis focused on EEG power within the alpha and theta frequency bands, employing repeated-measures ANOVA to assess the impact of cognitive load on brain activity.

Results:

A repeated-measures ANOVA revealed significant changes in EEG power across different task difficulty levels. The theta and alpha bands in F3, F4, and Fz, as well as the alpha, beta, and gamma bands in P3, P4, and Pz, emerged as promising indicators for differentiating between varying levels of cognitive load in human-robot tasks.

Conclusions:

Electroencephalography spectral power, particularly within the alpha and theta frequency bands, is a reliable indicator of human MWL. These frequency bands exhibit dynamic changes in response to fluctuating cognitive demands, especially in human-robotic interaction tasks.

1. Background

Technological advancements and developments in information communication have transformed the nature of work at both organizational and industry levels. A shift has occurred from predominantly physical labor to work characterized by mental and cognitive demands (1). In recent years, robots have emerged as essential components for attaining competitiveness in manufacturing, particularly when they can collaborate with humans within a shared workspace and foster a collaborative partnership. The current scenario emphasizes a collaborative environment where robots and humans can work together and interact seamlessly (2).

Human-robot interaction (HRI), an interdisciplinary field encompassing robotics, data science, psychology, and cognitive science, has gained prominence within the context of the Fourth Industrial Revolution. Fostering effective human-machine interaction, characterized by reliable relationships and seamless communication, is crucial for navigating this era of technological advancement and facilitating user skill acquisition (3). Robots can assist humans by relieving them of physically demanding tasks, carrying heavy loads, and conducting repetitive tasks. Most applications in the industrial domain refer to assembly processes, where robots are responsible for tasks that require high precision or cause repetitive strain injuries when done manually, such as carrying heavy objects (4).

Robot integration often necessitates workforce adaptation, requiring workers to relearn tasks and adjust to new workflows. This challenge is exacerbated by the prevalent nature of repetitive tasks and the demanding work environments in industrial settings, characterized by factors such as noise, hazards, dim lighting, time pressure, supervision, and performance monitoring (5, 6). Consequently, human factors, including mental stress induced by prolonged interaction with robots, must be carefully considered and integrated into the planning and design of robotic applications in industrial settings (7).

The term "mental workload (MWL)" describes the amount of cognitive effort required by a person or user to complete one or more tasks (8). Different researchers define MWL as the measure of cognitive resources expended during task execution within a specified timeframe, used to forecast individual or system performance (9, 10). Mental workload is a critical determinant of task performance (11) and is a key factor considered in the development and assessment of complex human-machine interfaces (12). There is broad consensus among researchers about the multifaceted nature of mental burden, which includes both individual effort and attention (13, 14). Mental workload assessment typically employs a combination of subjective, performance-based, and physiological measures (15).

Physiological measures serve as natural indicators of MWL, as increased cognitive demand necessitates greater resource allocation (16). A variety of physiological measures, including electrocardiography (ECG), eye movement, electroencephalography (EEG), respiration, and electromyography (EMG), can be employed to assess MWL. For example, since the brain is the organ responsible for information processing and decision-making, MWL that is cognitively demanding should directly affect brain functions and be associated with electrical activities (17).

The EEG is a non-invasive technique used to capture and record the brain's electrical activity in response to stimuli and behavioral tasks (18). Cognitive theories posit a profound neural basis for emotions, implicating specific cortical and subcortical brain regions in the processing of primary emotional states. These neural correlates of emotion are differentiated from the brain's general electrical and metabolic functions (19). To elucidate the relationship between neural activity and observed voltage patterns, frequency band analysis is commonly employed, with alpha (8 - 12.5 Hz), theta (4 - 8 Hz), and beta (12.5 - 30 Hz) bands representing key spectral features associated with various cognitive and neural processes (20). Previous studies have indicated that the frontal and parietal bands are associated with workload levels (21, 22).

The integration of EEG technology into the assembly line workforce presents significant challenges. The EEG power indicates the level of synchronous neuronal activity. Klimesch's research suggests that EEG power is not simply a linear indicator of cognitive function but rather exhibits a more intricate and nonlinear relationship with cognitive and memory performance (23). The study found a significant correlation between MWL and EEG power in the alpha and theta frequency bands. It is suggested that alpha power is correlated with changes in MWL due to its association with arousal levels, idle states, and cortical inhibition (24).

In a study conducted by Diaz-Piedra et al. on army combat drivers, it was concluded that the theta band is a well-established indicator of cognitive workload (CWL). Increases in theta power within the frontal and occipital regions are directly correlated with task difficulty (25). Aksu et al. conducted a study involving 15 participants where they employed EEG and eye tracking for assessing MWL. They concluded that as task difficulty increased, alpha power exhibited a differential pattern: It rose in the frontal regions while decreasing in the temporal, parietal, and occipital regions (26).

Industrial tasks often involve dynamic and unpredictable elements. Research is needed to explore how MWL changes in response to these dynamic conditions, particularly in high ecological validity task designs. While industrial ergonomics research has extensively examined MWL in human-computer interaction scenarios, it has largely overlooked tasks that require simultaneous motor and cognitive demands. Particularly within the domain of HRI, studies that combine movement and cognition assessments are conspicuously absent.

2. Objectives

As different professions experience various types of human error resulting from mental fatigue and the development of robotic systems, the importance of human interaction and collaboration with these systems is critical. Quantifying MWL for workplace fatigue planning and management can potentially reduce the impact of human error. Therefore, this study seeks to measure the cognitive load experienced by human operators within simulated industrial settings. The research will investigate the impact of integrating a robotic assistant into the workflow, with a specific focus on how this technology influences the cognitive demands placed on the operator.

3. Methods

3.1. Participants

In this study, 17 male participants aged between 25 and 35 years (mean ± SD: 30.82 ± 2.35) were recruited from Tehran University of Medical Sciences. There was a required variability in participants’ educational backgrounds, ranging from postgraduate to PhD as the highest degree, and in visual-spatial intelligence. Although the relatively small sample size of seventeen participants restricts the generalizability of findings, the in-depth analysis of individual data offers valuable insights into the studied phenomena and establishes a solid foundation for future research with larger cohorts.

Participants were carefully screened to exclude individuals with visual or neurological impairments and were all right-handed novices to this experimental paradigm. The present study was approved by the Ethical Committee of Tehran University of Medical Sciences (IR.TUMS.SPH.REC.1401.173). Informed consent was obtained from all individuals.

3.2. Task

The study design includes one independent variable: The robotic task, which was manipulated by having participants perform the task at three levels of difficulty — low workload, medium workload, and high workload. The defined task involves participants performing a simulated task using a pre-designed and built robotic arm controlled remotely through manual controllers, as shown in Figure 1.

Participant wearing the single electrode electroencephalography (EEG) device conducting an experiment.
Figure 1.

Participant wearing the single electrode electroencephalography (EEG) device conducting an experiment.

A robotic arm with 5 degrees of freedom and an AVR microcontroller was utilized to execute the HRI task. The robotic arm is equipped with a gripper that receives motion commands from manual controllers (joysticks). Through these commands, it can manipulate objects of sizes proportional to the gripper's surface. The maximum radius of motion for the robotic arm is 60 centimeters.

The simulated task is designed to have high ecological validity, requiring participants to process visual and spatial information, mirroring real-world industrial robot operation. To enhance ecological validity, the task incorporated both motor and cognitive components. Three levels of CWL (low, medium, and high) were established by manipulating task complexity. The maximum duration for each task block was set at 10 minutes.

To balance and neutralize the learning effect and order effect, each participant randomly selected one of the six predefined sequences or conditions and performed the robotic task. The conclusion of each block was determined through a trigger installed in the robotic framework.

Working memory load was manipulated by requiring participants to concurrently memorize a series of visual stimuli, including industry-related images and two-digit numbers, while performing the robotic task. The number of stimuli was systematically varied across three conditions to induce varying levels of cognitive demand: Low (2 stimuli), medium (5 stimuli), and high (7 stimuli).

The initial task involved manipulating cubes with embedded industrial images and numbers. To increase cognitive load, the task incorporated a Stroop-like interference effect by mismatching cube colors, written words, and box colors. Three levels of difficulty were introduced: Color-matching, color-word matching, and color-word-color matching. To enhance ecological validity, industrial sounds were played concurrently during task execution.

The task conditions for the robotic task in the medium and high levels were similar to the low level, with the difference being that the number of industrial shapes and digit numbers attached to the cubes increased to 5 and 7, respectively.

3.3. Research Design and Procedure

A repeated-measures design was employed, with each participant completing tasks at three cognitive load levels. Prior to the main experiment, participants provided informed consent, underwent pre-testing, and received training on the robotic arm. A baseline resting state measurement was collected before each experimental block. Participants engaged in each task condition for 10 minutes, followed by a debriefing session. The experimental setup during the robotic assembly task is depicted in Figure 2.

Study protocol of robotic task
Figure 2.

Study protocol of robotic task

3.4. Stroop Method

The Stroop color word test is a cognitive assessment that evaluates an individual's ability to manage conflicting information. By presenting words printed in incongruent colors, the test induces interference between automatic and controlled processing. This incongruency demands increased cognitive effort from participants. Imposing a time constraint on responses further amplifies task difficulty (27).

To increase the cognitive engagement of the participants, boxes designed for cube placement were configured according to the Stroop effect paradigm. At the low level, participants were required to match the color of the cubes with the color of the box's front face and perform the necessary movements. At the medium level, the cubes had to be moved based on the written color on the box. At the high level, the cube color needed to correspond to the background color of the text on the box.

3.5. Data Acquisition and Measurement

The EEG data were acquired at a 500 Hz sampling rate using a 64-channel dry Ag/AgCl electrode cap (64-channel EEG system - eego™mylab, ANT Neuro, Netherlands) conforming to the International 10-20 system. Two additional electrodes were placed on the left and right mastoid processes for subsequent offline re-referencing (Figure 3). Ground and reference electrode impedances were maintained below 5 kΩ.

Arrangement of the electrodes used in data acquisition as a single electrode
Figure 3.

Arrangement of the electrodes used in data acquisition as a single electrode

Raw EEG data underwent offline preprocessing using EEGLAB (28). The EEG preprocessing involves a series of techniques designed to enhance the quality and interpretability of raw EEG data. This is a critical step due to the pervasive presence of noise in scalp-recorded EEG signals, stemming from sources such as movement artifacts, powerline interference, and ocular activity. By effectively removing these artifacts, EEG preprocessing improves the spatial information of neural activity and prepares the data for subsequent analysis.

In our study, a bandpass filter (0.5 - 80 Hz) was applied to mitigate noise, including DC drift, powerline interference, and muscle artifacts. Epoch extraction was followed by manual rejection of contaminated segments. Independent component analysis (ICA) was utilized to decompose the EEG signal into independent components, with subsequent removal of components related to eye blinks and other artifacts.

The acquired signals were analyzed in single-channel (or single-electrode) data. A spatial and frequency analysis was performed on the EEG data, focusing on the absolute power considering the frontal and parietal electrodes (F3, F4, F7, F8, Fz, P3, P4, P7, P8, Pz) as shown in Figure 3.

Participants were recruited using a convenience sampling method. Following a pre-screening process, individuals who met the inclusion criteria were included in the final study phase. Repeated measures ANOVA models were employed, treating load condition as a within-subjects factor. For degrees of freedom where Mauchly's test revealed a breach of the assumption of sphericity, Greenhouse-Geisser estimations were used. Partial eta squared was used to report effect sizes. Holm-adjusted and P-values (< 0.05) were used for pairwise comparisons between load conditions.

4. Results

All participants were male. The mean age of the participants was 30.82 years (SD = 2.35), with an age range of 25 to 35 years. Tables 1 and 2 provide a summary of the qualitative and quantitative demographic characteristics of the participants.

Table 1.Descriptive Statistic of Quantitative Demographics Data
VariablesMin - MaxMean ± SD
Age26 - 3530.82 ± 2.35
Visual acuity/right eye8 - 109.64 ± 0.78
Visual acuity/left eye8 - 109.82 ± 0.25

Descriptive Statistic of Quantitative Demographics Data

Table 2.Descriptive Statistics of Qualitative Demographics Data
VariablesNo. (%)
Education level
Post graduate2 (11.8)
Ph.D.15 (88.2)
Marital status
Married7 (41.2)
Single10 (58.8)
Medical history (disease)
Yes2 (11.8)
No15 (88.2)

Descriptive Statistics of Qualitative Demographics Data

For each brain rhythm analyzed, a two-way repeated-measures ANOVA was conducted with 24 electrodes and three levels of task load as within-subject factors. Given the established association between frontal electrodes and theta band activity, and parietal electrodes and alpha band activity in previous research, subsequent analyses focused on a subset of 10 electrodes (F3, F4, F7, F8, Fz, P3, P4, P7, P8, Pz) centered within these respective brain regions.

The results of the study revealed a significant increase in the maximal power of theta and alpha waves in the frontal lobe (F3, Fz, F4) during the robotic task. Additionally, the maximal power of alpha, beta, and gamma waves in the parietal lobe (P3, P4, Pz) showed significant differences across the three task load conditions. Specifically, alpha power in the designated channels exhibited a decreasing trend, while beta, gamma, and theta power in the determined channels showed an increasing trend as the task load progressed through the three predetermined levels (Tables 3 and 4).

Table 3.Electroencephalography Absolute Power Measured at Different Task Levels for a Specific Set of Channels a
Brain Wave (Regions) and ChannelsLowMediumHighP-Valueηp2Pholm b
Theta (frontal)
F315.59 ± 0.5519.22 ± 0.4621.86 ± 0.460.00000.975P1= 3.7861E-12; P2 = 6.6851E-16; P3 = 1.3371E-10
F415.92 ± 0.619.49 ± 0.5821.93 ± 0.50.00000.963P1 = 4.1906E-11; P2 = 1.9112E-14; P3 = 5.0737E-9
F716.39 ± 0.5416.34 ± 0.3816.61 ± 0.370.2--
F816.44 ± 0.616.41 ± 0.416.41 ± 0.490.8--
Fz15.6 ± 0.3719.42 ± 0.5422 ± 0.530.00010.988P1 = 3.867E-14; P2 = 7.8737E-19; P3 = 3.2343E-13
Alpha (parietal)
P311.25 ± 0.519.11 ± 0.387 ± 0.770.00020.937P1 = 6.9468E-11; P2 = 3.097E-12; P3 = 2.2435E-7
P411.36 ± 0.439.11 ± 0.477.28 ± 0.50.00030.948P1 = 1.2573E-10; P2 = 5.457E-13; P3 = 5.8784E-8
P713.27 ± 0.4813.36 ± 0.6213.21 ± 0.410.6--
P813.11 ± 0.4213.15 ± 0.4513.47 ± 0.530.07--
Pz11.31 ± 0.499.11 ± 0.517.15 ± 0.540.000070.959P1 = 3.9762E-10; P2 = 3.7379E-13; P3 = 4.8679E-11
Alpha (frontal)
F311.13 ± 0.439.07 ± 0.427.13 ± 0.670.00020.938P1 = 7.0232E-12; P2 = 1.2355E-12; P3 = 9.149E-8
F411.25 ± 0.579 ± 0.437.08 ± 0.610.00010.933P1 = 4.3172E-9; P2 = 5.7815E-12; P3 = 8.5047E-10
F713.37 ± 0.5413.35 ± 0.513.15 ± 0.50.28--
F813.49 ± 0.413.51 ± 0.4813.43 ± 0.530.39--
Fz11.07 ± 0.449.09 ± 0.347.32 ± 0.40.00010.953P1 = 9.7632E-10; P2 = 5.458E-14; P3 = 2.434E-10

Electroencephalography Absolute Power Measured at Different Task Levels for a Specific Set of Channels a

Table 4.Electroencephalography Absolute Power Measured at Different Task Levels for a Specific Set of Channels a
Brain Wave (Regions) and ChannelsLowMediumHighP-Valueηp2Pholm b
Beta (Parietal)
P310.73 ± 0.5113.45 ± 0.5716.85 ± 0.360.00050.973P1 = 2.8383E-10; P2 = 1.7455E-17; P3 = 2.9694E-12
P410.58 ± 0.4813.45 ± 0.3116.8 ± 0.640.00020.976P1 = 5.2999E-13; P2 = 7.3577E- 17; P3 = 2.0943E-11
P712.61 ± 0.5712.37 ± 0.5312.43 ± 0.50.2--
P812.32 ± 0.3912.52 ± 0.5112.48 ± 0.430.43--
Pz10.7 ± 0.5713.37 ± 0.4516.79 ± 0.470.00030.976P1 = 9.0472E-11; P2 = 8.6309E-17; P3 = 2.7439E-13
Gamma (parietal)
P310.84 ± 0.5913.58 ± 0.5816.74 ± 0.380.00010.974P1 = 1.7269E-10; P2 = 2.0546E-16; P3 =2.7259E-13
P410.77 ± 0.5813.75 ± 0.3716.68 ± 0.620.00020.967P1 = 4.2864E-11; P2 = 2.8998E-16; P3 = 1.8809E-10
P710.7 ± 0.4910.83 ± 0.510.81 ± 0.470.6--
P810.74 ± 0.6410.7 ± 0.3810.61 ± 0.560.8--
Pz10.54 ± 0.4813.43 ± 0.4416.62 ± 0.560.00010.973P1 = 1.5129E-12; P2 = 1.7475E-15; P3 = 5.3886E-12

Electroencephalography Absolute Power Measured at Different Task Levels for a Specific Set of Channels a

A two-way repeated-measures ANOVA was conducted to examine theta, alpha, beta, and gamma oscillatory activity in the frontal and parietal cortices and related electrodes as a function of varying cognitive demands during a robotic task. Post-hoc analysis revealed a significant difference between low and medium, medium and low, and also between low and high cognitive load concerning all four frequency bands.

For theta in the frontal region, channel F3 [F(2,32) = 614.78, P < 0.001, ηp2 = 0.975], channel F4 [F(2,32) = 412.31, P < 0.001, ηp2 = 0.963], and channel Fz [F(2,32) = 1281.86, P < 0.001, ηp2 = 0.988] showed significant differences. Regarding alpha band power, channel P3 [F(2,32) = 236.44, P < 0.001, ηp2 = 0.937], channel P4 [F(2,32) = 290.41, P < 0.001, ηp2 = 0.948], and channel Pz [F(2,32) = 369.58, P < 0.001, ηp2 = 0.959] indicated significant differences between the load conditions. This significance for alpha band power in the frontal region for channels P3, P4, and Pz was [F(1.48,23.8) = 240.64, P < 0.001, ηp2 = 0.938], [F(1.63,26.17) = 240.64, P < 0.001, ηp2 = 0.933], and [F(1.88,30.15) = 324.273, P < 0.001, ηp2 = 0.953], respectively.

One of the most notable findings of our study demonstrates a significant relationship between varying workload levels and changes in parietal beta and gamma band power. The main effects of the gamma band were observed in channel P3 [F(2,32) = 594.84, P < 0.001, ηp2 = 0.974], channel P4 [F(2,32) = 473.87, P < 0.001, ηp2 = 0.967], and channel Pz [F(2,32) = 582.81, P < 0.001, ηp2 = 0.973]. For the beta band, significant effects were found in channel P3 [F(2,32) = 587.58, P < 0.001, ηp2 = 0.973], channel P4 [F(2,32) = 646.8, P < 0.001, ηp2 = 0.976], and channel Pz [F(2,32) = 654.69, P < 0.001, ηp2 = 0.976].

The average power of the signals for all frequency bands is shown in Figures 4 to 6.

Mean theta band power per frontal channel for all subjects
Figure 4.

Mean theta band power per frontal channel for all subjects

Mean alpha band power per parietal channel for all subjects
Figure 5.

Mean alpha band power per parietal channel for all subjects

Changes in alpha, beta, and gamma band power within the selected channels. The figure illustrates only those channels that demonstrated statistically significant differences between the three task load conditions. F<sub>7</sub>, F<sub>8</sub>, P<sub>7</sub>, and P<sub>8</sub> channels didn’t show any significant differences in frequency bands.
Figure 6.

Changes in alpha, beta, and gamma band power within the selected channels. The figure illustrates only those channels that demonstrated statistically significant differences between the three task load conditions. F7, F8, P7, and P8 channels didn’t show any significant differences in frequency bands.

5. Discussion

Objective assessment of MWL, particularly using physiological measures, is well-established in fields such as surgery, aviation, and driving. However, industrial operators interacting with automated systems have received less attention. With the rise of robotics, HRI in industrial settings is increasingly common. The primary objective of this analysis was to extend the scope of research in industrial HRI by concentrating on human MWL and to further advance the development of efficient and safe human-robot interface workstations.

The CWL, or MWL as synonymously mentioned above, is a fundamental concept in the field of HRI workstations. Despite its long history of study, establishing robust evaluation methods for CWL remains a challenge. The EEG, given its accessibility, has become a popular choice for investigating CWL-related cognitive processes (29). This study employs EEG to evaluate the MWL in a simulated human-robot industrial task. Employing EEG devices to assess workers' cognitive abilities offers a promising avenue for examining the MWL associated with robotic tasks and the transitions between them. This approach provides a valuable tool for understanding the cognitive demands of HRIs and optimizing task design to minimize cognitive load.

Based on previous research, three potential explanations for the inconsistent findings were hypothesized. First, the workload manipulations employed in prior studies may have been insufficient to induce significant changes in neural activity or may have reached a saturation point. Second, individual differences in baseline brain activity, task-related cognitive strategies (including attention allocation and working memory utilization), and performance levels might have obscured systematic relationships between CWL and EEG spectral power. Third, considering the growing use of robotic assistants in simulated industrial contexts, it is proposed that cognitive demand may arise as a consequence of HRI.

Theta band power in the frontal cortex is susceptible to increasing CWL. Higher task demands correlate with elevated theta activity in this region. Given the established link between frontal theta and cognitive processes such as working memory and executive function, these findings suggest that theta power may serve as a Neural Index of the brain's effort to manage and manipulate information under demanding conditions (30). A prevailing consensus in the literature suggests that elevated CWL is often accompanied by increased theta band power during ecologically valid tasks (31). However, a more in-depth examination of the empirical evidence revealed inconsistent findings regarding the relationship between CWL and theta and alpha band oscillations (32). In contrast, laboratory-based studies have demonstrated more consistent associations between theta and alpha band frequency modulations and cognitive processes such as working memory and attention (33).

In our study, the experimental findings revealed a statistically significant relationship between theta and alpha band activity in the F3, F4, and Fz channels and fluctuations in workers' mental load. Furthermore, distinct patterns were observed in the alpha, beta, and gamma bands of the P3, P4, and Pz channels during task transitions from low to high, supporting the association between cognition and the parietal region, which is in line with the study results of Klimesch, Roux and Uhlhaas (33, 34).

The observed effect sizes, as measured by partial eta-squared (ηp2), were large, indicating that cognitive load had a considerable impact on EEG activity in frontal and parietal regions. Specifically, theta power in frontal regions and alpha, beta, and gamma power in parietal regions were significantly influenced by the varying levels of task demand. Alpha brainwave power exhibits a negative correlation with CWL; as task demands increase, alpha power tends to decrease (35, 36). In our study, as the workload increased from low to high, a decrease in alpha band power was observed at the P3, P4, and Pz electrodes.

In contrast with our results, John et al., who worked on cognitive complexity during unexpected robot interaction under different MWL states in a physical human-robot collaboration, reported an increase in central alpha and beta power with increasing cognitive load (37). Traditionally associated with a state of wakeful relaxation, alpha desynchronization is a well-documented neural correlate of cognitive engagement (33, 38).

To bolster the ecological validity of our research, future investigations will be conducted in authentic HRI settings. While laboratory experiments can impose stringent restrictions on participant behavior to mitigate the influence of artifacts on EEG recordings, such constraints are infeasible in real-world work environments. Consequently, the presence of artifacts in EEG data collected in naturalistic settings presents significant challenges for data analysis. Recent investigations have emphasized the potential of particular EEG device configurations to reduce the impact of artifacts and noise stemming from physical movement. For instance, Hwang et al. employed an ear canal EEG system, a technology known for its resilience to artifacts generated by physical movement (39).

This study sought to enhance our understanding of the fluctuations in CWL experienced by industrial robot operators during simulated tasks. Power spectral density (PSD) was utilized to extract valuable insights from the participants' brain activity while they performed tasks of varying complexity. The EEG spectral power, particularly within the alpha and theta frequency bands, is a reliable indicator of human MWL. These frequency bands exhibit dynamic changes in response to fluctuating cognitive demands (3).

Future work can extend the current EEG studies to better understand the evolution of workload. This study conducted a series of measures utilizing a highly simulated remote robotic control environment to systematically investigate the cognitive demands associated with industrial automation. By employing a comparative analysis of multiple physiological measures, it is anticipated that a more nuanced understanding of the factors contributing to MWL in industrial operators will be achieved.

Finally, the overarching benefit of this research lies in enhancing the safety of operators and workers in industrial settings. By gaining a deeper understanding of the CWL experienced by robot operators during different tasks, training programs can be optimized. This optimization can also be considered for designing assistive robots and refining systems to foster successful outcomes.

Footnotes

References

  • 1.
    Feder K, Michaud D, McNamee J, Fitzpatrick E, Davies H, Leroux T. Prevalence of Hazardous Occupational Noise Exposure, Hearing Loss, and Hearing Protection Usage Among a Representative Sample of Working Canadians. J Occup Environ Med. 2017;59(1):92-113. [PubMed ID: 28045804]. [PubMed Central ID: PMC5704673]. https://doi.org/10.1097/JOM.0000000000000920.
  • 2.
    Romero D, Stahre J, Wuest T, Noran O, Bernus P, Fasth FA, et al. Towards an Operator 4.0 Typology: A Human-Centric Perspective on the Fourth Industrial Revolution Technologies. Tianjin, China: CIE46 Proceedings; 2016.
  • 3.
    Borghini G, Astolfi L, Vecchiato G, Mattia D, Babiloni F. Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neurosci Biobehav Rev. 2014;44:58-75. [PubMed ID: 23116991]. https://doi.org/10.1016/j.neubiorev.2012.10.003.
  • 4.
    Freire IT, Guerrero-Rosado O, Amil AF, Verschure PFMJ. Socially adaptive cognitive architecture for human-robot collaboration in industrial settings. Front Robot AI. 2024;11:1248646. [PubMed ID: 38915371]. [PubMed Central ID: PMC11194424]. https://doi.org/10.3389/frobt.2024.1248646.
  • 5.
    Rao JV, Chandraiah K. Occupational stress, mental health and coping among information technology professionals. Indian J Occup Environ Med. 2012;16(1):22-6. [PubMed ID: 23112503]. [PubMed Central ID: PMC3482704]. https://doi.org/10.4103/0019-5278.99686.
  • 6.
    Stanton JM. Reactions to Employee Performance Monitoring: Framework, Review, and Research Directions. Human Perform. 2000;13(1):85-113. https://doi.org/10.1207/s15327043hup1301_4.
  • 7.
    Hopko S, Wang J, Mehta R. Human Factors Considerations and Metrics in Shared Space Human-Robot Collaboration: A Systematic Review. Front Robot AI. 2022;9:799522. [PubMed ID: 35187093]. [PubMed Central ID: PMC8850717]. https://doi.org/10.3389/frobt.2022.799522.
  • 8.
    Carissoli C, Negri L, Bassi M, Storm FA, Delle Fave A. Mental Workload and Human-Robot Interaction in Collaborative Tasks: A Scoping Review. Int J Human–Computer Interact. 2023;40(20):6458-77. https://doi.org/10.1080/10447318.2023.2254639.
  • 9.
    Rizzo L, Longo L. Inferential models of mental workload with defeasible argumentation and non-monotonic fuzzy reasoning: a comparative study. 2296. Trento, Italy: Proceedings of the 2nd Workshop on Advances In Argumentation In Artificial Intelligence; 2019.
  • 10.
    Hancock GM, Longo L, Young MS, Hancock PA. Mental Workload. In: Salvendy G, Karwowski W, editors. Handbook of Human Factors and Ergonomics. New Jersey, USA: John Wiley & Sons; 2021. p. 203-26. https://doi.org/10.1002/9781119636113.ch7.
  • 11.
    Galy E. Consideration of several mental workload categories: perspectives for elaboration of new ergonomic recommendations concerning shiftwork. Theo Issues Ergonomics Sci. 2017;19(4):483-97. https://doi.org/10.1080/1463922x.2017.1381777.
  • 12.
    Yu D, Antonik CW, Webber F, Watz E, Bennett W. Correction to: Multi-modal physiological sensing approach for distinguishing high workload events in remotely piloted aircraft simulation. Human-Intelligent Syst Integ. 2021;3(2):201-11. https://doi.org/10.1007/s42454-021-00033-3.
  • 13.
    Longo L. Formalising Human Mental Workload as a Defeasible Computational Concept[dissertation]. Dublin, Ireland: University of Dublin; 2014.
  • 14.
    Estes S. The Workload Curve: Subjective Mental Workload. Hum Factors. 2015;57(7):1174-87. [PubMed ID: 26130676]. https://doi.org/10.1177/0018720815592752.
  • 15.
    Watson AB, Boff KR, Kaufman L, Thomas JP. Handbook of perception and human performance. Michigan, USA: John Wiley & Sons; 1986.
  • 16.
    Fairclough SH, Houston K. A metabolic measure of mental effort. Biol Psychol. 2004;66(2):177-90. [PubMed ID: 15041139]. https://doi.org/10.1016/j.biopsycho.2003.10.001.
  • 17.
    Charles RL, Nixon J. Measuring mental workload using physiological measures: A systematic review. Appl Ergon. 2019;74:221-32. [PubMed ID: 30487103]. https://doi.org/10.1016/j.apergo.2018.08.028.
  • 18.
    Tatum WO, Husain AM, Benbadis SR, Kaplan PW. Handbook of EEG interpretation. New York, USA: Demos Med; 2007.
  • 19.
    Castiblanco Jimenez IA, Marcolin F, Ulrich L, Moos S, Vezzetti E, Tornincasa S. Interpreting Emotions with EEG: An Experimental Study with Chromatic Variation in VR. In: Open Access Maced J Med Sci, editor. Advances on Mechanics, Design Engineering and Manufacturing IV. 7. Skopje, Republic of Macedonia: Open Access Maced J Med Sci; 2023. p. 318-29. https://doi.org/10.1007/978-3-031-15928-2_28.
  • 20.
    Jafari MJ, Khosrowabadi R, Khodakarim S, Mohammadian F. The Effect of Noise Exposure on Cognitive Performance and Brain Activity Patterns. Open Access Maced J Med Sci. 2019;7(17):2924-31. [PubMed ID: 31844459]. [PubMed Central ID: PMC6901841]. https://doi.org/10.3889/oamjms.2019.742.
  • 21.
    Novak D, Beyeler B, Omlin X, Riener R. Workload Estimation in Physical Human–Robot Interaction Using Physiological Measurements. Interact Computers. 2015;27(6):616-29. https://doi.org/10.1093/iwc/iwu021.
  • 22.
    Spüler M, Walter C, Rosenstiel W, Gerjets P, Moeller K, Klein E. EEG-based prediction of cognitive workload induced by arithmetic: a step towards online adaptation in numerical learning. Zdm. 2016;48(3):267-78. https://doi.org/10.1007/s11858-015-0754-8.
  • 23.
    Klimesch W. EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. Brain Res Brain Res Rev. 1999;29(2-3):169-95. [PubMed ID: 10209231]. https://doi.org/10.1016/s0165-0173(98)00056-3.
  • 24.
    Brouwer AM, Hogervorst MA, van Erp JB, Heffelaar T, Zimmerman PH, Oostenveld R. Estimating workload using EEG spectral power and ERPs in the n-back task. J Neural Eng. 2012;9(4):45008. [PubMed ID: 22832068]. https://doi.org/10.1088/1741-2560/9/4/045008.
  • 25.
    Diaz-Piedra C, Sebastian MV, Di Stasi LL. EEG Theta Power Activity Reflects Workload among Army Combat Drivers: An Experimental Study. Brain Sci. 2020;10(4). [PubMed ID: 32231048]. [PubMed Central ID: PMC7226148]. https://doi.org/10.3390/brainsci10040199.
  • 26.
    Aksu SH, Cakıt E, Dagdeviren M. Mental Workload Assessment Using Machine Learning Techniques Based on EEG and Eye Tracking Data. App Sci. 2024;14(6). https://doi.org/10.3390/app14062282.
  • 27.
    Scarpina F, Tagini S. The Stroop Color and Word Test. Front Psychol. 2017;8:557. [PubMed ID: 28446889]. [PubMed Central ID: PMC5388755]. https://doi.org/10.3389/fpsyg.2017.00557.
  • 28.
    Delorme A, Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods. 2004;134(1):9-21. [PubMed ID: 15102499]. https://doi.org/10.1016/j.jneumeth.2003.10.009.
  • 29.
    Chikhi S, Matton N, Blanchet S. EEG power spectral measures of cognitive workload: A meta-analysis. Psychophysiology. 2022;59(6). e14009. [PubMed ID: 35128686]. https://doi.org/10.1111/psyp.14009.
  • 30.
    Gartner M, Grimm S, Bajbouj M. Frontal midline theta oscillations during mental arithmetic: effects of stress. Front Behav Neurosci. 2015;9:96. [PubMed ID: 25941479]. [PubMed Central ID: PMC4403551]. https://doi.org/10.3389/fnbeh.2015.00096.
  • 31.
    Tsang PS, Vidulich MA. Mental Workload and Situation Awareness. In: Salvendy G, editor. Handbook of Human Factors and Ergonomics. New Jersey, USA: John Wiley & Sons; 2006. p. 243-68. https://doi.org/10.1002/0470048204.ch9.
  • 32.
    Puma S, Matton N, Paubel PV, Raufaste E, El-Yagoubi R. Using theta and alpha band power to assess cognitive workload in multitasking environments. Int J Psychophysiol. 2018;123:111-20. [PubMed ID: 29017780]. https://doi.org/10.1016/j.ijpsycho.2017.10.004.
  • 33.
    Klimesch W. alpha-band oscillations, attention, and controlled access to stored information. Trends Cogn Sci. 2012;16(12):606-17. [PubMed ID: 23141428]. [PubMed Central ID: PMC3507158]. https://doi.org/10.1016/j.tics.2012.10.007.
  • 34.
    Roux F, Uhlhaas PJ. Working memory and neural oscillations: alpha-gamma versus theta-gamma codes for distinct WM information? Trends Cogn Sci. 2014;18(1):16-25. [PubMed ID: 24268290]. https://doi.org/10.1016/j.tics.2013.10.010.
  • 35.
    Li Y, Shen L, Sun M. Electroencephalography Study of Frontal Lobe Evoked by Dynamic Random-Dot Stereogram. Invest Ophthalmol Vis Sci. 2022;63(5):7. [PubMed ID: 35522304]. [PubMed Central ID: PMC9078048]. https://doi.org/10.1167/iovs.63.5.7.
  • 36.
    Li H, Zhu P, Shao Q. Rapid Mental Workload Detection of Air Traffic Controllers with Three EEG Sensors. Sensors (Basel). 2024;24(14). [PubMed ID: 39065975]. [PubMed Central ID: PMC11281270]. https://doi.org/10.3390/s24144577.
  • 37.
    John AR, Singh AK, Gramann K, Liu D, Lin CT. Prediction of cognitive conflict during unexpected robot behavior under different mental workload conditions in a physical human-robot collaboration. J Neural Eng. 2024;21(2). [PubMed ID: 38295415]. https://doi.org/10.1088/1741-2552/ad2494.
  • 38.
    Fallahi M, Motamedzade M, Heidarimoghadam R, Soltanian AR, Miyake S. Assessment of operators' mental workload using physiological and subjective measures in cement, city traffic and power plant control centers. Health Promot Perspect. 2016;6(2):96-103. [PubMed ID: 27386425]. [PubMed Central ID: PMC4932229]. https://doi.org/10.15171/hpp.2016.17.
  • 39.
    Hwang T, Kim M, Hong S, Park KS. Driver drowsiness detection using the in-ear EEG. Annu Int Conf IEEE Eng Med Biol Soc. 2016;2016:4646-9. [PubMed ID: 28269310]. https://doi.org/10.1109/EMBC.2016.7591763.
comments

Leave a comment here


Crossmark
Crossmark
Checking
Share on
Cited by
Metrics

Purchasing Reprints

  • Copyright Clearance Center (CCC) handles bulk orders for article reprints for Brieflands. To place an order for reprints, please click here (   https://www.copyright.com/landing/reprintsinquiryform/ ). Clicking this link will bring you to a CCC request form where you can provide the details of your order. Once complete, please click the ‘Submit Request’ button and CCC’s Reprints Services team will generate a quote for your review.
Search Relations

Author(s):

Related Articles