Non-technical Skill Assessment Tools for Surgical Team Members in the Operating Room: A Critical Review

authors:

avatar Peigham Heidarpoor ORCID 1 , avatar Shahram Yazdani ORCID 2 , avatar Leila Sadati ORCID 2 , *

Department of Community-Based Health Education, Virtual School of Medical Education & Management, Shahid Beheshti University of Medical Sciences, Tehran, Iran
Department of Medical Education, Virtual School of Medical Education & Management, Shahid Beheshti University of Medical Sciences, Tehran, Iran
Shiraz E-Medical Journal: Vol.22, issue 2; e101291
published online: November 9, 2020
article type: Review Article
received: January 26, 2020
revised: May 10, 2020
accepted: May 11, 2020
DOI: https://doi.org/10.5812/semj.101291

how to cite: Heidarpoor P, Yazdani S, Sadati L. Non-technical Skill Assessment Tools for Surgical Team Members in the Operating Room: A Critical Review. Shiraz E-Med J. 2021;22(2):e101291. https://doi.org/10.5812/semj.101291.

Abstract

Context:

The operating room is a unique environment in which learning happens through interactions with others and facing complex situations. The results of studies show that non-technical skills play a key role in successful technical performance in critical situations. Therefore, the identification and assessment of them are very valuable. This review study was performed to identify and critique non-technical skill assessment tools for surgical team members.

Evidence Acquisition:

The Carnwell and Daly critical review method was used for the study design. Articles and documents were searched in databases of PubMed, Google Scholar, Scopus, and ScienceDirect from 1999 to 2019.

Results:

This literature review yielded 13 assessment tools of non-technical skills in the operating room, including ANTS, AS-NTS, N-ANTS, ANTS-AP for the anesthesia team, OTAS, Oxford NOTEHS, Oxford NOTECHS II, and Revised NOTECHS for the surgical team, SPINTS for scrub nurses, and OSANTS, NOTSS, SDM-RS, and SLI for surgeons.

Conclusions:

Since the development of the tools depends entirely on the context and task analysis of any member of the surgical team, all countries must customize the available tools and develop similar tools for other members of surgical teams.

1. Context

The term ‘non-technical skills’ (NST) has been derived from the guidelines developed by European Joint Aviation Authorities in 1990, known as Crew Resource Management (CRM), in response to the then-emerging importance of teaching skills that enabled airline pilots to manage the supply of services in critical emergencies and maintain flight safety (1-3). These skills have attitudinal, behavioral, and cognitive characters and are known as human factors, soft skills, or NTS (4). A group of psychologists believes that NTS should not be regarded as equivalent to human factors because NTS, including situation awareness, decision-making, leadership, teamwork, communication, and stress and fatigue management, are personal, social, and cognitive skills that complement technical skills and have a major role in the personnel's effective, safe, and efficient performance of tasks. This is while human factors are only associated with the cultural characteristics ruling the workplace, including relationships with patients and colleagues, teamwork, reflection, and creation of a balance between work and life (5-7). By another definition, NTS refers to non-clinical skills, which include 10 key skills, namely communication skills, critical thinking, emotional intelligence, ethical conduct, rational curiosity, organizational skills, resilience, personal promotion, teamwork, and occupational commitment (8). Undoubtedly, in any job, determining necessary NTS requires the careful identification of delegated responsibilities and the understanding of workplace features and conditions and organizational culture in place (9).

The operating room environment is a unique clinical setting with characteristics such as diverse, sophisticated technological equipment and tools, innovative techniques in performing various procedures, restrictions on traffic, stressful emergencies, work in a busy and congested environment, and the need for teamwork along with attention to patient safety. Identifying the components associated with non-technical cognitive and behavioral skills in this setting can have an effective role in providing an efficient skill instructional model in the curricula of all surgical team members (10).

The results of previous studies have shown that poor NTS in the operation room weakens the level of technical adeptness, leading to unwanted and unfavorable postoperative complications and outcomes (11, 12). For example, surgeons’ lack of attention and poor communication with members of the surgical team has accounted for 43% of surgical errors (13). Therefore, NTS training can lead to improved acquisition of technical skills, which ultimately improves the quality of services provided, reduces medical errors, and enhances patient safety (14-18). Although teaching such skills still has no formal place in the surgical curricula of many universities, in the ACGME core competencies, more than half of the 210 points are designated to medical competencies about NTS, including communication and inter-professional skills, professionalism or professional mannerism, and system-based performance, while interpersonal and communication skills are emphasized as one of the core competencies of surgeons (19, 20). Over the last two decades, the importance of NTS for medical students to acquire professional competencies has encouraged researchers and authorities of educational systems to develop and introduce effective models for teaching and evaluating the NTS (21).

The results of a systematic review study conducted by Dietz et al. (2014) on the models introduced for registering behavioral skills in medical sciences showed that 75% of these models were designed for specialized clinical fields, with the highest number in the fields of surgery and anesthesia (45%) and the rest in trauma and resuscitation (30%) in adults and infants, which indicates the importance of teaching and evaluating these skills in the operating room environment (22). Given the complexity and unique conditions of the operating room and the need for teamwork in such an environment to maintain patient and personnel safety, identifying NTS and models developed for their teaching and assessment can improve the current curricula of surgery and anesthesia by offering a valid model. The present study was conducted to critically review the tools developed to assess NTS among surgical team members in the environment of the operating room.

2. Evidence Acquisition

Since the present study aimed to identify and assess the weaknesses and strengths of the tools developed to assess NTS in surgical team members in the operating room environment, a critical review method was considered appropriate for the study. The critical review method proposed by Carnwell and Daly was used, and the study followed their framework (23). Given that the main focus of the study was on identifying tools and models developed to assess NTS according to a systematic method, the literature search was based on the PRISMA guidelines (24). A search was carried out for relevant articles and documents published from 1999 to 2019 in databases including PubMed, Google Scholar, ScienceDirect, and Scopus using the following English keywords: "Human factors" OR "soft skills" AND "non-technical skills in operating room” OR “non-technical skills of surgical team" OR "non-technical skills of the scrub nurse" OR "non-technical skills of the surgeon” OR "non-technical skills in anesthesia”.

3. Results

A total of 1,397 articles were extracted from the first stage of the search, with 360 being repetitive. The titles of the remaining 1037 articles were assessed by two of the researchers in terms of relevance to the study objectives, which resulted in the selection of 431 articles. At this stage, the references provided by these articles were manually checked, and three new articles were added to the previous list. The abstracts of these 434 articles were then translated and reviewed by the two researchers, and any doubts about the selection of an article were resolved by seeking the opinion of a third researcher for final confirmation. Ultimately, 52 articles were selected based on their relevance to the subject of ‘assessment tools for NTS in surgical team members’, and were fully reviewed. After excluding the repetitive articles dealing with the same tool and those with a poor methodology, 19 articles were finally approved (Figure 1).

Steps to select and approve the articles
Steps to select and approve the articles
Figure 1.

Steps to select and approve the articles

The review of these 19 final articles led to the identification of 13 NTS assessment tools that were related to the operating room environment, which had been developed and used specifically for assessing NTS. Carnwell and Daly proposed four stages in the development of critical reviews, including (1) Defining the scope of the review, (2) Identifying and selecting the sources of relevant information, (3) Organizing the results of the review into categories, and (4) Concluding and informing further studies (23). Accordingly, the subject of ‘non-technical skills assessment tools’ was critically evaluated as the main category in two subcategories, namely individual and group assessment tools for different people and groups working in the operating room, as shown in Table 1. Table 2 presents the chronology of the development of NTS assessment tools over the last two decades.

Table 1.

NTS Assessment Tools Based on Different People and Groups Working in the Operating Room

No.Non-Technical Skills Evaluation ToolsSurgical TeamAnesthetic PractitionersAnesthesiology StudentsNurse AnesthetistsScrub NursesAnesthesiologistsSurgeons
1Anaesthetists non-technical skills (ANTS)*
2Anaesthetists non-technical skills (OTAS)*
3Non-technical skills for surgeons (NOTSS)*
4Revised non-technical skills (Revised NOTECHS)*
5Scrub practitioners’ intraoperative non-technical skills (SPINTS)*
6Oxford non-technical skills (Oxford NOTEHS)*
7Nurse anaesthetists’ non-technical skills (N-ANTS)*
8Oxford non-technical skills II (Oxford NOTECHS II)*
9Anaesthetic non-technical skills for anaesthetic practitioners (ANTS-AP)*
10Objective structured assessment of technical skill (OSANTS)*
11Surgical decision-making rating scale (SDM-RS)*
12Surgeons' leadership inventory (SLI)*
13Anaesthesiology students’ non-technical skills (AS-NTS)*
Table 2.

NTS Assessment Tools Based on Chronology of Assessment Tools Development

No.AuthorsCountry and Year of Tool DevelopmentAssessment ToolKind of MeasurementScoringAssessment AttributionTarget GroupParticipants in Tool DevelopmentValidityReliability
1Fletcher et al.2000, ScotlandANTSScaleScore: 0 - 4Task management, Team working, Situation awareness, Decision-makingAnesthesiologistsAnesthesiologists and PsychologistsContent validityGood Internal consistency
2Healey et al., Sevdalis et al. (25, 26)2004, LondonOTASChecklist, scaleScore: 0 - 6Cooperation, Leadership, Coordination, Awareness, and CommunicationSurgical team membersSurgical residents, human factor experts, psychologists, expert surgeons, nurses, anesthesia supervisors, anesthesiologists, and scrub nursesConstruct validity Content validityICC = 0.4 - 0.9
3Yule, R. Flin, S. et al. (27)2005, ScotlandNOTSSScaleScore:1 - 4Situation awareness, Decision-making, Task management, Leadership, Communication and teamworkSurgeons, assistant surgeons, and scrub nursesConsultant surgeons, surgical students, surgical care practitioners, scrub nurses, anesthesiologists, clinical supervisors, and Professor Rhona FlinConstruct validity Content validity Face validityICC = 0.8
4Sevdalis et al. (26)2008, LondonRevised NOTECHSScaleScore: 1 - 6Cooperation, Leadership and Managerial, Situation Awareness and Vigilance, Decision-making, Communication and InteractionSurgeons, anesthesiologists, and scrub nursesSurgeons, anesthesiologists, and scrub nursesContent validityICC = 0.78 - 0.88
5Lucy,Michel,flin, et al. (28)2008, ScotlandSPINTSScaleScore: 1 - 4 Poor-GoodSituation awareness, Communication and teamwork, Task managementScrub nursesScrub nurses and psychologistsContent validityICC = 0.7
6Suman et al. (29)2008, CanadaSDM-RSScaleScore: 1 - 5Decision-makingSurgeonsSurgeonsContent validityCronbach's alpha = 0.98
7A. Mishra, et al. (30)2009, United KingdomOxford NOTEHSScaleScore: 1 - 4Leadership and management, Teamwork and cooperation, Problem-solving and decision-making, Situation awarenessSurgeons, anesthesiologists, and scrub nursesSurgical residents, operating room nurses, and anesthesiologistsConstruct validity Content validity Face validityHigh
8LYK-Jensen (31) 2011, DenmarkN-ANTSScaleScore: 1 - 5Situation awareness, Decision-making, Task management, Team workingNurses AnesthetistsAnesthesiology nurses, anesthesiologists, surgeons, scrub nurses, clinical supervisors, and educational supervisorsContent validityHigh
9Sarah Henrickson (32)2013, ScotlandSLIScaleScore: 1-4, Poor = GoodMaintaining standards, Managing resources, Making decisions, Directing, Training, Supporting others, Communicating, and Coping with pressureSurgeonsSurgeons and psychologistsContent validity Face validityInter-rater agreement = 0.7
10Robertson et al. (33)2013, The United Kingdom and the USAOxford NOTECHS IIScaleScore: 1 - 8Leadership and management, Teamwork and cooperation, Problem-solving and decision-making, Situation awarenessSurgeons, Anesthesiologists, and scrub nursesHuman factor experts, psychologists, expert surgeons, anesthesia experts, anesthesiologists, scrub nursesConstruct validity Content validity Face validity Criterion validityHigh Inter-rater agreement
11Nicolas J et al. (34)2014, Canada.OSANTSScaleScore: 1 - 5Decision-making, Leading, Directing, Managing, Coordinating, Professionalism, Situation awareness, Teamwork, CommunicationSurgeonsSurgeonsContent validityCronbach's alpha = 0.8
12John Rutherford,flin (35)2015, Great Britain and IrelandANTS-APScaleScore: 1-4, Poor = GoodSituation awareness, Teamwork and communication, Task managementAnesthetic PractitionersConsultants, experts anesthesiologist assistants, anesthesiologists, faculty members and trainersConstruct validity Content validityHigh Cronbach's alpha
13Parisa Moll-Khosrawi, Anne Kamphausen (36)2019, GermanyAS-NTSScaleScore: 1-5, Very poor-very goodPlanning tasks, Prioritizing and problem-solving, Teamwork and leadership, Team orientationAnesthesiology studentsPsychometric psychologists anesthesiologistsContent validityICC = 0.89

4. Discussion

According to the results of this study, six of the above-mentioned tools were developed under the supervision of Dr. Rhona Flin, a professor in the School of Psychology, the University of Aberdeen, which is considered one of the strengths of the development process of these tools due to the attention to the psychometric aspects of the tools (3, 27, 32, 35, 37). Five of these tools were developed in Scotland (3, 27, 32, 35, 37), one in Denmark (31), one in Germany (36), two in Canada (29, 34), and five in the UK (26, 30, 31, 33, 35), which can limit their generalizability, given the different context of operating rooms in developed and developing countries, particularly since a job analysis was used in the process of development of these tools, which reduces their generalizability. The reason is that, for example, in some countries, scrub nurses are operating room technologists who perform the same duties as an assistant surgeon in the surgical team; also, the role of anesthesia nurses differs widely in different countries.

4.1. Scrub Nurses’ NTS Assessment Tools

SPINTS is a specialized tool for assessing NTS in scrub nurses. Scrub nurses are among the core sterile members of the surgical team who assume various responsibilities in different operating rooms according to their job description. The development of this tool dates back to over 10 years ago, and several researchers around the world have used it to assess scrub nurses' NTS and confirmed its validity and reliability. This tool was developed by a tool development team at the University of Aberdeen in Scotland and assesses four aspects of NTS in scrub nurses, including Situation awareness, Communication and teamwork, and Task management (28, 37-39). Since the process of tool development has been based on the occupational analysis of scrub nurses in these countries, the generalizability of these skills to surgical technologists is not possible in countries with similar jobs to scrub nurses, who assume the role of surgeon's first assistant due to the lack of a designated surgeon's first assistant in most centers, especially at non-university centers. For example, in critical situations, scrub nurses may have an important role in stress management in stages such as exposure and homeostasis even though this category has not been foreseen in the four categories extracted for scrub nurses based on SPINTS.

4.2. Anesthesia Group’s NTS Assessment Tools

In the anesthesiology group, ANTS was developed in 2000 as a specialized NTS assessment tool for anesthesiologists and has since been used in several studies, including studies by Flin in 2011, Jirativanont in 2017, and Boet et al., as a valuable tool for assessing NTS in anesthesiologists (3, 40, 41). Besides ANTS, N-ANTS was developed in 2014 to assess anesthesiology nurses (29), ANTS-AP in late 2015 to assess anesthetic practitioners (35), and AS-NTS in 2019 (36) to assess anesthesiology students. Since anesthesiology nurses have a similar job description to anesthesiologists in some centers and induce anesthesia independently, the NTS is approximately the same in these two tools, such that the four main categories of NTS include task management, team working, situation awareness, and decision making. Nonetheless, in ANTS-AP, due to the lack of direct responsibility for the patients, the decision-making skill has been removed from the four main skills, and only three skills (situation awareness, teamwork and communication, and task management) are assessed by the tool. Although the categories extracted in N-ANTS and ANTS largely match due to the similarity of the duties of anesthesiologists and anesthesiology nurses in most countries, given the particular conditions of patient admission to operating rooms in different countries in terms of structural and managerial differences, stress management may be among the main categories of NTS in the anesthesiology group that has not been included in this tool. Moreover, the decision-making skill has not been foreseen in ANTS-AP, while in operating rooms admitting large numbers of patients concurrently, the anesthesiologist cannot be simultaneously present for all the admitted cases, and assistants should also have the ability to make correct decisions. AS-ANTS also emphasizes the assessment of NTS, such as planning tasks, prioritizing and problem-solving, teamwork and leadership, and team orientation, which requires further contemplation due to differences in anesthesiology students' education in different countries. In this tool, the issue of the students' ability to communicate with patients, and the medical team appears to have been overshadowed by teamwork.

4.3. Surgeons’ NTS Assessment Tools

In the category of tools developed for surgeons, NOTSS was the first tool developed to assess the four main skills (situation awareness, decision-making, leadership and communication, and teamwork) (27, 42, 43). The validity and reliability of this tool were confirmed in studies conducted by Dedy et al. in 2016 (44), Jung et al. in 2018 (42), Rao et al. in 2016 (45), and Yule et al. in 2015 (7). OSANTS is another tool developed in 2014 to specifically assess surgeons’ skills. Compared to the previous tools, which mostly assess the four main skills (situation awareness, decision-making, leadership and communication, and teamwork), this tool assesses other aspects of NTS, including professionalism, management, and coordination (34). In a few years, researchers conducted several trial studies and confirmed the effectiveness of this tool in assessing residents' learning (44, 46). The categories in this tool appear to agree with the competencies determined in ACGME. As discussed in the previous sections, the majority of the NTS assessment tools for surgical and anesthesia team members in the operating room have considered some cognitive, behavioral, and attitudinal skills together, with the most prominent being situation awareness, communication and teamwork, leadership and management, decision-making, and problem-solving. Nevertheless, some researchers have only discussed the assessment of one of these skills and have developed tools consistent with the given skill, including SDM-RS, which only assesses surgeons' decision-making skills (29), or SLI, which assesses management skills under subjects such as maintaining standards, managing resources, making decisions, directing, training, supporting others, communicating, and coping with pressure (32). Although focusing on only one competency can enable a more valid assessment, due to the key role and effectiveness of other aspects of NTS, including communication and situation awareness skills, in decision-making, tools that assess these skills together seem to have a greater validity in determining surgeons' competence.

4.4. Surgical Team Members’ NTS Assessment Tools

Based on the results presented in Table 1, given the requirement for teamwork in the operating room and the assessment of members’ performance as the surgical team, four of the identified tools concurrently assess the NTS of three main surgical team members, namely the surgeons, anesthesiologists, and scrub nurses (25, 26, 30, 33, 47). OTAS (25, 47), Revised NOTECHS (26), Oxford NOTECHS (30), and Oxford NOTECHS-II (33) are listed in this group. Only has the Oxford NOTECHS-II been psychometrically assessed in a population of orthopedics in an Asian country (40). Like other tools introduced in the above, due to differences in duties and job descriptions of surgical team members, these skills may be different in various countries.

5. Conclusions

According to the present findings, the empowerment of each member of the operating room department in utilizing the required NTS is imperative. Attention to these skills has resulted in the development of several tools over the last two decades in different countries. Since the development of tools and the identification of these skills depend entirely on the context and job description of the operating room team members, all countries must customize the available tools and develop similar tools for other members of their surgical teams, including the circular nurses, surgical assistants, and surgical technologists. Undoubtedly, improving NTS alongside technical skills can only be realized when the curriculum development authorities for these disciplines prioritize the training and assessment of these skills in their educational agenda.

References

  • 1.

    Civil Aviation Authority. Crew resource management (CRM) training: Guidance for flight crew, CRM instructors (CRMIS) and CRM instructor-examiners (CRMIES). 737. Great Britain: CAA; 2006.

  • 2.

    Flin R, Martin L, Goeters K, Hörmann H, Amalberti R, Valot C, et al. Development of the NOTECHS (non-technical skills) system for assessing pilots’ CRM skills. Hum Factors Aerosp Safety. 2003;3:97-119. https://doi.org/10.4324/9781315194035-1.

  • 3.

    Flin R, Patey R. Non-technical skills for anaesthetists: developing and applying ANTS. Best Pract Res Clin Anaesthesiol. 2011;25(2):215-27. [PubMed ID: 21550546]. https://doi.org/10.1016/j.bpa.2011.02.005.

  • 4.

    Cahan MA, Larkin AC, Starr S, Wellman S, Haley HL, Sullivan K, et al. A human factors curriculum for surgical clerkship students. Arch Surg. 2010;145(12):1151-7. [PubMed ID: 21173288]. https://doi.org/10.1001/archsurg.2010.252.

  • 5.

    Traynor O. Human factors in the surgical arena: a programme of personal skills for surgical practice. Royal college of surgeons in ireland web site; 2008. Available from: http://www.rcsi.ie/index.jsp?1nID=93&2nID=95&3nID=122&4nID=300&5nID=424&6nID=424&nID=432&aID=904..

  • 6.

    Nasir ANBMD, Ali DF, Noordin MKB, Nordin MSB. Technical skills and non-technical skills: predefinition concept. Proceedings of the IETEC’11 Conference, Kuala Lumpur, Malaysia. 2011. p. 1-p17.

  • 7.

    Yule S, Parker SH, Wilkinson J, McKinley A, MacDonald J, Neill A, et al. Coaching non-technical skills improves surgical residents' performance in a simulated operating room. J Surg Educ. 2015;72(6):1124-30. [PubMed ID: 26610355]. https://doi.org/10.1016/j.jsurg.2015.06.012.

  • 8.

    Wolf SJ, Lockspeiser TM, Gong J, Guiton G. Identification of foundational non-clinical attributes necessary for successful transition to residency: a modified Delphi study with experienced medical educators. BMC Med Educ. 2018;18(1):150. [PubMed ID: 29929502]. [PubMed Central ID: PMC6013951]. https://doi.org/10.1186/s12909-018-1247-6.

  • 9.

    Gordon M, Darbyshire D, Baker P. Non-technical skills training to enhance patient safety: a systematic review. Med Educ. 2012;46(11):1042-54. [PubMed ID: 23078681]. https://doi.org/10.1111/j.1365-2923.2012.04343.x.

  • 10.

    Flin RH, O'Connor P, Crichton M. Safety at the sharp end: a guide to non-technical skills. Ashgate Publishing, Ltd; 2008.

  • 11.

    Alken A, Luursema JM, Weenk M, Yauw S, Fluit C, van Goor H. Integrating technical and non-technical skills coaching in an acute trauma surgery team training: Is it too much? Am J Surg. 2018;216(2):369-74. [PubMed ID: 28882359]. https://doi.org/10.1016/j.amjsurg.2017.08.011.

  • 12.

    Anderson O, Davis R, Hanna GB, Vincent CA. Surgical adverse events: a systematic review. Am J Surg. 2013;206(2):253-62. [PubMed ID: 23642651]. https://doi.org/10.1016/j.amjsurg.2012.11.009.

  • 13.

    Xyrichis A, Ream E. Teamwork: a concept analysis. J Adv Nurs. 2008;61(2):232-41. [PubMed ID: 18186914]. https://doi.org/10.1111/j.1365-2648.2007.04496.x.

  • 14.

    McCulloch P, Mishra A, Handa A, Dale T, Hirst G, Catchpole K. The effects of aviation-style non-technical skills training on technical performance and outcome in the operating theatre. Qual Saf Health Care. 2009;18(2):109-15. [PubMed ID: 19342524]. https://doi.org/10.1136/qshc.2008.032045.

  • 15.

    Lingard L, Espin S, Whyte S, Regehr G, Baker GR, Reznick R, et al. Communication failures in the operating room: an observational classification of recurrent types and effects. Qual Saf Health Care. 2004;13(5):330-4. [PubMed ID: 15465935]. [PubMed Central ID: PMC1743897]. https://doi.org/10.1136/qhc.13.5.330.

  • 16.

    Mazzocco K, Petitti DB, Fong KT, Bonacum D, Brookey J, Graham S, et al. Surgical team behaviors and patient outcomes. Am J Surg. 2009;197(5):678-85. [PubMed ID: 18789425]. https://doi.org/10.1016/j.amjsurg.2008.03.002.

  • 17.

    Barach P, Johnson JK, Ahmad A, Galvan C, Bognar A, Duncan R, et al. A prospective observational study of human factors, adverse events, and patient outcomes in surgery for pediatric cardiac disease. J Thorac Cardiovasc Surg. 2008;136(6):1422-8. [PubMed ID: 19114184]. https://doi.org/10.1016/j.jtcvs.2008.03.071.

  • 18.

    Gawande AA, Zinner MJ, Studdert DM, Brennan TA. Analysis of errors reported by surgeons at three teaching hospitals. Surgery. 2003;133(6):614-21. [PubMed ID: 12796727]. https://doi.org/10.1067/msy.2003.169.

  • 19.

    Swing SR. The ACGME outcome project: retrospective and prospective. Med Teach. 2007;29(7):648-54. [PubMed ID: 18236251]. https://doi.org/10.1080/01421590701392903.

  • 20.

    Kieu V, Stroud L, Huang P, Smith M, Spychal R, Hunter-Smith D, et al. The operating theatre as classroom: a qualitative study of learning and teaching surgical competencies. Educ Health (Abingdon). 2015;28(1):22-8. [PubMed ID: 26261110]. https://doi.org/10.4103/1357-6283.161845.

  • 21.

    Agha RA, Fowler AJ, Sevdalis N. The role of non-technical skills in surgery. Ann Med Surg (Lond). 2015;4(4):422-7. [PubMed ID: 26904193]. [PubMed Central ID: PMC4720712]. https://doi.org/10.1016/j.amsu.2015.10.006.

  • 22.

    Dietz AS, Pronovost PJ, Benson KN, Mendez-Tellez PA, Dwyer C, Wyskiel R, et al. A systematic review of behavioural marker systems in healthcare: what do we know about their attributes, validity and application? BMJ Qual Saf. 2014;23(12):1031-9. [PubMed ID: 25157188]. https://doi.org/10.1136/bmjqs-2013-002457.

  • 23.

    Carnwell R, Daly W. Strategies for the construction of a critical review of the literature. Nurse Educ Pract. 2001;1(2):57-63. [PubMed ID: 19036245]. https://doi.org/10.1054/nepr.2001.0008.

  • 24.

    Selcuk AA. A guide for systematic reviews: PRISMA. Turk Arch Otorhinolaryngol. 2019;57(1):57-8. [PubMed ID: 31049257]. [PubMed Central ID: PMC6461330]. https://doi.org/10.5152/tao.2019.4058.

  • 25.

    Healey AN, Undre S, Vincent CA. Developing observational measures of performance in surgical teams. Qual Saf Health Care. 2004;13 Suppl 1:i33-40. [PubMed ID: 15465953]. [PubMed Central ID: PMC1765793]. https://doi.org/10.1136/qhc.13.suppl_1.i33.

  • 26.

    Sevdalis N, Davis R, Koutantji M, Undre S, Darzi A, Vincent CA. Reliability of a revised NOTECHS scale for use in surgical teams. Am J Surg. 2008;196(2):184-90. [PubMed ID: 18558392]. https://doi.org/10.1016/j.amjsurg.2007.08.070.

  • 27.

    Yule S, Flin R, Paterson-Brown S, Maran N, Rowley D. Development of a rating system for surgeons' non-technical skills. Med Educ. 2006;40(11):1098-104. [PubMed ID: 17054619]. https://doi.org/10.1111/j.1365-2929.2006.02610.x.

  • 28.

    Mitchell L, Flin R. Non-technical skills of the operating theatre scrub nurse: literature review. J Adv Nurs. 2008;63(1):15-24. [PubMed ID: 18598248]. https://doi.org/10.1111/j.1365-2648.2008.04695.x.

  • 29.

    Chatterjee S, Ng J, Kwan K, Matsumoto ED. Assessing the surgical decision making abilities of novice and proficient urologists. J Urol. 2009;181(5):2251-6. [PubMed ID: 19296971]. https://doi.org/10.1016/j.juro.2009.01.032.

  • 30.

    Mishra A, Catchpole K, McCulloch P. The Oxford NOTECHS System: reliability and validity of a tool for measuring teamwork behaviour in the operating theatre. Qual Saf Health Care. 2009;18(2):104-8. [PubMed ID: 19342523]. https://doi.org/10.1136/qshc.2007.024760.

  • 31.

    Lyk-Jensen HT, Jepsen RM, Spanager L, Dieckmann P, Ostergaard D. Assessing nurse anaesthetists' non-technical skills in the operating room. Acta Anaesthesiol Scand. 2014;58(7):794-801. [PubMed ID: 24673620]. https://doi.org/10.1111/aas.12315.

  • 32.

    Henrickson Parker S, Flin R, McKinley A, Yule S. The surgeons' leadership inventory (SLI): a taxonomy and rating system for surgeons' intraoperative leadership skills. Am J Surg. 2013;205(6):745-51. [PubMed ID: 22867725]. https://doi.org/10.1016/j.amjsurg.2012.02.020.

  • 33.

    Robertson ER, Hadi M, Morgan LJ, Pickering SP, Collins G, New S, et al. Oxford NOTECHS II: a modified theatre team non-technical skills scoring system. PLos One. 2014;9(3). e90320. [PubMed ID: 24594911]. [PubMed Central ID: PMC3942429]. https://doi.org/10.1371/journal.pone.0090320.

  • 34.

    Dedy NJ, Szasz P, Louridas M, Bonrath EM, Husslein H, Grantcharov TP. Objective structured assessment of nontechnical skills: Reliability of a global rating scale for the in-training assessment in the operating room. Surgery. 2015;157(6):1002-13. [PubMed ID: 25704419]. https://doi.org/10.1016/j.surg.2014.12.023.

  • 35.

    Rutherford JS, Flin R, Irwin A, McFadyen AK. Evaluation of the prototype Anaesthetic Non-technical Skills for Anaesthetic Practitioners (ANTS-AP) system: a behavioural rating system to assess the non-technical skills used by staff assisting the anaesthetist. Anaesthesia. 2015;70(8):907-14. [PubMed ID: 26152252]. https://doi.org/10.1111/anae.13127.

  • 36.

    Moll-Khosrawi P, Kamphausen A, Hampe W, Schulte-Uentrop L, Zimmermann S, Kubitz JC. Anaesthesiology students' Non-Technical skills: development and evaluation of a behavioural marker system for students (AS-NTS). BMC Med Educ. 2019;19(1):205. [PubMed ID: 31196070]. [PubMed Central ID: PMC6567593]. https://doi.org/10.1186/s12909-019-1609-8.

  • 37.

    Mitchell L, Flin R, Yule S, Mitchell J, Coutts K, Youngson G. Development of a behavioural marker system for scrub practitioners' non-technical skills (SPLINTS system). J Eval Clin Pract. 2013;19(2):317-23. [PubMed ID: 22502593]. https://doi.org/10.1111/j.1365-2753.2012.01825.x.

  • 38.

    Mitchell L, Flin R, Yule S, Mitchell J, Coutts K, Youngson G. Evaluation of the Scrub Practitioners' List of Intraoperative Non-Technical Skills system. Int J Nurs Stud. 2012;49(2):201-11. [PubMed ID: 21974792]. https://doi.org/10.1016/j.ijnurstu.2011.08.012.

  • 39.

    McClelland G. Assessing scrub practitioner non-technical skills: a literature review. J Perioper Pract. 2015;25(1-2):12-8. [PubMed ID: 26016259]. https://doi.org/10.1177/1750458915025001-203.

  • 40.

    Kalantari R, Farahani AZ, Garosi E, Badeli H, Jamali J. Translation and psychometric properties of the Persian version of oxford non-technical skills 2 system: Assessment of surgical teams’ non-technical skills in orthopedic surgery wards. Arch Bone Jt Surg. 2019;7(2):173.

  • 41.

    Association of Surgical Technologists. Surgical technology for the surgical technologist: A positive care approach Cengage Learning. 5th ed. 2018. 1328 p. English.

  • 42.

    Jung JJ, Borkhoff CM, Juni P, Grantcharov TP. Non-technical skills for surgeons (NOTSS): Critical appraisal of its measurement properties. Am J Surg. 2018;216(5):990-7. [PubMed ID: 29478826]. https://doi.org/10.1016/j.amjsurg.2018.02.021.

  • 43.

    Yule S, Flin R, Maran N, Rowley D, Youngson G, Paterson-Brown S. Surgeons' non-technical skills in the operating room: reliability testing of the NOTSS behavior rating system. World J Surg. 2008;32(4):548-56. [PubMed ID: 18259809]. https://doi.org/10.1007/s00268-007-9320-z.

  • 44.

    Dedy NJ, Bonrath EM, Ahmed N, Grantcharov TP. Structured training to improve nontechnical performance of junior surgical residents in the operating room: A randomized controlled trial. Ann Surg. 2016;263(1):43-9. [PubMed ID: 25775073]. https://doi.org/10.1097/SLA.0000000000001186.

  • 45.

    Rao R, Dumon KR, Neylan CJ, Morris JB, Riddle EW, Sensenig R, et al. Can simulated team tasks be used to improve nontechnical skills in the operating room? J Surg Educ. 2016;73(6):e42-7. [PubMed ID: 27491863]. https://doi.org/10.1016/j.jsurg.2016.06.004.

  • 46.

    Dedy NJ, Fecso AB, Szasz P, Bonrath EM, Grantcharov TP. Implementation of an effective strategy for teaching nontechnical skills in the operating room: A single-blinded nonrandomized trial. Ann Surg. 2016;263(5):937-41. [PubMed ID: 26079900]. https://doi.org/10.1097/SLA.0000000000001297.

  • 47.

    Hull L, Arora S, Kassab E, Kneebone R, Sevdalis N. Observational teamwork assessment for surgery: content validation and tool refinement. J Am Coll Surg. 2011;212(2):234-243 e1-5. [PubMed ID: 21276535]. https://doi.org/10.1016/j.jamcollsurg.2010.11.001.

Technical Skills Assessment
2022-09-02 10:29:22
You made some good points there. I did a search on the issue and found most persons will go along with your blog.

reply