https://doi.org/10.5812/ijcm.91020

The Critical Assessment of the Quality of Common Clinical Guidelines for Administering Chemotherapy Drugs by Using AGREE II Tool

authors:

avatar Tayebeh Karimi ORCID 1 , avatar Masoud Bahrami ORCID 2 , * , avatar Ghasem Yadegarfar ORCID 3

Faculty of Nursing and Midwifery, Isfahan University of Medical Sciences, Isfahan, Iran
Cancer Prevention Research Center, Faculty of Nursing and Midwifery, Isfahan University of Medical Sciences, Isfahan, Iran
Heart Failure Research Centre and Biostatistics Department, School of Public Health, Isfahan University of Medical Sciences, Isfahan, Iran
International Journal of Cancer Management: Vol.12, issue 8; e91020
published online: August 27, 2019
article type: Review Article
received: February 25, 2019
revised: June 12, 2019
accepted: July 29, 2019

How To Cite Karimi T, Bahrami M, Yadegarfar G. The Critical Assessment of the Quality of Common Clinical Guidelines for Administering Chemotherapy Drugs by Using AGREE II Tool. Int J Cancer Manag. 2019;12(8):e91020. https://doi.org/10.5812/ijcm.91020.

Abstract

Context:

Administering chemotherapy drugs in patients with cancer is a complicated and risky process that may have the potential to harm patients and the healthcare team if not done safely.

Objectives:

This study aimed to evaluate the quality of current clinical guidelines for the administration of chemotherapy drugs by using the appraisal of guidelines for research & evaluation II (AGREE II) tool.

Evidence Acquisition:

After searching through authoritative databases and publishing clinical guidelines by institutions, the articles which were published between 2007 to August 2018 with selected keywords were reviewed and the clinical guidelines were evaluated independently by 5 experts using AGREE II tool based on the inclusion and exclusion criteria. The consensus between appraisers was examined using the ICC (intraclass correlation coefficient).

Results:

Among 51 clinical guidelines, 4 of them were matched with the inclusion criteria. All 4 clinical guidelines had a high level of quality, with a minimum score above 60% in their 5 domains. Domain 1(scope and purpose) obtained the highest score (95.27 %) and domain 3 (rigor of development) obtained the lowest score (84.89%). The degree of agreement between evaluators was very good (0.84 – 0.95).

Conclusions:

The overall quality of current clinical guidelines for administering chemotherapy drugs which were evaluated by AGREE II tool, is satisfactory. Therefore, their clinical implementation is recommended to the entire health care team and nurses working in oncology departments, in order to improve the safety of patients and the health care team, and also to prevent the complications of chemotherapy.

1. Context

Chemotherapy is an effective but potentially dangerous treatment (1) and presents risks to the health of patients and healthcare team (2). Chemotherapy drugs are considered as high-risk medications due to their cytotoxic properties. These drugs have limited therapeutic index and have to be administered based on complicated protocols and programs (3). Maintaining the patient’s safety during the administration of chemotherapy drugs is an important goal for health institutions (4). Nurses’ knowledge and skill play a vital role in managing chemotherapy and preventing complications, so nurses should have adequate knowledge in this area and follow evidence-based recommendations and clinical guidelines (5, 6).

Clinical guidelines include advice and recommendations that are systematically developed so they can help the care providers with decision making in specific clinical conditions (7). In fact, clinical guidelines include a set of care recommendations developed for enhancing the effectiveness and safety of interventions, based on national needs and conditions (8). A good clinical guideline should be scientifically valid, practical, and consistent with clinical situations as well as it should eventually improve the patient’s condition (9). However, methods of the developing clinical guidelines are different in every organization, and developers are not always faithful to applying the best evidence while formulating guidelines. Accordingly some guidelines are often below the basic standards, which have raised concerns about the quality of these methods (10, 11). The purpose of quality of clinical guidelines is to ensure that potential biases in the process of developing guidelines are well prevented, the internal and external validity of recommendations are met, and the recommendations are clinically applicable (12). Increase in the number, complexity, and heterogeneity of clinical guidelines and concerns about their quality has led to an increased need to develop internationally-recognized criteria for determining the quality of clinical guidelines (11, 13). The appraisal of guidelines for research & evaluation (AGREE) instrument is a valid evaluation tool that has been approved by the leading critics and authors of international clinical guidelines (14) and is the only clinical guideline assessment tool that has been formally developed and internationally validated. It has been approved by several organizations, including the advisory committee on health research of World Health Organization (WHO) (15) and provides a framework for evaluating the quality of clinical guidelines (16). So far, no study has been conducted in order to assess the quality of clinical guidelines for the administration of chemotherapy drugs. Therefore, this study aimed to assess the quality of clinical guidelines for the administration of chemotherapy drugs by using AGREE II tool in order to clinically apply them for the patients with cancer with the aim of improving evidence-based decisions and increasing the safety of the patient and the health care team.

2. Evidence Acquisition

This study is part of the nursing PhD dissertation entitled “Designing Clinical Guideline for Preparation and Administration of Injectable Chemotherapy Drugs and Prescription Care in Adult Cancer Patients for Nurses Working in Iranian Oncology Centers” with the confirmation of the vice-chancellor in research affairs (code: 395553) and using the adaptation method.

2.1. Selection Criteria

Inclusion criteria were: (1) clinical guidelines for the administration of chemotherapy drugs; (2) the target population consisting of adult cancer patients (above 18 years old); (3) guidelines with English language; (4) clinical guidelines that were developed by institutions, associations, and groups associated with cancer; (5) clinical guidelines which were developed based on evidence-based and systematic reviews (the quality of evidence was examined). The latest updated version of the guideline for assessment was also selected and for the guidelines which were published in several forms, the one with the most details about the development methodology was selected.

Exclusion criteria were: (1) clinical guidelines without accessible full text; (2) service packs, care plans, systematic reviews, patients’ guides, and books.

2.2. Search Strategy and Screening of Guidelines

In order to choose clinical guides, English language articles which were published from 2007 to August 2018 were reviewed. We searched through robust databases such as Scopus, PubMed, ProQuest, Cochrane Library, MEDLINE, Web of Science, and clinical guideline development centers including the Guidelines International Network (GIN), the National Institute for Clinical Excellence (NICE ), The Cancer Care Ontario (CCO), the National Guidelines Clearinghouse (NGC), the Scottish Intercollegiate Guidelines Network (SIGN), and also Iranian article databases such as Irandoc, SID, and Magiran with a combination of keywords: clinical guideline, administration, chemotherapy, and cancer.

After a systematic search, related clinical guidelines to the search keywords were found our research team which was included a nursing PhD student, and two academic members in cancer nursing and familiar with the guideline development, evaluated the obtained clinical relevant guidelines based on the title, abstract, and text, as well as the inclusion and exclusion criteria after excluding repetitive titles. In case of disagreement, the research team reached a consensus by more discussions.

2.3. Appraising Screened CPGs by Applying AGREE II Instrument

Selected clinical guidelines were presented to the relevant experts to be evaluated with AGREE II. The experts reviewed the clinical guidelines individually and separately, in terms of publication year, developer institution, target population, quality of evidence, and main instructions using AGREE II tool. The researcher contacted the health system staff via phone or in-person and persuaded them to co-operate and participate in the appraisal of clinical guidelines. Then, carrying an invitation letter for co-operation containing a brief description of goals and methodology of the research, she visited their offices or workplaces and after obtaining informed consent forms from the appraisers, they were provided with a CD containing 4 clinical guides along with the AGREE II tool, its completion guide and necessary explanations on how to evaluate the guidelines, and their questions were answered. According to the recommendations in AGREE II manual, each guideline is evaluated by at least 2 and preferably 4 appraisers, as increasing the number of appraisers will increase the reliability of the assessment (17). Therefore, in this study, each guideline was appraised by 5 health care workers from different disciplines in order to improve the reliability of the assessment. The review and the appraisal of clinical guidelines took from June to late October 2018.

AGREE II is a standard tool for appraisal of clinical guidelines and helps health care providers to assess a clinical guideline before following its recommendations. The AGREE II tool is a general tool. It can be used to evaluate guidelines in regard to each aspect of the disease, including diagnostic, health promotion, treatment, and intervention guidelines. This tool is proper for assessing provided guidelines in paper or electronic format (software).

AGREE II tool consists of 23 key classified items in the 6 domains of Scope and Purpose (items 1 - 3), Stakeholder Involvement (items 4 - 6), Rigor of Development (items 7 - 14), Clarity of Presentation (items 15 - 17), Applicability (items 18 - 21), and Editorial Independence (items 22 - 23) besides 2 global rating items. Each section addresses one aspect of the guideline’s quality. After completing the 23 items, verified appraisers will answer the 2 global items. The overall assessment requires that the evaluators judge the overall quality of the clinical guideline according to the criteria considered in the appraisal process. They should also state whether they recommend its clinical implementation or not. In this tool, each item is rated and assigned a score of 1 (strongly disagree) to 7 (strongly agree). The score depends on the precision and the quality of the report and is independent for each of the 6 domains. The results were analyzed by standardizing the obtained score for each domain by the following formula, as well as calculating the overall mean score in each domain.

Equation 1.(Obtained score-Minimum possible score)×100Maximum possible score-Minimum possible score

In addition, the scores of the domains should not be added as a single score. In the final evaluation, the overall quality of the guidelines and the appraisers’ final recommendation on their clinical implementation are also presented (18).

The agreement between appraisers was assessed using the intraclass correlation coefficient (ICC) and was defined as slight (0.20 ≥), fair (0.21 - 0.40), moderate (0.41 - 0.60), strong (0.61 - 0.80), or almost perfect agreement (0.81 - 1). According to the previous studies, the overall quality of each clinical guideline was calculated by using a threshold of 60% for the final score of each domain (19, 20). The guidelines with scores above 60% for 5 or more domains were defined as high quality guidelines, for 3 or 4 domains as moderate quality, and for 2 or fewer domains as poor quality. In addition, the overall quality has been measured as (mean ± standard deviation). Recommendations for the clinical implementation of the guidelines have been expressed as recommended, recommend with modifications, and not recommended.

According to the previous articles, the scores of the domains were classified as good (> 80%), acceptable (60% - 79%), low (40% - 59%), and very low (< 40%) (21, 22).

The AGREE II tool is widely used to evaluate the quality standard of clinical guidelines to assess methodological rigor and transparency of guideline development. This tool has been validated and tested for high reliability with detailed framework to assess the quality of guidelines in 6 standardized domains but also provides a methodological strategy for guideline development and content (18, 23, 24). Terrace validated this tool in a study as an international assessment tool for evaluating the quality of clinical guidelines in which 95% of evaluators considered the instrument useful for the appraisal of clinical guidelines. Moreover, the reliability of its mentioned parts was acceptable with the score of 64% - 88% (19). In Iran, Rashidian and Yousefi-Nooraie translated the AGREE tool into Persian, and its validity was confirmed by the joint cooperation committee of Tehran University of Medical Sciences and the Ministry of Health and Medical Education. In addition, the reliability of Persian version of the instrument and its English version was not found to be significantly different after being compared with each other (25).

3. Results

3.1. Selecting the Clinical Guidelines

After performing a systematic search, 51 clinical guidelines associated with search keywords were found, a total of 36 clinical guidelines remained after excluded repetitive titles. Then the abstracts of the found clinical guidelines were investigated by the research team and 21 of them were excluded. Next, the full texts of the 15 remaining guidelines were evaluated based on the inclusion and exclusion criteria, and 11 of them were removed. Finally, 4 guidelines on the administration of chemotherapy drugs were presented to the relevant experts in order to be appraised using AGREE II tool. The flow chart for selecting the clinical guidelines is shown in Figure 1.

The flow chart of the identification process for guideline selection
The flow chart of the identification process for guideline selection
Figure 1.

The flow chart of the identification process for guideline selection

3.2. The Characteristics of Clinical Guidelines

An Australian clinical guideline developed by the Clinical Oncology Society of Australia aims the safe prescribing, dispensing, providing, and administration of cancer treatment drugs (26). A Canadian guideline, developed by the Cancer Care Ontario Institute, seeks to provide guidance on the processes, technologies, and instruments of administering chemotherapy drugs (27). Another guideline is developed by the London Cancer Alliance from the UK, with the purpose of prescribing cytotoxic drugs (28). The other clinical guideline is from Ireland, created by Northern Ireland Cancer Network, which focuses on administering anti-cancer drugs (29). Only in one of the clinical guidelines, the conflict of interest has been mentioned (27). All 4 clinical guidelines are evidence-based (26-29). The updated versions of 2 guidelines are not available, but are current (27, 28). The updated version of one guideline is for 2017 (26) and another one has been planned for 2019 (29). Only one guideline has mentioned its funding source (27). The Characteristics of Clinical Guidelines are presented in Table 1.

Table 1.

Characteristics of Clinical Guidelines

Guideline TitlePublication YearCountryInstitutionPurposeConflicts of InterestEvidence-Based GuidelineUpdateFundingSize of Complete Guideline (Pages)
1. COSA guidelines for the safe prescribing, dispensing, and administration of systemic cancer therapy2010AustraliaClinical Oncological Society of Australia (COSA)Safe prescribing, dispensing, and administration of cancer therapyNot availableYesJune 2017Not disclose247
2. Safe Administration of Systemic Cancer Therapy. Part 2: Administration of Chemotherapy and Management of Preventable Adverse Events2014CanadaCancer Care OntarioProvide guidance on processes, technologies, and devices for the administration of chemotherapyStatement about conflicts of interest of group members presentYesNot availableOntario Ministry of Health and Long-Term Care83
3. London Integrated Care Systems (ICSs), Guidelines for Safe Prescribing, Handling, and Administration of Systemic Anti-Cancer Therapy2015EnglandLondon Cancer AllianceAdministration of cytotoxic drugsNot availableYesNot availableNot disclose98
4. Guidelines for the safe prescribing, handling and administration of Systemic Anti-Cancer Therapies2017IrelandNorthern Ireland Cancer NetworkAdministration of Systemic Anti-Cancer TherapiesNot availableYesMay 2019Not disclose92

3.3. The Appraisal of the Clinical Guidelines Using AGREE II

A total of 4 clinical guidelines were reviewed and evaluated by 5 expert appraisers and analyzed after standardizing the scores obtained for each domain, as well as calculating the overall mean score of each. In addition, the overall quality of the clinical guidelines and the final opinions of appraisers about the clinical practice of the guidelines are presented in Table 2.

Table 2.

Summary of the Average of Domain Scores of Guidelines According to AGREE IIa

Guideline TitleDomain 1 (Scope and Purpose)Domain 2 (Stakeholder Involvement)Domain 3 (Rigor of Development)Domain4 (Clarity of Presentation)Domain 5 (Applicability)Domain 6 (Editorial Independence)Total Score MeanOverall QualityOverall Recommendation (to Use Guideline)
1. COSA guidelines for the safe prescribing, dispensing and administration of systemic cancer therapy100% (good)b77.78% (acceptable)b86.25% (good)b96.67% (good)b80% (good)b61.67% (acceptable)b80.58%HighYes (4 reviewers) yes, with modifications (1 reviewer)
2. Safe Administration of Systemic Cancer Therapy. Part 2: Administration of Chemotherapy and Management of Preventable Adverse Events95.56% (good)b93.33% (good)b81.25% (good)b88.89% (good)b89.17% (good)b98.33% (good)b80.87%HighYes (2 reviewers) yes, with modifications (3 reviewer)
3. London Integrated Care Systems (ICSs), Guidelines for Safe Prescribing, Handling and, Administration of Systemic Anti-Cancer Therapy92.22% (good)b91.11% (good)b83.75% (good)b95.56% (good)b92.5% (good)b91.67% (good)b82.46%HighYes (3 reviewers) yes, with modifications (2 reviewer)
4. Guidelines for the safe prescribing, handling, and administration of Systemic Anti-Cancer Therapies93.33% (good)b93.33% (good)b88.33% (good)b93.33% (good)b94.17% (good)b95% (good)b84.49%HighYes (4 reviewers) yes, with modifications (1 reviewer)
Total domain score; mean95.27% (good)88.88% (good)84.89 (good)93.61% (good)88.96% (good)86.66% (good)

aDomain scores ≥ 80% = good, 60% - 79% = acceptable, 40% - 59% = low; < 40% = very low.

bTotal score of domain > 60%.

All 4 clinical guidelines had a high level of quality, with scores above 60% in at least 5 domains. Among them, the guidelines for the safe prescribing, handling, and administration of systemic anti-cancer therapies (29) earned the highest score (84.49%). The range of the scores of the domains falls between 61.67% (the lowest score which belonged to the domain 6 (editorial independence) in COSA guidelines for the safe prescribing, dispensing and administration of systemic cancer therapy (26)), and 100% (the highest which belonged to the domain1 (score and purpose) in COSA guidelines for safe prescribing, dispensing, and administration of systemic cancer therapy (26)). By comparing the mean scores of each domain among the guidelines, it is observed that domain 1 had obtained the highest scores (95.27%), while domain 3 (rigor of development) had the lowest mean score (84.89%).

In domain 1 with the highest mean score (95.27%), COSA guidelines for the safe prescribing, dispensing, and administration of systemic cancer therapy had, the highest score (26) (100% = good) and the London Integrated Care Systems (ICSs) guidelines for safe prescribing, handling, and administration of systemic anti-cancer therapy (28)had the lowest score (92.22% = good).

In domain 2 (stakeholder involvement) with a mean score of 88.88%, the 2nd and the 4th guidelines had the highest scores (93.3% = good) and guideline 1 had the lowest score (77.78% = acceptable). In domain 3 (rigor of development) with the lowest mean score (84.89%), the 4th guideline had the highest score (83.33% = good) and the 2nd guideline had the lowest score (81.25% = good). In domain 4 (clarity of presentation), with a mean score of 93.61%, guideline 1 had the highest score (96.67% = good) and guideline 2 had the lowest one (88.89% = good).

In domain 5 (applicability), with a mean score of 88.96%, the 4th guideline had the highest score (94.17% = good) and guideline 1 had the lowest one (80% = good). In domain 6 (editorial independence), with a mean score of 86.66%, guideline 2 had the highest score (98.33% = good) and guideline 1 had the lowest (61.67% = acceptable).

The highest agreement among appraisers was belonged to domain 1 (scope and purpose) (0.95 = very good) and the lowest one was belonged to domain 3 (rigor of development) (0.84 = very good).

4. Conclusions

The findings of this study showed the appraisal of common clinical guidelines for the administration of chemotherapy drugs using AGREE II instrument is acceptable and the overall quality of all four clinical guidelines is high. All domains had a score above 60%, and the scores of 6 domains in all guidelines varied from good to acceptable. These results may reflect the improving trend in clinical guidelines methodology over the past decade. The criteria used to appraise clinical guidelines have also been enhanced.

The obtained results are somehow steady considering the scores of the domains. In addition, domain 1 (scope and purpose) achieved good scores with the least variability in all guidelines. Furthermore, this domain has been well explained in all the evaluated guidelines.

Domain 1 (scope and purpose) and domain 4 (clarity of presentation) have obtained the highest scores, both of which with a score above 90%. This finding was consistent with the results of the previous studies on the appraisal of clinical guidelines using AGREE II tool, which covered various clinical topics (19, 21, 22, 30-33). Such high scores are a result of full explanations of these domains presented by clinical guidelines developers. These domains also consist of items such as the general objectives of the guidelines, the clarity and characteristics of recommendations, population and health questions, the clarity of health management items, and key recommendations that cannot be easily excluded.

Domain 3 (rigor of development) had the lowest score among the domains. This was in line with the study conducted by Gavriilidis et al. that they assessed the clinical guidelines for the resection of hepatocellular carcinoma (15). In other studies, this domain had also the lowest scores (14, 34), which may be a result of the lack of methodological expertise and insufficient information on the methods of developing clinical guidelines and the fact that two of these clinical guidelines lacked the updated versions. Therefore, it is recommended to use expert librarians in the process of developing clinical guidelines, to explicitly describe the information on search strategies, to explore databases, the process of selecting articles, the method of evidence quality assessment, and recommendations rating.

Four of the appraisers recommended the clinical implementation of COSA guidelines for the safe prescribing, dispensing and administration of systemic cancer therapy, as well as for safe prescribing, handling, and administration of systemic anti-cancer therapies. However, studies have shown that the implementation of evidence-based information in many healthcare settings remains a challenge, and it is often difficult to appraise the clinical applicability and performance of clinical guidelines. It takes almost 5 years for a clinical guideline to be clinically used as a routine. Even most of the approved guidelines are not fully pursued by staff (35, 36).

Most of the domains had high reliability. Therefore, the degree of agreement between the appraisers showed a strong correlation (0.84 - 0.95). The highest degree of agreement among the evaluators was found in domain 1 (scope and purpose) (0.95 = very good) and the lowest, in domain 3 (rigor of development) (0.84 = very good) this domain has 8 items and this is the most important reason for the ICC value being less compared to other domains.

Clinical guidelines are known as the main means of knowledge transfer, which require expertise and teamwork for their successful and sustainable implementation. The application and implementation of clinical guidelines is also dependent on the staff’s complete trust in their quality and credibility. Since nurses are responsible for the clinical administration of drugs and play a major role in chemotherapy, and due to the high overall quality of common clinical guidelines for prescribing chemotherapy drugs, their clinical use is recommended to nurses and all treatment team members who work in oncology wards in order to improve the safety of the patients and the treatment team and also to prevent complications during chemotherapy.

The present study had some limitations. We performed an inclusive, accurate and complete search although some clinical guidelines and their updated versions may have been overlooked. Another limitation was that only the guidelines in English language were appraised, due to the research team members did not fluent in other languages. The advantage of this study was using AGREE II tool, which has a high reliability and validity. The appraisers used instructions on how to use the tool, and all the guidelines were independently evaluated.

Acknowledgements

References

  • 1.

    Weingart SN, Li JW, Zhu J, Morway L, Stuver SO, Shulman LN, et al. US cancer center implementation of ASCO/oncology nursing society chemotherapy administration safety standards. J Oncol Pract. 2012;8(1):7-12. [PubMed ID: 22548004]. [PubMed Central ID: PMC3266320]. https://doi.org/10.1200/JOP.2011.000379.

  • 2.

    Kreidieh FY, Moukadem HA, El Saghir NS. Overview, prevention and management of chemotherapy extravasation. World J Clin Oncol. 2016;7(1):87-97. [PubMed ID: 26862492]. [PubMed Central ID: PMC4734939]. https://doi.org/10.5306/wjco.v7.i1.87.

  • 3.

    Griffin MC, Gilbert RE, Broadfield LH, Easty AE, Trbovich PL, Griffin MC, et al. ReCAP: Comparison of independent error checks for oral versus intravenous chemotherapy. J Oncol Pract. 2016;12(2):168-9. e180-7. [PubMed ID: 26420888]. https://doi.org/10.1200/JOP.2015.005892.

  • 4.

    Leung M, Bland R, Baldassarre F, Green E, Kaizer L, Hertz S. Safe administration of of Chemotherapy: safety during chemotherapy ordering, transcribing, dispensing, and patient identification. 2012. Available from: https://www.cancercareontario.ca/en/guidelines-advice/types-of-cancer/1196.

  • 5.

    O'Grady NP, Alexander M, Burns LA, Dellinger EP, Garland J, Heard SO, et al. Guidelines for the prevention of intravascular catheter-related infections. Clin Infect Dis. 2011;52(9):e162-93. [PubMed ID: 21460264]. [PubMed Central ID: PMC3106269]. https://doi.org/10.1093/cid/cir257.

  • 6.

    Cicolini G, Simonetti V, Comparcini D, Labeau S, Blot S, Pelusi G, et al. Nurses' knowledge of evidence-based guidelines on the prevention of peripheral venous catheter-related infections: A multicentre survey. J Clin Nurs. 2014;23(17-18):2578-88. [PubMed ID: 24354398]. https://doi.org/10.1111/jocn.12474.

  • 7.

    National Collaborating Centre for Methods and Tools (NCCMT). Critically appraising practice guidelines: The AGREE II instrument. Hamilton: McMaster University; 2013. Available from: http://www.nccmt.ca/resources/search/100.

  • 8.

    Harbour R, Lowe G, Twaddle S. Scottish intercollegiate guidelines network: The first 15 years (1993-2008). J R Coll Physicians Edinb. 2011;41(2):163-8. [PubMed ID: 21677923]. https://doi.org/10.4997/JRCPE.2011.209.

  • 9.

    Lo Vecchio A, Giannattasio A, Duggan C, De Masi S, Ortisi MT, Parola L, et al. Evaluation of the quality of guidelines for acute gastroenteritis in children with the AGREE instrument. J Pediatr Gastroenterol Nutr. 2011;52(2):183-9. [PubMed ID: 20808248]. [PubMed Central ID: PMC3713480]. https://doi.org/10.1097/MPG.0b013e3181e233ac.

  • 10.

    Burgers JS, Grol R, Klazinga NS, Makela M, Zaat J, Agree Collaboration. Towards evidence-based clinical practice: An international survey of 18 clinical guideline programs. Int J Qual Health Care. 2003;15(1):31-45. [PubMed ID: 12630799]. https://doi.org/10.1093/intqhc/15.1.31.

  • 11.

    Melnyk BM, Fineout-Overholt E. Evidence-based practice in nursing & healthcare: A guide to best practice. Third ed. Lippincott Williams & Wilkins; 2015.

  • 12.

    Dans AL, Dans LF. Appraising a tool for guideline appraisal (the AGREE II instrument). J Clin Epidemiol. 2010;63(12):1281-2. [PubMed ID: 20605571]. https://doi.org/10.1016/j.jclinepi.2010.06.005.

  • 13.

    Burgers JS, Fervers B, Haugh M, Brouwers M, Browman G, Philip T, et al. International assessment of the quality of clinical practice guidelines in oncology using the Appraisal of Guidelines and Research and Evaluation Instrument. J Clin Oncol. 2004;22(10):2000-7. [PubMed ID: 15143093]. https://doi.org/10.1200/JCO.2004.06.157.

  • 14.

    Yang C, Zhang Z, Zhang L, Tian J, Yu D, Wang J, et al. Quality assessment of clinical practice guidelines on tic disorders with AGREE II instrument. Psychiatry Res. 2018;259:385-91. [PubMed ID: 29120847]. https://doi.org/10.1016/j.psychres.2017.08.060.

  • 15.

    Gavriilidis P, Roberts KJ, Askari A, Sutcliffe RP, Huo TL, Liu PH, et al. Evaluation of the current guidelines for resection of hepatocellular carcinoma using the Appraisal of Guidelines for Research and Evaluation II instrument. J Hepatol. 2017;67(5):991-8. [PubMed ID: 28690176]. https://doi.org/10.1016/j.jhep.2017.06.028.

  • 16.

    Zhang ZW, Liu XW, Xu BC, Wang SY, Li L, Kang YY, et al. Analysis of quality of clinical practice guidelines for otorhinolaryngology in China. PLoS One. 2013;8(1). e53566. [PubMed ID: 23349719]. [PubMed Central ID: PMC3548820]. https://doi.org/10.1371/journal.pone.0053566.

  • 17.

    Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G. The appraisal of guidelines for research & evaluation (AGREE) instrument. 2017. Available from: https://www.agreetrust.org/resource-centre/agree-ii/.

  • 18.

    Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: Advancing guideline development, reporting and evaluation in health care. CMAJ. 2010;182(18):E839-42. [PubMed ID: 20603348]. [PubMed Central ID: PMC3001530]. https://doi.org/10.1503/cmaj.090449.

  • 19.

    Agree Collaboration. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: The AGREE project. Qual Saf Health Care. 2003;12(1):18-23. [PubMed ID: 12571340]. [PubMed Central ID: PMC1743672]. https://doi.org/10.1136/qhc.12.1.18.

  • 20.

    Ou Y, Goldberg I, Migdal C, Lee PP. A critical appraisal and comparison of the quality and recommendations of glaucoma clinical practice guidelines. Ophthalmology. 2011;118(6):1017-23. [PubMed ID: 21507487]. https://doi.org/10.1016/j.ophtha.2011.03.038.

  • 21.

    Sekercioglu N, Al-Khalifah R, Ewusie JE, Elias RM, Thabane L, Busse JW, et al. A critical appraisal of chronic kidney disease mineral and bone disorders clinical practice guidelines using the AGREE II instrument. Int Urol Nephrol. 2017;49(2):273-84. [PubMed ID: 27804080]. https://doi.org/10.1007/s11255-016-1436-3.

  • 22.

    Messina C, Bignotti B, Bazzocchi A, Phan CM, Tagliafico A, Guglielmi G, et al. A critical appraisal of the quality of adult dual-energy X-ray absorptiometry guidelines in osteoporosis using the AGREE II tool: An EuroAIM initiative. Insights Imaging. 2017;8(3):311-7. [PubMed ID: 28432574]. [PubMed Central ID: PMC5438319]. https://doi.org/10.1007/s13244-017-0553-6.

  • 23.

    Chua M, Ming J, Chang SJ, Santos JD, Mistry N, Silangcruz JM, et al. A critical review of recent clinical practice guidelines for pediatric urinary tract infection. Can Urol Assoc J. 2018;12(4):112-8. [PubMed ID: 29319481]. [PubMed Central ID: PMC5905538]. https://doi.org/10.5489/cuaj.4796.

  • 24.

    Polus S, Lerberg P, Vogel J, Watananirun K, Souza JP, Mathai M, et al. Appraisal of WHO guidelines in maternal health using the AGREE II assessment tool. PLoS One. 2012;7(8). e38891. [PubMed ID: 22912662]. [PubMed Central ID: PMC3418264]. https://doi.org/10.1371/journal.pone.0038891.

  • 25.

    Rashidian A, Yousefi-Nooraie R. Development of a Farsi translation of the AGREE instrument, and the effects of group discussion on improving the reliability of the scores. J Eval Clin Pract. 2012;18(3):676-81. [PubMed ID: 21410842]. https://doi.org/10.1111/j.1365-2753.2011.01649.x.

  • 26.

    Clinical Oncological Society of Australia. COSA guidelines for the safe prescribing, dispensing and administration of systemic cancer therapy. 2010. Available from: https://www.cosa.org.au/publications/guidelines/.

  • 27.

    Leung M, Bland R, Baldassarre F, Green E, Kaizer L, Hertz S. Safe administration of systemic cancer therapy part 2: Administration of chemotherapy and management of preventable adverse events. 2014. Available from: https://www.cancercareontario.ca/en/guidelines-advice/types-of-cancer/1201.

  • 28.

    London Cancer Alliance. London ICS guidelines for the safe prescribing, handling and administration of systemic anti-cancer treatment drugs. 2015. Available from: http://www.londoncanceralliance.nhs.uk/media/104582/pan-london-cytotoxic-handbook-july-2015.pdf.

  • 29.

    Northern Ireland Cancer Network. Guidelines for the safe prescribing, handling and administration of systemic anti-cancer therapies (SACT). 2017. Available from: http://nican.hscni.net/content/guidelines-safe-prescribing-handling-and-administration-systemic-anti-cancer-therapies-sact.

  • 30.

    Vasse E, Vernooij-Dassen M, Cantegreil I, Franco M, Dorenlot P, Woods B, et al. Guidelines for psychosocial interventions in dementia care: A European survey and comparison. Int J Geriatr Psychiatry. 2012;27(1):40-8. [PubMed ID: 21370278]. https://doi.org/10.1002/gps.2687.

  • 31.

    Zeng L, Zhang L, Hu Z, Ehle EA, Chen Y, Liu L, et al. Systematic review of evidence-based guidelines on medication therapy for upper respiratory tract infection in children with AGREE instrument. PLoS One. 2014;9(2). e87711. [PubMed ID: 24586287]. [PubMed Central ID: PMC3930557]. https://doi.org/10.1371/journal.pone.0087711.

  • 32.

    Holvoet T, Raevens S, Vandewynckel YP, Van Biesen W, Geboes K, Van Vlierberghe H. Systematic review of guidelines for management of intermediate hepatocellular carcinoma using the Appraisal of Guidelines Research and Evaluation II instrument. Dig Liver Dis. 2015;47(10):877-83. [PubMed ID: 26250948]. https://doi.org/10.1016/j.dld.2015.07.005.

  • 33.

    Gillespie BM, Bull C, Walker R, Lin F, Roberts S, Chaboyer W. Quality appraisal of clinical guidelines for surgical site infection prevention: A systematic review. PLoS One. 2018;13(9). e0203354. [PubMed ID: 30212487]. [PubMed Central ID: PMC6136720]. https://doi.org/10.1371/journal.pone.0203354.

  • 34.

    Chen Y, Hu S, Wu L, Fang X, Xu W, Shen G. Clinical practice guidelines for hypertension in China: A systematic review of the methodological quality. BMJ Open. 2015;5(7). e008099. [PubMed ID: 26179649]. [PubMed Central ID: PMC4513449]. https://doi.org/10.1136/bmjopen-2015-008099.

  • 35.

    Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004;8(6):iii-v. 1-72. [PubMed ID: 14960256]. https://doi.org/10.3310/hta8060.

  • 36.

    Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients' care. Lancet. 2003;362(9391):1225-30. [PubMed ID: 14568747]. https://doi.org/10.1016/S0140-6736(03)14546-1.