A Review of the Quality of Extant Clinical Practice Guidelines in Cancer Therapy-Induced Mucositis

1. Context

Oral mucositis is a common complication for patients undergoing cancer therapy and has a significant negative effect on a patient’s quality of life (1, 2) as it disrupts eating ability, makes oral hygiene too difficult, and may result in low treatment adherence (3). Severe mucositis is a restricting factor of chemotherapy dose which can have a negative effect on prognosis (1) and bring about increase in overall costs for patient care and health care system (3). For this reason, oral and dental care should be considered as vital components of caring before, during and after chemotherapy (4).

Evidence-based guidelines are considered as powerful instruments to achieve effectiveness and efficient care by professionals, authorities, policy makers, and are valuable instruments in cancer patient care (5). Clinical practice guidelines (CPGs) are systematically developed statements to help people with decision making in the clinical setting (6). But the quality of currently used methods is worrying (7, 8). The quality of the CPGs is defined as: “Ensuring that the biases are likely in developing CPGs process, appropriately have been addressed and recommendations are validated internally and externally and feasible in practice” (6, 9). Given this, CPGs must have special quality criteria that have been defined and validated by AGREE collaboration (5). The aim of AGREE II instrument is providing a framework for assessing the quality of guidelines, presenting a methodological strategy for developing guidelines, promoting that information and how it should be reported in CPGs (6, 9) and helping health care providers to assess a clinical practice guideline before application of its recommendations (8). In doing literature searches, only one study was found on this subject conducted by Potting, et al. which assessed the quality of related CPGs with cancer therapy-induced mucositis, which mentioned documents as guidelines which are lacking effective guideline characteristics (10). This study aimed to assess the methodological quality of existing CPGs about cancer therapy-induced mucositis from different countries and websites.

2. Evidence Acquisition

We conducted a review of published CPGs for the management of cancer therapy-induced mucositis.

2.1. Selection Criteria

Inclusion criteria were: 1, having a clear statement introducing itself as “guideline”; 2, The target population is the adult (above 18 years old); 3, Mucositis induced by cancer therapy; 4, The language of the guideline is English; 5, CPGs were selected which had recommendations including prevention, diagnosis and management for mucositis; 6, CPGs developed by related institutions, communities, associations, cancer care groups, and societies; and 7, Having a guideline structure and characteristics. Included guidelines were original reports based on systematic review and research evidence. We searched whether guidelines were evidence based, whether they reported search strategy, stratified quality of evidence and graded rigor of evidence. If CPGs have been updated, we assessed the last version of the guideline.

Exclusion criteria were: clinical pathways, systematic reviews, protocols, instructional booklets, editorial, narrative reviews, patients’ guides and books.

The present study is a part of a Ph.D. thesis, with the confirmation of the vice-chancellor in research affairs (Code: 395651).

2.2. Search Strategy and Screening of Guidelines

For selecting guidelines, we did a comprehensive search of websites and databases: National comprehensive cancer network (NCCN), the national guideline clearinghouse (NGC), the national institute for health and clinical excellence (NICE), the scottish intercollegiate guidelines network (SIGN), cancer care ontario (CCO), oncology nursing society (ONS), NHSC (http://www.health.gov.au/), guidelines international network (GIN), MEDLINE, Science Direct, Cochrane Library, Scopus, ProQuest, EMBASE, PubMed, Google Scholar, Google, Yahoo, cumulative index to nursing and allied health literature (CINAHL). The mentioned sites were used to find acceptable guidelines published from observation time until April 2017. Also, Persian databases such as SID, Magiran and Iranian medical universities’ websites with keywords Mucositis or oral care and cancer or cancer therapy or chemotherapy and guideline or recommendations and/or combinations of these keywords in the English language were searched.

After finding relevant studies, our research team perused them. Titles, abstracts and full text of the guidelines were assessed and the related ones were screened based on the inclusion and exclusion criteria.

2.3. Appraising Screened CPGs by Applying AGREE II Instrument

After the research team screened guidelines based on the inclusion and exclusion criteria, 5 appraisers assessed screened CPGs by AGREE II instrument independently. The AGREE II instrument updated in 2013 (8) was used for assessing quality of CPGs.

AGREE II instrument contains 6 domains, including “Scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability, editorial independence” (Table 1), and 23 items. At last, in the Overall Assessment domain, an overall score of guideline quality and appraisers’ recommendations for using it are provided. The scoring of this instrument is as follows: a quality score is calculated for each of the 6 domains in the AGREE II instrument. A 7 point scale from 1-strongly disagree (lack of item)- to 7-strongly agree (having expected quality)- was applied to measure the extent that each criterion is provided by the guideline (8, 11, 12). Domain scores were given in percentages. Although they can be compared among guidelines, they are not designed for providing a total score. In other words, scores of the 6 domains are independent and should not be added as a single quality score. The calculated score for each domain is based on this formula: obtained score - minimum possible score/ maximum possible score - minimum possible score (8).

For increasing reliability of assessment according to AGREE II instrument, there were 5 experienced appraisers who scored each guideline independently. For enhancing consistency with existing studies that have used AGREE II instrument and identifying clearly potential useful guidelines in cancer therapy-induced mucositis, overall assessment of AGREE II instrument (3-recommended, 2-recommended with modifications, 1-not recommended) were determined independently by each appraiser. Then, if assigned scores of each domain demonstrated significant variation, we asked appraisers to revise them, but minor differences were ignored. Next, the agreement was achieved by the research team. Finally, the median score was calculated for each item. We practiced according to appraisers’ final scores and AGREE II instrument instruction. Also, we determined satisfactory quality for any guideline that scored above 60% in all six domains, median scores under 30% (unfavorable), median scores between 30– 60% (relatively unfavorable) and above 60% (favorable), as defined by AGREE-II.

3. Results

3.1. Study Selection

After an extensive search, 34 documents were found whose titles, abstracts and full-text the research team assessed according to eligibility criteria. Eventually, 3 CPGs included (Figure 1).

Screening of Found Guidelines

Figure 1.

Screening of Found Guidelines

3.2. Characteristic of CPGs

A summary of characteristics of included CPGs is shown in Table 1. All of the CPGs were developed by an evidence-based approach, were updated versions, and focused on prevention, assessment and management for mucositis.

CPGs were supported financially by professional organizations, governments and academic societies, except for two CPGs which did not disclose their funding sources. All of the three selected CPGs were developed by medical societies, care groups, and professional organizations.

3.3. Appraising CPGs by Using the AGREE-II Instrument

Five appraisers assessed 3 included CPGs using the AGREE-II instrument; Table 2 shows the summaries of appraising each guideline. Assessed CPGs, gained highest scores in the clarity of presentation domain, scope and purpose (median score, 63.3%), stakeholder involvement (median score, 54.4%) respectively, and the lowest scores in editorial independence domain. The scores in applicability (median score, 30%), rigor of development (median score, 28.8%), and editorial independence (median score, 26.7%) domains were unfavorable. Also, every appraiser scores the overall quality of guidelines, guideline number 3, obtained higher scores for recommending and in all of the domains’ scores than two other guidelines. Overall results of 6 domains of AGREE II instrument in 3 guidelines with descending respectively were clarity of presentation (median score, 92.2%), scope and purpose (median score, 63.3%), stakeholder involvement (median score, 54.4%), applicability (median score, 30%), rigor of development (median score, 28.8%), editorial independence (median score, 26.7%), and overall assessment (63.3%).

In general, amongst all of the AGREE II instrument’s domains, in assessed guidelines, clarity of presentation, scope and purpose domains had favorable scores and other domains had unfavorable and relative favorable scores.

This study aimed to assess the methodological quality of existing CPGs about cancer therapy-induced mucositis from different countries and websites and suggested the quality assessment of existing CPGs of cancer therapy-induced mucositis by AGREE II instrument. Scope and purpose domain are related to the overall aim, the target population, and the specific healthcare questions (items 1 - 3) (8). The present study indicated that the scores of this domain for the 3 assessed guidelines were favorable (median score, 63.3%). In the study of Sabharwal, et al. also scope and purpose domain had the highest score (median score, 95%) in comparison to other domains of AGREE II instrument (13). But in our study, the obtained highest score was in the clarity of presentation (median score, 92.2%). Clarity of presentation is concerned with the language, structure, and framework of the guideline and emphasizes clarity, specificity, and unambiguous recommendations (items 15 - 17) (8). The study of Birken, et al. and Deng, et al. showed that assessed guidelines obtained the highest score in this domain (14, 15). Also, in the study of Sabharwal, et al., another domain taking high scores was clarity of presentation (median score, 92%) as appraisers found guidelines providing clear recommendations were easily identifiable (13).

In stakeholders’ involvement domain, it was assessed whether the professional group is presented, patients’ preference and perspective have been acquired, and intended users of guideline clearly has been defined (items 4 - 6) (16). In the present study, stakeholders involvement domain had a higher score than other domains except for clarity of presentation and scope and purpose domains (median score, 54.4%) but its score was not favorable. The study of Sabharwal, et al. showed stakeholders involvement domain (median score, 83%) obtained a favorable score and after 3 domains: scope and purpose, rigor of development and clarity of presentation, had a higher score than other domains (13). Applicability domain focuses on potential barriers and facilitators for implementing guidelines, strategies for promoting, and implications of resources for using the guideline (items 18 - 21) (8). In the present study, applicability (median score; 30%) took an unfavorable score. In two studies of Sabharwal, et al., Xie, et al. and Deng, et al., this domain obtained the lowest score amongst all domains of the AGREE II instrument and in other studies; applicability domain took weak score among the 6 domains of AGREE II instrument (11, 13, 15-18). The weak scores in applicability domain place emphasis on the necessity of improving the conceptualization in guideline implementation during the development process.

The rigor of development is the core of the methodology of guidelines and continued search processes of evidence, grading, briefing and formulation of recommendations (Items 7 - 14) (16). The present study showed domain of rigor of development (median score, 28.8%) was in unfavorable condition. The findings of the study of Cranney, et al. showed that consistency with the methodological quality of current osteoporosis CPGs was low and practically none of CPGs cover dissemination subjects and a few guidelines were judged as acceptable for applying in their current format (19). Moreover, other studies found that most of the CPGs had serious methodological defects (10, 17).

The domain of editorial independence focuses on the investment of issues and conflict of interest for all of the involvement members (Items 22 - 23) (16). In the present study, we found that assessed guideline had the lowest score in this domain. The median scores were not at a favorable level (< 60%). Two other studies showed that almost all of the assessed CPGs were evaluated as weak in the domain of editorial independence (11, 17). Given that the conflict of interest is the most common source of bias in guideline development (20), in the present study, only one of the CPGs reported the conflict of interest.

4. Conclusions

In the overall quality assessment in the present study, obtaining a score (median score, 63.3%) has had a relative favorable level by appraisers (the lowest level of favorable score) and it necessitates undertaking modification and promoting the quality of CPGs. The Potting’s study showed that it is necessary to promote the methodological quality of guidelines if they were applied in clinical practice (10). The results of Briken, et al. indicated that the quality of survivorship of CPGs was weak (14). In addition, the overall recommendation domain is related to rating overall quality of guidelines and whether the guideline is recommended for use in clinical practice (8). In the present study, guideline (number 3) obtained the highest score for recommendation. Several studies indicated a variation to the recommendation of CPGs (10, 13, 14, 16-18). It was concluded in this study that the quality of cancer therapy-induced mucositis CPGs needs to be improved and designing high-quality CPGs must be considered. Incorporating the process of applying resources and monitoring and standardization of the implementation of the CPGs needs to be improved. Therefore, studies conducted to analyze barriers and facilitators of guideline implementation will be useful.