1. Context
Research builds upon previously acquired knowledge (1), obtained by critically evaluating past published materials, referred to as "literature." Research information literacy entails abilities to find and access information (literature search), evaluate information, and organize and use information (literature review) (2). The literature review is to examine, explain, interpret, and summarize past research on a topic of interest (2, 3) to elaborate a research hypothesis/question (4), synthesize new concepts (5), identify knowledge gaps (2, 6) and methodological biases (2), develop new theories (6), and identify new research directions (2, 4).
The literature review is an integral part of the research process, from developing research ideas to disseminating findings (4, 7). In addition to providing a firm foundation for advancing knowledge (6), a thoughtful, focused, and up-to-date literature review impresses the readers (8) and maximizes relevance, originality, generalizability, and impact of the work, and prevents research duplications (9). Ignorance, misunderstanding, or misapplying the literature are the most frequent pitfalls of a literature review (10). In scientific writing, the lack of an efficient literature review is one of the top reasons for rejecting the submitted works (8).
This review provides a practical guide on conducting an effective literature review. In addition, literature search and evaluation are also briefly discussed.
2. Functions of Literature Review
The output of a literature review can be a standalone report (11) as a genre of paper, named a literature review paper (2), or incorporated into a research proposal, a thesis, or other formats of scientific papers (11). Literature review papers are helpful for researchers to get an up-to-date and well-structured overview of the literature on a specific topic (12). They have several added values, including explicating research gaps, which is helpful for researchers who design a new study in the same area for the first time, outlining the advantages and disadvantages of the methods, and discussing the implications of the findings (12). For writing a research proposal, the literature review helps the authors establish the link between what they are proposing to conduct and what has been already investigated (13).
At the stage of research dissemination, the information gathered for a literature review will be used for writing different sections of an original research paper (i.e., introduction, methods, discussion) (14), enabling authors to justify the importance and novelty of their study, rationalize the study design and methods, and elaborate and contextualize study findings (4, 7).
3. Components of Research Information Literacy
Research information literacy has three main components (5, 11, 15): (1) literature search, including (a) defining the question and (b) finding the relevant literature, (2) literature evaluation, and (3) literature review, including (a) compelling reading, (b) analyzing previous information and synthesizing new information, and (c) providing literature review output. Figure 1 summarizes the main components of research information literacy.
Figure 1.
Components of research information literacy. Created with BioRender.com
3.1. Literature Search
3.1.1. Defining the Question
The starting point in information literacy is that researchers determine the nature and extent of information needed (16). During a literature search, authors may seek relevant responses to various kinds of questions in mind (queries), including contextual questions (i.e., historical background, contemporary debates, open questions, relevant theories, and concepts in the field), descriptive questions (to describe a phenomenon, e.g., definition, pathophysiology, and prevalence of a disease), explanatory questions (to explain a phenomenon or examine reasons for and associations between what exists), exploratory questions (e.g., seeking for the little-known aspects of a topic), and generative questions (which try to provide new ideas for the development of theories) (17, 18). These queries should not be confused with "original research questions," which is the uncertainty that investigators want to answer empirically (19, 20). Indeed, researchers may consult the literature to refine an original research question to be answered empirically (16) or ask a question to be answered through the information available in the literature (16).
3.1.2. Finding the Relevant Literature
Before beginning a literature search, a general topic should be converted into a well-defined, clear, and concise search question (21), including a maximum of 2 - 3 main concepts (e.g., population, exposure, and outcome). Only a well-considered search question may result in a brilliant literature review (22), whereas a more broadly phrased search query may generate endless lists of publications (23). Two approaches can be used for information retrieval: searching by content (non-systematic or conventional) and searching by methodology (i.e., systematic search) (23, 24). A conventional literature search is conducted by content on a specific topic (e.g., biological subjects, diseases, or drugs); in this method, search terms are put together in various logical combinations (23). On the other hand, search by content may be a workable approach if the purpose of search is to find "everything" (23, 24).
A systematic literature search is based on a systematic approach to literature retrieval and should be documented and verifiable (24). Therefore, choosing the right keywords, creating a valid and reliable search strategy (i.e., an organized structure of keywords used to search a database), and selecting the suitable bibliographic databases related to the field are essential elements of a systematic search (25, 26). In a systematic approach, search questions are formulated using various models, including PICO (Patient/Problem, Intervention, Comparison, and Outcome) for clinical questions (27, 28), SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, and Research type) for qualitative/mixed method questions (29), and SPICE (Setting, Perspective, Intervention, Comparison, and Evaluation) for evidence-based practice (18).
In addition to automatic search, snowballing helps researchers determine other references they have missed (30) due to lack of indexing in the databases searched or incomprehensiveness of search strategy (26). Snowballing has been defined as "recursively pursuing relevant references cited in already-retrieved literature and adding them to the search results" (31). In snowballing, authors examine references included in other relevant studies (backward snowballing) or papers that cite these studies (forward snowballing) (30). In addition, following and keeping up with the newest textbooks in the field and searching for books and book chapters may be helpful to ensure the comprehensiveness of the literature search (26). Starting a literature search using published review articles or textbooks is easy to acquire background knowledge on a topic, pick up keywords, or identify immature areas (4). However, books and reviews should not be the foundation of the authors' literature review (4), and searching for peer-reviewed original research articles is essential to formulate an efficient literature review (4).
Upon searching the biomedical literature, three types of literature sources can be found: (1) primary (i.e., original studies based on direct observations, use of statistical records, interviews, or experimental methods of actual practices or the actual impact of practices or policies), secondary (i.e., interpretations or evaluations derived from or refer to the primary literature, e.g., meta-analyses and systematic reviews), and tertiary (i.e., collection of primary and secondary sources, e.g., textbooks, encyclopedia articles, guidebooks, or handbooks) (32). Peer-reviewed original articles are the most valuable and valid sources for citing (33), and citations of reviews should be limited to situations in which a summary of the topic is more effective (34, 35).
For more details on "how to conduct an efficient and complete method to develop literature searches" and "how to perform a systematic search," valuable references are available (25, 26). In addition, several biomedical databases, e.g., Medline/PubMed, Embase, Europe PubMed Central (PMC), Web of Science, Ovid Medline, and Cochrane Library and Clinical Evidence, are available for searching the literature (24, 36). Databases can be categorized as primary and secondary (value-added databases); the primary databases contain data in their original form, e.g., the Swiss protein database, while secondary databases contain sub-classified data, e.g., Single nucleotide polymorphisms (37).
3.2. Literature Evaluation
Upon searching the scientific sources, authors must ensure whether the documents collected from the literature are accurate and valid. Not as formal as a systematic review, a quality assessment of studies is essential to give the literature review credit and reduce bias (4). Retrieved sources should be examined to decide which papers, books, or other sources would be read in detail (23). No simple way is available to determine whether a source meets the quality requirements; however, some key elements may help to decide whether the paper is of sufficient quality to be included (23). Factors needed to be evaluated during the quality and validity assessment of the retrieved documents include the soundness of the study methods (i.e., research design, sampling, data collection, and analyses), the date and source of publication (i.e., characteristics of the journal), and the origins of financial support (11). For laboratory-based studies, considering experimental protocols, data collection and processing, and statistical analyses indicates data quality, reliability, and reproducibility (4). Readers should highlight the strengths and weaknesses of the sources, whether the results can provide sufficient power for generalization, and whether the evidence can be used in clinical practice (38). Fake and retracted sources or those suspected of fraud or misconduct (i.e., manipulation of data or image, fabrication or falsification of data and conclusions, or duplication of publications (39)) should be discarded.
Guidelines like those developed by the Centre for Evidence Based-Medicine (CEBM) may be helpful for literature evaluation (4). Other tools include Newcastle-Ottawa Quality Assessment Form for Cohort Studies, Minimum Criteria to Assess Risk of Bias in Case Series, Cochrane Collaboration modified tool for assessing the risk of bias for clinical trials, checklist for quasi-experimental studies, and Clinical Appraisal Checklist for Experimental Studies (40-42).
3.3. Literature Review
3.3.1. Effective Reading of the Literature
To conduct a literature review successfully, researchers must read scientific documents effectively (5, 43), reading with questions in mind, and search for answers (44). Readers must think about how the paper connects to their idea or study question or how it challenges or supports their hypothesis (45). It is suggested that instead of reading the whole article in one piece, the readers should employ active reading (46). An efficient active task requires three steps: skimming, re-reading, and interpreting. By skimming the paper, readers can understand what the authors did, develop a framework, and get a broad idea. Skimming a paper enables researchers/authors to identify a paper's category, context, validity, clarity, and contribution to the field (43). Box 1 addresses how to read a scientific paper and provides essential tips for active reading.
| Essential Tips |
|---|
| 1. Follow the below three steps: |
| Skim the paper (get a big picture by reading the title and abstract, scan the whole paper and focus on headings and sub-headings, read the conclusion, take notes for any ambiguity, and go through the references) |
| Re-read the paper and focus on the following issues: study question, support of conclusion by study results and other works in the field, repeatability of methodology, reliability of findings, and factors affecting results |
| Interpret results (focus on details, tables, and figures and try to interpret results, read the discussion and lookup for challenging and controversial issues and persuasiveness of arguments, and make sure about getting the main points) |
| 2. Connect the document to other pieces of literature |
| 3. Write down interesting pieces of information and insights |
| 4. Draw interference based on prior knowledge and experiences |
| 5. Visualize the information (draw a diagram to categorize ideas, questions, and connections) |
| 6. Summarize information |
To figure out the presented data accurately, readers need to scrutinize the figures and tables of a paper, either before reading the main text or using a backward-forward approach to the text (45). For figures, they should understand each x and y axis, color scheme, statistical method, and particular plotting approach; for tables, identifying the experimental groups and variables and focusing on the legend or caption are essential to conclude and understand the take-home message (45).
The readers also need to distinguish a paper's objective vs. subjective sections (i.e., the results section compared to other sections are intended to present the author's interpretation of the data). It is also essential to criticize the author's interpretation to evaluate the strength of their conclusions (45). They should also try to make an accurate judgment on the significance of results (based on the magnitude of the effect size and statistical indicators, e.g., P-values and confidence intervals) and the level of evidence provided by the paper (48). They also must clearly distinguish between opinions, theories, and facts (15).
3.3.2. Analyzing and Synthesizing
The most complex step of a literature review is its analysis and synthesis of new concepts around the topic (49, 50). Critical literature analysis consists of carefully examining an issue's main ideas and relationships and providing a critique of existing literature (50). It requires deconstructing the topic into its basic elements, including the history and origins, main related concepts, the key relationships and interactions, research methods, and applications of the topic (50). It should determine the strengths and weaknesses of the literature evidence, whether the available documents are sufficient for generalization, and whether the evidence provides a firm conclusion (38). It should identify the aspects of the topic that remained less investigated, inconsistencies among current perspectives, and address future perspectives on the topic (50).
Synthesis of new concepts or ideas results from critical analysis of multiple sources and includes summarizing and integrating knowledge on the topic (38, 45). Some models of synthesizing new concepts from the literature review have been provided by Torraco (50); these models include (1) a research agenda (which results from the critical analysis of the literature and provides challenging questions and directions for future research), (2) a taxonomy or conceptual classifications of theories (as a means for classification of previous research and a foundation for the development of new theories), (3) alternative models or conceptual frameworks (i.e., rethinking about established issues in the literature), and (4) metatheory (i.e., the integration and synthesis of a literature review to provide a basis for developing metatheory across theoretical domains through future research) (50).
The synthesis follows four primary purposes (49): (1) describing current knowledge around a topic or body of literature, (2) supporting the need for and significance of new research, (3) explaining research findings, and (4) describing the quality of a body of research. An accurate synthesis must provide key information from various scientific papers, reflecting different academic viewpoints on a topic (21). Finally, a critical synthesis demonstrates insight into the current state of knowledge in the field and identifies major remained open questions (51).
A sound synthesis critically discusses literature, identifies methodological problems of the available evidence, and points out research gaps (22). Conducting an analytical synthesis relies on the researcher's knowledge and experience and needs to be based on sound logical and conceptual reasoning. The breadth and depth of the synthesized content may differ depending on the maturity of the research field, compared to an infant field of research where few studies exist. Authors must analyze and synthesize many documents for a mature topic (52). In the latter, focusing on breadth instead of depth and the concepts rather than studies is a practical suggestion (52). Authors must be careful about their own biases and be objective; they should support their viewpoints and justify them clearly to the readers (38). They must be critical of both positive and negative literature and follow a consistent approach (38). Focusing on the topic during the analysis and synthesis is crucial in keeping the literature content constructive and concise (38).
3.3.3. Literature Review Output
The last step is writing a literate review. At this step, authors should go beyond a merely descriptive summary of published literature; however, providing a descriptive overview of the topic and retrieved documents may be helpful. Different approaches can be used to write a literature review note. The use of a "concept-centric" approach helps authors to summarize literature efficiently and identify the research debates they wish to contribute to; this is in contrast to an "author-centric" approach, which essentially presents a summary of the relevant documents and fails to synthesize the literature (6, 52). Chronologically writing the literature is a less common approach used for a topic that has evolved over periods in which theories have been developed, tested, and refined over several decades (51). Sometimes, using tables and figures is a practical way to convey essential concepts and information, summarize the literature, or provide synthesized ideas (52).
Reporting (references to previous research) is essential for writing a literature review output (53). To write a good literature review, authors should choose the best verbs and be careful regarding syntax. Using appropriate reporting verbs is one of the most straightforward ways of attributing content to another source and represents the writer's viewpoints to take either a supportive, tentative, critical, or neutral stance towards the reported findings/ideas (54). Reporting verbs may refer to the author acts (author's act verbs) or the writer/reporter acts (writer's act verbs) (55). The purpose of an academic writer is conveyed by choice of information from other authors to present (research space) and the manner for presenting information to include the writer's view of the information status (evaluative space) (55). The writer's act reporting verbs are divided into comparing and theorizing verbs (55).
Based on different kinds of activity referred to or processes involved, the author's act reporting verbs are categorized as (Figure 2): (1) "real-world or experimental activity verbs," (2) "discourse activity verbs," and (3) "cognition activity verbs." Real-world/experimental activity verbs make up more than half of all the verbs used to report information (53). Based on the kind of information that is being communicated, these verbs are further categorized into (a) finding verbs that are associated with the findings of the study and (b) procedural verbs that are associated with experimental procedures (53). Whether the action referred to affected reporting writer, finding verbs are categorized as (i) objective verbs that mainly refer to a specific result/finding and in which reporting writer is neutral toward the reported information and (ii) effect verbs that mainly refer to generalized conclusions of studies and indicating non-neutral communication of reported information by the reporting writer.
Discourse activity verbs refer to linguistic activities and involve interaction through speech or writing (53). These verbs are categorized as tentative, non-tentative (certainty), and qualification verbs; tentative discourse verbs are associated with generalized conclusions, whereas certainly discourse verbs are related to specific results (53). Whether the event referred to was before the actual experimental activity or not, tentative verbs are categorized as pre-experiment verbs, referring to hypotheses that need to be tested by an experimental study, and post-experiment verbs, referring to results obtained from the study (53). Certainty verbs are associated with reported propositions in more conclusive terms and are categorized as informing/recording verbs and argument verbs (53). Informing verbs are associated with the neutral passing of information without interpretation (53). Argument verbs support the reported proposition of reporting writer, include writer interpretation, and are not neutral but imply that a particular stance is being reported (53). Qualification verbs limit the claim's venerability and indicate the need for further work (53). Overall, informing verbs and objective verbs refer to specific results, whereas effects verbs and tentative verbs refer to generalized statements. Cognition verbs refer to the mental activities of the reporting writer and are associated with reports of consensus ideas generally held by many researchers in the field without attribution to any particular source (53).
The "present simple tense" or "present perfect tense" are generally used to introduce the literature (56). The "present simple tense" is appropriate to discuss established knowledge, i.e., laws, theorems, definitions, and proofs (56). The "present perfect tense" is used where a study was conducted at an undetermined time in the past or an investigation that began in the past and continues today (56, 57). Using the "present perfect tense," authors may communicate that something is very recent or has just been completed (57). The "past tense" is more suitable for referring to a completed activity (e.g., part of a methodology) or referring to information that was once considered true but has now been discounted (57). Furthermore, the "past tense" must be used where an exact date is given in the sentence or a specific piece of information is mentioned (e.g., initial approach or method that has been abandoned) (56, 57). In describing one specific study that was finished before another, the use of "past perfect tense" to emphasize a difference in timing is recommended (57). Keeping consistency throughout the text for choosing verb tenses is essential (22, 57).
4. Do's and Don'ts of Conducting a Literature Review
The poor quality of a literature review, i.e., providing a list of studies followed by statements of what was done in the paper without any direction, critical analysis, or identification of knowledge gaps, is the most common compliance of reviewers (58). An effective and in-depth literature review is much more than a collection of summarized papers' results or even an elaborated "annotated bibliography" of multiple research manuscripts (6). A high-quality literature review is rigorous, consistent, concise, analytical, and synthetic (7). To conduct a well-crafted literature review, authors should balance breadth and depth by including all relevant studies but only describing critical studies in more detail in a structured way (52). A literature review should be coherent and cover the research field of interest, not being endless and full of descriptive summaries of all studies identified through search (52). A literature review should be informative, personal but unbiased, and knowledge-rich content, including conflicting and inconsistent findings and established and current thinking (4). A literature review should be integrative, represent current knowledge, and provide new frameworks and perspectives (50).
Authors should avoid intentional or unconscious preconceived ideas about the literature; it can affect conducting a literature review entirely, from identifying literature sources, selecting papers, or evaluating the evidence (4). They should follow an unbiased approach to paper selection and avoid "cherry-picking" documents that support hypotheses that agree with authors' opinions or support their research findings (4).
5. Conclusions
The literature review is an integral part of the research process and is considered a "knowledge map," which analyzes prior literature and synthesizes new concepts (5). A literature review is like a whole completed puzzle in which each piece of the evidence or information is only one piece of the puzzle (3). The literature review can be a prerequisite to a subsequent empirical study or provide a standalone output (59). To conduct an effective literature review, authors need to determine the required information (search question), translate the search question into keywords, find and evaluate the relevant information, and analyze the information to synthesize new information or refine the study hypothesis. The most challenging task is literature analyzing and synthesizing new concepts around a topic. Concept-centric, authors-centric, and chronological approaches are used to summarize the literature; among them, concept-centric is preferred. A well-crafted literature review impresses the readers (8), maximizes the relevance of the work, and prevents research duplications (9).
