 Research Insights Rapid Evidence Reviews for Health Policy and Practice Summary Organizations such as the Cochrane and Campbell Collaborations, Health policymakers in the United States and elsewhere are increas- among others, have created programs to produce high-quality, ingly interested in making evidence-based policy decisions, but objective, and policy-relevant systematic reviews. For example, over require assistance navigating the large volume of material in peer-re- a 10-year period, the Robert Wood Johnson Foundation’s Synthesis viewed and grey literature. This trend has led many organizations to Project summarized what was known about various health policy identify methods for producing high-quality evidence reviews faster topics, weighed the strength of research findings, identified gaps in than traditional systematic reviews to accommodate policymakers’ knowledge, and emphasized the policy implications of key findings.7,8 timeframes. After examining several existing rapid evidence review Organizations have also developed methodological guidelines to help program models and the insights of experienced evidence reviewers, others conduct high-quality SRs to inform policymaking.9,10,11 we find that there are methodological and organizational strategies that may help organizations produce rapid reviews that are high Organizations using these SR methods produce comprehensive, quality, timely, and relevant to policymakers’ decisions. relevant, and defensible results, but often require significant time and other resources to complete them.12,13 Research has documented Introduction the challenges for those seeking to use SRs to inform time-sensitive Health services and policy research (HSR) is meant to be useful policy decisions.14 In response, organizations in the United States to policy and practice.1 In recent years, policymakers and health and elsewhere have developed programs and methods for rapid systems leaders have become increasingly interested in making evidence reviews (RERs). As part of an effort to develop a rapid evidence-based policy decisions.2,3,4 One way to help policymakers review capability to support the Robert Wood Johnson Foundation’s understand and benefit from large bodies of HSR is through the commitment to building a Culture of Health,15 this brief examines use of systematic reviews of evidence.5 Systematic reviews (SRs) use what defines RERs, different approaches to conducting them, and explicit and pre-specified methods to identify, select, critically ap- the implications of applying RERs to health and social policies. In praise, and analyze data from research studies relevant to a particu- addition, the brief describes several examples of RER programs in lar research question.66 the United States and elsewhere. Genesis of this brief: This brief is part of AcademyHealth’s Rapid Evidence Review project, an initiative supported by the Robert Wood Johnson Foundation to pilot alternative approaches to quickly, efficiently, but rigorously synthesize and communicate evidence to inform public and private decision-making.  During 2015, AcademyHealth reviewed how others in the United States and beyond have approached this task. This brief summarizes the results of that effort. 1 Rapid Evidence Reviews for Health Policy and Practice Box 1: Terminology Used to Describe the Rapid Review of Evidence The words used to describe rapid reviews of evidence vary. Here we present several examples from the published academic literature along with explanations of the rationale behind each choice of terminology. Term Source Rationale Brief review Abrami et al, 2010 Emphasizes that timeframe and scope, rather than timeframe alone, distinguish these reviews from comprehensive reviews. Rapid review Polisena et al, 2015 Describes evidence syntheses that streamline systematic review methods to achieve shorter turnaround than required for full systematic reviews. Evidence summary Khangura et al, 2012 Adopted by Ontario Hospital Research Institute’s Knowledge to Action program to describe their explicit process after a lack of agreement about methodology underlying “rapid reviews” caused some to question the validity of products using that term. Rapid review Featherstone et al, 2015; Borrowed from Andradas et al, 2008 to describe “brief, readable, and usable responses” completed in Hartling L. et al, 2015. six months to guide decision-making. Rapid evidence Thomas, et al, 2013 Describes processes that speed up systematic reviews “to dovetail more closely with policy and assessment practice decision-making timescales.” Authors see “rapid evidence assessment” as synonymous with “rapid review” and “brief review.” Rapid syntheses Wilson MG et al, 2014 Distinguished from rapid systematic reviews and rapid realist reviews in that: (1) they are not comprehensive systematic reviews completed in a condensed timeline; (2) questions can take many forms and relate to problems, options, or implementation considerations rather than the effects of a single option; and (3) evidence reviewed can include systematic reviews and other evidence, rather than just single studies. What are “Rapid Evidence Reviews”? • RERs employ a variety of strategies to accelerate the review pro- RERs are efforts to assess and synthesize evidence in less time than cess. Reviewers may expedite administrative processes, dedicate traditional or “full” SRs. There is no single, definitive terminol- more staff and resources to conduct a full SR in a shorter time- ogy or definition of this capacity (see Box 1).While SRs provide frame, form specialized teams skilled at conducting RERs, narrow comprehensive examinations of the evidence base and are often or target the review’s focus to be more manageable, and/or modify reviewer-driven, RERs emphasize efficiency and responsiveness to or eliminate traditional SR steps.19 the needs of policymakers or other stakeholders. In some cases, such • The end products generated by rapid review processes vary. decision makers directly commission them. In health policy, RERs RERs may produce reference lists, annotated bibliographies, can inform government policy or clinical practice, influence deci- summaries of abstracts or summaries of key findings from the sions about insurance coverage and reimbursement, advise about the evidence, evidence overviews without policy recommendations, or structure and delivery of care in health systems, and offer guidance meta-analyses, or even comprehensive reviews like SRs.20 ,21 on implementation of new care models or system changes. Approaches to Making Rapid Reviews Rapid Because the precise needs of health policymakers vary, RERs employ Methods for conducting RERs are commonly adapted or modified a broad spectrum of streamlined secondary research methodologies. from standard SR methods. Modifications to SR methods can make Boxes 2 through 5 below provide examples of RER programs de- RERs more efficient or responsive, but they carry tradeoffs. Because signed to inform policy in the United States and elsewhere, illustrat- policymakers often commission RERs to respond to a specific policy ing methods RERs employ to achieve efficiency and responsiveness: issue, these policymakers’ needs are a guiding focus of the reviews •RERs encompass a wide range of timeframes. RERs can take themselves. In some cases, a policymaker’s deadline for making a anywhere from a few business days to a year to complete.16,17 The decision determines what is possible. This must be balanced against time required may reflect the time and resources available to do the risks of making a wrong decision. the work, the information needed by a decision maker, or the date of an impending policy decision. The timeframe chosen has impli- Reviewers commonly choose among several strategies for conducting cations for what a reviewer can credibly produce.18 an RER that reflect the particular context and resource constraints. 2 Rapid Evidence Reviews for Health Policy and Practice These strategies may focus on the review’s relevance rather than Box 2: Different Approaches for Different its speed, and reviewers may cut out or shorten steps that are not Timeframes: The McMaster University Health necessary to meet requestor’s needs. McMaster University’s Rapid Forum’s Rapid Response Program Response program offers requestors a variety of different approach- The McMaster Health Forum (www.mcmasterhealthforum.org) supports es and products to choose from (see Box 2). evidence-informed policymaking about health systems through several programs. These include: a Rapid Response Program; Health Systems Evidence (a continuously updated repository of syntheses of research To meet time and resource constraints, RER programs can: evidence about governance, financial and delivery arrangements within • Use specialized review teams to mobilize and complete RERs health systems, and about implementation strategies); stakeholder dialogues that bring together policymakers, leaders, citizens and quickly. Review teams who have prior experience conduct- researchers for off-the-record deliberations about pressing health challenges; and citizen panels that engage groups of 10 to 14 people ing SRs and/or RERs can more easily respond to requests for in off-the-record discussions about their views and experiences related evidence reviews on short notice than teams who need to train or to health challenges. The Rapid Response Program was developed in 2014 as a way to “fill the gap” between policymakers using Health recruit additional staff.22,23,24 Having review staff knowledgeable Systems Evidence to identify relevant syntheses on their own and in the topic area of a proposed RER may also be helpful. This ap- when they need them and a longer-term process (typically between proach requires an organization to already employ reviewers with three to six months) for convening stakeholder dialogues that provide the opportunity for health-system policymakers, stakeholders and the necessary skills and experience. researchers, who are informed by a pre-circulated evidence brief to deliberate as a group about a health-system problem, options to • Restrict evidence searches to particular contexts, geographic address it, and key implementation considerations and to identify next areas, or other settings relevant to the requestor’s circum- steps for different constituencies. The Rapid Response Program fills this gap by responding to requests from decision makers who need stances. This way, it is possible to reduce the amount of evidence support with identifying research evidence about questions related reviewers need to evaluate for inclusion. The approach limits a to health system governance, financing, and delivery arrangements, but who also need support with identifying and synthesizing research review’s generalizability, but it saves time.25,26 evidence. • Focus review topics to facilitate more efficient reviews. RERs with narrowly focused topics may not require the location or To accommodate the timelines of policymakers, the Forum’s Rapid Response Program offers rapid syntheses that can be completed analysis of as much evidence as reviews with broader topics. One in three, 10, or 30 business days.52 Rapid syntheses produced in strategy used to focus a review topic is to narrowly define the longer time frames are able to incorporate more and broader kinds of evidence and provide more detailed summaries of that evidence search terms used to identify relevant data. The tradeoff is that than those produced in shorter timeframes.53 For example, syntheses reviews of narrow topics may identify less evidence of relevance produced in three business days provide summary tables of key findings from existing SRs and economic evaluations, but exclude or provide less generalizable conclusions than broader and more individual primary research studies, grey literature, and environmental comprehensive reviews.27 Overly narrow topics or search terms scans from the report. In contrast, 30-day syntheses provide summary tables of relevant systematic reviews, economic evaluations and may even result in reviews that do not locate relevant studies.28 primary studies (where no systematic reviews are available), quality • Conduct selective evidence searches to speed up a review. After appraisals of all included systematic reviews and a detailed narrative summary of key findings. External topical experts also review the draft deciding upon the question or topic and relevant search terms, final reports produced within the 10 and 30 business day timeframes. reviewers must decide where to search for evidence (e.g., which Regardless of the timeframe, Rapid Response reports do not make policy recommendations nor do they replicate full SR methods. bibliographic databases) and criteria for what types of studies to include in the review. Rather than including comprehensive and statistical conclusions about the body of evidence.32 RERs literature searches, RERs may involve searching only a small may include less extensive data analysis or synthesis than SRs. number of relevant databases, or only include previous system- RERs may focus only on findings most relevant to policymakers’ atic reviews, or studies with specific study designs, languages, or needs. The tradeoff is that important observations or reflections years of publication. Restricted searches may identify fewer ir- on the quality of the evidence may be missed. relevant studies than comprehensive searches, but may also miss important evidence.29 • Automate parts of the process. Advances in technology such as natural language processing and machine learning may offer • Conduct selective data extraction to make RERs more effi- opportunities to streamline the review process, particularly study cient. After selecting which studies to include in a review, RER selection, data extraction, although there is limited research on reviewers need to extract data from them. This process entails how well these tools can replicate the work human reviewers,33,34 collecting and recording study results and features or character- istics such as research design, and sample size.30 RER reviewers • Use of fewer people to complete RER review steps. To ensure may decide a priori to extract less data from each study. Again, comprehensiveness and accuracy, SRs generally have two or more selective data extraction may miss relevant information.31 individuals sequentially replicate important review procedures. RERs may save time by omitting these crosschecks.35,36 Alterna- • Perform less extensive data analysis or synthesis to save time. tively, reviewers can work in parallel rather than sequentially, the After extracting data from studies, SRs synthesize it into narrative tradeoff is an increased likelihood of errors.37,38 3 Rapid Evidence Reviews for Health Policy and Practice • Present the information in a manner that takes less time to Box 3: Producing Reviews through Knowledge produce. An annotated bibliography or simple evidence table ar- Brokering: Sax Institute Evidence Checks raying only basic study information and results requires less time The Sax Institute (https://www.saxinstitute.org.au/) is a nonprofit, than the extended discussion usually present in a full SR. Shorter, nonpartisan organization that works closely with both researchers and policymakers in Australia to promote the use of research evidence in less comprehensive final products can also be more appropriate health policy.54 Since 2006, the Sax Institute’s “Evidence Check” program than a full report if an RER is based on more limited searches, has analyzed, summarized, and synthesized HSR evidence to produce more than 160 reviews at the request of government policymakers.55 Each inclusion criteria, extraction, analysis, or synthesis. review takes approximately three months to complete. Evidence Checks inform decisions about whether to adopt new policies and programs, and how to develop them. They can examine how programs or policies Does Speeding Up a Review Affect the Results? have been implemented in other places, whether specific programs are Recent research suggests that decisions RER reviewers make to speed effective, and assess whether the evidence is reliable. Recent reviews have examined the effectiveness of public health interventions, maternal up the results may not necessarily lead to lower-quality reviews. One and neonatal outcomes following in vitro fertilization pregnancies, and how study concluded that restricting literature searches to a small number to improve antibiotic use in community settings. of highly-relevant databases does not adversely impact a review.39 An- A defining element of the Sax Institute’s Evidence Check program is other study found that, even though employing two evidence screen- the use of “knowledge brokering” to facilitate reviews. Sax Institute staff function as intermediaries –“knowledge brokers” – between requesting ers maximizes the inclusiveness of evidence searches, a single evidence policymakers and the external health researchers who conduct the screener is likely to identify the majority of relevant records, which reviews. The knowledge broker works with requestors to clarify their may be sufficient for drawing valid conclusions.40 On the other hand, issues of concern into researchable topics and questions. Once the requestors and broker agree on review questions and scope, Sax reviews that only search for evidence electronically may fail to identify Institute staff recruit reviewers with appropriate expertise from the up to half of all relevant research studies, highlighting the importance organization’s network of public health and HSR member organizations. Interested reviewers submit expressions of interest to the Sax Institute of seeking input from topical experts.41 In terms of the conclusions and the requester selects their preferred team. The reviewers spend one reviews reach, one study found little difference between RERs and SRs month collecting and evaluating research evidence and producing a draft report. Requestors review the draft report for two weeks and provide conducted on the same topics, despite their different methodologies.42 feedback, which reviewers incorporate into the final Evidence Check report. To confirm that all parties are satisfied with their experiences and final products, Sax Institute staff interview requestors and reviewers six How Are RER Programs Organized? months and twelve months after the report is completed. Both reviewers Organizations interested in conducting RERs need to decide not and policymakers have found the knowledge brokering process valuable for producing rigorous and relevant reviews to inform decisions.56 only what methods to use, but also how to structure and implement the reviews.43 Existing RER programs suggest a number of different approaches to funding, staffing, and governance. • Funding: Many RER programs rely primarily on ongoing govern- Box 4: Dissemination Strategies for Informing ment funding or external grants. Other programs charge requestors Clinical Practice: The Department of Veterans a fee for their commissioned reviews based on the complexity of the Affairs’ (VA) Responsive Innovation Evidence review and the expected difficulty of meeting requestors’ dead- Review (RIER) Program57 lines. While an ongoing grant may provide a more stable source of In 2011, a Southern California Veterans Affairs (VA) program known long-term funding, the size of such grants could limit the number as the Veterans Assessment and Improvement Laboratory – Patient Aligned Care Team initiative (VAIL-PACT) created the Responsive of reviews a program can afford to produce each year. “User-pay” Innovation Evidence Review (RIER) project, a rapid evidence review models may ensure that the demand for reviews does not outstrip program intended to help design evidence-based quality improvement (QI) programs for primary care delivered by the VA. An Evidence Review the budget available to fund them, but such approaches may pro- Workgroup consisting of VA and external researchers from the Southern vide less predictable revenue streams that make it difficult to retain California Evidence-based Practice Center (EPC), a professional reference librarian, and a project assistant with clinical experience staff and cover fixed costs. conducted the reviews. During the first 16 months of the project, 13 • Staffing: There are two approaches to staffing: Organizations can RIERs were completed, each taking between two and six weeks to complete. rely on their own staff (see Box 2, 4 and 5), or they can com- mission outside experts to conduct the review (see Box 3). The Each RIER addresses a problem, a set of questions, and specific innovations chosen by QI teams. RIERs presented their findings advantage of hiring one’s own staff is the accumulation of domain in six,15-page evidence overviews designed to be accessible and knowledge and reviewing skills in the organization, at the cost of understandable to a broad clinical audience. In addition to delivering finished reports to requestors, the review team presented many reports having a larger payroll to meet. Organizations that commission at a “project collaborative” attended by VA providers, including quality outside experts often serve as “knowledge brokers,” helping the councils, workgroups, and other innovation teams interested in the RIER service. Online surveys of those who read the reviews suggested requesting agency formulate an appropriate question, match- that RIERs were useful to quality improvement professionals, and many ing the right reviewer to that question, and facilitating sufficient respondents expressed interest in requesting an RIER in the future. conversation between the reviewer and requester to assure the relevance of the final product. However, this method requires releasing some control over the product. 4 Rapid Evidence Reviews for Health Policy and Practice Challenges for RERs to Inform Social Policy Box 5: Evidence-based Answers to Social Policy The literature examining the impact of RER approaches has focused Questions: EPPI-Centre Systematic Reviews on reviews of clinical interventions or health technologies. They The Evidence for Policy and Practice Information and Coordinating Centre (EPPI-Centre) (http://eppi.ioe.ac.uk/cms/) located at the Social have focused less on policies related to the organization and financ- Science Research Unit of University College London’s Institute of ing of care or the social determinants of health, which may include Education focuses on “informing policy and professional practice with sound evidence” in fields such as education, public health, and a broad range of social science and social policy research. Some international health systems development.58 One way they pursue this RER programs, such as University College London’s EPPI-Centre goal is by conducting systematic reviews of public policy research evidence. Although no timeframe is specified for EPPI-Centre SRs, (see Box 5), have specialized in social policy reviews. Their experi- requestors’ needs and deadlines typically determine the duration of ence highlights several unique challenges such RERs face compared the research.59 Established in 1993 primarily to perform SRs for UK government agencies, the EPPI-Centre has widened its focus to include to those performed on clinical topics. more international contexts and broadly generalizable knowledge through a relationship with the Alliance for Health Policy and Systems • Evidence reviews covering social issues or social science research Research, housed at the World Health Organization. To date, the EPPI- are often less generalizable than clinical or biomedical reviews. Centre has conducted 190 evidence reviews. Many social issues are inseparable from specific social, political, or To address some of the challenges of reviewing social policy evidence, geographic contexts, making it difficult to generalize their conclu- the EPPI-Centre tailors SRs to the priorities of requesting policymakers. sions.46 In contrast, clinical research questions often address physi- When decision makers request a review, EPPI-Centre staff help to articulate their needs along with “reviewable” questions. To keep their ological phenomena or biomedical impacts, which can more easily results relevant, the review team limits their evidence searches by topic, be generalized to broader populations or alternate contexts. study methodology, date of publication, type of publication, and/or publication language. Then, as findings emerge, EPPI-Centre staff meet • The scope of evidence for social policy reviews may be hard with requestors to find out which results matter to the review’s users. The review team uses this feedback to synthesize their findings into an to define. Social policies often target multifaceted problems and approximately 25-page report, which includes a four-page executive include diverse portfolios of interventions, rather than specific summary and a one-page summary of main findings.60 policies or procedures.47 • Social policy research is difficult to search. Biomedical research • Transparency. Transparent reporting makes reviews more repro- is usually found in a small number of well-indexed databases ducible and inspires greater trust in the results. Explicit descrip- such as PubMed. However, social policy research tends to be tions of the steps taken in an RER and any caveats or limitations dispersed over many databases, often poorly-indexed, and may of the review make it easier for readers to interpret findings include large amounts of unpublished “grey” literature.48 and decide how much confidence to place in the evidence.44,45 • Social policy research is more heterogeneous than biomedi- Additionally, broad dissemination of the program’s products cal research. While randomized controlled trials are common in and approach through publically-accessible websites, databases, clinical research, social policy research may adopt many different mailing lists, and traditional and social media can also help create qualitative, observational, experimental, and quasi-experimental transparency and trust. Box 4 describes how the U.S. Department designs. This diversity of approaches to social policy research of Veterans’ Affairs’ Responsive Innovation Evidence Review makes it more difficult to combine evidence across studies. (RIER) Program disseminates findings among stakeholders to improve the usefulness and impact of evidence reviews. However, Conclusion transparency presents challenges for maintaining the confidenti- Policymakers have looked for ways to review HSR evidence in a ality of politically-sensitive requests. rigorous yet timely fashion in order to provide the best information • Governance: The governance structure of a review program to inform health policy decisions. RERs attempt to meet this need. plays a role in ensuring that people can trust the objectivity and Because RER results are usually less comprehensive than those rigor of the program’s products. Approaches to ensuring such from full SRs and often carry a greater degree of uncertainty, they credibility include establishing a representative group of stake- are not simply an alternative to conducting an SR when funding holders to advise or govern the program, instituting mechanisms and resources are limited.49 Simpler review methods can keep the to identify and address potential conflicts of interest, maintaining amount of information reviewers need to process manageable, transparency of the review process, and inviting external, objec- shortening the time needed to complete those steps.50 RERs involve tive experts to review the validity and objectivity of RERs. tradeoffs for efficiency that limit either the generalizability of the results or the confidence readers can place in an RER’s conclu- sions.51 However, the experience of existing RER programs suggests it is possible to tailor approaches to fit the particular topic, policy context, and time and resources available to produce insights that are both relevant and credible. 5 Rapid Evidence Reviews for Health Policy and Practice About the Authors 14 Tricco AC, Cardoso R, Thomas SM, Motiwala S, Sullivan S, Kealey MR, Hem- melgarn B, Ouimet M, Hillmer MP, Perrier L, Shepperd S, Straus SE. Barriers Jared Hite is an MPP/MBA candidate at the Heller School for Social and facilitators to uptake of systematic reviews by policy makers and health care Policy and Management at Brandeis University. He was a graduate managers: a scoping review. Implement Sci. 2016 Jan 12;11(1):4. intern with the Translation and Dissemination Institute at Acad- 15 The Robert Wood Johnson Foundation. In it together: building a Culture of Health [Internet]. 2015 [cited 26 September 2015]. Available from: http://www. emyHealth during summer 2015. rwjf.org/en/library/annual-reports/presidents-message-2015.html 16 Watt A, Cameron A, Sturm L, Lathlean T, Babidge W, Blamey S et al. Rapid Michael E. Gluck, Ph.D., M.P.P., is Senior Director of Evidence versus full systematic reviews: validity in clinical practice? ANZ Journal of Generation and Translation at AcademyHealth, where he co-directs Surgery. 2008;78(11):1037-1040. the Translation and Dissemination Institute. 17 Polisena J, Garritty C, Kamel C, Stevens A, Abou-Setta A. Rapid review pro- grams to support health care and policy decision making: a descriptive analysis of processes and methods. Systematic Reviews. 2015;4(1). Suggested Citation 18 Wilson M, Lavis J, Gauvin F. Developing a rapid-response program for health Hite J and Gluck ME. “Rapid Evidence Reviews for Health Policy system decision-makers in Canada: findings from an issue brief and stakeholder dialogue. Systematic Reviews [Internet]. 2015 [cited 26 September 2015];4(1). and Practice,” AcademyHealth, January 2016. Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4373100/ 19 Tsertsvadze A, Chen Y, Moher D, Sutcliffe P, McCarthy N. How to conduct Endnotes systematic reviews more expeditiously? Syst Rev. 2015 Nov 12;4(1):160. 1 AcademyHealth defines health services and policy research as the multidisci- 20 Polisena J, Garritty C, Kamel C, Stevens A, Abou-Setta A. Rapid review pro- plinary field of scientific investigation that studies how social factors, financing grams to support health care and policy decision making: a descriptive analysis systems, organizational structures and processes, health technologies, and of processes and methods. Systematic Reviews. 2015;4(1). personal behaviors affect access to health care, the quality and cost of health care, and ultimately our health and well-being. AcademyHealth. What is 21 Tsertsvadze A, Chen Y, Moher D, Sutcliffe P, McCarthy N. How to conduct HSR - About Us - AcademyHealth [Internet]. Washington, D.C. 2015 [cited 29 systematic reviews more expeditiously? Syst Rev. 2015 Nov 12;4(1):160. September 2015]. Available from:http://www.academyhealth.org/About/content. 22 Thomas J, Newman M, Oliver S. Rapid evidence assessments of research to in- cfm?ItemNumber=831&navItemNumber=514 form social policy: taking stock and moving forward. Evid policy. 2013;9(1):5- 2 Perrier L, Mrklas K, Lavis J, Straus S. Interventions encouraging the use of 27. systematic reviews by health policymakers and managers: A systematic review. 23 Hartling L, Guise JM, Kato E, et al. EPC methods: an exploration of methods Implementation Science. 2011;6(1):43. and context for the production of rapid reviews [Internet]. Rockville (MD): 3 Mays N, Pope C, Popay J. Systematically reviewing qualitative and quantitative Agency for Healthcare Research and Quality (US); 2015 Feb. Available from: evidence to inform management and policy-making in the health field. Journal http://www.ncbi.nlm.nih.gov/books/NBK274092/ of Health Services Research & Policy. 2005;10(suppl 1):6-20. 24 Cochrane Canada. CCC introduction to rapid reviews, the different evidence, 4 Haskins R, Margolis G. Show me the evidence. Washington, D.C.: Brookings different synthesis series [Internet]. 2014 Available from: https:/www.youtube. Institution Press; 2014. com/watch?v=egoNzTa_HE4. 5 Lavis J, Davies H, Oxman A, Denis J, Golden-Biddle K, Ferlie E. Towards sys- 25 Thomas J, Newman M, Oliver S. Rapid evidence assessments of research to in- tematic reviews that inform health care management and policy-making. Journal form social policy: taking stock and moving forward. Evid policy. 2013;9(1):5- of Health Services Research & Policy. 2005;10(suppl 1):35-48. 27. 6 National Research Council. Finding what works in health care: standards for 26 Tovey D. Interviewed by: Gluck M, Hite J. 29 June 2015. systematic reviews. Washington, DC: The National Academies Press. 2011. 27 Khangura S, Polisena J, Clifford TJ, Farrah K, Kamel C. Rapid review: an Available from: http://www.nap.edu/catalog/13059/finding-what-works-in- emerging approach to evidence synthesis in health technology assessment. Int J health-care-standards-for-systematic-reviews Technol Assess Health Care. 2014;30(1):20–7. 7 The Robert Wood Johnson Foundation. The Synthesis Project [Internet]. 2015 28 Ganann R, Ciliska D, Thomas H. Expediting systematic reviews: methods and [cited 18 August 2015]. Available from: http://www.rwjf.org/en/library/collec- implications of rapid reviews. Implementation Sci. 2010;5(1):56. tions/the-synthesis-project.html 29 National Research Council. Finding what works in health care: standards for 8 Schone E, Brown R. Risk Adjustment: What is the current state of the art, systematic reviews. Washington, DC: The National Academies Press. 2011. and how can it be improved? [Internet]. The Robert Wood Johnson Founda- Available from: http://www.nap.edu/catalog/13059/finding-what-works-in- tion; 2013. Available from: http://www.rwjf.org/content/dam/farm/reports/ health-care-standards-for-systematic-reviews reports/2013/rwjf407046 30 Thomas J, Newman M, Oliver S. Rapid evidence assessments of research to in- 9 National Research Council. Finding what works in health care: standards for form social policy: taking stock and moving forward. Evid policy. 2013;9(1):5- systematic reviews. Washington, DC: The National Academies Press. 2011. 27. Available from: http://www.nap.edu/catalog/13059/finding-what-works-in- 31 Ganann R, Ciliska D, Thomas H. Expediting systematic reviews: methods and health-care-standards-for-systematic-reviews implications of rapid reviews. Implementation Sci. 2010;5(1):56. 10 Institute of Medicine. Knowing what works in health care: A roadmap for the 32 Green S, Higgins J, Alderson P et al. Cochrane handbook of systematic reviews nation. Washington, DC: The National Academies Press. 2008. Available from: of interventions. J. Higgins, S. Green, eds. West Sussex, England: The Cochrane http://www.nap.edu/catalog/12038/knowing-what-works-in-health-care-a-road- Collaboration and John Wiley, 2008. map-for-the 33 Tsertsvadze A, Chen Y, Moher D, Sutcliffe P, McCarthy N. How to conduct 11 Green S, Higgins J, Alderson P et al. Cochrane handbook of systematic reviews systematic reviews more expeditiously? Syst Rev. 2015 Nov 12;4(1):160. of interventions. J. Higgins, S. Green, eds. West Sussex, England: The Cochrane Collaboration and John Wiley, 2008. 34 Hartling L, Guise JM, Kato E, et al. EPC methods: an exploration of methods and context for the production of rapid reviews [Internet]. Rockville (MD): 12 Tricco A, Antony J, Strauss S. Systematic reviews vs. rapid reviews: what’s the Agency for Healthcare Research and Quality (US); 2015 Feb. Available from: difference? Presentation presented at CADTH Rapid Review Summit; 2015: http://www.ncbi.nlm.nih.gov/books/NBK274092/ Vancouver, BC. 35 Hartling L, Guise JM, Kato E, et al. EPC methods: an exploration of methods 13 Tsertsvadze A, Chen Y, Moher D, Sutcliffe P, McCarthy N. How to conduct and context for the production of rapid reviews [Internet]. Rockville (MD): systematic reviews more expeditiously? Syst Rev. 2015 Nov 12;4(1):160. 6 Rapid Evidence Reviews for Health Policy and Practice Agency for Healthcare Research and Quality (US); 2015 Feb. Available from: 48 Thomas J, Newman M, Oliver S. Rapid evidence assessments of research to in- http://www.ncbi.nlm.nih.gov/books/NBK274092/ form social policy: taking stock and moving forward. Evid policy. 2013;9(1):5- 36 Tricco A, Antony J, Strauss S. Systematic reviews vs. rapid reviews: what’s the 27. difference? Presentation presented at CADTH Rapid Review Summit; 2015: 49 Thomas J, Newman M, Oliver S. Rapid evidence assessments of research to in- Vancouver, BC. form social policy: taking stock and moving forward. Evid policy. 2013;9(1):5- 37 Ganann R, Ciliska D, Thomas H. Expediting systematic reviews: methods and 27. implications of rapid reviews. Implementation Sci. 2010;5(1):56. 50 Thomas J, Newman M, Oliver S. Rapid evidence assessments of research to in- 38 Tsertsvadze A, Chen Y, Moher D, Sutcliffe P, McCarthy N. How to conduct form social policy: taking stock and moving forward. Evid policy. 2013;9(1):5- systematic reviews more expeditiously? Syst Rev. 2015 Nov 12;4(1):160. 27. 39 Watt A, Cameron A, Sturm L, Lathlean T, Babidge W, Blamey S et al. Rapid 51 Thomas J, Newman M, Oliver S. Rapid evidence assessments of research to in- versus full systematic reviews: validity in clinical practice? ANZ Journal of form social policy: taking stock and moving forward. Evid policy. 2013;9(1):5- Surgery. 2008;78(11):1037-1040. 27. 40 Watt A, Cameron A, Sturm L, Lathlean T, Babidge W, Blamey S et al. Rapid 52 McMaster Health Forum. Rapid Response program [Internet]. 2015 [cited versus full systematic reviews: validity in clinical practice? ANZ Journal of 1 September 2015]. Available from: https://www.mcmasterhealthforum.org/ Surgery. 2008;78(11):1037-1040. policymakers/rapid-response-program 41 Ganann R, Ciliska D, Thomas H. Expediting systematic reviews: methods and 53 Wilson M, Lavis J, Gauvin F. Developing a rapid-response program for health implications of rapid reviews. Implementation Sci. 2010;5(1):56.35. system decision-makers in Canada: findings from an issue brief and stakeholder dialogue. Systematic Reviews [Internet]. 2015 [cited 26 September 2015];4(1). 42 Watt A, Cameron A, Sturm L, Lathlean T, Babidge W, Blamey S et al. Rapid Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4373100/ versus full systematic reviews: validity in clinical practice? ANZ Journal of Surgery. 2008;78(11):1037-1040. 54 Sax Institute. About us - Sax Institute [Internet]. 2015 [cited 4 September 2015]. Available from: https://www.saxinstitute.org.au/about-us/ 43 Wilson M, Lavis J, Gauvin F. Developing a rapid-response program for health system decision-makers in Canada: findings from an issue brief and stakeholder 55 Sax Institute. Evidence Check Brochure [Internet]. 2012 [cited 4 September dialogue. Systematic Reviews [Internet]. 2015 [cited 26 September 2015];4(1). 2015]. Available from: https://www.saxinstitute.org.au/wp-content/uploads/ Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4373100/ Evidence-Check-brochure-2012.pdf 44 Abrami P, Borokhovski E, Bernard R, Wade C, Tamim R, Persson T et al. Issues 56 Campbell D, Donald B, Moore G, Frew D. Evidence Check: knowledge broker- in conducting and disseminating brief reviews of evidence. Evidence & Policy: ing to commission research reviews for policy. Evid Policy. 2011;7(1):97-107. A Journal of Research, Debate and Practice. 2010;6(3):371-389. 57 Danz M, Hempel S, Lim Y, Shanman R, Motala A, Stockdale S et al. Incorporat- 45 Featherstone R, Dryden D, Foisy M, Guise J, Mitchell M, Paynter R et al. ing evidence review into quality improvement: meeting the needs of innovators. Advancing knowledge of rapid reviews: an analysis of results, conclusions and BMJ Quality & Safety. 2013;22(11):931-939. recommendations from published review articles examining rapid reviews. 58 EPPI-Centre. About the EPPI-Centre [Internet]. 2015 [cited 5 September 2015]. Systematic Reviews. 2015;4(1). Available from: http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=63 46 Oliver S. Interviewed by: Gluck M, Hite J. 25 June 2015. 59 Oliver S. Interviewed by: Gluck M, Hite J. 25 June 2015. 47 Thomas J, Newman M, Oliver S. Rapid evidence assessments of research to inform 60 Gough D. Learning from research: systematic reviews for informing policy social policy: taking stock and moving forward. Evid policy. 2013;9(1):5-27. decisions: a quick guide [Internet]. 1st ed. London, England: Nesta; 2013 [cited 2 September 2015]. Available from: http://www.alliance4usefulevidence.org/ assets/Alliance-FUE-reviews-booklet-3.pdf 7