By: Kristin Nelson, Associate, International Regulatory Affairs
Reviewing published literature on your subject device is a key component of meeting post market surveillance and clinical evaluation requirements for medical devices marketed in the European Union (EU), Australia, China and many other countries as well. Under the EU Medical Device Regulation 2017/745 (MDR), systematic scientific literature review is a necessary (and some might even argue evil) part of the clinical evaluation process to ensure adequate monitoring of a medical device and identify potential gaps in clinical evidence. Additionally, updates to literature reviews must be performed periodically, with greater frequency for higher risk medical devices. Searches are veritable balancing acts to include all relevant literature sources while minimizing the number of irrelevant articles yielded in the search. In addition, once a literature search is conducted and the relevant articles are identified, key data must be systematically extracted for analysis. Literature reviews therefore entail what can be an extensive and lengthy process, especially when data screening and extraction are conducted manually and captured using programs such as Excel.
Fortunately, to streamline this process, artificial intelligence (AI) software has emerged and gained popularity in recent years. AI powered services provide varying degrees of automation, user interface, database integration, quality-checks, pre-built algorithms, and categorization to increase efficiency in literature search execution, screening and/or data extraction.
When selecting a software service, it is important to consider the databases that may be needed or preferred by a researcher, author, reviewer, and/or client. For example, for search execution, software services vary in compatibility with different databases. They may allow for searches directly within an AI software or for subsequent importation of search results. AI programs that can search directly within a database, allow the literature search criteria to be saved, and can automatically execute new searches and import new publications over time. Other services seek to streamline the literature search process by providing cluster-based searches already labeled with descriptive keywords. Users can then select various clustering algorithms and even select multiple cluster-based searches at once. Some databases have more limited compatibility with an AI software. In these cases, as well as in most others, users can execute their own searches and import the resulting references for later steps.
The literature screening step has significant potential to be streamlined using AI and automation. Some software services provide complete automation of article inclusion and exclusion using selected keywords to identify relevant articles. Additionally, many services also provide a de-duplication feature, automatically excluding duplicate articles. For software services that automatically exclude articles based on relevance, the automation relies on the thoroughness of the terms, and may require multiple iterations to ensure that the correct articles are being identified. Other services provide assisted automation. For example, some services rely on user-populated PICO (population/patient, intervention, comparison, outcome) terms to rank all the articles, providing percentages of certainty or confidence that each article may be included or excluded. Users can also select a cut-off point to automatically exclude articles. Other services continuously reorder references based on relevance to help verify exclusion decisions.
Possibly the most time-consuming step, depending on the quantity of included articles, is data extraction. Using templates (either customized or pre-built), AI software can automatically extract key data from articles. Like populating PICO terms or keywords for data screening, the efficacy and reliability of the automated extraction relies on the template used. Anticipating all relevant safety and performance outcomes, study design and demographic information is essential to ensure accurate data extraction. Troubleshooting data extraction with several articles is recommended to maximize the potential of AI automation. Some AI services automatically extract data without user verification, whereas others will flag key endpoints for the user to confirm. Other data extraction timesavers may include automatic unit conversions, performance of needed calculations, and text-field input validation.
While AI software services provide several means to potentially streamline the literature review effort, harnessing the most power from AI requires thoughtful user inputs. Templates, keywords, PICO terms, and other parameters crafted by the user influence the output of the AI. Many AI software services are also relatively new which comes with the expected glitches or unknown anomalies, especially when looking to integrate with an already established workflow. In this regard, the upfront time needed to learn a software and create necessary parameters may, at least initially, outweigh the time saved compared to completing a manual literature review. Of course, the intent is that future literature searches, which must be conducted regularly, including annual updates for high-risk devices under MDR, will be more effective and more efficient overall. That said, conducting a literature review manually under a well-established process with tried-and-true techniques may provide greater confidence in the accuracy and consistency of the output as compared to use of a new assistive technology.
Though the prospects of AI software services are encouraging, extensive vetting and training is still required to ensure the correct service is selected to meet the needs of the literature review and to ensure the service is used correctly. AI software services are also expected to continue to evolve to improve existing features, offer new functions, and provide a more intuitive user experience. Therefore, though still developing, AI software may, with more experience and demonstration, provide another tool to help curtail a traditionally cumbersome literature review process. In the meantime, new users are wise to remain skeptical and open-minded, and to know that manual literature reviews are still widely conducted and accepted.
Learn more about our Global Regulatory Affairs services here.
Learn more about our European Regulatory services here.