Table 1 Search techniques and analytic tools

Berry Picking

Berry Picking is a retrieval model where obtaining evidence is not a linear path, but an iterative process where each newly identified piece of information can result in a modification of the information base required. Various techniques are used to identify the next piece of relevant information such as footnote chasing, journal browsing or database searching. Where it differs, is that information is not returned as a complete set, but in bits and pieces (the berries) informing the information base as one goes along [24].

Pearl growing

In the process of pearl growing, relevant articles to the topic of interest are identified and they enable researchers to isolate keywords and index terms on which the researchers can base their search. By using these identified keywords and index terms to build the search, the corpus of relevant articles will grow. This process is repeated for all initial papers and newly identified relevant papers for either a predetermined number of times or until no new relevant papers are identified [10, 25].

Citation tracking

For citation tracking, researchers search for all articles which were cited by relevant articles (backward citation tracking) and for all articles which cite the relevant articles (forward citation tracking). Every found reference has been deemed relevant after careful consideration by the researchers. As such, researchers make use of the ‘knowledgeable crowd’. That is, a corpus can grow through citation tracking based on the knowledge present within the literature by peers based on their knowledge and judgement of the content of the full article [11, 22].

‘Golden bullets’

‘Golden bullets’ are articles that align with the inclusion criteria for the systematic literature review and, therefore, undoubtedly should be part of the review. The ‘golden bullets’ are used for feature extraction to inform the Boolean search strategy. Furthermore, the ‘golden bullets’ are used in the validation test of the search. During the validation, the reviewer is checking whether the ‘golden bullets’ are included in the outcome of the search, ensuring a suitable search strategy to identify relevant studies.

Software

During the iterative method, some text analysis tools can be used. For instance, during the analysis of the ‘golden bullets’ the analysis tools present in Eppi reviewer [33] can be used. A possible tool for word frequency is the TF*IDF option, which helps to identify relevant terms and PubReMiner (http://hgserver2.amc.nl/cgi-bin/miner/miner2.cgi). PubReMiner is an online resource to which PubMed search queries can be submitted to produce a list and frequency counts for all keywords (subheadings, title-words etc.) and MeSH-terms associated with the articles in that query.

Swift-review is an interactive workbench that provides numerous tools to assist with literature prioritization. The software utilizes recently developed statistical modelling and machine learning methods that allow users to identify over-represented topics within the literature corpus and to rank-order titles and abstracts for manual screening [34].

To identify multi-word phrases, n-grams, the Termine tool can be used [35]. For identifying concepts within the ‘golden bullet’ set, it can be helpful to use cluster analysis [36] within Eppi reviewer, which is an application of the Lingo3 engine. Results of the search can be loaded in Endnote X7 (or any other suitable program for managing references) for deduplication. In the absence of Eppi reviewer a plethora of tools is available on the web like voyant-tools (https://voyant-tools.org) for term frequency analysis, termine on the web for n-grams (http://www.nactem.ac.uk/software/termine/) and vos-viewer for cluster analysis (http://www.vosviewer.com). For more information see http://systematicreviewtools.com.