Hacking the Literature Review: Opportunities and Innovations to Improve the Research Process

Jennifer Lubke, Virginia G. Britt, Trena M. Paulus, and David P. Atkins

Jennifer Lubke (Jennifer-Lubke@utc.edu) is Assistant Professor, the University of Tennessee at Chattanooga, Chattanooga, Tennessee; Virginia G. Britt (virginia.britt@wcs.edu) is Library Media Specialist, Nolensville High School, Nolensville, Tennessee; Trena M. Paulus (tpaulus@uga.edu) is Professor, University of Georgia, Athens, Georgia; David P. Atkins (david.atkins@utk.edu) is Professor, the University of Tennessee Libraries, Knoxville, Tennessee.

Thanks to Ann Viera, Elizabeth Pope, and the QUAL8545 students at the University of Georgia for their helpful comments on early drafts of this article.

Research outputs across the academic disciplines are almost exclusively published electronically. Organizing and managing these digital resources for purposes of review, and with the technical savvy to do so, are now essential skills for graduate study and life in academia. Paradoxically, digital and web-based technologies provide greater ease and efficiency with which to gather mass amounts of information, while at the same time presenting new challenges for reading, analyzing, organizing, and storing resources. Students, scholars, and the librarians who support them must adopt and refine practices to convert from paper-full to paperless literature review. This article proposes a methodical, reproducible, three-stage process that harnesses the power digital tools bring to the research cycle, regardless of the user’s preferred platform or operating system. Focusing just on the literature review phase, we develop a conceptual framework, illustrated with concrete tips and advice for storing and organizing, reading and annotating, and analyzing and writing. We demonstrate how a researcher’s self-selected suite of tools may be used to complement and even overcome the limitations of comprehensive academic literature and composition platforms such as Docear and F1000Workspace, especially regarding qualitative data analysis software for analyzing and coding research literature. Using these techniques, librarians can become teachers and research partners supporting the skill development of faculty and students.

A decade ago, Boote and Beile lamented the quality of dissertation literature reviews in educational research, suggesting that their criteria are part of the “hidden curriculum” and “tacit knowledge passed on from mentors to candidates.”1 As educational researchers, research methodologists, and librarians, we understand Boote and Beile’s arguments, as we have tried to engage our students and ourselves in strategies for a more “systematic literature review.”2 Consistent with the growing interest in digital tools to support research,3 library scholarship has investigated the role of digital tools in scholarly publication workflows,4 including “personal digital libraries,”5 “personal information management,”6 tools for discovery,7 and collaborative practices for information management.8 Digital tools are changing the nature of the research process, including the literature review, and have the potential to improve the quality of the outcomes by creating an entirely paperless process.

“Paper-full” literature reviews, characterized by numerous print copies of publications, sheets of hand- and typewritten notes, and lists of bibliographies, can pose several problems for scholars. Many publications are distributed in digital formats and the scholar’s final product is submitted in digital form. Moving between the print and digital environments, and managing and annotating journal articles, books, and other sources in a methodical way, can prove cumbersome. For born-digital materials, creating a paper version eliminates many advantages of the electronic versions. Transporting hard copies between office, home, and travel is difficult, and it is not easy to share these materials with collaborators. Librarians and the educators they serve see that many of today’s students are already paperless in how they interact with the published literature. While mid- to late-career scholars may find the idea of going paperless daunting, this may be less true of those early in their careers. Librarians have an opportunity to leverage the technical affordances of born-digital composition, data management, and publishing to support the creation and use of wholly digital literature review processes by scholars in all phases of their careers.

Most of the scholarship in this area has described various aspects of the literature review process, such as organizing and downloading PDFs;9 describing various citation management systems (CMSs) like Zotero,10 Mendeley,11 EndNote,12 and RefWorks;13 reporting on surveys of CMS users;14 comparing CMS features,15 including those for collaboration and social networking;16 and reporting the accuracy of “cite as you write” features.17 What is still missing, though, is guidance on how to reenvision or hack the literature review using new tools to create an entirely paperless workflow—including organizing relevant sources, annotating them, and synthesizing ideas across the sources to create an academic argument. Librarians can step into this void, imparting both guidance and instruction during the research process. Librarians can teach practical skills and offer advice regarding technologies that leverage the potential within electronic collections, within analytical software applications, and within composition and publication tools to create effective literature reviews.

In this paper we describe an innovative, purely digital, “paperless literature review” workflow18 that will lead to improved outcomes for scholars who must store, organize, read, annotate, and analyze the work of others before writing their own. Librarians can apply these workflows into their own publishing and outreach practices, adding value to the research process that goes beyond just connecting researchers with information resources. Using these techniques, librarians can become teachers and research partners supporting the skill development of faculty and students.

Literature Review

In a user-needs study conducted in a collaboration between Cornell University Library and Columbia University Libraries, doctoral students in the humanities “questioned the continued utility of librarians, asking why a librarian’s assistance is still needed, given the convenience of online research tools and availability of faculty expertise.”19 In the past, libraries were responsible for storing information, and users were on their own to build knowledge from it. Now, however, as Favaro and Hoadley have noted, the Internet plays the information storage and retrieval role, requiring libraries and librarians to take on new roles, one of which is helping patrons effectively manage their personal information collections.20 As Favaro and Hoadley asked, “How can we envision the role of libraries in a much broader process of knowledge building that extends beyond a trip to the library or constructing a bibliography?”21 These authors argued that tools for information discovery and access can no longer be segregated from those used in the subsequent steps of the research process and that librarians need a better understanding of digital workflows in order to support early-career scholars. These scholars need assistance selecting the best tools for various phases of the research process—from information retrieval to knowledge building—be they library-provided or freely available.

These digital workflows build primarily on PDF texts such as scholarly articles and book chapters. Research outputs, most commonly in the form of PDFs, are readily accessible online via Open Access or a library’s collections. Coupled with the advent of social media and mobile computing, this suggests new routes for researcher workflow, beginning with the literature review.22 For example, in a two-part cross-disciplinary study of nearly two hundred Pennsylvania State University (Penn State) faculty, Antonijević and Cahoy found that participants preferred commercial search engines over academic databases for discovery.23 A vast number of the study’s respondents reported that they work predominantly with PDFs, which they organize, back up, and archive in numerous ways, including hard drives, thumb drives, cloud-based solutions, and “old-fashioned” paper print outs and files. Old-fashioned data management practices persist for a number of reasons but are becoming increasingly cumbersome as PDFs dominate as the preferred digital format for publishing texts.24

Scholars’ PDF annotation practices are increasingly idiosyncratic and hybridized, owing to the proliferation of digital tools that enable researchers to create searchable and shareable notes.25 For instance, Bjarnason described his process of “marking up PDFs and writing corresponding notes in a single text file, used for all note keeping, and using keywords to allow topic searches and visualization.”26 One tool that has an expanded role to play is the digital citation manager or citation management system (CMS). Childress noted: “It could be said that citation management is the foundation for scholars to begin collecting, managing, and archiving their research findings as well as their own scholarly output.”27 Antonijević and Cahoy reported that a large number of social science and humanities faculty do, in fact, manage citations with EndNote and Zotero. Yet, some participants questioned the robustness of CMS for indexing and filing, opting instead for cloud-based solutions—even as they voiced privacy and security concerns about using nonuniversity servers.28

As the Penn State study illustrated, integrating any new tool, such as CMS, into a paperless workflow is not without its challenges. These challenges are not limited to CMS and include: developing and maintaining requisite technology skills,29 staying organized and avoiding information overload,30 adapting to changing file formats,31 choosing from a multiplicity of storage options,32 overcoming the familiarity bias of manual bibliographies,33 and managing discipline-specific concerns.34

A persistent challenge is integrating search and retrieval with organizing and archiving practices to avoid losing track of resources. Researchers report a sense of disintegration caused by the proliferation of digital tools for data management.35 Antonijević and Cahoy suggested this finding “implies that although institutional support and training programs are vital for the uptake of digital tools, such programs are not necessarily sufficient for effective integration of those tools into scholarly practice.”36 This lack of integration might be viewed as a two-pronged challenge of overcoming “supply-side influences,” or the lingering effect of practices cultivated in analog environments, and “demand-side characteristics,” or the unique habits and dispositions imposed by the scholar’s home discipline.37 For example, participants in Antonijević and Cahoy’s study reported that “integration of digital tools into their search activities resulted in a complete breakdown of their systems for organizing information, which were developed for print-based materials.”38 They go on to conclude that “while implementation of digital tools into one phase of the workflow might be rewarding, it might also become a challenge in other phases of the work.”39 The breakdown seems most acute when involving CMS, the capacities and functionalities of which are not fully trusted or understood by librarians40 nor the broader academic community.41

Citing “ubiquitous disconnects” between tools, Favaro and Hoadley wrote, “imagine the loss of focus and the distraction created whenever a student needs to move from a highlighted PDF to typing the metadata into EndNote or Zotero . . . to moving the document and inviting collaborators into a space such as Google Docs or Dropbox.”42 As researchers continue to seek a transformational killer app, a “one-stop shop” for the literature review workflow, the potential for information discovery through tagging, sharing, and networking groups available in Mendeley,43 Zotero,44 and CiteULike45 remains an area of active inquiry.46

Librarians must also consider disciplinary differences when helping their patrons convert to a paperless workflow.47 For example, at the University of Pennsylvania, humanities and social sciences faculty are chiefly concerned with archiving publications, whereas faculty in the hard sciences also want to archive their process.48 Disciplines that value qualitative methods of research are familiar with the utility of qualitative data analysis software (QDAS) for documenting the process,49 a functionality that we argue can be applied to analysis of literature, regardless of discipline or research paradigm.

The Paperless Literature Review Workflow

Scholars’ publishing suites such as Docear, which brands itself as an open source “literature suite that cuts the (paper) clutter,” and the F1000Workspace, which supports collecting, writing, and collaborating among scientists, both demonstrate potential in creating a transformational “one-stop shop” to support the writing workflow. However, they do not necessarily provide the most robust tools for literature reviews. To illustrate these workflows and concepts, we focus solely on the literature review, proposing paperless workflows which are not dependent on specific tools or platforms. We illustrate the process with the tools we use in our own work as well as describing innovative, open source alternatives, proposing models for librarians and researchers alike.

We use the topic of crowdfunding (the practice of fundraising on the Internet for personal need or professional projects, on sites such as GoFundMe, Kickstarter, or Indiegogo) to contextualize our discussion of the literature review process. Figure 1 provides an overview of a paperless workflow. Stage 1 begins with selecting a digital tool to store and organize sources—either a CMS or a cloud storage device. The choice may, in part, depend on whether the selected CMS provides: (1) robust PDF annotation tools and (2) a mobile app to be used during the reading phase. If so, sources can be both organized and annotated thoroughly within the CMS. If not, it may be better to opt for cloud storage and then move to a digital reader on a mobile device in Stage 2 to annotate the sources. Finally, in Stage 3, annotated sources can be uploaded to a QDAS program to synthesize and begin writing the literature review.

Stage 1. Store and Organize Sources

With a paperless workflow, the lifecycle of a literature source begins at harvesting and continues with storage. Born-digital publications readily synchronize with this process. Any paper-based sources or any digital formats that are not in optical character recognition (OCR) format may be reformatted and added to this workflow. Any born-print documents must be converted into machine-readable text, which can be accomplished by using an OCR scanning/conversion application such as is available in Adobe Acrobat Pro or a PDF-to-OCR conversion such as that available in Google Drive. Another option is to take notes from a book in a word processing program, convert into a PDF, and include it with the other files for the literature review.

A CMS can provide a robust hub through which all paperless activity may flow. Examples of CMS include EndNote, Mendeley, RefWorks, Zotero, and Papers. A CMS serves as a personal database of resources which can be organized, searched and shared. Figure 2 shows the Mendeley interface, with organization tools in the left-hand pane, including search and filtering tools, organizational folders, and groups. Sources within the crowdfunding folder appear in the middle pane and can be sorted by author, title, year, journal name, etc. Details of the selected source appear in the right-hand pane, and the PDF can be read, and to varying degrees, annotated within the CMS as well.

All CMS tools import PDFs and citation metadata directly from publisher databases and even web browsers. Literature searches can be conducted from within the CMS and/or sources can be downloaded directly into the software. For example, figure 3 shows Zotero, which shares a split screen with the web browser. Within the notes editor interface, new sources can be recorded, tagged, and linked to related resources in the Zotero database.

Each CMS has a note-taking feature, similar to the one seen in figure 3. When a new source is added to the library, a brief annotation about when and where the item was located, how the item will be used in the research project, and so on can be included, which helps ensure methodical tracking. The notes field can support more substantial annotations and memos, upwards of several pages in length. These annotations are searchable, and, while sharing utilities vary from system to system, they typically can be exported to the desktop or emailed to collaborators.

CMS selection depends on the needs and preferences of the user. Factors to consider include availability, cost, online storage capacity, collaboration features, and PDF annotation support (important in Stage 2). Discipline-specific concerns also will influence the user’s choice of CMS, so it is advisable to consult with collaborators and others in the field when making a selection. The Penn State University Libraries maintain a current and comprehensive online comparison chart of the major citation managers (http://guides.libraries.psu.edu/CitationStyles/Tools).

Cloud-based content management services (for example, Box, Dropbox, or Google Drive) can be used as an alternative to a CMS. Cloud storage services provide browser-based interfaces and downloadable desktop iOS- and Android-compatible mobile apps that enable drag-and-drop movement of files. This seamless integration can easily support a collaborative workflow and allows working with files on mobile devices. Built-in utilities allow easy sharing of files with collaborators as well as file integration with other desktop and mobile apps. Within the Dropbox iOS app, for example, the user can transfer a PDF to his or her preferred PDF reader or citation manager using the “Open In…” button (see figure 4).

Whether a CMS or a cloud storage system is selected, storing and organizing documents is an important first step in a paperless literature review workflow. Creating a system for naming files and folders is also important. We recommend naming files by author and publication date, for example, “Smith 2016,” so that files can be sorted alphabetically for fast retrieval. Since storing sources in the cloud lacks CMS functionality for creating multiple copies of the same citation in multiple libraries for various projects, it will be necessary to organize sources into separate folders by project, topic or some other scheme. Finally, it is important to ensure that the system is backed up, preferably in the cloud.

Stage 2. Read and Annotate Sources

Once sources have been organized and stored, the next step is to thoroughly read the literature, annotating as necessary, in preparation for synthesizing the sources and writing an academic argument. As Bjarnason has noted, the “physical filing cabinet for paper journal articles has been replaced by an online repository of marked up PDFs.”50 This repository can be either the CMS or a collection of sources in cloud storage. The ideal scenario is that the selected CMS provides an app so that sources can be read and annotated on a mobile device, such as Mendeley’s iPad app illustrated in figure 5.

Mendeley provides highlighting and note-taking features, but not underlining or some of the more robust features available in other PDF reading applications or devices (see figure 6). In this case, users should probably choose to annotate PDFs outside of the CMS and then import them.

For scholars who are used to reading and annotating on paper and find it too cumbersome to do so on a desktop or laptop computer, we recommend using a tablet device (such as an iPad) in conjunction with a digital reader. The GoodReader App (iOS) is one such digital reader; others include iAnnotate (iOS), Mac Preview (Mac), or Adobe Acrobat PDF Reader (Windows, Mac, iOS, Android, Windows Phone). Tablets can make the transition to a paperless workflow less distressing because it is possible to curl up with the readings, use a finger or stylus to annotate much like one would with a highlighter or pen, and otherwise physically engage with the texts in similar ways as on paper. Mobile devices also allow the text to be enlarged for visibility and to focus on the most meaningful segments of the source for deeper engagement. By using a lightweight tablet reader or a mobile device synchronized either with the CMS or cloud storage, resources become much more portable and comfortable to review.

Digital readers can easily be synchronized with cloud storage; for example, GoodReader can be connected and synchronized with Dropbox. PDFs can be uploaded and downloaded between the two programs. Sources that need to be read are downloaded from Dropbox into GoodReader, organized into folders as needed, annotated, “flattened” (to retain the annotations), and then uploaded back into Dropbox (see figure 7). In a collaborative project, this step of the process also allows any member of the team to pull down a source, create an annotated version, and easily return the file to the cloud for others on the team to read and view annotations and notes. GoodReader works similarly with Google Drive and other cloud storage services.

When the annotation feature is selected, digital readers will automatically suggest creating a second copy of the file for annotation purposes. This way the original, clean copy of the source is retained and a second, annotated copy is stored in GoodReader. Annotations can include marginalia or collapsible “sticky notes,” which allow for more detailed responses and memos on the text. In addition, these readers often support searching for keywords, highlighting (in a variety of colors), underlining, adding shapes and arrows, and selecting from a variety of other annotation tools as illustrated in figure 6.

After reading is complete, all notes and annotations are saved and can be revisited at any time. Once the source is ready for further analysis in QDAS or by other research collaborators, the app provides the option to send the PDF with all annotations “flattened.” Flattening saves the annotations within the PDF, ensuring they are readable on other devices and in other applications, such as QDAS introduced in Stage 3. The flattened source may be emailed, returned to the cloud storage program, or, at this point, put into a CMS.

Stage 3. Upload, Analyze and Write

Once sources have been stored, organized, read and annotated, it is time to analyze, synthesize themes, and create arguments in preparation for writing. Qualitative data analysis software programs are highly useful in this stage. First developed in the 1980s, QDAS programs were initially designed by researchers to help organize and analyze the massive amounts of data typically generated in a qualitative study, such as interview recordings and transcripts, observational field notes, digital photographs, and journal entries. Performing a literature review is in many ways a type of qualitative data analysis—with the literature serving as the data that needs to be described, analyzed, and interpreted.51 Just as QDAS programs such as QSR NVivo, ATLAS.ti and MAXQDA can make the analysis of research data more systematic and transparent, they can do the same for literature reviews.52 Here librarians can make real contributions to improve the quality of literature review workflows by ensuring that scholars understand what QDAS programs are and the tools and techniques they offer. QDAS developers themselves have taken up this application of their software, with QSR NVivo, for example, partnering with Endnote and offering webinars on how to conduct literature reviews. MAXQDA version 11 allows users to import RIS data from Endnote, Citavi and Zotero; and Pope (2016) recently published her strategies for conducting literature reviews in ATLAS.ti. QDAS programs are quite complex, but, for those conducting qualitative studies, the time invested in learning the software will pay off when it is time to use the software again for the data analysis. To select a QDAS program we recommend checking which ones are supported at your institution with site licenses and training, as well as which ones are being used by colleagues. There are also free webinars and video tutorials online for most of the programs.

In this workflow, QDAS provides a digital workspace to aggregate, code, organize, and compose all elements of the literature review. For example, QSR NVivo supports the importation of entire citation manager libraries, including both bibliographic metadata and the associated annotated PDFs. Alternately, it is possible for the researcher to upload entire folders of PDFs or simply to drag and drop all the annotated sources from a computer or cloud-based folder into the QDAS program to begin analysis. Uploading all the annotated sources into a QDAS package not only allows quick clicking from one source to the next for easy retrieval and review, but also provides tools and features to further organize the sources by any criteria relevant to the literature review. In ATLAS.ti, for example, the “documents family” feature allows grouping of sources according to characteristics such as the type of source it came from, or whether it has been reviewed. In figure 8, the sources are organized by source: journal articles, popular press articles, and theses and dissertations. Clicking on the journal articles family will immediately display only those sources for further review. Document families can also be used to organize sources by the major themes of literature review.

Each source that has been uploaded into the software can be described in the comment field. In figure 8, for example, the full reference citation for the source has been entered for ease of retrieval. Unfortunately, at the moment, QDAS programs do not have a “cite as you write” feature, so reference lists cannot be generated in the same way they are in a CMS. This makes it necessary to use both a CMS and a QDAS program for an ideal paperless workflow. Alternately, Docear and F1000Workspace provide CMS features, some limited analysis of sources through memos, and reference list generation.

One advantage of reading and annotating the sources prior to uploading them into a QDAS program is that after the initial reading, the major categories and themes of the literature review might be relatively clear to the researcher as the analysis phase begins. In other words, the beginnings of the argument might already be crystallizing. The highlights and annotations made while reading can guide the coding and retrieving of relevant sections of the sources. “Coding” is relatively easy to do in all the programs and simply means tagging a particular sentence, paragraph or entire section of a source with a label created by the researcher. This way, when it is time to write, all the other sections of the sources related to that topic can be retrieved at once. With source materials coded in the software, it is possible to retrieve all the literature sections on the same topic, reread them, and begin writing.

To illustrate, figure 9 shows yellow highlights that were made in GoodReader while reading the source on an iPad. During the initial reading and annotating it became clear that the various sources about crowdfunding often mentioned the variety of fields (journalism, filmmaking, science) that engage in the practice. This would no doubt be important to mention in the literature review. The sources also noted that successful crowdfunding efforts are often due to the strong existing social networks of those requesting the funds. Codes were then created in ATLAS.ti for both of these ideas (“fields using crowdfunding” and “existing social networks—building an audience”). These codes were attached to relevant sections of the source (called “quotations”) for easy retrieval when it was time to write. Such codes can be created before the sources are reviewed in QDAS or created along the way. QDAS keeps track of how often each code has been used so that it becomes clear which topics are being discussed most often in the literature. In figure 10, “criteria for success” (of crowdfunding efforts) was used across the sources sixteen times, and “existing social networks” thirteen times. “Fields using crowdfunding” was used eight times. This provides an initial idea of how much support there is in the literature for each topic to be covered in the review.

Once the sources have been coded, scholars may retrieve all the coded sections of the literature on each topic and view them together. To review all the literature that was coded as “criteria for success,” the code is clicked and a window appears with all the quotations on that topic across the literature sources. In figure 11, the sixteen quotations for “criteria for success” have been retrieved and are visible in the new window. Clicking on each quotation will make it appear in the context of the full source so it can be reviewed for additional insights.

It is easy to uncode, change code names, merge codes, or reorganize codes while reviewing them to be sure they reflect the findings of the literature review and will be helpful in creating the argument. QDAS tools also provide features that will run reports displaying all the quotations for each code, as illustrated in figure 12 with the ten quotations coded “definitions of crowdfunding.” The quotations can then be reviewed or copied and pasted right into a word-processing document as the literature review is being written.

Once codes have been finalized it can be helpful to visually display them to organize them into an argument structure in preparation for writing. In figure 13, codes were dragged and dropped into a network view in ATLAS.ti and meaningful links were created. The researcher used network links to show that the literature review would begin by defining “crowdfunding” and that this definition would include a discussion of its history, how it works, and which fields are using it. Another section of the review would discuss the criteria that make crowdfunding efforts successful. A major factor in successful crowdfunding efforts is having existing social networks that can build a strong audience for the crowdfunding request. By using the network view, the argument of the literature review is visible.

Another useful feature for writing is the memo tool provided by each QDAS package. This tool is a text editor that can be used to compose and therefore capture insights and ideas and even write initial drafts of the argument while staying spatially close to the literature being reviewed. In figure 14, some initial thoughts about “criteria for success” of crowdfunding campaigns are captured in a memo while looking at a section of one of the sources. It is possible to write entire sections of the literature review by using the memo tool, before bringing it back into word processing software to ready it for publication.

Finally, QDAS saves all this analysis in an archival and shareable file. At a later date, a researcher can update the literature review or extract portions to create new literature reviews or bibliographies. Biomedical researchers compiling systematic literature reviews and meta-analyses of literature could use QDAS as an efficient platform for managing these very large and very complex reviews. QDAS tools support many of the steps required in completing systematic reviews such as appraising the literature, extracting data from the literature, analyzing results across publications, and composing the written findings. While we have illustrated Stage 3 using ATLAS.ti, other QDAS programs such as QSR NVivo and MAXQDA provide very similar features and functionality.

At this point in the process the workflow moves back to the CMS. CMSs were designed to extract metadata from sources, which is then used to generate a list of references in whatever citation style the researcher prefers (APA, MLA, Chicago) through a plug-in for word processing programs. It is often this “cite as you write” feature that motivates researchers to adopt a CMS tool, in their effort to more easily generate accurate bibliographies and references pages from their cited sources. One way to choose between CMS tools is to compare how well each one performs metadata extraction. However, it is generally the case that a CMS offers only varying degrees of formatting accuracy and is highly dependent on how precisely the metadata is entered.53 As we have illustrated, the power of the CMS resides in a host of other functions to support literature reviews (for example, organizing, sharing, annotating sources), of which outputting references is but one.

Conclusion

With born-digital literature sources eclipsing paper-based sources, a paperless workflow seems inevitable and preferable. In this paper we have illustrated one such workflow to support a more methodical approach to creating literature reviews, including using digital tools for storing and organizing sources for the literature review, reading and annotating the sources, analyzing the sources, and writing up the arguments. While tools such as Docear and F1000Workspace are proposing comprehensive platforms to support an integral part of this workflow, neither yet includes the analytic robustness of QDAS software to systematically code and retrieve parts of the literature that support the overall argument.

This workflow was developed based on our own literature review processes, and we continue to learn about new strategies and tools from our colleagues and students. There is a great opportunity for librarians to collaborate with and to provide guidance to scholars and researchers, both new and established, interested in adopting paperless workflows, helping them identify tools that will be of greatest value. Librarians can build upon their expertise and experience with collecting literature and move beyond just helping scholars create the corpus of literature for review. Librarians are well situated to learn how the collections we build and curate can best be used by those we serve. Just as librarians embraced CMS as transformational tools for citation management, librarians can embrace and advocate QDAS and paperless review processes. Librarians can develop the skills, create the training, and foster collaborations required to become more fully embedded in the research process.

Librarians can expand their own research instruction portfolio to use article discovery and citation management as springboards into innovative literature review practices. Instruction doesn’t have to end with “…and now you can import your sources into EndNote.” Instruction can move into the next phase, exploring how a researcher can manage digital bibliographies and apply digital tools, perhaps the same tools used in their data analysis, to create literature reviews. Librarians can also partner with academic departments or research technology support operations who are vested in teaching research methods. At The University of Tennessee, Knoxville, librarians partner with instructors in the campus’s Office of Information Technology to offer training sessions which bridge citation management tools with NVivo, a campus-supported QDAS application. For librarians working at institutions that lack this level of institution-wide research support, learning more about the technologies used in each step of the research process and then stepping in to fill the need for research methods support could be an opportunity for outreach to academic departments or individual patrons that demonstrates the library’s worth and adds value to the institution. To better explore issues and options, it may be worthwhile to replicate Antonijević and Cahoy’s ethnographic study of the information management practices of scholars in the context of literature reviews to discover whether the workflow we propose here is similar to what scholars are currently doing.54 Also, the Kramer and Bosman survey of researchers’ tools and workflows provides valuable data and promotes innovation by illustrating technologies and workflows that support open source, open access and open science.55

Through the research and exploration of the literature review process, we have proposed a methodical, reproducible, paperless literature review processes that harnesses the power of digital tools. We also see potential in expanding the changing role of librarians as both teachers and research partners, supporting faculty and students in their research and publishing skills. Our hope is that through presenting this information, we might influence information behaviors and updates to the scholarly workflow through the use of digital tools.

References

1. David N. Boote and Penny Beile, “Scholars before Researchers: On the Centrality of the Dissertation Literature Review in Research Preparation,” Educational Researcher 34, no. 6 (August 2005): 13.
2. Sebastian K. Boell and Dubravka Cecez-Kecmanovic, “On Being ‘Systematic’ in Literature Reviews in IS,” Journal of Information Technology 30, no. 2 (June 2015): 1.
3. Trena Paulus, Jessica Lester, and Paul Dempster, Digital Tools for Qualitative Research, (Thousand Oaks, CA: Sage, 2014).
4. Sharon Favaro and Christopher Hoadley, “The Changing Role of Digital Tools and Academic Libraries in Scholarly Workflows: A Review,” Nordic Journal of Information Literacy in Higher Education 6, no. 1 (2014); Thomas L. Mead and Donna R. Berryman, “Reference and PDF-Manager Software: Complexities, Support and Workflow” Medical Reference Services Quarterly 29, no. 4 (2010); Bianca Kramer and Jeroen Bosman, “Innovations in Scholarly Communication—Global Survey on Research Tool Usage [version 1; referees: awaiting peer review]” F1000Research 5 (2016).
5. Navid Aghakhani, Fatemeh Lagzian, and Bidyut Hazarika, “The Role of Personal Digital Library in Supporting Research Collaboration,” Electronic Library 31, no. 5 (2013): 548–60.
6. Ina Fourie, “Collaboration and Personal Information Management (PIM),” Library Hi Tech 30, no. 1 (2012): 186–93; Steve Whittaker, “Personal Information Management: From Information Consumption to Curation,” Annual Review of Information Science and Technology 45, no. 1 (2011): 1–62; Antoinette Lush, “Fundamental Personal Information Management Activities—Organisation, Finding and Keeping: A Literature Review,” Australian Library Journal 63, no. 1 (2014): 45–51.
7. Favaro and Hoadley, “Changing Role of Digital Tools.”
8. Fourie, “Collaboration and Personal Information Management.”
9. Vedana Vaidhyanathan et al., “Making Bibliographic Researchers More Efficient: Tools for Organizing and Downloading PDFs, Part 1,” Journal of Electronic Resources in Medical Libraries 9, no. 1 (2012): 47–55.
10. Khue Duong, “Rolling Out Zotero across Campus as a Part of a Science Librarian’s Outreach Efforts,” Science & Technology Libraries 29, no. 4, (2010): 315–24; Peter Fernandez, “Library Values That Interface with Technology: Public Service Information Professionals, Zotero, and Open Source Software Decision Making,” Library Philosophy and Practice (e-journal), 2012, http://digitalcommons.unl.edu/libphilprac/803.
11. Alison Hicks and Caroline Sinkinson, “Examining Mendeley: Designing Learning Opportunities for Digital Scholarship,” portal: Libraries and the Academy 15, no. 3 (2015): 531–49.
12. Anya McKinney, “EndNote Web: Web-Based Bibliographic Management,” Journal of Electronic Resources in Medical Libraries 10, no. 4 (2013): 185–92.
13. Mariela Hristova, “RefWorks Usage Patterns: Exploring the First Four Semesters of Use by Faculty, Graduate Students, and Undergraduates,” Internet Reference Services Quarterly 17, no. 2 (2012): 45–64; Cynthia Vaughn, “Citation Management: RefWorks,” Journal of Electronic Resources in Medical Libraries 10, no. 1, (2013): 25–31.
14. Jenny Emanuel, “Users and Citation Management Tools: Use and Support,” Reference Services Review 41, no. 4 (November 25, 2013): 639–59; Jamie Salem and Paul Fehrmann, “Bibliographic Management Software: A Focus Group Study of the Preferences and Practices of Undergraduate Students,” Public Services Quarterly 9, no. 2 (2013): 110–20; Hannah Gascho Rempel and Margaret Mellinger, “Bibliographic Management Tool Adoption and Use: A Qualitative Research Study Using the UTAUT Model,” Reference & User Services Quarterly 54, no. 4 (Summer 2015): 43–53.
15. Dawn Childress, “Citation Tools in Academic Libraries: Best Practices for Reference and Instruction,” Reference & User Services Quarterly 51, no. 2 (Winter 2011): 143–52; Merinda Kaye Hensley, “Citation Management Software: Features and Futures,” Reference & User Services Quarterly 50, no. 3 (Spring 2011): 204–8; Michael Steeleworthy and Pauline Theresa Dewan, “Web-Based Citation Management Systems: Which One Is Best?,” Partnership: The Canadian Journal of Library Information Practice and Research 8, no. 1 (2013): 1–8; Mangesh S. Talmale and Surya Nath Singh, “Social Citation 2.0 Tools: An Overview,” International Journal of Information Dissemination and Technology 3, no. 2 (2013): 80–85.
16. Amit K. Mahajan and D. Kyle Hogarth, “Taking Control of Your Digital Library: How Modern Citation Managers Do More Than Just Referencing,” CHEST: Topics in Practice Management 144, no. 6 (2013): 1930–33; Holt Zaugg et al., “Mendeley: Creating Communities of Scholarly Inquiry through Research Collaboration” TechTrends 55, no 1 (Jan. 2011): 32–36.
17. Lindley Homol, “Web-Based Citation Management Tools: Comparing the Accuracy of Their Electronic Journal Citations,” The Journal of Academic Librarianship 40, no. 6 (Nov. 2014): 552–57.
18. Thorarin Bjarnason, “Digital to-Do: Paperless Literature Review,” IEEE Potentials 30, no. 4 (July 2011): 27–30.
19. Gabriela Castro Gessner et al., Supporting Humanities Doctoral Student Success: A Collaborative Project between Cornell University Library and Columbia University Libraries (Washington, DC: Council on Library and Information Resources, 2011): 11, http://academiccommons.columbia.edu/item/ac:140506.
20. Favaro and Hoadley, “The Changing Role of Digital Tools.”
21. Ibid., 30.
22. Ellen Collins, Monica E. Bulger, and Eric T. Meyer, “Discipline Matters: Technology Use in the Humanities,” Arts and Humanities in Higher Education 11, no. 1–2 (2012): 76–92
23. Smiljana Antonijević and Ellysa Stern Cahoy, “Personal Library Curation: An Ethnographic Study of Scholars’ Information Practices,” portal: Libraries and the Academy 14, no. 2 (2014): 287–306.
24. Bjarnason, “Digital to-Do.”
25. Bjarnason, “Digital to-Do”; Favaro and Hoadley, “Changing Role of Digital Tools”; Gessner et al., Supporting Humanities.
26. Bjarnason, “Digital to-Do,” 28.
27. Childress, “Citation Tools in Academic Libraries,” 150.
28. Antonijević and Cahoy, “Personal Library Curation.”
29. Favaro and Hoadley, “Changing Role of Digital Tools”; Gessner et al., Supporting Humanities.
30. Antonijević and Cahoy, “Personal Library Curation”; Favaro and Hoadley, “Changing Role of Digital Tools.”
31. Ibid.
32. Antonijević and Cahoy, “Personal Library Curation.”
33. Antonijević and Cahoy, “Personal Library Curation”; Childress, “Citation Tools in Academic Libraries”; Gessner et al., Supporting Humanities.
34. Antonijević and Cahoy, “Personal Library Curation”; Childress, “Citation Tools in Academic Libraries”; Collins, Bulger, and Meyer, “Discipline Matters.”
35. Collins, Bulger, and Meyer, “Discipline Matters”; Favaro and Hoadley, “Changing Role of Digital Tools.”
36. Antonijević and Cahoy, “Personal Library Curation,” 302.
37. Collins, Bulger, and Meyer, “Discipline Matters,” 78
38. Antonijević and Cahoy, “Personal Library Curation,” 303.
39. Ibid., 302–3.
40. Childress, “Citation Tools in Academic Libraries.”
41. Antonijević and Cahoy, “Personal Library Curation”; Gessner et al., Supporting Humanities.
42. Favaro and Hoadley, “Changing Role of Digital Tools,” 27.
43. Emanuel, “Users and Citation Management Tools”; Hicks and Sinkinson, “Examining Mendeley.”
44. Emanuel, “Users and Citation Management Tools.”
45. Bjarnason, “Digital to-Do;” Emanuel, “Users and Citation Management Tools.”
46. Antonijević and Cahoy, “Personal Library Curation.”
47. Ibid., 303.
48. Ibid., 299.
49. Ibid.
50. Bjarnason, “Digital to-Do,” 28.
51. Harry F. Wolcott, Transforming Qualitative Data: Description, Analysis, and Interpretation (Thousand Oaks, CA: Sage, 1994).
52. Paulus, Lester, and Dempster, Digital Tools.
53. Childress, “Citation Tools in Academic Libraries.”
54. Antonijević and Cahoy, “Personal Library Curation.”
55. Kramer and Bosman, “Innovations in Scholarly Communication.”