Learning Outcomes
By the end of this section, you will be able to:
- Integrate your ideas with ideas from related sources.
- Locate, compile, and evaluate primary, secondary, and tertiary research materials related to your topic.
A bibliography is a list of the sources you use when doing research for a project or composition. Named for the Greek terms biblion, meaning “book,” and graphos, meaning “something written,” bibliographies today compile more than just books. Often they include academic journal articles, periodicals, websites, and multimedia texts such as videos. A bibliography alone, at the end of a research work, also may be labeled “References” or “Works Cited,” depending on the citation style you are using. The bibliography lists information about each source, including author, title, publisher, and publication date. Each set of source information, or each individual entry, listed in the bibliography or noted within the body of the composition is called a citation.
Bibliographies include formal documentation entries that serve several purposes:
- They help you organize your own research on a topic and narrow your topic, thesis, or argument.
- They help you build knowledge.
- They strengthen your arguments by offering proof that your research comes from trustworthy sources.
- They enable readers to do more research on the topic.
- They create a community of researchers, thus adding to the ongoing conversation on the research topic.
- They give credit to authors and sources from which you draw and support your ideas.
Annotated bibliography expand on typical bibliographies by including information beyond the basic citation information and commentary on the source. Although they present each formal documentation entry as it would appear in a source list such as a works cited page, an annotated bibliography includes two types of additional information. First, following the documentation entry is a short description of the work, including information about its authors and how it was or can be used in a research project. Second is an evaluation of the work’s validity, reliability, and/or bias. The purpose of the annotation is to summarize, assess, and reflect on the source. Annotations can be both explanatory and analytical, helping readers understand the research you used to formulate your argument. An annotated bibliography can also help you demonstrate that you have read the sources you will potentially cite in your work. It is a tool to assist in the gathering of these sources and serves as a repository. You won’t necessarily use all the sources cited in your annotated bibliography in your final work, but gathering, evaluating, and documenting these sources is an integral part of the research process.
Compiling Sources
Research projects and compositions, particularly argumentative or position texts, require you to collect sources, devise a thesis, and then support that thesis through analysis of the evidence, including sources, you have compiled. With access to the Internet and an academic library, you will rarely encounter a shortage of sources for any given topic or argument. The real challenge may be sorting through all the available sources and determining which will be useful.
The first step in completing an annotated bibliography is to locate and compile sources to use in your research project. At the beginning, you do not need to be highly selective in this process, as you may not ultimately use every source. Therefore, gather any materials—including books, websites, professional journals, periodicals, and documents—that you think may contain valuable ideas about your topic. But where do you find sources that relate to your argument? And how do you choose which sources to use? This section will help you answer those questions and choose sources that will both enhance and challenge your claim, allowing you to confront contradictory evidence and synthesize ideas, or combine ideas from various sources, to produce a well-constructed original argument. See Research Process: Accessing and Recording Information for more information about sources and synthesizing information.
Primary, Secondary, and Tertiary Sources
In your research, you likely will use three types of sources: primary, secondary, and tertiary. During any research project, your use of these sources will depend on your topic, your thesis, and, ultimately, how you intend to use them. In all likelihood, you will need to seek out all three.
Primary Sources
Primary sources allow you to create your own analysis with the appropriate rhetorical approach. In the humanities disciplines, primary sources include original documents, data, images, and other compositions that provide a firsthand account of an event or a time in history. Typically, primary sources are created close in time to the event or period they represent and may include journal or diary entries, newspaper articles, government records, photographs, artworks, maps, speeches, films, and interviews. In scientific disciplines, primary sources provide information such as scientific discoveries, raw data, experimental and research results, and clinical trial findings. They may include published studies, scientific journal articles, and proceedings of meeting or conferences.
Primary sources also can include student-conducted interviews and surveys. Other primary sources may be found on websites such as the Library of Congress, the Historical Text Archive, government websites, and article databases. In all academic areas, primary sources are fact based, not interpretive. That is, they may be commenting on or interpreting something else, but they themselves are the source. For example, an article written during the 1840s condemning the practice of enslavement may interpret events occurring then, but it is a primary source document of its time.
Secondary Sources
Secondary sources, unlike primary sources, are interpretive. They often provide a secondhand account of an event or research results, analyze or clarify primary sources and scientific discoveries, or interpret a creative work. These sources are important for supporting or challenging your argument, addressing counterarguments, and synthesizing ideas. Secondary sources in the humanities disciplines include biographies, literary criticism, and reviews of the fine arts, among other sources. In the scientific disciplines, secondary sources encompass analyses of scientific studies or clinical trials, reviews of experimental results, and publications about the significance of studies or experiments. In some instances, the same item can serve as both a primary and a secondary source, depending on how it is used. For example, a journal article in which the author analyzes the impact of a clinical trial would serve as a secondary source. But if you instead count the number of journal articles that feature reports on a particular clinical trial, you might use them as primary sources because they would then serve as data points.
Table 14.1 provides examples of how primary and secondary sources often relate to one another.
| Discipline | Primary Source Example | Secondary Source Example |
|---|---|---|
| Literature | Wilfred Owen’s poem “Dulce et Decorum est” | Essay analyzing World War I poetry |
| Psychology | Raw data from a study testing the effects of a medication on bipolar disorder | Book evaluating different approaches to treating bipolar disorder in patients |
| Politics and Government | Transcript of John F. Kennedy’s inauguration speech | Website analyzing the themes present in John F. Kennedy’s inauguration speech |
| History | Diary of a soldier who fought in the Civil War | Textbook entry about the battles of the Civil War |
| Fine Art | Native American pottery | Newspaper article about the importance of honoring Native American art |
| Performing Arts | Recording of a live concert | Critical review of a concert published in a magazine |
Tertiary Sources
In addition to primary and secondary sources, you can use a tertiary source to summarize or digest information from primary and/or secondary sources. Because tertiary sources often condense information, they usually do not provide enough information on their own to support claims. However, they often contain a variety of citations that can help you identify and locate valuable primary and secondary sources. Researchers often use tertiary sources to find general, historical, or background information as well as a broad overview of a topic. Tertiary sources frequently placed in the secondary-source category include reference materials such as encyclopedias, textbooks, manuals, digests, and bibliographies. For more discussion on sources, see The Research Process: Where to Look for Existing Sources.
Authoritative Sources
Not all sources are created equally. You likely know already that you must vet sources—especially those you find on the Internet—for legitimacy, validity, and the presence of bias. For example, you probably know that the website Wikipedia is not considered a trustworthy source because it is open to user editing. This accessibility means the site’s authority cannot be established and, therefore, the source cannot effectively support or refute a claim you are attempting to make, though you can use it at times to point you to reliable sources. While so-called bad sources may be easy to spot, researchers may have more difficulty discriminating between sources that are authoritative and those that pose concerns. In fact, you may encounter a general hierarchy of sources in your compilation. Understanding this hierarchy can help you identify which sources to use and how to use them in your research.
Peer-Reviewed Academic Publications
This first tier of sources—the gold standard of research—includes academic literature, which consists of textbooks, essays, journals, articles, reports, and scholarly books. As scholarly works, these sources usually provide strong evidence for an author’s claims by reflecting rigorous research and scrutiny by experts in the field. These types of sources are most often published, sponsored, or supported by academic institutions, often a university or an academic association such as the Modern Language Association (MLA). Such associations exist to encourage research and collaboration within their discipline, mostly through publications and conferences. To be published, academic works must pass through a rigorous process called peer review, in which scholars in the field evaluate it anonymously. You can find peer-reviewed academic sources in library catalogs, in article databases, and through Google Scholar online. Sometimes these sources require a subscription to access, but students often receive access through their school.
Academic articles, particularly in the social and other sciences, generally have most or all of the following sections, a structure you might recognize if you have written lab reports in science classes:
- Abstract. This short summary covers the purpose, methods, and findings of the paper. It may discuss briefly the implications or significance of the research.
- Introduction. The main part of the paper begins with an introduction that presents the issue or main idea addressed by the research, establishes its importance, and poses the author’s thesis.
- Review. Next comes an overview of previous academic research related to the topic, including a synthesis that makes a case for why the research is important and necessary.
- Data and Methods. The main part of the original research begins with a description of the data and methods used, including what data or information the author collected and how the author used it.
- Results. Data and methods are followed by results, detailing the significant findings from the experiment or research.
- Conclusion. In the conclusion, the author discusses the results in the context of the bigger picture, explaining the author’s position on how these results relate to the earlier review of literature and their significance in the broad scope of the topic. The author also may propose future research needs or point out unanswered questions.
- Works Cited or References. The paper ends with a list of all sources the author used in the research, including the review of literature. This often-overlooked portion of the composition is critical in evaluating the credibility of any paper that involves research.
Credible Nonacademic Sources
These sources, including articles, books, and reports, are second in authority only to peer-reviewed academic publications. Credible nonacademic sources are often about current events or discoveries not yet reviewed in academic circles and often provide a wider-ranging outlook on your topic. Peer-reviewed texts tend to be narrow and specific, whereas nonacademic texts from well-researched sources are often more accessible and can offer a broader perspective. These three major categories generally provide quality sources:
- Information, white papers, and reports from government and international agencies such as the United Nations, the World Health Organization, and the United States government
- Longer articles and reports from major newspapers, broadcast media, and magazines that are well regarded in academic circles, including the New York Times, the Wall Street Journal, the BBC, and the Economist
- Nonacademic books written by authors with expertise and credentials, who support their ideas with well-sourced information
To find nonacademic sources, search for .gov or .org sites related to your topic. A word of caution, however: know that sources ending in .org are often advocacy sites and, consequently, inherently biased toward whatever cause they are advocating. You also can look at academic article databases and search articles from major newspapers and magazines, both of which can be found online.
Short Informational Texts from Credible Websites and Periodicals
The next most authoritative sources are shorter newspaper articles or other pieces on credible websites. These articles tend to be limited in scope, as their authors report on a single issue or event. Although they do not often provide in-depth analysis, they can be a source of credible facts to support your argument. Alternatively, they can point you in the direction of more detailed or rigorous sources that will enhance your research by tracing the original texts or sources on which the articles are based. Usually, you can find these sources through Internet searches, but sometimes you may have difficulty determining their credibility.
Judging Credibility
To judge credibility, begin by looking for the author or organization publishing the information. Most periodical compositions contain a short “About the Author” blurb at the beginning or end of the article and often include a link to the author’s credentials or to more information about them. Using this information, you can begin to determine their expertise and, potentially, any agenda the author or organization may have. For example, expect a piece discussing side effects of medical marijuana written by a doctor to present more expertise than the same piece written by a political lobbyist. You also can determine whether bias is present; for example, the organization may promote a particular way of thinking or have an agenda that will influence the content and language of the composition. In general, look for articles written with neutral expertise.
The CRAAP Test
You may find the CRAAP test a helpful and easy-to-remember tool for testing credibility. This checklist provides you with a method for evaluating any source for both reliability and credibility. CRAAP stands for Currency, Relevance, Authority, Accuracy, and Purpose. The CRAAP test, as shown in Table 14.2, includes questions that can be asked of any source.
| Currency | When was the information published, revised, or updated? Does your topic require current information? Are links within the source current? |
| Relevance | Does the information relate to your topic or support your thesis? Who is the intended audience of the source? What is the purpose of the source? |
| Authority | Who is the publisher, sponsor, or source? What are the author’s credentials and/or qualifications? Does the URL reveal anything about the source? |
| Accuracy | Where does the information come from? What evidence is used to support the information, and can it be verified? Are there elements of bias? Has the information been reviewed? |
| Purpose | What is the author’s purpose for creating the source? Is the information based on facts, opinion, or propaganda? What biases are present? Are biases recognized? |
Sources with Clear Bias or Unclear Authority
The final type of source encompasses nearly everything else. Although they cannot be considered credible or valid to support your argument or claims, these sources are not necessarily useless. Especially when you are compiling sources at the beginning of a project, those with clear bias or unclear authority can be useful as you explore all facets of a topic, including positions within an argument. These sources also can help you identify topics on which to base your search terms and can even point you toward more credible sources.
Locating Sources
Academic article databases are the best starting places for finding sources. There are too many databases to cover them all in this chapter, but you would be wise to familiarize yourself with those to which you have access through your school or program. For further information on databases, see The Research Process: Where to Look for Existing Sources. In the long run, this knowledge will save you a good deal of time and a possible headache.
You will want to start with your college library website, which includes access to sources paid for by your institution. As a student, you should be able to access these quickly and easily. Another popular and wide-ranging database is Google Scholar. Google Scholar is helpful for finding sources across a wide range of topics. One drawback, however, is that it catalogues nearly all disciplines, so the results can be vast and unfocused. Therefore, when using Google Scholar, be as specific as possible, and add your academic discipline as a keyword. For example, when searching for information on climate change, add the keyword “environment” or “politics” depending on your research angle; otherwise, the results will include all disciplines and potentially bury the articles you seek. Google Scholar also has a feature labeled “Cited by,” which shows you other papers that cite the article in their review of literature relate to the topic. Writing Process: Informing and Analyzing contains more information about focusing your searches. Like clues to a mystery, one search can lead you to a wealth of related articles.
When you are able to identify potential sources by reading their abstracts or using Google Scholar, you may at times land on a publisher’s website that requires you to pay to read the full article. When you find yourself in a situation such as this, record information about the article—author(s), article title, journal title, publication date. It is likely that you will be able to use your school’s database to access the article. For information about other databases, consult The Research Process: Where to Look for Existing Sources.
Just as writing is recursive, requiring you to go back and forth between different stages of the process, you will likely return to your annotated bibliography at different points. You may begin by looking for sources related to your topic, or you may choose or narrow your topic after an initial database search for sources. If your project has a variety of possible topics, you may even start with a current issue of a leading journal in the field, find an article that interests you, and use that article to shape your topic selection. As a bonus, you will have your first reputable source. Later, as you refine your thesis, reasoning, and evidence, you may find yourself returning to your search for sources. Consider this hypothetical situation: You are developing an argument that examines the risk factors of childhood trauma that surface in later life. As you analyze the data from your sources, it occurs to you to find out whether any documented correlation exists between early trauma and resilience. So you return to Google Scholar and your university’s academic database to find more research based on this idea in order to revise your analysis by adding the new viewpoint.
One difficulty may be homing in on the keywords that will lead you to the sources you need. At this point, sources from the last two categories discussed may come into play: short pieces from credible websites and newspapers and other texts with clear bias or unclear authority. Less credible sources may lead you to better ones, particularly if you can identify the keywords used in them and then apply those keywords within academic databases. For more on developing useful keywords, consult The Research Process: Where to Look for Existing Sources.
Boolean Operators
Keyword searches can become frustrating, either yielding so much information that it seems impossible to sort through or narrowing the search so much that you miss important potential sources. One way to remedy this situation is to become familiar with Boolean operators, the basis of mathematical sets and database logic. Rather than searching with natural language only, you can use these operators to focus your search. The three basic Boolean operators are AND, OR, and NOT. Using these operators helps you search by linking necessary information, excluding irrelevant information, and focusing information. For example, if you have some pieces of information from tertiary sources, you may be able to use Boolean operators to find additional useful sources. A search string such as artificial intelligence (title) AND Buiten (author) AND 2019 (year) can yield the exact journal source you need. Here is a brief review of how to use the three operators:
- Use AND to narrow search results and tell the database to include all search terms in finding sources. If you want to find sources that include all of the search terms entered, use the AND operator. In Figure 14.11, the darkest blue triangular section in the center of the Venn diagram represents the result set for this search, including all three terms. In many databases, including Google, AND is implied between each word. To exclude AND, use quotation marks. For example, Google would translate the search term ethics artificial intelligence as ethics AND artificial AND intelligence. To make your phrases more specific, use the AND operator combined with quotation marks: “ethics” AND “artificial intelligence”.
- Use OR to connect two or more similar concepts and broaden your results, telling the search engine that any of your search terms can appear in the results it gives you. The Boolean operator OR is represented by Figure 14.12. Using the OR operator gives you a very large set of results.
- Use NOT to exclude results from a search. This operator can help you narrow your search, telling the search engine to ignore names or words you do not want included in your results. For example, if you know you don’t want self-driving cars in your search results, you might search for “artificial intelligence” NOT “self-driving cars”.
Choosing Sources
Choosing sources to include in your annotated bibliography may seem overwhelming. However, if you can find a few good academic articles as a starting point, use them to guide your research. Academic articles are efficient, scrutinized by experts in their fields, and organized in ways that aid readers in identifying key findings that relate to their argument. The following tips will help you choose solid sources to guide your research:
- Look for relevant scholarly articles. Even the briefest Google search can yield an overwhelming amount of content. Sift through it by looking first through academic databases to find high-quality sources relevant to your research.
- Read abstracts. As you sift through scholarly articles, you can get a good idea of what each one is about by reading the abstract. It includes the findings and will show you in about 100 words whether the paper holds relevance to your research.
- Skim. Once you have determined that an article may be useful, skim each section to glean the information you need. Closer and more extensive reading can come later as you develop and support your argument.
- Avoid getting bogged down in technical information or industry-specific jargon. The benefit of reading peer-reviewed research is that you know the reviewers have determined it to be solidly constructed. Therefore, even if you don’t understand some portions completely, you can still feel confident about using relevant information from the article.
- Work smarter by using the research provided. Once you have identified an article that is helpful to your research, use it to find more like it. Search for other publications by the authors; researchers often spend much of their careers researching one overarching topic or theme. Use the review of literature to identify related articles that may add to your research. You can also use the article’s bibliography to find additional sources. Or reverse engineer the process: use article databases to find other articles that cite the article in their literature reviews.