Chase References Backward and Forward · Method

Purpose

Once you have one good source, it can lead you to most of the others. Going backward means reading its reference list: the earlier work it relied on. Going forward means finding the later work that cites it: who built on it, tested it, corrected it or argued with it. Researchers also call this citation chasing or snowballing.

It often beats keyword searching. When Trisha Greenhalgh and Richard Peacock audited the 495 sources in a large review they had written on how new practices spread through health-care organizations, they found that only 30% had come from their planned database and hand searches. About half (51%) came from snowballing, such as following references of references, and 24% from personal knowledge and contacts. Tracking citations electronically took them about a day; hand-searching 271 journal-years took a month and turned up only 24 papers they used.

Keyword searches only find work that uses your words. Citations connect work that uses different words for the same thing.

When you need it

You’ve found one or two sources that are exactly on target and want more like them.
Keyword searches return junk, because the field uses different terms than you do.
You want to know whether a finding held up: was it repeated, challenged or corrected?
You’ve read a review article and want to follow its leads.
You want to know whether you’ve missed something important.

How to do it

1Pick good starting points

Start from one to three sources that sit right on your question. A recent review is ideal, because its reference list covers the field. A well-known older study works too, because a lot of later work will cite it.

2Go backward through the reference list

Read the reference list, but don’t stop at the titles. Find where each reference is cited in the text and what it’s cited for. Mark the ones cited for claims that matter to you, the older works that everything else seems to build on, and any other reviews. Those are your next reads.

3Go forward with “Cited by”

In Google Scholar, each result has a Cited by link that lists later work citing it. On that page, tick Search within citing articles and add a keyword to narrow a long list. Semantic Scholar’s paper pages also list a paper’s references and the papers that cite it, and flag some citations as “highly influential” (a machine judgment of how much the cited work shaped the citing one). Web of Science and Scopus, usually available through a university or public library, do the same with their own data. Each tool covers a different set of publications, so the counts and lists won’t match. That’s normal.

4Repeat from what you find

Each relevant new source becomes a new starting point: check its references and its citations in turn. Keep a log, or use a reference manager such as Zotero, so you know what you’ve already checked and why you kept or dropped it.

5Look for challenges and corrections

Forward chasing is how you find out what happened to a finding after it was published. Look in the citing works for replications, failed replications, critiques, corrections and replies from the original authors. A finding that later work quietly abandoned can still look solid if you only read the original.

6Stop when the same names keep coming back

You’re near the end when new rounds mostly turn up sources you already have: the same authors, the same key studies, the same few reviews. That’s saturation. Claes Wohlin’s guidelines for snowballing in systematic reviews work the same way: repeat backward and forward rounds until no new relevant papers appear. For smaller projects, also set limits in advance, such as a date range or a number of rounds.

Try it

Your task

Start from one of the most famous papers in science: J. D. Watson and F. H. C. Crick, “Molecular Structure of Nucleic Acids: A Structure for Deoxyribose Nucleic Acid,” Nature 171: 737–738, April 1953. Go backward: what is the first reference in its list, and what does the paper say about it? Then go forward two steps: find a paper by the same two authors, published later in 1953, that cites it; then find a famous 1958 experiment whose reference list includes that second paper.

Show a hint

The reference list is short. For the forward steps, the second paper is about what the structure means for genetics, and the 1958 experiment used a heavy form of nitrogen to test how DNA copies itself. Google Scholar’s “Search within citing articles” can help narrow the citing list.

Show the answer and the path

Backward: reference 1 is Linus Pauling and Robert Corey’s 1953 proposal for the structure of nucleic acids (Nature 171: 346, and Proceedings of the National Academy of Sciences 39: 84). It was a three-chain structure with the phosphates near the axis and the bases on the outside. Watson and Crick open their paper by saying they find it unsatisfactory, for two reasons: the X-ray data came from the salt rather than the free acid, so nothing obvious would hold the negatively charged phosphates together; and some of its atomic distances look too small. Their other references are to Furberg, Chargaff, Wyatt, Astbury, and Wilkins and Randall.

Forward, step 1: Watson and Crick, “Genetical Implications of the Structure of Deoxyribonucleic Acid,” Nature 171: 964–967 (1953), which cites the April paper as its first reference.

Forward, step 2: Matthew Meselson and Franklin Stahl, “The Replication of DNA in Escherichia coli,” Proceedings of the National Academy of Sciences 44(7): 671–682 (1958). Its reference list includes “Genetical Implications.”

Two lessons. First, the important follow-up isn’t always the most obvious citation of your starting paper: here it ran through a second paper, so it takes two forward steps. Second, the April paper is cited so widely that its “Cited by” list is too long to read; searching within the citing articles is what makes it usable.

Common mistakes

Only going backward. Everything a reference list leads to is older than your starting point. Forward chasing finds the updates, critiques and corrections.
Treating citation counts as a quality score. Papers get cited for being wrong, controversial or early, as well as for being right.
Following every reference. Chase the ones cited for claims that matter to your question, and skim the rest.
Staying inside one circle. Research groups tend to cite each other. Run a keyword search too, and look on purpose for critics and rival views.
Not keeping a log. Without one you’ll check the same papers twice and lose track of why you dropped others.

Limits

No tool covers everything. Google Scholar doesn’t index all scholarly work, older work may not be digitized, and some databases can’t show a paper’s references because the publisher hasn’t shared them. When a list looks incomplete, open the paper itself.

Forward chasing can’t find work that hasn’t been cited yet, so the newest research is invisible to it. Combine it with keyword searches, and set up alerts if your project runs for months.

Citation networks can also mislead. In a 2009 study in the BMJ, Steven Greenberg mapped 242 papers and 675 citations about one claim in muscle disease research. Papers supporting the claim received about 94% of the citations to primary data; six papers with data that weakened it received about 6%. If you only follow citations, you inherit the field’s habits of attention, including what it ignores. Seek out the papers that disagree.