Evaluating Scientific Papers: Details are Important

In a previous Buzz Words column I advised readers to evaluate the conclusions and assertions in scientific papers carefully. For example, there are scientific publications asserting that dead bird reporting is a good indicator of human risk for West Nile transmission. Recall, for reasons stated in previous Buzz Words articles, "it ain't necessarily so." If there is one consistent theme in reading scientific papers critically it is "pay attention to the details." After more careful consideration of the details, critical readers may find that many papers lead the reader to conclusions that are not as clear cut as the authors believe.

A recent publication by Tonry et al. (2005. West Nile Virus detection in urine. Emerging Infectious Diseases 11: 1294-1296) can be read at:

In this interesting paper, the authors detected WNV RNA by RT-PCR in human urine from a patient 8 days after the onset of symptoms. Live WNV was not isolated from the urine with standard virus isolation techniques. Although the authors state that the implications of their findings are unclear, the authors believe this finding may lead to a rapid diagnostic technique for WNV in humans. They are conducting further studies on how long WNV can be found in human urine samples from patients with clinical and subclinical WNV infections. Based on their findings, work with other flaviviruses, and findings of actual WNV in hamster urine, the authors conclude that WNV is shed in human urine as well. It is straightforward, and certainly important, that WNV RNA can be detected in human urine.

Why are the details important? Richard Lewontin (in his book It Ain't Necessarily So: The Dream of the Human Genome and Other Illusions. 2000. New York Review of Books) relates the story of the wonder-rabbi of Chelm. This wonder-maker had a vision where he saw the fiery destruction of a school in the city of Lublin more then 50 miles away. His vision solidified and enhanced his stature as the Chelm wonder-rabbi. Several days later, a visitor to Chelm arrived from Lublin. Everyone offered sympathies at the tragic loss of the school. The visitor told everyone that there was no fire, the school is just fine, and he asked "what kind of wonder-rabbi is that?" The reply from one of the rabbi's disciples was, "Well, well, burned or not burned, it's only a detail. The wonder is he could see so far."

Now back to scientific publications, details, and critical reading. The paper detecting WNV RNA in human urine is interesting and raises many issues for readers to consider. Consider the details.

  • WNV RNA is not necessarily live WNV and may in fact be RNA fragments of WN virions.
  • A sample size of 1 raises the concern that the circumstances of this single individual may not be generally applicable to humans.
  • What procedures were used to attempt to isolate live virus? How much urine was subjected to PCR testing, how many tests, what volumes were used to attempt virus isolations?
  • Why was WNV RNA not found in any of the other (how many?) daily samples? One needs details of how many samples were processed on each day, the volumes tested and an assessment of the probability of detecting specific quantities or levels of RNA in specific sample volumes. Details. Details.

The finding of WNV RNA in this single patient's urine is interesting and indeed does show the presence of WNV RNA in this individual. Therefore PCR testing of urine from individuals infected with WNV may lead to a new diagnostic test for West Nile virus, and certainly more work is warranted. This preliminary observation will need to be validated through more sampling and more attention to details in order to accurately assess the importance of this preliminary first report.

Remember, "It's only a detail. The wonder is he could see so far."

Pay careful attention to details.

Walter J. Tabachnick, Professor
Florida Medical Entomology Laboratory
University of Florida/IFAS