Science is self-correcting: Lessons from the arsenic controversy

This raises an interesting question: just who is critiquing the NASA study? It turns out that many of the critics are also NASA-funded. In fact, many prominent critics of this study are funded by the NASA Astrobiology Institute – the very same program that funded the arsenic study.

Carl Zimmer gives us several examples:

  • Norm Pace offers the critique: “Low levels of phosphate in growth media, naive investigators and bad reviewers are the stories here”.
  • Shelley Copley suggested, “this paper should not have been published”
  • Roger Summons remarked that a critical experiment was left undone, and backed the critical blog analysis of his NASA-funded former student.
  • Michael Russell agreed with blogosphere critics, and offered his own critique of the study based on cosmic ratios of phosphorus to arsenic. Russell is a member of the Astrobiology Institute, as well as an employee of NASA’s Jet Propulsion Laboratory.
  • Forest Rohwer observed, “the experimental evidence in the paper is pretty weak.”
  • George Cody says he “cannot accept this claim until such an experiment [mass spectrometry] (easily done) is performed.”
  • Steven Benner was an early skeptic. To NASA’s credit, they invited him to present his criticisms at the press conference. He has said “we are not expecting this result to survive”.

Each of these scientists is affiliated with NASA Astrobiology.

Lesson three: Scientists offer opinions based on their scientific knowledge and a critical interpretation of data. Scientists willingly critique what they think might be flawed or unsubstantiated science, because their credibility – not their funding – is on the line.

Regardless of whether or not ‘arseno-DNA’ survives the test of time and further study, scientists have shown that they will rigorously criticize science perceived as flawed, with no fear of reprisal from funding agencies.

This is the key lesson to take from this incident, and it applies to all scientific disciplines: peer-review continues after publication. Challenges to consensus are seriously entertained – and are accepted when supported by rigorous data. Poorly substantiated studies may inspire further study, but will be scientifically criticized without concern for funding opportunities. Scientists are not “afraid to lose their grant money”.

Finally, there is the issue of how scientists who publish papers that generate credible blog reactions should in turn react. In times past, it was simple to wait for properly crafted letters and comments to be sent in to the journal. This gave fixed targets to deal with and allowed for considered reflection and response; discussions would perhaps be published 6 months to year later. But today, serious criticisms can arrive immediately (as seen above). Nature (perhaps with a little schadenfreude) had an op-ed suggesting that the authors on this (Science) paper should be more strongly engaged in the reaction, while Science had a plea from the lead author for a little patience, since they were clearly a little overwhelmed.

Page 2 of 3 | Previous page | Next page