Skip to content

Can science become too big to fail?

March 25, 2013

I recently had an interesting discussion with my colleagues Dan Graur and Ricardo Azevedo about their recent critique of the ENCODE project. If you have not read the article, I recommend it highly. I can guarantee that you will be entertained more than by almost any scientific article you’ve read in your life.

I believe that the authors make very good points here – they argue that the claims that 80% of the genome is functional (i.e. there is very little “junk DNA”) are unsupported by the data. A very interesting point is that the data was mined somewhat blindly. Evolutionary biologists who may have had a different perspective and interpretation were not consulted, for instance. I am not a specialist in these areas, and will leave this debate to others.  But the article, and the response it received raise a number of other questions.

How best to interpret the large amounts of data that are becoming available is a problem of tremendous importance. However, I wanted to address another question here. One of the main criticism of the paper by Graur, et al. is about the tone they chose to adopt. Some have deemed it too aggressive or disrespectful, or simply unbecoming of a scientific publication.

I know the people who wrote the paper well. I am certain they would not have written a controversial paper for the sake of controversy – they believe deeply and stand behind what they wrote. This was a small group of scientists in a good, but not super-famous department. They were calling into question the findings of a project that cost $200 million and involved over 400 scientists, or more. Earlier publications have questioned the results of the ENCODE project, but none have gotten the attention that the paper by Graur, et al. did – the snarky tone certainly caught people’s attention, and that was the point.

However, this also exposes a fundamental problem with Big Science. By definition, Big Science projects cost millions or even billions of dollars. They can fund hundreds of labs and thousands of scientists. Could such projects be too big to fail? If the projects do not achieve the desired results, will it be possible to admit so? Or suppose even that great new data is generated, and our knowledge is advanced by the project. But perhaps the results are not easily summarized in a headline or a soundbite. Perhaps they are complicated, not easily interpreted, or simply not definitive. However, if billions were spent, will there be a need to try to come up with headline worthy claims, and pronouncements that textbooks will have to be rewritten?

When so much money and so many careers are at stake, the rules under which science is done probably become different. To openly challenge such results becomes a questionable career move – it becomes likely that you have just made hundreds or thousands of enemies. If you have a disagreement with one or a few colleagues, that may not mean that your next grant will not get funded. But if you offended hundreds of them things may look different.

I don’t want to say that big scientific projects should not exist. Some such projects have been amazingly successful in the past – big science gave us the map of the human genome, and nuclear weapons (you can argue with the benefit to humanity, but not with the success), and Big Engineering landed people on the Moon. However, there is a political side of such projects that sets them apart. The goals of the projects will need to be carefully defined, and whenever possible, all steps of the project will need to be open to the entire scientific community.

In almost all cases, controversy is nothing to be afraid of in science. Controversy is frequently what drives science forward. If the result of Big Science is to tame or extinguish controversy, we should be worried indeed.


From → Uncategorized

  1. What is the role of the funding agency in all of this? In this case, the NIH. I am not aware of what they have actually done, or are even required to do, with regard to evaluating the merits of *results*, as opposed to just the merits of a proposal. Personally, I dislike the “snarky” tone, and I wonder if it’s a shortcut for attention that undermines the credibility of the process (an ends-justifies-the-means mentality). But it’s no big deal.

  2. Nicolas Le Novère permalink

    I think there is a confusion here. There is the large data-generation project, that costed a very large amount of money. And there is the analysis done by some of the researchers working on the project. What Dan Graur hurled abuse at was the analysis, not the data. The data is out there, ready to be analysed by anyone who disagree on the conclusions. The project did not succeed or failed, like the sequencing of the human genome did not fail because it did not provide answers to the causes of most diseases or phenotypes.

    I know many scientists who worked on the ENCODE project. They are very good scientists, most of them not under the spotlights. They did not tried to sale unsupported conclusions just to get publicity. They thought they did a good job. All the hundreds of them. They did not expect to be insulted, that is all. Dan Graur may have a point (I kind of disagree a bit But using tone and words used by creationists, climage change skeptics or anti-GMO activists does not help.

    The point is that there was nothing in the “article” of Dan Graur that was food for a scientific article. It was a piece of opinion, valid for a correspondance or something like that. We can absolutely criticise big science. And I am fairly certain that if Dan Graur or somebody else comes with a very solid statistical analysis of ENCODE data, and show conclusively that a part of the 80 is not “functional” or that most of the 62% of transcribed one is also junk RNA, this will be published in a very good journal.

    • josic permalink

      You raise a good point. These are two separate issues, and they should be treated as such. However, from my discussions with the group that did the reanalysis (and Graur, et al. did reanalyze the data), the data was not all that easy to work with. Making the data openly available, and making it easy to use by the community is not the same. Again, I am only reporting the experiences of others who have tried this, and neither I nor my collaborators tried it. Perhaps someone outside of the project who has tried to access and work with the data can comment.

  3. Obama just officially announced the $100 million big brain project. Talk about too big to fail.

  4. josic permalink

    Perhaps I have not looked at the right places, but I still don’t really know what the goal of the project will be. And if the goals are not clearly defined, then it really is impossible to fail.

Trackbacks & Pingbacks

  1. Hypothesis-free big science: is it good for you? | The Computational Neuroscientist

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: