to BioTechniques free email alert service to receive content updates.
How to Predict Scientific Impact

09/12/2012
Jim Kling

A new tool promises to predict a scientist’s future success better than the traditional h-index.


Attempts to predict a young scientist’s future success are nothing new. Search committees attempt to do it. So do funding agencies. But now researchers have described a new tool in this week’s Nature that they claim improves upon the traditional h-index (1).

A scientist’s h-index is calculated by weighing the number of papers published by that researcher and the number of citations that those papers have garnered. The higher the h-index, the better the scientist, or so the theory goes. But while a scientist’s score is good at predicting his or her h-index in a year or two, it’s a poor predictor of the h-index 10 years out.

Researchers have described a new tool in this week’s Nature that they claim can predict a researcher’s future h-index. Source: Kording Lab website





Konrad Kording and his colleagues at Northwestern University decided they could improve the measure. They analyzed a database of 3085 neuroscientists, 57 Drosphila scientists, and 151 evolutionary biologists, from which they could glean a publication, citation, and funding history.

Simply counting a scientist’s papers is a mistake, because individuals can knock out a high number of low-quality papers. Even relying on citation numbers is risky, because a researcher might have been a middle author on a paper that garnered many citations instead of the key driver of the research. “(The scientist) happened to have been in the lab at the right time,” said Kording, who is an associate professor in physical medicine and rehabilitation at the university’s Feinberg School of Medicine.

The new tool uses the scientist’s h-index, but also includes number of articles written, the number of years since publishing the first article, the number of distinct journals published in, and the number of articles in high-impact journals. “They are the same factors that search committees use. What is new is that we [have determined] how much you should weigh them,” said Kording.

It is about twice as accurate as a scientist’s current h-index when it comes to predicting the h-index at 10 years out, said Kording. But it remains imperfect, explaining only about half the variance in h-index scores.

One of the more important factors is the number of unique journals a scientist has published in. Kording believes that scientists who publish broadly tend to be doing very well. “If you publish in many different journals, there are more scientists who could cite you, and more people will read your papers. You're also probably a better [scientist], because oftentimes the editor of the journal asks you to [submit], because they like your research,” he said.

So, if your score is low, should you worry? Well, the tool isn’t perfect, so there’s no need to panic just yet. “Maybe you’ve gotten better training and you just haven’t published yet. Or maybe your field is about to have a big [breakthrough], and you will be washed to success with the rest of the field,” said Kording.

The team has plans to develop algorithms for other aspects of a scientific career, including teaching ability (using data from a website where students rate their teachers), funding success, and patents.

References

  1. Acuna, D. E., S. Allesina, and K. P. Kording. 2012. Future impact: Predicting scientific success.Nature 489(7415):201-202.

Keywords:  career