Quantifying creativity in research work
Society has accepted that revolutionary scientific advances and innovation are the engine of prosperity and, as a result, research spending has grown over the years. In the United States, funding for research increased more than tenfold (excluding inflation) in the period from 1955 to 2020. And, according to UNESCO records, over the past few years, investment in global research has grown faster than the economy, with the most advanced countries allocating more than 3% of their GDP to research.
Accordingly, it is relevant to ask about the well-being of research activity and how its output is changing, i.e., whether the increase in funding and in the number of scientists has led to a proportional increase in scientific discoveries.
A study published in Nature[1] last January focuses on this issue, that is to say, on how to define whether a scientific paper establishes new research directions and how to quantify this characteristic of creativity in scientific output over the course of time, noting “a marked decline in disruptive science and technology over time”. Russell J. Funk, co-author of the paper, explains that disruptive science is “work that breaks with the status quo and carves out new directions for scientific inquiry.”[2]
An original study
The paper, entitled Papers and Patents are Becoming Less Disruptive Over Time, authored by professors from the Universities of Minnesota and Arizona, examines how to discern scientific contributions that simply improve existing knowledge streams from those that disrupt current knowledge, rendering them obsolete and pushing science and technology in new directions. And, how to quantify the development over the course of time.
The authors argued that, if a study was very groundbreaking, subsequent research would be less likely to cite the study references and would instead cite the study itself. Based on this concept, a new indicator that measures the disruptive capacity was created, the CD index. Using this, they analyzed 25 million scientific papers in all fields of science, which is hundreds of millions of citations covering a period of more than 60 years, from 1945 to 2010, and 3.9 million patents, from 1976 to 2010.
The same study notes that disruptive papers can also be identified from a language analysis. While disruptive papers use words that evoke discovery or creation (“produce”, “determine”), articles more based on consolidation use terms that evoke the improvement of an existing process (“enhance”, “improve”, “associate”). The language itself reveals the more or less innovative character of the work.
The authors concluded that, throughout the period analyzed, there was a sharp and sustained decline over time in disruptive research, truly revolutionary studies, and a predominance of incremental research.
Confirmation of what is already known
This is not the first analysis that has attempted to explore how innovative activity evolves, reaching similar conclusions. As the article itself states, there are “studies [that] document declining research productivity in semiconductors, pharmaceuticals and other fields.” An extensive study conducted in 2020 by a group of researchers from Northwestern University and the University of Chicago[3] on 1.8 billion citations among 90 million papers on 241 topics affirmed that “[a] novel idea that does not fit within extant schemas will be less likely to be published, read, or cited,” and found that “much more research effort and money are now required to produce similar scientific gains—productivity is declining precipitously.” Another article, also published in 2020 by a research group from Stanford University and MIT[4], determined: “Our robust conclusion is that research productivity is falling sharply everywhere we look.”
No agreement on the causes
But while the idea of major and lasting regression in innovation is consolidated, there is no consensus explanation for its causes. The authors of the paper published in Nature attribute this trend in part to “scientists’ and inventors’ reliance on a narrower set of existing knowledge”, since “relying on narrower slices of knowledge [does not benefit] scientific progress more generally.”
Others cite, for example, the increase in the amount of knowledge that new scientists must acquire to develop their work.[5] It is also attributed to the widespread trend of growth of large research teams to the detriment of solitary researchers and small teams. Researchers at Northwestern University and the University of Chicago, who analyzed 65 million papers spanning the past 60 years, show that smaller teams have tended to bring new ideas and more revolutionary results, while larger teams have tended to develop existing theories.[6]
There is greater agreement that the prevailing system of allocating funding for research is the determining factor in this decline. Peer review is the main mechanism responsible for the evaluation and development of scientific research and has been for decades. This is the system of allocating funding for the billions of dollars that support research around the world.
The proper functioning of this funding allocation mechanism is in the crosshairs of many scientists and organizations, which identify a multitude of defects. As Richard Smith, director of United Health Europe, stated, “peer review is a flawed process, full of easily identified defects with little evidence that it works. Nevertheless, it is likely to remain central to science and journals because there is no obvious alternative.” It is comparable to democracy: “a system full of problems but the least worst we have.”[7]
Anna Meier specifies the cause of this decline in what she defines as “grant culture”: the “endless pursuit” of research funding, which proves to be a deterrent to proposing objectives that break with the status quo since when “you have to get a grant and the competition for grants is so extensive, you propose the project that is most likely to get funded. Proposing something off the beaten path is an enormous risk in time and energy potentially wasted.”[8]
Because, indeed, it is a reality that ideas or projects that deviate from the canon are less favored, as confirmed by a well-documented paper entitled Looking Across and Looking Beyond the Frontier of Knowledge: Intellectual Distance, Novelty and Allocation of Resources in Science[9] authored by Kevin J. Boudreau et al. and published in 2016 in Management Science. In the article, the author concludes that “more novel proposals are associated with lower evaluations.” Their reasoning leads them to state that “experts extrapolating beyond the knowledge frontier to comprehend novel proposals are prone to systematic errors, misconstruing novel work. This implies that rather than receiving unbiased assessments (with zero-mean errors), novel proposals are discounted relative to their true merit, quality and potential.”
Something needs to change
The article by Park, Leahey, and Funk is having considerable impact, since they have managed to gauge, using data, the negative sense that the comparison with the last century in terms of discoveries that actually change the world is truly unfavorable. And if indeed the number of disruptive discoveries is not falling, it is because the investment effort in research is constantly being doubled.
The decline in innovative work is probably the product of many factors, only some of which we can control. Undoubtedly among them is the fact that current science is more dedicated to publishing papers and seeking funding than to doing the kind of in-depth work that leads to breakthroughs.
Since science is the engine of productivity and prosperity, this problem is shifted to the real economy of nations. In the aforementioned paper written by a research group from Stanford University and MIT,[10] the authors concluded that “just to sustain constant growth in GDP per person, the United States must double the amount of research effort every 13 years to offset the increased difficulty of finding new ideas.”
It is clear that things need to change in the way scientific research is conducted.
Biothics Observatory -Institute of Life Sciences
Catholic University of Valencia
References:
[1] Park, M., Leahey, E. & Funk, R.J. Papers and patents are becoming less disruptive over time. Nature 613, 138–144 (2023). https://doi.org/10.1038/s41586-022-05543-x
[2] Colleen Flaherty The Decline of ‘Disruptive’ Science Inside Higher Ed January 17, 2023 https://www.insidehighered.com/news/2023/01/17/study-disruptive-science-decline
[3] Johan S. G. Chua and James A. Evans Slowed canonical progress in large fields of science. PNAS 2021 Vol. 118 No. 41 e2021636118
[4] Nicholas Bloom et al. Are Ideas Getting Harder to Find? American Economic Review 2020, 110(4): 1104–1144 https://doi.org/10.1257/aer.20180338
[5] Derek Tompson, The Consolidation-Disruption Index Is Alarming, The Atlantic JANUARY 11, 2023
[6] Wu, L., Wang, D. & Evans, J.A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019).
[7] Smith R. Peer review: a flawed process at the heart of science and journals. J R Soc Med. 2006 Apr;99(4):178-82. doi: 10.1177/014107680609900414. PMID: 16574968; PMCID: PMC1420798
[8] Colleen Flaherty The Decline of ‘Disruptive’ Science. Inside Higher Ed. January 17, 2023
[9] Boudreau KJ, Guinan EC, Lakhani KR, Riedl C. Looking Across and Looking Beyond the Knowledge Frontier: Intellectual Distance, Novelty, and Resource Allocation in Science. Manage Sci. 2016 Oct;62(10):2765-2783. doi: 10.1287/mnsc.2015.2285. Epub 2016 Jan 8. PMID: 27746512; PMCID: PMC5062254.
[10] Nicholas Bloom et al. Are Ideas Getting Harder to Find? American Economic Review 2020, 110(4): 1104–1144 https://doi.org/10.1257/aer.20180338
Leave A Comment