Yves here. KLG below describes how science publications have become much less reliable over time, with the proliferation of \u201copen access\u201d journals part of the problem. As he said via e-mail:<\/p>\n
As science has hypertrophied, the bad is supplanted the good as spurious quantification of scientific research has come to rule the practice of science. This has in many ways ruined the practice, the art, and the craft of scientific research. Yes, I am one of those.<\/p>\n
This has been facilitated by the rise of open-access, mostly virtual publishing, which was not originally considered a threat. On the contrary, open-access online publishing was expected to be a positive good. And it often is. But it often is not. Predatory publishers were not long in appearing as online publishing became accepted. Neo-predatory, if not outright predatory, \u201cjournals\u201d are now the repositories of an ever growing mountain, and a form of Gresham\u2019s Law \u2013 the bad drives out the good \u2013 has taken over much of the scientific literature, which has become \u201cpay-to-play.\u201d<\/p>\n
And the finish is something my colleagues and I have only recently determined. Pay-to-publish has extended its tentacles into medical education, probably with equally deleterious effects.<\/p>\n
By KLG, who has held research and academic positions in three US medical schools since 1995 and is currently Professor of Biochemistry and Associate Dean. He has performed and directed research on protein structure, function, and evolution; cell adhesion and motility; the mechanism of viral fusion proteins; and assembly of the vertebrate heart. He has served on national review panels of both public and private funding agencies, and his research and that of his students has been funded by the American Heart Association, American Cancer Society, and National Institutes of Health<\/p>\n
The continuing COVID-19 syndemic has done nothing if not shaken the foundations of biomedical science, from the basic biology of viral infection and the immune response to the redefinition of what a vaccine should do.\u00a0 Facts that have been well known for more than fifty years have been ignored.\u00a0 For example, lasting immunity to coronaviruses such as SARS-CoV-2 is elusive, whether through previous exposure or vaccination.\u00a0 Nevertheless, Biomedicine went all-in on vaccines that made a lot of money for Big Pharma.\u00a0 Yes, these are said to have prevented millions of deaths, which may be true.\u00a0 But that is not the common understanding or expectation of how vaccines work, and legitimate concerns about the Covid vaccines were largely ignored.<\/p>\n
Four years into the current pandemic there are few medical interventions that work well in and of themselves, but clinical management of disease has undoubtedly improved.\u00a0 Given that airborne transmission of SARS-CoV-2 is an established fact now, despite considerable institutional dithering, we know how to prevent the virus from spreading. \u00a0Effective ventilation\/air exchange and the use of functional masks in crowded indoor spaces should be universal.\u00a0 The looming questions now are (1) what of long-Covid for which the only good intervention is prevention of infection in the first place, (2) how will repeated infections with emerging variants of SARS-CoV-2 affect the long-term health of afflicted patients, and (3) when, not if, will a more virulent variant emerge?\u00a0 COVID-19 is not the common cold.\u00a0 We knew in less than four years into the HIV\/AIDS epidemic nearly forty years ago how to prevent the spread of HIV.\u00a0 This made all the difference even if there was some resistance to recommendations, from testing the blood supply to nonpharmaceutical interventions that prevented transmission of HIV.\u00a0 Perhaps in the coming months COVID-19 research will begin to provide the needed answers.<\/p>\n
My thesis here is the foundations of my profession were already weakened beyond resistance to a global crisis when COVID-19 emerged in late 2019.\u00a0 It has been plain for years that the scientific literature is in distress, from physics to biochemistry to cell biology.\u00a0 The latter are weekly stops for me, and I often do not know what to believe.\u00a0 When I began my career as a scientific worker\/scientist, this was not a problem.\u00a0 The literature that I read then was reliable.\u00a0 But that literature also had distinct levels.\u00a0 At the basic but rarely inconsequential end of the spectrum were the \u201carchival\u201d journals, as they were recognized at the time.\u00a0 This is where most science is published, and without this foundation very few of the \u201cbreakthroughs\u201d published at the so-called \u201chigh end\u201d would be possible.<\/p>\n
My career has been contemporaneous with the development of tools to analyze the scientific literature quantitatively.\u00a0 This began with the scientometric [1] work of Eugene Garfield that culminated in the Science Citation Index and the Journal Impact Factor.\u00a0 Without going into detail, the former allows one to track the citations of any given paper in the literature (also available through PubMed).\u00a0 As a crude measure of the importance of a scientist\u2019s work, the citation index has marginal utility.\u00a0 As a practical tool for the working scientist, the Web of Science is useful in keeping track of where her or his work is read and by whom and how it is used as a foundation for further research.\u00a0 The Journal Impact Factor is not nothing, but it is primarily a marketing tool that can be manipulated by publishers to inflate the importance of its products.\u00a0 For example, the high-end (and archival) journal of my primary field of research added \u201cminireviews\u201d to the front matter of each issue about thirty years ago,\u00a0 Citations went up markedly, because these reviews were cited frequently in the introductions to papers that required a current summary of a given field.\u00a0 Good business, yes, but not an original contribution to new knowledge.\u00a0 Still, it is widely but mistakenly believed the journal can be the judge of the science.<\/p>\n
In my view the quantitative analysis of the scientific literature has been most effective at providing a mechanism by which the practice of science can be gamed by those willing to confuse \u201cscientific research activities\u201d with scientific research (see below).\u00a0 This has been covered in the literature read by scientists, but the rot at the core is beginning to show up in more visible places, such as an article in The Wall Street Journal last week: Flood of Fake Science Forces Multiple Journal Closures.\u00a0 The backstory is long and a consequence of spurious quantification of scientific \u201coutput.\u201d \u00a0This has led to corruption of the scientific literature.\u00a0 What is new is that legacy publishers such as Wiley are now being affected.<\/p>\n
In the case of John Wiley & Sons (established in New York in 1807), the publisher has closed 19 journals under its banner that \u201cwere infected by large-scale research fraud\u201d and in the past two years \u201chas retracted more than 11,300 papers that appeared compromised.\u201d\u00a0 As noted in the WSJ, these papers are the product of \u201cpaper mills\u201d that specialize in the production of fake science that \u201cfor a price, will list a scientist as an author of a wholly or partially fabricated paper.\u00a0 The mill then submits the work, generally avoiding the most prestigious journals in favor of publications such as one-off special editions that might not undergo as thorough a review and where they have a better chance of getting bogus work published.\u201d\u00a0 Indeed, and paper mills are adept at evolving to elude detection.\u00a0 But Wiley is a legacy scientific publisher that could be and should be viewed as a bulwark against sketchy authors who publish the results of sketchy \u201cresearch activities.\u201d\u00a0 On the other hand, maybe not.\u00a0 Academic publishing is a $30 billion dollar a year \u201centerprise.\u201d\u00a0 There is a lot of money to be made by someone. [2]<\/p>\n
Although outright fraud has been and still remains a vanishingly small fraction of the scientific literature, its persistence, including the gaming of the literature, erodes trust in science while dissolving a foundation that has been built since Philosophical Transactions of the Royal Society was first published in 1665.\u00a0 My question here is whether science is being harmed by a version of Gresham\u2019s Law \u2013 Bad Money Drives out the Good: \u201cWhenever coins containing precious metals have been used along with base metal coins of the same denomination, both legally accepted as tender, the bad coins have driven the good coins out of circulation.\u201d\u00a0 A more general description of Gresham\u2019s Law comes from Mackenzie King, longtime Prime Minister of Canada in the first half of the twentieth century:<\/p>\n
Something analogous to Gresham\u2019s Law will be found to obtain in the case of competing standards in Industry (or Science).\u00a0 Assuming there is indifference in the matter of choice between competing commodities or services, but that in the case of such commodities or services the labor standards involved vary, the inferior standard, if brought in this manner into competition with a higher standard, will drive it out, or drag the higher down to its level.\u00a0 This is effected by the opportunity of under-selling which comes, where in such cases human well-being is sacrificed to material ends. The superior standard, not being recognized or demanded, is unable to hold its own, and in time disappears. This Law is just as real and relentless in its operation in Industry (Science) as Gresham\u2019s Law of the precious metals is with respect to money and the mechanism of exchange. \u00a0Indeed, a more accurate exposition would describe both as manifestations of one and the same law, which I propose to call the Law of Competing Standards. \u00a0I see no reason why economists (scientists) should not recognize the existence of such a law, and incorporate it immediately in (their practice) as being quite as significant as the Law of Supply and Demand, the Law of Diminishing Returns, or any other Law accorded a place in its nomenclature.<\/p>\n
Thus, my answer to my own question is an unqualified \u201cyes,\u201d largely because I have seen it with my own eyes during the past 25-30 years.\u00a0 Bad publishing may not be driving out the good, but it certainly overwhelms the good in volume.\u00a0 Not coincidentally, this period has seen the rise of open-access, mostly virtual scientific publishing.<\/p>\n
There is absolutely nothing wrong with open-access publishing.\u00a0 A strong case has been made that all scientific research published with public support should be \u201copen to access by everyone.\u201d Much is available, even from for-profit legacy publishers, immediately upon publication or after an embargo of six months to a year.\u00a0 But papers published 50-80-100 years ago by many journals (Science and Nature included) require a subscription or access through a library with a subscription or a one-time payment, with $32 seeming to be a popular price.\u00a0 Absurd, although my subscription to Nature journals available online at $30 per month is a bargain for me.<\/p>\n
Along with virtual publication online came predatory publishers not very long after.\u00a0 This is not the place to discuss predatory publishers at length.\u00a0 Some of them may have cleaned up their act, but any scientist of a certain age who pays attention (apparently a distressingly low number) recognizes them on sight, even if the pdf downloaded from one of these journals is identical on the surface to a paper from the Journal of the American Chemistry Society (1879) or Proceedings of the National Academy of Sciences (1915). [3] \u00a0An archived link to Beall\u2019s List (a good story from an observant librarian that is not covered fairly in this Wikipedia entry) is here.\u00a0 Scroll to the bottom for information on a very prominent provider of open-access, online journals whose products appear often in current PubMed searches, one of which claims to be based in Basel, which is the home of Big Pharma.<\/p>\n
So, why is this a problem?\u00a0 Bad science is just that and ultimately inconsequential, while good science will lead to scientific progress, however this is defined, for example in previous posts here and here.\u00a0 The answer comes back to the specious quantification of scientific output by individual scientists and their research groups.\u00a0 Although this may be changing slowly at the margin, the only thing promotion committees, institutional administrators, and sometimes peer review panels at funding agencies seem to be able to do is count: The number of papers published, and the dollars included in a grant award (especially the indirect costs \u2013 \u201coverhead\u201d that goes to the institution for its services).\u00a0 The quality of the work is seldom considered.\u00a0 Although this may be changing, too, quality requires judgment and imagination informed by long and deep experience.\u00a0 Neither of these attributes is easily quantified.<\/p>\n
In any case, what I describe here as Gresham\u2019s Law as Applied to Science continues to have malign effects.\u00a0 It is clear that much of published science is now done \u201cquick and dirty\u201d with the payoff having little to do with science as we should expect.\u00a0 For example, I repeat myself, but as of 20 May 2024 there are 422,367 results returned in a search of PubMed using \u201cCovid\u201d (case-insensitive) as the query in the 4.5 years since late-2019 (~8,000 per month, ~270 per day, which is absurd).\u00a0 \u201cHIV AIDS\u201d (case insensitive) returns 180,613 results in the 42 years since 1982 (~360 per month).\u00a0 The likely explanation, and the only plausible explanation to me, is that the scientific literature has become so large as to include much that shouldn\u2019t exist.\u00a0 Open access, which is an absolute good in principle, and publication online, another near-absolute good in principle, have facilitated the dilution and subsequent diminution of value in scientific research and publishing.\u00a0 But the numbers are way up, and they can be counted.<\/p>\n
The resultant imperatives felt by scientists, including yours truly, are strong but not irresistible.\u00a0 The bad has driven out the good, and this has not been limited to newer, online publications.\u00a0 Many of the early papers in legacy journals (e.g., NEJM) on COVID-19 and vaccines were rushed.\u00a0 In the emergency, that might have been expected but the result has been less than optimum.\u00a0 Publication during the early days of HIV-AIDS was also rapid, with intense rivalries obvious even to the most casual observer.\u00a0 But progress was made with dispatch, especially in identification of HIV as the agent of AIDS (using technology that for COVID took hours to days instead of months to years with AIDS) and the recommendation of nonpharmaceutical interventions that prevented transmission of HIV.\u00a0 Different virus, different mode of transmission, different pathology, although immune system dysregulation is common to both AIDS and Covid.\u00a0 Even so, good adherence to long established precepts of epidemiology and public health were the order of the day in the 1980s.<\/p>\n
I realize that I could be taken as an \u201cman yelling at clouds,\u201d but I recently came across another excrescence of the qualitative decline of scientific publication and by extension biomedical science that is having a deleterious effect on medical education.\u00a0 Beginning in 2022 the first board examination taken by medical students (USMLE Step 1), usually at the end of their second year of medical school, was changed from being a scored exam to pass\/fail.\u00a0 The reasons for this have never been explained very well to my knowledge, but it will be an interesting story eventually.\u00a0 Before this transition, a student\u2019s Step 1 score was used as an objective, if imperfect, measure of how well a medical student had mastered the material required as the scientific foundation for the practice of medicine (anatomy, biochemistry, systems physiology, pharmacology, etc.). \u00a0Step 1 was an achievement test, with a very strong correlation between performance in the preclinical curriculum and the Step 1 score.\u00a0 Naturally, a high score on Step 1 became a prerequisite for a student to match into the most competitive, largely because of a dearth of positions, residency programs.\u00a0 Examples include Neurosurgery, ENT, Dermatology, Urology, and Orthopedic Surgery.<\/p>\n
This ill-considered change caused an apparent crisis for medical students as they realized that another mechanism would now be required to distinguish themselves from their peers.\u00a0 The answer was, naturally, \u201cresearch.\u201d \u00a0A recent article, Medical Students lose in the research arms race for residency slots, by Anmol Shrestha explains:<\/p>\n
Before (medical school), I had known publish-or-perish as something for those seeking tenure at universities.\u00a0 I believed that medical school was a place where I would be learning about the human body and learning bedside manner through classes like medical Spanish\u2026But as my first year went on and I saw more and more classmates skip patient-facing classes to spend time conducting research, I realized how much the publish-or-perish environment also applied to medical students like me who are hoping to get into residency programs. [4]<\/p>\n
Yes, there are medical students who do serious research as students.\u00a0 Outside of large medical schools with extensive internal research programs funded largely by NIH, this remains rare unless the student is in a program that emphasizes research, such as an MD-PhD Program that includes three years of research between the two preclinical and the two clinical years.\u00a0 It is coming to light that an extensive apparatus has been assembled to fill this perceived need.\u00a0 As several colleagues and I have learned recently, it attracts students even before they begin medical school.\u00a0 The article at the link notes \u201cthe average number of publications medical students have under their belts by the end of medical school has skyrocketed\u2026and why applicants for certain specialties, like neurosurgery, average up to 29 research activities \u2013 a number higher than many PhD candidates publish by the end of their programs.\u201d<\/p>\n
This requires major unpacking.\u00a0 The first thing to notice is that \u201cpublications\u201d are conflated with \u201cresearch activities.\u201d \u00a0In this case, as my colleagues and I have discovered, these research activities are no more than lines on a curriculum vitae in the vast majority of cases.\u00a0 Publications, such as they exist, are generally case studies, which might be useful, or irrelevant \u201creviews\u201d put together, not written, by a group of medical students from disparate medical schools in the Americas, Europe, and Asia.\u00a0 They do not seem to be the products of paper mills as defined above, but they are artificial assemblies facilitated by what can only be called \u201cringleaders.\u201d\u00a0 Few of the publications have been cited or are likely to be cited more than less than a handful of times in the future.\u00a0 Yet they exist and can be counted in PubMed, which is their only reason for being.<\/p>\n
As for 29 research activities being more than most PhD students will list as graduate students, yes, that is laughably true.\u00a0 An outstanding graduate student in an outstanding institution who is blessed with a project equal to her abilities will publish 3-5 papers and present her work at another 5-6 scientific meetings as a student (4-5 years in the US).\u00a0 Most will be credited with less, including that outstanding student with a difficult project.\u00a0 \u201cResearch activities\u201d have little meaning in graduate school but not medical school.\u00a0 Gresham\u2019s Law has reached into medical education as an unintended but perfectly predictable consequence of the brave new pass\/fail world.\u00a0 In any case, a Residency Program Director who believes a precocious medical student has legitimately published, as a student, between 10 and 25 entries indexed in PubMed deserves what is coming.\u00a0 It is not clear that the resident\u2019s future patients deserve it, though, whatever the count is.<\/p>\n
This is all still playing out, but many medical students\/graduates have some explaining to do, since some have participated in these \u201cresearch activities\u201d as free agents without the knowledge of their institution.\u00a0 This is unacceptable, and I confess to never imagining this could be a thing.\u00a0 How has this happened?\u00a0 The current climate of rapid, open-access, online publication with cursory peer review opened the gates for those who can pay the submission fees, which are seldom waived as far as I can tell.\u00a0 I anticipate this will be a big mess, to use a not-so-technical term, the resolution of which may be ugly.<\/p>\n
Finally, my recommendation to anyone, scientist and nonscientist alike, who reads the scientific literature is caveat emptor.\u00a0 When a group of collaborators seems too disparate to have actually worked together, this is probably true and an alarm should ring.\u00a0 And if even one author uses an email address other than an institutional email, stop reading.\u00a0 That goes for all scientific papers, by the way, unless the author(s) explicitly state that their work is their own, alone, and unaffiliated with their home institution (a very rare event).<\/p>\n
In the meantime, it will be necessary to go back to the future and stop counting and start reading and thinking, with care and attention.\u00a0 Gresham\u2019s Law should not apply to science or medical education.\u00a0 Nor should it be valid in an economics true to the origin of the term as \u201cthe art of managing the household,\u201d which is covered here at Naked Capitalism every day.<\/p>\n
Notes<\/p>\n
[1] There is a journal, which I have looked at on occasion.\u00a0 Scientometrics can be useful but also defines \u201cmeta.\u201d<\/p>\n
[2] As a beginning faculty member, I was apparently too obtuse for a senior colleague.\u00a0 He asked me if I would be willing to pay a $100 submission fee to a leading journal of which he was an Associate Editor in our common field.\u00a0 I answered \u201cyes\u201d but then asked if the page charges and surcharges for color figures\/diagrams would still be charged.\u00a0 He answered \u201cyes.\u201d\u00a0 When I began submission to all journals was completely analog and the manuscript had to be typeset and figures converted from actual photographs for publication.\u00a0 This was expensive.\u00a0 At the time of our conversation, submission requirements were very strict, and all text and figures digital-ready at the proper resolution.\u00a0 The page charges that amounted to several thousand dollars were still collected even though the new technology did not require any special handling of the manuscript.\u00a0 Yes, someone was making a lot of money.\u00a0 We submitted our work to an equivalent journal published by the American Chemical Society.\u00a0 For free, with no charge for color if the editor agreed that it was necessary.\u00a0 In retrospect, perhaps wasting the $2,000 (17 years ago) unnecessarily might have been a better career move.<\/p>\n
[3] Yes, I am aware that all journals and institutions are vulnerable to fraud, and that scientists are only people of every sort.\u00a0 Nevertheless, while the wheels may grind slowly, a scientist caught in a fraud eventually loses all credibility, as in the case of Andrew Wakefield.<\/p>\n
[4] Note that to practice medicine in the United States as a licensed physician, a residency must be completed.\u00a0 These range from three years, including the \u201cintern\u201d year, in Internal Medicine and Family Medicine to five years in Urology and General Surgery and eight years in Neurosurgery.<\/p>\n
![\"Print](\"https:\/\/cdn.printfriendly.com\/buttons\/print-button-gray.png\")
<\/div>\n<\/div>\n