《科学幻想》评论：滚蛋吧，科学骗子

A review of Science Fictions: How Fraud, Bias, Negligence, and Hype Undermine the Search for Truth by Stuart Ritchie, Bodley Head, 353 pages (July, 2020).
对《科学虚构》的评论：斯图尔特-里奇（Stuart Ritchie）著，Bodley Head出版社出版，353页（2020年7月）。

As I sat down to review Stuart Ritchie’s new book, Science Fictions, I was interrupted immediately by mournful texts from a young man who was being hosed for his write-up of the results from a study. He’d asked me to take a look at it. A charity wanted to improve literacy in poor children. Children’s literacy had been measured before and after a “treatment” or intervention. There was no “control group” in the design. No similar sample of children who trundled along without the intervention, nor an intervention designed to match the treatment in all but the supposed crucial component. Had literacy increased at the second assessment because of the treatment or because the children were a year older? Your guess is as good as mine. The young man fed this problem back to his superiors and was called, peremptorily, to an online meeting. The charity had wanted a glowing report and were unhappy they didn’t get it. The young man said he felt like his bones were filling up with lead. I’ll send him a copy of Stuart’s book. I hope his bosses at the charity will read it too. After all, they are spending (wasting) tens of thousands of dollars on a bad study.
当我坐下来评论斯图尔特-里奇（Stuart Ritchie）的新书《科学幻想》（Science Fictions）时，立即被一个年轻人发来的哀伤短信打断了。他让我帮他看看。一家慈善机构希望提高贫困儿童的识字率。在 "治疗 "或干预前后，对儿童的识字率进行了测量。设计中没有 "对照组"。没有类似的儿童样本，这些儿童在没有干预措施的情况下蹒跚学步，也没有设计出与干预措施相匹配的干预措施，只有所谓的关键部分。在第二次评估中，识字率提高是因为治疗还是因为孩子们大了一岁？你的猜测和我的一样。这位年轻人把这个问题反馈给了他的上司，并被强制性地叫去参加一个在线会议。慈善机构想要一份光彩夺目的报告，但没有得到，他们很不高兴。这位年轻人说，他感觉自己的骨头里都是铅。我会给他寄一本斯图尔特的书。我希望他在慈善机构的上司也能读读这本书。毕竟，他们在一项糟糕的研究上花费（浪费）了数万美元。

Science Fictions bites down hard on four key problems that beset the institution of science. It begins with the spooky story of a paper, which appeared in a top-flight psychology journal, showing that the fundamental laws of physics had been broken by undergraduates; they had reversed time.
《科学虚构》对困扰科学机构的四个关键问题进行了深入探讨。它首先讲述了一篇发表在顶级心理学杂志上的论文的神奇故事，这篇论文显示，物理学的基本定律被大学生们打破了；他们逆转了时间。

In one experiment, in this now infamous paper, students were shown words on a screen, one at a time. Then they were asked to type as many as they could recall. Next 20 randomly chosen words were shown to the students from the original list. The surprising finding was that the students were more likely to remember the 20 words they were about to see for a second time, even though they had only psychic intuition to guide them. Astonishing! And, yes, it was parapsychological nonsense, but the story warms us up for Ritchie’s key themes.
在这篇臭名昭著的论文中，有一个实验是让学生在屏幕上一次显示一个单词。然后要求他们键入尽可能多的单词。接着，从原来的单词表中随机抽取 20 个单词给学生看。令人惊讶的发现是，学生们更有可能记住他们即将第二次看到的 20 个单词，尽管他们只有心理直觉来引导自己。令人吃惊！是的，这只是超心理学的无稽之谈，但这个故事让我们对里奇的关键主题有了更深的了解。

Chapters begin with tales of various kinds of malfeasance, followed by cogent analyses of what went wrong and why. Sometimes the fault lies in an individual rogue player; after all villains exist in art, commerce, finance, and writing, so why shouldn’t some scoundrels haunt the halls of science and medicine? Sometimes it’s a matter of over-egging, or a hapless mistake. When the research question concerns the placing of Whalleyanidae in the Lepidoptera Tree of Life, mistakes may be forgiven and the dear Whalleyanidae (a genus of moth endemic to Madagascar) will find its correct taxonomic home eventually. But when the research question concerns medicine or a surgical protocol (as in the shameful tale of Dr Paolo Macchiarini, with his fraudulent plastic tracheas), bad practice leads to lost lives.
各章以各种渎职行为的故事开头，然后是对出错原因的有力分析。毕竟艺术、商业、金融和写作领域都有恶棍，为什么科学和医学领域就没有恶棍呢？有时候，这只是一个揠苗助长的问题，或者是一个无奈的错误。当研究问题涉及到如何将Whalleyanidae列入鳞翅目生命树时，错误可能会被原谅，亲爱的Whalleyanidae（马达加斯加特有的蛾属）最终会找到它正确的分类归宿。但是，当研究问题涉及医学或外科手术方案时（如保罗-马基亚里尼博士（Dr Paolo Macchiarini）伪造塑料气管的可耻故事），错误的做法就会导致生命的丧失。

Given the constellation of demanding behaviours that get you to the start line as a professional scientist (such as protracted, detailed work which is regularly scrutinised by others), one might expect that fraud-inclined people would find the going easier elsewhere. Yet microbiologist Elisabeth Bik, who conducted a painstaking search through papers in 40 biology journals, found evidence of cheating in many images of Western blots (those blurry blobs in columns produced by proteins in gel). She found many published images that had been manipulated in something like Photoshop. When you look at some of her examples, the duplication seems obvious. Why wasn’t it picked up by reviewers? It was probably partly due to our biases: We expect honesty. Intentional data distortion is such a weird thing to do, it simply wouldn’t occur to most reviewers to check for it. Fraud harms trust in science; how common is it? The largest study to date, using data from seven pooled self-report surveys found that 1.97 percent of scientists admitted to faking their data at least once. Low prevalence, but reason for greater vigilance, better training, and maintenance of scepticism—especially when the laws of physics are overturned by undergraduates.
作为一名专业科学家，要想站在起跑线上，必须具备一系列高要求的行为（例如，长期、细致的工作，并定期接受他人的审查），因此，人们可能会认为，有欺诈倾向的人在其他地方会更容易上当受骗。然而，微生物学家伊丽莎白-比克（Elisabeth Bik）对 40 种生物学期刊的论文进行了艰苦的搜索，在许多 Western 印迹（凝胶中蛋白质在柱子上形成的模糊斑点）图像中发现了作弊的证据。她发现许多发表的图片都经过了类似 Photoshop 的处理。当你看到她的一些例子时，重复似乎是显而易见的。为什么审稿人没有发现呢？部分原因可能是我们的偏见：我们期待诚实。故意篡改数据是一件很奇怪的事情，大多数审稿人根本不会想到去检查它。造假损害了人们对科学的信任；这种现象有多普遍？迄今为止最大规模的研究使用了七项自我报告调查的数据，发现1.97%的科学家承认至少伪造过一次数据。虽然发生率不高，但我们有理由提高警惕、加强培训并保持怀疑态度--尤其是当物理定律被大学生推翻时。

Science Fictions contains elaborations of p-values, p-hacking, statistical significance, and the importance of including null results. It is written in such everyday language it could lead any Ariadne out of a dark jargon maze and into the daylight. The section in the “Bias” chapter explaining why we’d expect to see a funnel-shaped set of data points in a meta-analysis, and what to worry about when we don’t, is handled with great clarity. The appendix on “How to read a scientific paper” is a superb crib sheet for students, journalists, or anyone who wants to paddle around Google Scholar in their own canoe. References are expanded, informative, and worth their own read; there’s a lovely quote from Ronald Fisher: “[T]o consult the statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. He can perhaps say what the experiment died of.” Short headings at the top of each page in the Notes section would make it easier to navigate between main text and references, but if you own the book you can fix that with a pencil.
科学幻想》详细阐述了P值、P黑客、统计意义以及包含无效结果的重要性。这本书用非常日常化的语言写成，可以带领任何阿里阿德涅走出黑暗的术语迷宫，重见天日。在 "偏倚 "一章中，有一节解释了为什么我们希望在荟萃分析中看到一组漏斗状的数据点，以及当我们没有看到这些数据点时应该担心什么。关于 "如何阅读科学论文 "的附录为学生、记者或任何想在谷歌学术上一展身手的人提供了极好的参考资料。参考文献内容丰富，信息量大，值得一读；罗纳德-费舍尔（Ronald Fisher）有一段话说得很好：罗纳德-费舍尔（Ronald Fisher）有一段话说得很好："在实验结束后向统计学家请教，往往只是要求他进行尸检。他也许能说出实验死于什么"。注释部分每页顶部的简短标题会使正文和参考文献之间的导航更加方便，但如果你拥有这本书，你可以用铅笔解决这个问题。

I have some criticisms. I found a hint of hype in this anti-hype book. The journal that published the parapsychological paper is shamed for rejecting Ritchie’s own study (which contradicted the paranormal findings), but his study included many fewer subjects (50 in each of three experiments) whereas the original offending article looked stronger with 1,000 subjects and nine experiments. A study with 3,289 subjects showing the laws of physics are intact (phew!) was published by Galak et al only a year after the super-sensory paper, and in the offending journal. A year is quick for science publishing.
我有一些批评意见。在这本反炒作的书中，我发现了一丝炒作的痕迹。发表超心理学论文的期刊因拒绝里奇自己的研究（该研究与超自然现象的发现相矛盾）而感到羞愧，但他的研究包括了更少的受试者（三个实验各50名受试者），而最初的违规文章有1000名受试者和9个实验，看起来更有说服力。在超感觉论文发表一年后，加拉克等人在违规期刊上发表了一项有 3289 名受试者参与的研究，显示物理定律完好无损（呼！）。对于科学出版来说，一年时间太快了。

The “replication crisis” could just as well be called the Replication Revolution. Stale old blunders are being called out, and the record corrected. Figuring out what went wrong and how to do better is a sign of health and vigour in science practice.
复制危机 "也可以称为 "复制革命"。陈旧的错误正在被揭露，记录正在被纠正。找出问题所在以及如何做得更好，是科学实践健康和充满活力的标志。

This is a golden age for learning how to do science well. It has never before been easier to obtain free help from qualified experts. Eager, informed, scientists on Twitter help strangers with readings, methods, and statistical approaches. The open science movement, the reproducibility project, and registered report repositories all foster greater transparency and better research.
这是一个学习如何做好科学的黄金时代。从合格专家那里获得免费帮助从未像现在这样容易。推特上热心、消息灵通的科学家们帮助陌生人解决阅读、方法和统计方法方面的问题。开放科学运动、可重现性项目和注册报告库都促进了更高的透明度和更好的研究。

In the chapter on “Negligence,” the hunt for candidate genes (that may influence any trait: height, extraversion, chocolate preference) is presented as a sorry tale of failed science. Is that fair? Seems like business as usual: formerly we knew less, now we know more. Most people who worked on candidate genes jumped on new techniques, with their greater statistical power, as soon as they nosed out of the gate. Genetics is a fast-moving field; it’s a sign of success, not failure, that we make horse-eyes at work done 10 years ago. Isn’t this how science has always worked historically? It wasn’t a failure that physicist Albert Michelson spent years fiddling with an interferometer designed to demonstrate the existence of the luminiferous aether, it was a win when he concluded there was no aether. He went on to measure the speed of light with spectacular accuracy; Nobel Prize deserved and given.
在 "疏忽 "一章中，寻找候选基因（可能影响任何性状：身高、外向性、巧克力偏好）的过程被描述为一个失败的科学的遗憾故事。这公平吗？似乎一切照旧：以前我们知道的更少，现在我们知道的更多。大多数研究候选基因的人在新技术一问世时，就立即采用了统计能力更强的新技术。遗传学是一个飞速发展的领域；我们对 10 年前的工作瞠目结舌，这是成功的标志，而不是失败。历史上的科学不都是这样发展的吗？物理学家阿尔伯特-迈克尔逊（Albert Michelson）花了数年时间摆弄一个旨在证明发光乙太存在的干涉仪，这并不是失败，当他得出不存在乙太的结论时，这就是胜利。他继续以惊人的精确度测量了光速；诺贝尔奖实至名归。

Science vs. Purpose: The College Board’s New Adversity Index
科学与目的：大学委员会的新逆境指数

Claims over what is being measured by a standardized assessment (referred to as a test’s construct) are where claims of scientific rigor can distort understanding of social goals associated with a particular test.
对标准化评估所测量的内容（被称为测试的结构）的质疑，就是对科学严谨性的质疑，可能会扭曲对与特定测试相关的社会目标的理解。

QuilletteJonathan Haber 乔纳森-哈伯

Have the most important crimes against science been identified in Science Fictions? I can’t speak to the whole of science, but I can see two muggings being played out on the scientific stage. They were known to ancient rhetoricians as suggestio falsi and suppressio veri. Let’s talk first about suppressing the truth.
《科学幻想》中是否已经确定了最重要的反科学罪行？我不能对整个科学界说三道四，但我可以看到科学舞台上正在上演两场抢劫。古代修辞学家将其称为 "暗示"（suppestionio falsi）和 "压制"（suppressionio veri）。让我们先谈谈压制真相。

Science can expose uncomfortable facts. Ritchie raises concerns about political bias in science, but too softly. Science needs ruthless defenders. No liberal or conservative bias will do.
科学可以揭露令人不安的事实。里奇提出了对科学中政治偏见的担忧，但过于轻描淡写。科学需要无情的捍卫者。自由派或保守派的偏见都不行。

Science should generate new knowledge about the world. Those who are institutionally involved in knowledge-generation (universities, scientists, and others) should be disinterested in research questions and findings. It takes guts. Every scholar knows, especially after recent publicised sackings and so on, that it would be tough to get a grant, or a paper in a top journal, for a study showing: that teams comprising men from elite academic institutions function well; that women are better at looking after babies than men; that fracking is fine; that intelligence is mostly genetic (insert your own worst nightmare top-line). No caveats like “on average”; I’m illustrating a point here—those findings would press our buttons (they press mine, too). But as scientists, our social and political perspectives belong at the coat check. Academia, scientific institutions, and much of the press strongly favour some answers and detest others. This hurts our capacity for generating knowledge about the world. The world does not have a social conscience. We do. Institutions in science and education should be clear about the distinction. Careers have been ruined because institutions have failed to grasp it. I’m looking at you, Cambridge.
科学应该产生关于世界的新知识。那些参与知识创造的机构（大学、科学家和其他人）应该对研究问题和研究成果不感兴趣。这需要勇气。每一位学者都知道，尤其是在最近公开的解雇事件之后，如果一项研究表明：由精英学术机构男性组成的团队运作良好；女性比男性更善于照顾婴儿；水力压裂没有问题；智力主要是遗传的（请插入你自己最糟糕的噩梦的顶线），要想获得资助或在顶级期刊上发表论文是非常困难的。没有 "平均 "之类的说明；我在此说明一个问题--这些发现会让我们很兴奋（我也很兴奋）。但作为科学家，我们的社会和政治视角属于 "衣帽间"。学术界、科研机构和大部分媒体都强烈倾向于某些答案，而厌恶另一些答案。这损害了我们创造世界知识的能力。世界没有社会良知。我们有。科学和教育机构应该明确两者之间的区别。由于机构未能把握这一点，我们的职业生涯已经毁于一旦。我在看着你，剑桥。

There’s another kind of suppression. When a truth has become generally known, is it ethical for funding agencies, universities, researchers to ignore it? In my view, this flirts with fraud. We have known for decades that people who are more closely genetically-related are more similar to each other. Human behaviour is heritable; all of it. This is possibly the most reproduced datum in the whole of the human behavioural sciences. So why do large-scale, longitudinal, social science studies that are not genetically-informative still get funded?
还有另一种压制。当一个真相已广为人知时，资助机构、大学和研究人员对此视而不见，这符合道德规范吗？在我看来，这近乎欺诈。我们几十年前就知道，基因关系更密切的人彼此更相似。人类的行为是可以遗传的，所有的行为都是如此。这可能是整个人类行为科学中被重复最多的数据。那么，为什么那些与遗传信息无关的大规模纵向社会科学研究仍能获得资助呢？

The answer is that many social scientists and their funders don’t like genes. Even a top UK university whose motto is “rerum cognoscere causas” (to know the causes of things) appears to conduct little, if any, empirical research in criminology, economics, gender studies, health policy, or psychology that incorporates genetically-informative studies. Whether genetics are relevant or not depends on what problem you want to solve, but if your research question includes causes and human behaviour, a genetic component ought to be essential. So why don’t genes more often have a place at the table? And why is this not a causum for concernum to boards and trustees?
答案是，许多社会科学家及其资助者都不喜欢基因。即使是以 "erum cognoscere causas"（了解事物的原因）为座右铭的英国顶尖大学，在犯罪学、经济学、性别研究、卫生政策或心理学等领域开展的实证研究中，似乎也很少有基因信息研究。基因是否相关取决于你想解决什么问题，但如果你的研究问题包括原因和人类行为，那么基因成分应该是必不可少的。那么，为什么基因不经常占据一席之地呢？为什么这不是董事会和受托人关心的问题？

What about suggestio falsi—not a pasta, but a statement of untruth? What do we do when ideas that once seemed useful (think of phlogiston) have expired? Social science may need something analogous to the Cochrane Reviews which synthesize medical evidence. Power Posing, the Implicit Association Test, the Myers-Briggs test, Stereotype Threat, and most priming studies would vanish like stains in sunlight.
suggestio falsi--不是意大利面，而是不真实的陈述，又是怎么回事？当那些曾经似乎有用的想法（想想phlogiston）已经过时时，我们该怎么办？社会科学可能需要一些类似于科克伦评论（Cochrane Reviews）的东西来综合医学证据。强力装腔作势、内隐联想测试、迈尔斯-布里格斯测试、刻板印象威胁以及大多数引导研究都会像阳光下的污渍一样消失。

The last chapter is on “Fixing Science.” It locates the problem in the wrong place. Science is fine; it’s our tendency to game it that needs an armed guard. Evolutionary anthropologist Richard McElreath recently tweeted: “Science is one of humanity’s greatest inventions. Academia, on the other hand, is not.” I’m with Richard on that.
最后一章是 "修复科学"。它把问题找错了地方。科学是没有问题的；需要武装警卫的是我们游戏科学的倾向。进化人类学家理查德-麦克埃利雷特（Richard McElreath）最近在推特上写道："科学是人类最伟大的发明之一："科学是人类最伟大的发明之一。而学术界却不是"。我赞同理查德的观点。

Ritchie correctly identifies problems with organising the outputs from science. Friends in other professions scratch their heads on hearing that all the content, and all reviewing, are supplied gratis to journals. The original author hands over copyright, the publisher retains 100 percent of the profit and the public, who subsidise the whole shamoozle via income tax, can’t read it without paying. It seems a tad unfair.
里奇正确地指出了科学成果的组织问题。其他行业的朋友一听说所有内容和审稿都是免费提供给期刊的，就挠头不已。原作者交出版权，出版商保留百分之百的利润，而通过所得税补贴整个骗局的公众，不付费就无法阅读。这似乎有点不公平。

Current remedies include open access publishing in which authors, or their funders, pay around $3,000 per article to the journal in order to make their work freely accessible to all. And pre-print servers to which scholars can upload manuscripts for anyone to read and comment on before peer review. This idea originated with physicist Paul Ginsparg, who founded the original pre-print server, the arXiv (pronounced archive), in 1991. The bioRxiv (biology archive server) followed in 2013, psyRxiv (psychology) and socRxiv (social science) in 2016, medRxiv (medicine) in 2019. Ginsparg reflected, after 20 years of the arXiv, that publishing is still in transition: “There is no consensus on the best way to implement quality control (top-down or crowd-sourced, or at what stage), how to fund it or how to integrate data and other tools needed for scientific reproducibility.”¹ Problems of how to organise the knowledge generated by science still exist, but there is now widespread agreement that well-designed, replicated, reproducible studies are essential.
目前的补救措施包括开放存取出版，即作者或其资助者向期刊支付每篇文章约 3000 美元的费用，以便让所有人都能免费获取其作品。还有预印本服务器，学者们可以在同行评审前将手稿上传给任何人阅读和评论。这个想法源于物理学家保罗-金斯帕格（Paul Ginsparg），他于1991年创建了最初的预印本服务器--arXiv（发音为档案）。随后，2013 年推出了 bioRxiv（生物学档案服务器），2016 年推出了 psyRxiv（心理学）和 socRxiv（社会科学），2019 年推出了 medRxiv（医学）。Ginsparg 认为，arXiv 推出 20 年后，出版业仍处于转型期："对于实施质量控制的最佳方式（自上而下还是众包，或者在哪个阶段）、如何为其提供资金，以及如何整合数据和科学可重复性所需的其他工具，目前还没有达成共识"。 ¹ 如何组织科学所产生的知识的问题依然存在，但现在人们普遍认为，精心设计、重复、可重现的研究是必不可少的。

Replication is not the only fruit. A second edition of this book could include the total evidence rule. We should probe our problems from different methodological approaches to see if the general findings converge. Our provisional knowledge about the world increases as we take into account what the Vienna Circle philosopher, Rudolf Carnap, described as the “total observational knowledge available to a person at the time of decision-making.”²
复制不是唯一的成果。本书的第二版可以加入总证据规则。我们应该从不同的方法论角度来探究我们的问题，看看一般的发现是否趋于一致。当我们考虑到维也纳圈哲学家鲁道夫-卡尔纳普（Rudolf Carnap）所描述的 "一个人在决策时可获得的全部观察知识 "时，我们对世界的临时知识就会增加。 ²

Science Fictions is engaging, story-led, and well-organised. It will equip my sad young friend to articulate what went wrong with his charity’s study on literacy and, as importantly, to do the next one well. If he absorbs Ritchie’s lessons, he will become a science ninja. His shuriken will chasten scammers and chisellers, he’ll lance those weaselly “trending” p-values, and forego the forking paths. Go slay the enemies of science young man.
《科学幻想》引人入胜、故事性强、条理清晰。它能让我这位伤心的年轻朋友清楚地说明他的慈善扫盲研究出了什么问题，更重要的是，它能让他做好下一次研究。如果他吸收了里奇的课程，他将成为一名科学忍者。他的短剑将诛杀骗子和凿子，他将刺穿那些狡猾的 "趋势 "P 值，并放弃岔路。去杀死科学的敌人吧，年轻人。

References: 参考资料

¹Ginsparg, P. ArXiv at 20. Nature 476. 2011. pp.145–147.
¹ Ginsparg, P. ArXiv at 20.Nature 476.2011. pp.145-147.
²Rieppel, Olivier. The philosophy of total evidence and its relevance for phylogenetic inference. Papéis Avulsos de Zoologia 45(8). 2005. pp. 77–89.
² Rieppel, Olivier.全证据哲学及其对系统发育推断的意义》。Papéis Avulsos de Zoologia 45(8).pp.