A Simple Replication Agreement Could Improve Trust in Science

2023-07-11
关注

Daphne, an eager computer-vision scientist, encounters a novel study highly relevant to her autonomous vehicles research. The study uses computational methods to reveal hidden driving patterns. Aware of autonomous driving’s safety implications, Daphne decides to act responsibly and replicate the study to verify its findings prior to relying on them.  

After carefully reading the paper’s methodology, however, Daphne realizes that the original researchers provided no access to the code and only partial access to the data sets they used in their study. Daphne emails Sasha, the corresponding author of the original study, and asks for assistance but gets no response. Daphne now faces a serious dilemma: conducting an independent study from scratch that requires tremendous resources, or relying on the existing study, although it is irreplicable.

Today, computational science, powered by complex models and vast data sets, drives research and innovation across almost every scholarly arena. Daphne’s illustrative story highlights a real challenge scientists face in replicating studies, and underscores the importance of enabling access to code, data and other information. Promoting replicability of scientific studies benefits the scientific community and fosters public trust in science.

Replication is vital to science, seen as fundamental by investigators as far back as the pioneering 17th-century chemist Robert Boyle. It ensures that published findings are valid. If other scientists can repeat the study and get the same results, we can trust the findings. If not, it's a red flag that something might be off. Science is cumulative, and replications build its comprehensive and credible body.

Unfortunately, the scientific community has struggled with a “replication crisis” in recent decades, where scientists in fields from economics to physics find it hard to reproduce results from published studies. In fact, some estimates of the severity of the crisis in specific fields, such as biology and psychology, found that a significant portion of scientific publications in these disciplines stand in question.

One of the major challenges driving the replication crisis is that scientists often do not share all information needed to replicate their work. Access to research materials is especially crucial for the replication of computational studies, given the increasing utilization of computational methods and the data-reliant nature of such studies on large data sets. Unfortunately, it is far from guaranteed.

There are many reasons why. Sometimes academic considerations, such as avoiding criticism, fear of retraction in case a mistake is revealed, and avoiding “scooping,” may drive the decision. In other cases, as the boundary between academic and applied research blurs, commercial considerations keep replication materials private to preserve the prospect of their commercialization. This creates a culture of secrecy, which goes against the fundamental values of openness and sharing in the scientific community.

Intellectual property (IP) law plays a significant role in creating a culture of secrecy in science. Patents and trade secrets secure the economic potential held in research and development (R&D). Patent law does so by providing exclusive rights to inventions. Yet a patented invention must be novel and nonobvious, which encourages inventors to conceal information (at least until they file a patent application). This can lead them to limit access to replication materials to maximize their chances of obtaining patent protection. The mechanism by which trade secrets contribute to nonsharing norms is more straightforward: trade-secret law can essentially protect any kind of information as long as it is kept secret.

Thus, while encouraging innovation, IP rights can also discourage sharing of research materials and exacerbate the replication crisis. Nevertheless, these rights matter to researchers, organizations and commercial firms, and benefit them (and the public that enjoys their efforts) in multiple ways, from the financial to the reputational.

To address this conflict, we propose a new policy instrument that could facilitate studies’ replicability without depriving scientists of their IP protection: the conditional access agreement (CAA). In short, the CAA establishes a private, controlled channel of communication for the transfer of replication materials between authors and replicators. This allows for on-demand replicability while maintaining the proprietary potential of a scientific study.

Under the CAA mechanism, when submitting a paper for publication, an author would execute an agreement with the journal, pledging to provide full access to replication materials upon demand by other researchers. The agreement would specify that anyone requesting access to the materials can only obtain it upon signing a nondisclosure agreement (NDA). The NDA would prohibit the use of the replication materials delivered by the original authors for any purpose other than replication. Since patent law and trade-secret law bar protection only in the case of public disclosure, information shared privately under the NDA would not nullify the possibility of obtaining patent protection, nor would it negate trade secrecy. The CAA policy is feasible thanks to the involvement of scientific journals as powerful intermediaries in the scientific ecosystem. Journals are the gatekeepers of responsible science, and as such, they are continuously involved in a publication’s life cycle, from submission to post-publication. A CAA policy also aligns with journals’ mission of promoting rigorous and credible science. Notably, the CAA policy imposes minimal costs on researchers, replicators and journals, as the infrastructures for such a mechanism already exist, including repositories of data and online manuscript submission systems for embedding the CAA.

Faced with a replication crisis, we believe that replication agreements facilitated by journals can improve science. Pursuing this path will enhance the replicability of scientific research and increase public trust in science.

This is an opinion and analysis article, and the views expressed by the author or authors are not necessarily those of Scientific American.

  • en
您觉得本篇内容如何
评分

相关产品

EN 650 & EN 650.3 观察窗

EN 650.3 version is for use with fluids containing alcohol.

Acromag 966EN 温度信号调节器

这些模块为多达6个输入通道提供了一个独立的以太网接口。多量程输入接收来自各种传感器和设备的信号。高分辨率,低噪音,A/D转换器提供高精度和可靠性。三路隔离进一步提高了系统性能。,两种以太网协议可用。选择Ethernet Modbus TCP\/IP或Ethernet\/IP。,i2o功能仅在6通道以太网Modbus TCP\/IP模块上可用。,功能

雷克兰 EN15F 其他

品牌;雷克兰 型号; EN15F 功能;防化学 名称;防化手套

Honeywell USA CSLA2EN 电流传感器

CSLA系列感应模拟电流传感器集成了SS490系列线性霍尔效应传感器集成电路。该传感元件组装在印刷电路板安装外壳中。这种住房有四种配置。正常安装是用0.375英寸4-40螺钉和方螺母(没有提供)插入外壳或6-20自攻螺钉。所述传感器、磁通收集器和壳体的组合包括所述支架组件。这些传感器是比例测量的。

TMP Pro Distribution C012EN RF 音频麦克风

C012E射频从上到下由实心黄铜制成,非常适合于要求音质的极端环境,具有非常坚固的外壳。内置的幻像电源模块具有完全的射频保护,以防止在800 Mhz-1.2 Ghz频段工作的GSM设备的干扰。极性模式:心形频率响应:50赫兹-18千赫灵敏度:-47dB+\/-3dB@1千赫

ValueTronics DLRO200-EN 毫欧表

"The DLRO200-EN ducter ohmmeter is a dlro from Megger."

评论

您需要登录才可以回复|注册

提交评论

广告

scientific

这家伙很懒,什么描述也没留下

关注

点击进入下一篇

What Does GPT Stand For?

提取码
复制提取码
点击跳转至百度网盘