Ethics and Society Review (ESR) facilitates researchers in mitigating negative ethical and societal aspects of their research by acting as a requirement to access funding.

TODO for Michael: remove robots indexing exclusion from before paper is out. No indexing right now to make sure no content accidentally leaks live.

Researchers in areas as diverse as computer science and political science must increasingly navigate the possible risks of their research to society. However, the history of medical experiments on vulnerable individuals influenced many research ethics reviews to focus exclusively on risks to human subjects rather than risks to human society. We created an Ethics and Society Review board (ESR), which fills this moral gap by facilitating ethical and societal reflection as a requirement to access grant funding: researchers cannot receive grant funding from participating programs until the researchers complete the ESR process for their proposal. Researchers author an initial statement describing their proposed research's risks to society, subgroups within society, and globally, and commit to mitigation strategies for these risks. An interdisciplinary faculty panel iterates with the researchers to refine these risks and mitigation strategies.

Download ESR Materials · Read Our Paper

Why the ESR exists

Whether research diffuses into society through technological adoption, through field experiments, or through policy, researchers must reflect on how to identify and mitigate the risks that the diffusion of their work presents to human society. These risks include, for example, the possibility that their contributions to artificial intelligence might exacerbate biases in the criminal justice system, that their urban planning concepts might backfire when implemented, or that their elections research might influence electoral outcomes. Through these projects and many others, researchers must grapple with not just the benefits of their work, but also the risks that their work presents to society: to forms of social organization ranging from groups to nations to humanity as a whole.

Research ethics review often focuses on risks to human subjects, not risks to human society, placing societal risks out of scope and out of jurisdiction. In the United States, ethics review is associated with Institutional Review Boards (IRBs) and is governed by the Common Rule. The Common Rule gives IRBs jurisdiction over risks to human subjects, who are the individuals directly engaged or studied in the research. However, the Common Rule governing IRBs specifically disallows review of consequences to human society:

The IRB should not consider possible long-range effects of applying knowledge gained in the research [...] as among those research risks that fall within the purview of its responsibility.

This regulation is generally interpreted to mean that IRBs should decline to review research risks to human society.

It is not unreasonable to worry about IRB overreach—almost every action carries potential risks of harms—yet it is inappropriate to ignore the risks that research poses for our collective future: the risks of artificial intelligence to the future of work, the risks of sustainability interventions to the societies that they are purported to support, the risks of the internet to professional media and accurate information. In the light of these risks, recent scholarship has argued that research carrying substantial societal risk should undergo ethics review. IRBs such as the Microsoft Research Ethics Review Program have adopted purviews that include these risks in their definition of "human subjects". The goal of our work is to leverage these conceptual and organizational insights to design a concrete process that engages researchers whose work typically falls outside the purview of their institution's current review processes.

How the ESR works

With the ESR, grant funding from participating institutions is not released until the researchers successfully complete an iterative review process on their proposed project. Conditioning funding on the ESR process helps engage researchers at the formative stages of their research, when projects are still open to change, and ensures broad engagement with the process rather than self-selection of just those who are motivated.

For funding organizations that incorporate the ESR in their grant process, researchers submit a brief ESR statement alongside their grant proposals. The ESR statement describes their project's most salient risks to society, to subgroups in society, and to other societies around the world. This statement articulates the principles the researchers will use to mitigate those risks, and describes how those principles are instantiated in the research design.

After the funding program conducts its grant merit review, it sends only the grants recommended for funding to the ESR for ethics review. The ESR convenes an interdisciplinary panel of faculty that considers the studies' risks and mitigations in the context of possible benefits to society, and determines the adequacy of the ESR statement provided by the investigators. Its goal is not to eradicate all potential or actual negative impacts---which is often impossible---but to work with the researchers to identify negative impacts and to devise reasonable mitigation strategies. Over one to two weeks, the faculty panel engages in iterative feedback to the researchers, which can include raising new possible risks, helping identify collaborators or stakeholders, and brainstorming additional mitigation strategies. PIs submit written responses to the ESR feedback as addenda to their original statement. These addenda can include replies to the panel's feedback as well as commitments to specific mitigation strategies.

When the process is complete, the ESR submits its recommendation to the funding program, and funds are released to the researchers.

Evaluation

We are always evaluating whether the ESR is helping. In our first year, we pursued a mixed method evaluation in partnership with Stanford HAI. More details are in the paper. Surveys and interviews of researchers who interacted with the ESR found 100% (95% CI: 87%–100%) were willing to continue submitting future projects to the ESR, and 58% (95% CI: 37%–77%) felt that it had influenced the design of their research project. The ESR panel most commonly identified issues of harms to minority groups, inclusion of diverse stakeholders in the research plan, dual use, and representation in datasets. These principles, paired with possible mitigation strategies, offer scaffolding for future research designs.

Organization

Team

Faculty co-chairs

2021-2022 ESR faculty panelists

Staff panelists

Alumni faculty panelists

Funding support

This work was supported by the Public Interest Technology University Network; Stanford's Ethics, Science, and Technology Hub; Stanford's Institute for Human-Centered Artificial Intelligence; and NSF ER2-2124734.