Update: EMNLP 2026 AI Reviewing Experiment

On May 10, we announced the EMNLP 2026 AI Reviewing Experiment. In total, 7,921 submissions opted into the experiment. We have been hard at work designing the experiment, creating author feedback surveys, and pilot testing the AI Reviewing Systems. We plan on posting the AI reviews to the OpenReview page of opted-in submissions around July 20.

Experiment Design Updates

We are still using two AI Reviewing Systems, namely ReviewerToo and the AAAI AI Reviewing System. However, the ReviewerToo system will use two different backbone LLMs: GPT-OSS-120B and GLM 4.7 Flash. Therefore, this gives a total of three possible AI reviewing systems. As previously stated, opted-in submissions will be randomly assigned to one of these AI reviewing systems. The authors will not know to which system their submission has been assigned.

Collecting Feedback

Opted-in submissions will see an AI Review Response Form attached directly to the AI review in OpenReview, which the authors can use to share feedback. Completing this survey is optional and has no impact on how your submission(s) are handled. It will be possible to complete this form until August 1.

Authors of submissions that did not opt-in will receive a single email inviting them to complete a short survey. The purpose of this survey is to better understand the reason(s) why authors did not opt-in. Once again, completing this survey is optional and has no effect on how your submission(s) are handled. This survey will also remain open until August 1.

Internal Testing

We have completed a round of internal testing of the AI reviewing systems on a small sample of submissions that were co-authored by the Program Chairs. In our internal testing, we identified an issue with the length of the AI reviews, which were sometimes too short or much too long. We also identified an issue with the AI reviewers being overly optimistic in their appraisal of the submissions, or not having a sense of which errors/weaknesses are critical vs which ones are not. Ultimately, we did not adjust the system prompts to address these issues, given the otherwise stable working state of the underlying systems.

Contact Information

For questions about the AI reviewing experiment, email: emnlp2026-ai-review-chairs@googlegroups.com

For questions about EMNLP 2026 commitment and other review related topics, email: emnlp2026-programchairs@googlegroups.com