Skip to content

Commit

Permalink
Few small updates - placeholder images, renamed some protocol section…
Browse files Browse the repository at this point in the history
…s, standardized *ACL use
  • Loading branch information
KDercksen committed Apr 19, 2024
1 parent db77209 commit 56f65a9
Show file tree
Hide file tree
Showing 3 changed files with 28 additions and 8 deletions.
4 changes: 2 additions & 2 deletions _pages/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,12 @@ Authors can sigh the license transfer agreement when they submit a paper in
OpenReview. The license agreement is available at the bottom of the submission
form.

![test image]()
<img src="https://via.placeholder.com/600x300" alt="Screenshot coming soon" title="Screenshot coming soon">

### How can authors revise their donation decision?

Once a paper is submitted, authors can revise their donation decision by going
to the paper forum and clicking the "Edit submission" button. In the dropdown
menu, there is an option to change the donation status.

![test image]()
<img src="https://via.placeholder.com/600x300" alt="Screenshot coming soon" title="Screenshot coming soon">
2 changes: 1 addition & 1 deletion _pages/home.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ community. The dataset will be used to study the peer review process in the ACL
community, and to develop new tools and methods to improve peer reviewing in our
conferences.

While peer review data collection is currently happening in the ACL venues
While peer review data collection is currently happening in the \*ACL venues
(Dycke et al. 2023), the resulting data is skewed towards papers with more
positive reviewer scores. By adapting the data collection protocol to create
more balanced data resources, we can improve the quality of this line of
Expand Down
30 changes: 25 additions & 5 deletions _pages/protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ toc_sticky: false
The data collection protocol for ARR-DC builds on the initiative by [Gurevych,
Kuznetsov and Dycke (2021)](/resources/). A general overview of the updated data
collection protocol for each role in the peer review process can be found
[below](#overview-of-april-2024-protocol).
[below](#overview-of-current-protocol).

The ARR-DC organization invests reasonable effort to allow reviewers and authors
to withdraw their consent and license agreement at any time. The data collection
Expand All @@ -24,8 +24,8 @@ Committee](https://www.aclweb.org/adminwiki/index.php?title=Formation_of_the_ACL

### Important changes

From April 2024 onwards, the protocol described on this page will be in place.
Below are the key novelties with respect to the pre-April 2024 protocol:
From April 2024 onwards, the new protocol described on this page will be in
place. Below are the key novelties with respect to the pre-April 2024 protocol:

- Authors now decide on data donation during submission and can revise this
decision throughout the reviewing process.
Expand All @@ -36,9 +36,11 @@ Below are the key novelties with respect to the pre-April 2024 protocol:
protocol.

_The pre-April 2024 protocol is described in detail in [this
blogpost](https://aclrollingreview.org/datacollection) on the ARR website._
blogpost](https://aclrollingreview.org/datacollection) on the ARR website. For a
short overview, refer to the respective [section
below](#overview-of-pre-april-2024-protocol)._

## Overview of April 2024 protocol
## Overview of current protocol

### For authors

Expand All @@ -63,3 +65,21 @@ Area Chairs (ACs) are asked to sign a license agreement to donate all their
meta-reviews for the ARR cycle, or they can grant the license for all future ARR
cycles until consent is withdrawn. Additionally, ACs can specify if they prefer
their meta-reviews to be included as-is or paraphrased.

## Overview of pre-April 2024 protocol

### For authors

Authors are requested to consent to the collection of their data after the
review process is completed and acceptance decisions are released. Collected
data includes the submission draft and the donated peer reviews of their draft.

### For reviewers

Reviewers are asked once per cycle if they are willing to donate their peer
review reports. The collected data consists of the review texts only.

### For meta-reviewers

ACs are asked to consent to the sharing of reviews of papers that are eventually
accepted at an \*ACL conference. Meta-review texts are **NOT** collected.

0 comments on commit 56f65a9

Please sign in to comment.