Reviews can be assigned either manually or automatically. Chairs and, if allowed by the conference, TPC members can add any number or reviewers to each paper by going to the main paper page or by checking off the boxes in the reviewer assignment table found in the 'My TPCs' section.
The review process typically goes through the following steps:
The basic steps for assigning reviews automatically are:
Details are below.
EDAS supports either a two-tier (single-level) or three-tier (two-level) review model, consisting of one or more conference chairs, TPC members and reviewers. In a two-tier model, the chair assigns reviewers to papers, who then complete or delegate their reviews. In a three-tier model, conference or track chairs assign TPC members to papers manually or automatically, while either the chairs or the TPC members can assign reviewers to papers. For journals, TPC chairs are renamed to editors instead, but are otherwise equivalent.
In a two-level model, EDAS has two kinds of reviews for each conference and paper: regular and meta reviews. Most smaller conferences, using the single-level review model, have only regular reviews, assessing the correctness, presentation and novelty of the paper. Some large conferences also add meta reviews to the review process, where one reviewer, a member of the TPC, for each paper is also assigned a meta review. Sometimes these meta reviews are called summary or TPC reviews. The TPC member in charge of the meta review summarizes the other reviews and any discussion among the reviewers. Typical questions might include a summary recommendation to the program chair, a summary of the discussion phase or regular reviews and, for papers where the assigned reviewers cannot arrive at a recommendation, suggestions for additional reviewers.
There are separate review forms (Reviews/Configure), notification letters (Conference/Templates) and deadlines for regular and meta reviews (Reviews/Configure).
Meta reviews can only be assigned to TPC members, while both TPC members and anybody else can serve as a reviewer.
If desired, TPC members can be assigned to a particular track and will only be able to claim and review papers from their track.
If allowed by the conference configuration, TPC members and reviewers can delegate the review to somebody else, e.g., a graduate student workind for a faculty member. Delegated reviews can still be viewed and edited by the original assigned reviewer; they can also be reclaimed if the person the review was delegated to fails to complete the review.
Smaller conferences typically have only the TPC members review the papers, with maybe a few delegations to outsiders. On the other hand, a very large conference, such as IEEE Infocom, may use a two or three-tier review model. The TPC members assign reviewers and provide a summary of the reviews, using the separate TPC review form defined in Reviews/Configure. They may also be a reviewer for the same paper and may initiate discussions among the reviewers for the paper. They are often also called lead reviewers and the TPC review is called a summary review.
To facilitate TPC meetings for large conferences, TPC members and papers can be assigned to TPC groups, with one or more groupleaders. Each paper can belong to at most one TPC group, but each TPC member can belong to any number of TPC groups.
Track chairs are members of the TPC that have chair-like privileges for papers in one or more designated tracks for the conference. They can assign reviewers manually, delete reviewers and view all reviews. They cannot run the automated review assignment mechanism for the conference and cannot notify all authors. They also cannot modify conference parameters, track parameters and email templates.
TPC members can be designated as groupleaders. They can review papers, but they can also be granted additional privileges (in Reviews/Configure) to assign papers, view reviews and see reviewer identities. In most cases, group leaders only have special privileges for those papers explicitly assigned to the same TPC group as the groupleader.
There are several ways to assign papers to TPC members and reviewers. These approaches are complementary, e.g., you can assign papers to reviewers automatically and then later add additional reviewers manually. Note that you need to first configure the review questions for your conference via Reviews/Configure (bottom of page).
Conferences can be configured, via Reviews/Configure, so that TPC members cannot assign reviews.
Conferences with a two-level review system can operate in two ways, depending on whether the reviewers are assigned from a fixed list, a technical program committee, or not. If there is a fixed list, reviewers can be entered as TPC members and selected from a list.
TPC members can declare conflicts of interest by adding and deleting names in a list. When creating a new user account, the list of conflicts is initially populated by all other EDAS users that share the same non-generic email domain, share the same name, have been co-authors or have the same affiliation. When reviewers claim papers for review, EDAS will omit papers where one or more of the authors is on the reviewer's conflict-of-interest list. For manual assignment, it will not assign such papers.
The goal of the automatic review assignment is to ensure that every paper has the required number of TPC and regular reviews. A secondary objective is to ensure that papers are reviewed by TPC members that are most interested in the paper. After those criteria, the algorithm tries to ensure that the review load is balanced between reviewers. Clearly, it's impossible to optimize for all three criteria at once, so some amount of unequal distribution is unavoidable.
The algorithm assigns papers up to the specified maximum number of reviews to TPC members. Each paper is assigned reviewers in turn, starting with the paper with the smallest number of claims. Reviews are assigned to be the most willing reviewer, i.e., if there's a reviewer that wants to review the paper, they will get precedence over reviewers who are merely willing to review the paper. If there's a tie between TPC members, the one with the smallest number of papers assigned gets it.
If a TPC member does not claim any papers, they are assigned based on their declared interests. If a TPC member does not indicate any interests, they are assumed to be interested in all topics.
After you have defined a review form (via Reviews/Configure), you can add reviewers. To add TPC members or reviewers manually, pull up the basic paper information, e.g., via Papers/List, and then click on 'Add reviewer' or 'Add TPC member' below the basic paper information. The TPC member or reviewer will receive an email asking him to review the paper. The TPC member or reviewer can accept or decline this invitation. Once he has accepted, he will be sent another email containing detailed review instructions.
Reviewers and TPC members can always check their EDAS home page to find the papers that they have been assigned.
Note that the notifications can be delayed until all reviewers have been assigned, by checking the appropriate box in the "Reviews" configuration menu. Pending notifications are sent via "Reviews/Notify reviewers".
Conferences can be configured for two kinds of review notifications. Reviewers can receive an email notification for each review they have been asssigned, or a single email if one or more new reviews have been assigned. For the latter, they need to consult their EDAS review listing to determine new reviews.
A TPC reviewer, if allowed by the review configuration, can assign additional non-TPC reviews for a paper, but no additional TPC reviews. Thus, there's one additional review outstanding after the TPC member assigns another review.
If allowed by configuration, both a TPC member and a reviewer can delegate ("hand off") their review to another person. They remain listed as the original reviewer for the paper and can check the status of the review and view the review itself once it is done. In the new interface, somebody who has been delegated the review cannot further delegate it; if he or she refuses the review, it reverts back to the original reviewer. (The old interface allows delegating delegated reviews, but that turned out to be confusing.)
Reviewers are reminded automatically or manually via Reviews/Reminders. Reminders are sent to reviewers that have not yet confirmed their review or where the review is almost due or beyond due. The reminder ("nag") interval and the warn-ahead period are configured in Reviews/Configure. Reminders for overdue reviews can be sent either in one message for each paper or in one message for each tardy reviewer, depending on which email template is defined. You can also send one message per reviewer to reviewers that have not yet confirmed all their reviews, via People/Email.