Key Concepts in Peer Review: Who are the “peers” in peer review?

One of the main activities of ESF-Science Connect is conducting peer review missions. In this post, the first in a series about key concepts in peer review, ESF shares some of its 40+ years of expertise in the peer review process, and explains the concept of a “peer” in the context of an ESF-led peer review exercise.

Peer Review – A quick definition

The term “Peer Review” in the ESF-Science Connect context refers to the integrated process for enabling the evaluation of research proposals, and ranking proposals against others in a group in order to recommend for funding the best proposals from a scientific quality point of view.

The profile and competencies of the “peers” involved in peer review play the most important role in achieving an effective, equitable and efficient selection of proposals for funding. The selection and allocation of these experts are grouped under the following four steps.

 

Each peer review process will be tailor-made according to the specificities of a given mission.  ESF will call on the services of the relevant types of experts as required by the exercise. There are three types of peers that can be involved in the ESF Peer Review process:

 

 

Peer #1 –Research Associates

Research Associates work for the ESF on a temporary project basis. When selecting ESF Research Associates we always look for the best-fit for the assignment and we take into consideration the area of expertise, discipline, experience and track record. Research Associates use their knowledge of the scientific community to identify External Reviewers that are best equipped to evaluate the quality of the proposals received. 

Peer #2 –External Reviewers

External Reviewers are recognised experts with in-depth scientific knowledge in a specific area, who have an outstanding record of current scholarship and strong credentials in their discipline.

These experts remotely assess the submitted proposals that have been assigned to them by the Research Associates. They do this on an individual basis, separately from other reviewers who may look at the same proposals. They do not discuss the proposals with anyone, and provide their assessments to ESF using known and clear criteria and scores.

Peer #3 –Panel Members

The constitution of Review Panels allows for the collective evaluation of a number of proposals by a group of more established/senior academics or researchers. These Panel Members have broad knowledge and experience, allowing a good overview of the State of the Art in their fields. 

To ensure the excellence, impartiality and transparency of the peer review process, Panel Members evaluate and, if required, consolidate scientific assessments made by external reviewers, and rank or prioritise proposals based on clear and stated criteria and parameters.

Rapporteurs:

All panel members act as rapporteurs, either as Lead or Secondary Rapporteur on a given proposal. Lead Rapporteurs initiate discussion on individual proposals, drawing on the external reviewers’ assessments (if applicable). Secondary Rapporteurs then present their views on the proposal.

Chair:

One of the panel members serves as chair. The Chair is an experienced panel member, and is asked to lead the panel meeting. The Chair ensures respect of the timeline and agenda, a fair treatment of all applications in respect of the assessment criteria and process, and arbitrates sensitive discussions.

A variety of profiles and an effective process

There are substantial differences between the roles of external reviewers and panel members. However, it is important to stress that there is no hierarchy between them; they are of equal standing within the process. The external reviewer considers the strengths and objectivity of a specific proposal based upon targeted expertise, whereas the panel member gives a subjective judgement across a group of proposals based upon broad expertise. The goal in constituting the peers for a mission is to ensure the availability of diverse viewpoints, scientific perspectives, and scholarly thinking. 

Peers are mostly selected from the ESF Community of Experts, a network of experts across all scientific disciplines. Once experts are invited and confirmed to a specific peer review exercise, proposals are assigned to them. Experts are informed of all the main elements and steps of the process. They are given a detailed description of their assignments, roles and responsibilities. In order to ensure equity and integrity in the assessment process, each expert signs a code of conduct before the start of the evaluation process .The code deals both with the requirement of declaring any conflict of interest, and with the obligation of respecting data confidentiality guidelines.

ESF is always looking for ways to improve the peer review process. Currently, we are working on the following initiatives:

  • Ensuring gender balance when selecting peer reviewers
  • Developing the online scientific platform (with a personal portal) enabling peer reviewers to follow the process more efficiently
  • Improving recognition of the vital contributions made by peer reviewers

For more information about our peer review process, please take a look at our website.

If you would like to join the Community of Experts, please find information on how to apply here.

0 Reply

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *