Why Collegial Feedback Matters in Academic Evaluation
In recent years, research assessment in higher education has increasingly leaned on quantitative metrics such as publication counts, impact factors, and h‑indices. While these indicators provide a quick snapshot of productivity, they often overlook the nuanced contributions that scholars make to their fields and to the academic community. At Eötvös Loránd University (ELTE) in Hungary, a new initiative is shifting the focus toward qualitative, peer‑based evaluation, placing collegial feedback at the center of the assessment process.
CoARA and the Push for a Qualitative Reform
The Coalition for Advancing Research Assessment (CoARA) is a network of more than 700 institutions committed to improving how research is evaluated. ELTE’s Faculty of Humanities became the first university in Hungary to secure CoARA funding in 2024, a milestone that underscores the institution’s willingness to experiment with alternative assessment models.
Associate Professor Csanád Bodó, who led the project, explains that the motivation came from a desire to align assessment practices with the principles of the Leiden Manifesto and the Declaration on Research Assessment (DORA). By integrating peer review into the evaluation framework, the faculty aims to reduce the overreliance on journal metrics and foster a culture of constructive dialogue among researchers.
From Idea to Implementation: The Funding Process
Applying for CoARA support involved a collaborative review process. Each proposal was evaluated by at least three experts, and consensus was reached through online discussions. Bodó notes that the flexibility of the application allowed the faculty to highlight its unique context while still meeting CoARA’s criteria for comparability across institutions.
Designing a Participatory Workshop Series
Central to the reform effort was a three‑day workshop series titled részvételi tanácsolás (participatory counsel). The format drew inspiration from citizen juries and was adapted to the academic setting to encourage open, structured dialogue.
Day One: Information and Context
Participants received an overview of current assessment practices, the limitations of metrics‑only approaches, and the CoARA principles. This foundation set the stage for deeper engagement in the following days.
Day Two: Deliberation and Debate
Using techniques such as the “six thinking hats” and “silent floor,” researchers explored how to balance selective assessment (e.g., promotion, funding decisions) with supportive assessment (e.g., career development, community impact). The goal was to identify criteria that could serve both purposes without compromising fairness.
Day Three: Decision‑Making and Recommendations
On the final day, participants drafted a set of actionable recommendations for the faculty leadership. These included proposals for a narrative CV format, peer‑reviewed self‑assessment, and a pilot study to test the new framework.
Key Findings: Selective vs. Supportive Assessment
One of the most significant insights was the need to treat selective and supportive assessment as complementary rather than separate. By applying the same qualitative criteria across both domains, researchers can receive feedback that informs career progression while also enhancing the quality of their work.
Moreover, the workshops highlighted the importance of peer review beyond manuscript evaluation. Continuous, collegial feedback can accelerate learning, improve research design, and strengthen collaborative networks.
Leadership Response and Institutional Commitment
Dean Dávid Bartus responded positively to the recommendations, acknowledging the potential benefits of a supportive assessment culture. However, he also raised concerns about the additional workload for faculty already stretched thin by teaching, research, and administrative duties.
To address this, the faculty plans to integrate the new assessment tools into existing workflows, such as incorporating narrative CVs into the annual review process and offering optional peer‑review workshops for interested staff.
Pilot Study: Testing the New Framework
The pilot involves twelve researchers who will complete an online training module on qualitative assessment. They will then draft narrative CVs covering research achievements, community engagement, and personal development. Each CV will be anonymously reviewed by two peers, and the feedback will be aggregated to assess the feasibility and impact of the new system.
Results will be published early next year, providing evidence to guide broader implementation across the faculty and potentially the entire university.
Implications for the Broader Academic Community
ELTE’s experience offers a blueprint for other institutions in Hungary and beyond. By demonstrating that qualitative, peer‑based assessment can coexist with traditional metrics, the university challenges the prevailing narrative that numbers alone can capture scholarly value.
Key takeaways for academic leaders include:
- Adopt a dual‑purpose assessment model that serves both career advancement and research quality.
- Facilitate structured, participatory dialogues to surface diverse perspectives.
- Provide training and resources to reduce the perceived burden of new assessment practices.
- Publish transparent recommendations and pilot results to build trust and encourage adoption.
Next Steps and Opportunities for Engagement
As the pilot concludes, ELTE will evaluate the outcomes and refine the framework. The faculty also plans to explore scaling the model to other departments and to collaborate with national research bodies to influence policy at a higher level.
Researchers interested in contributing to or learning from this initiative can access the full set of workshop materials and pilot documentation on the university’s research portal.
Take Action: Join the Conversation
Are you a faculty member or researcher looking to enhance your institution’s assessment practices? Consider the following steps:
- Explore ELTE’s pilot study results and workshop guidelines on the research portal.
- Contact the Faculty of Humanities’ research office to discuss potential collaborations or to request training materials.
- Share your own experiences with qualitative assessment in the comments below or on the university’s discussion forum.
- Apply for future CoARA funding opportunities to support similar projects at your institution.
- Stay informed by subscribing to the university’s research newsletter for updates on assessment reforms.
By engaging with these resources, you can help shape a more balanced, collegial, and effective research assessment culture.
