EHA European Hematology Exam 2018 Report
1. Method
a. Format
The second European Hematology Exam took place on Wednesday, June 13 in Stockholm, Sweden, and Bern, Switzerland. The Exam consisted of 100 multiple choice questions, covering all eight sections of the European Hematology Curriculum. The participants had 2,5 hours to complete the exam.
b. Candidates
73 candidates took part in the main session in Stockholm, Sweden, and 7 candidates attended the parallel session in Bern, Switzerland. Candidates originated from 26 countries and 6 regions/continents:
c. Pilot Switzerland
The Swiss Society of Hematology (SSH) took the initiative to organize a parallel session in Bern, Switzerland. This session was open to all Swiss candidates and served as a successful pilot for the implementation of the European Hematology Exam in the official Swiss end-of-training exam. National Hematology Societies or authorities of other countries are also welcome to contact EHA to discuss implementation of the European Hematology Exam as part of their national exams and/or organize local parallel sessions.
2. Quality assurance
a. Question writing and review process
To ensure that the items (questions, answer options and keys) are of high scientific and educational quality, adhere to the European recommended level, and cover the full spectrum of the Curriculum, a Question Writers Group has been established. For this group, 20 experienced hematologists have been selected and trained by educationalists on how to write good exam items. In pairs, these writers worked on their questions and reviewed each other’s questions. A second review round was performed by the Review Group, who also selected the final 100 questions, representing all Curriculum sections.
b. Psychometric analysis
To guarantee independent judgment of the quality of the exam and the passing score definition, EHA collaborates with CITO1, an internationally recognized professional research and knowledge institute in the field of testing and educational measurement. CITO’s Psychometric Research- and Knowledge Center performed the psychometric analysis of the results.
Reliability
In classical test theory, two well-known measures for reliability are Cronbach’s alpha and Guttman’s lambda-2. For this exam Cronbach’s alpha is 0.85 and Guttman’s lambda-2 is 0.86. These values are considered as good for a low stakes exam, and sufficient for high stakes exams by the European Federation of Psychologists’ Association (EFPA) Review Model2 and the standards of the Dutch Committee of Test Matters3 (COTAN).
Item analysis
To evaluate the quality of each item the correlation between the item score and the exam score has been calculated (Rit and Rir values4). The Rit-values were positive for 97 of the 100 items, with none of them significantly5 lower than 0. Items with negative Rit or Rir-values were checked, and no keying or other errors were found. The average Rit-value was 26. For general tests this is deemed sufficient, but it can be considered as good for an exam with such a broad scope.
Standard setting
To define the cut-off score, two standard setting methods were combined. Firstly, the Angoff method, in which 5 judges evaluated the difficulty of each question. And secondly, an equating procedure using item response theory, where the results of the 2018 exam could be placed on the same ability scale as the 2017 exam. This resulted into a cut-off score of 58 (or more) items correct to pass the exam. This means that 64 candidates (80,0%) passed.
Communication of the results
Within 4-6 weeks after the exam, the candidates have received an email with the pass-fail decision. The candidates who passed, received a certificate. In addition, all candidates have received information on their score per section of the European Hematology Curriculum. For this, CITO translated the overall cut-off score into a cut-off score per section, and the candidates received information on whether they scored above or below this cut-off score. Passing the exam is regarded an extra quality stamp for hematologists, the right to practice will still be granted by the national authorities.
1 https://www.cito.com/
2 See http://www.efpa.eu/professional-development/assessment
3 See https://www.cotandocumentatie.nl/cotan/beoordelingssysteem (in Dutch)
4 Rit: uses exam score with the evaluated item score, Rir: without the evaluated item score
5 At an alpha of 0.05
3. Candidate experience
At the end of the exam, the candidates were asked to complete a short evaluation survey. The results of this survey showed that:
- 97% of the candidates found the exam relevant (78%) or partly relevant (19%) to their hematology training
- 87% of the candidates experienced the exam setup as good or very good.
- 95% of the candidates was able to finish the exam within the 2,5 hours.
- Of the non-native English speakers, for 82% the English language did not pose a problem at all, while for the other 18% it was only a problem for some questions.
4. Acknowledgement
EHA would like to thank the following experts for their time and effort towards making the 2018
European Hematology Exam a success.
Exam Working Group
José Tomás Navarro, Spain (Chair Working Group)
Antonio Almeida, Portugal (Chair Curriculum Committee)
Gunnar Birgegård, Sweden (Chair Question Writers Group)
Cheng Hock Toh, United Kingdom
Janet Strivens, United Kingdom
Question Review Group
Gunnar Birgegård, Sweden
José Tomás Navarro, Spain
Antonio Almeida, Portugal
Hamdi Akan, Turkey
Anna Eriksson, Sweden
Janet Strivens, United Kingdom
Johanna Ungerstedt, Sweden
Question Writers Group
Gunnar Birgegård, Sweden
José Tomás Navarro, Spain
Antonio Almeida, Portugal
Hamdi Akan, Turkey
Cem Ar, Turkey
Gueorgui Balatzenko, Bulgaria
Anna Eriksson, Sweden
Carlos Fernández de Larrea, Spain
Margarita Guenova, Bulgaria
Gunnar Larfors, Sweden
Esperanza Lavilla Rubira, Spain
Estella Matutes, Spain
Gemma Moreno Jiménez, Spain
Esther Oliva, Italy
Alicia Rovó, Switzerland
Campbell Tait, United Kingdom
Peter te Boekhorst, The Netherlands
Cheng Hock Toh, United Kingdom
Johanna Ungerstedt, Sweden
Mariëlle Wondergem, The Netherlands
5. EHA European Hematology Exam 2019
The 2019 EHA Hematology Exam takes place in Amsterdam, during the EHA Congress. More information will be available soon on the EHA website.
Back