Information mismatch in PHH3-assisted mitosis annotation leads to interpretation shifts in H&E slide analysis

Jonathan Ganz; Christian Marzahl; Jonas Ammeling; Emely Rosbach; Barbara Richter; Chloé Puget; Daniela Denk; Elena A Demeter; Flaviu A. Tăbăran; Gabriel Wasinger; Karoline Lipnik; Marco Tecilla; Matthew J. Valentine; Michael J. Dark; Niklas Abele; Pompei Bolfa; Ramona Erber; Robert Klopfleisch; Sophie Merz; Taryn A. Donovan; Samir Jabari; Christof A. Bertram; Katharina Breininger; Marc Aubreville

doi:10.1038/s41598-024-77244-6

You are here:

PHAIDRA
Detail o:3754

Open in new window

Title (eng)

Information mismatch in PHH3-assisted mitosis annotation leads to interpretation shifts in H&E slide analysis

Author

Jonathan Ganz

Christian Marzahl

Jonas Ammeling

Emely Rosbach

Barbara Richter

Chloé Puget

Daniela Denk

Elena A Demeter

Flaviu A. Tăbăran

Gabriel Wasinger

Karoline Lipnik

Marco Tecilla

Matthew J. Valentine

Michael J. Dark

Niklas Abele

Pompei Bolfa

Ramona Erber

Robert Klopfleisch

Sophie Merz

Taryn A. Donovan

Samir Jabari

Christof A. Bertram

Katharina Breininger

Marc Aubreville

Abstract (eng)

The count of mitotic figures (MFs) observed in hematoxylin and eosin (H&E)-stained slides is an important prognostic marker, as it is a measure for tumor cell proliferation. However, the identification of MFs has a known low inter-rater agreement. In a computer-aided setting, deep learning algorithms can help to mitigate this, but they require large amounts of annotated data for training and validation. Furthermore, label noise introduced during the annotation process may impede the algorithms' performance. Unlike H&E, where identification of MFs is based mainly on morphological features, the mitosis-specific antibody phospho-histone H3 (PHH3) specifically highlights MFs. Counting MFs on slides stained against PHH3 leads to higher agreement among raters and has therefore recently been used as a ground truth for the annotation of MFs in H&E. However, as PHH3 facilitates the recognition of cells indistinguishable from H&E staining alone, the use of this ground truth could potentially introduce an interpretation shift and even label noise into the H&E-related dataset, impacting model performance. This study analyzes the impact of PHH3-assisted MF annotation on inter-rater reliability and object level agreement through an extensive multi-rater experiment. Subsequently, MF detectors, including a novel dual-stain detector, were evaluated on the resulting datasets to investigate the influence of PHH3-assisted labeling on the models' performance. We found that the annotators' object-level agreement significantly increased when using PHH3-assisted labeling (F1: 0.53 to 0.74). However, this enhancement in label consistency did not translate to improved performance for H&E-based detectors, neither during the training phase nor the evaluation phase. Conversely, the dual-stain detector was able to benefit from the higher consistency. This reveals an information mismatch between the H&E and PHH3-stained images as the cause of this effect, which renders PHH3-assisted annotations not well-aligned for use with H&E-based detectors. Based on our findings, we propose an improved PHH3-assisted labeling procedure.

Keywords (eng)

MitosisHistones MetabolismHumansDeep LearningAlgorithmsReproducibility of ResultsStaining and Labeling MethodsImage ProcessingComputer-Assisted Methods

Type (eng)

journal article

Language

[eng]

Persistent identifier

https://phaidra.vetmeduni.ac.at/o:3754

DOI

10.1038/s41598-024-77244-6

Is in series

Title (eng)

Scientific Reports

Volume