Advertisement
Original Article|Articles in Press

Clinicians’ perceptions of oral leukoplakia: a pitfall for image annotation in Supervised Learning

Published:March 06, 2023DOI:https://doi.org/10.1016/j.oooo.2023.02.018
Clinicians’ perceptions of oral leukoplakia: a pitfall for image annotation in Supervised Learning
Previous ArticleThe diagnostic efficacy of ultrasonographically-based measurements of minimum and maximum fascia-tumor distance in differentiating superficial from deep lobe parotid tumors
Next ArticleWorld Workshop on Oral Medicine VIII: Development of a Core Outcome Set for Dry Mouth: A Systematic Review of Outcome Domains for Xerostomia.

      Abstract

      Objective

      The present study aims to quantify clinicians’ perceptions of oral potentially malignant disorders (OPMD) when evaluating, classifying, and manually annotating clinical images, as well as to understand the source of interobserver variability when assessing these lesions. The hypothesis was that different interpretations can affect the quality of the annotations that are used to train a Supervised Learning (SL) model.

      Study design

      46 clinical images from 37 patients were reviewed, classified, and manually annotated at the pixel level by three labelers. We compared the inter-examiner assessment based on clinical criteria through the κ statistics (Fleiss's kappa). The segmentations were also compared using the mean pixel-wise intersection over union (IoU).

      Results

      The interobserver agreement for homogeneous/non-homogeneous criteria was substantial (κ=63, 95% confidence interval (CI): 0.47 to 0.80). For the subclassification of non-homogeneous lesions, the interobserver agreement was moderate (κ=43, 95% CI: 0.34 to 0.53) (p <0.001). The mean IoU of 0.53 (±0.22) was considered low.

      Conclusion

      The subjective clinical assessment (based on human visual observation, variable criteria that has suffered adjustments over the years, different educational backgrounds, and personal experience) may explain the source of interobserver discordance for classification and annotation of OPMD. Therefore, there is a strong probability of transferring the subjectivity of human analysis to artificial intelligence (AI) models. The use of large datasets and segmentation based on the union of all labelers’ annotations holds the potential to overcome this limitation.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Register: Create an account
      Institutional Access: Sign in to ScienceDirect

      Article info

      Publication history

      Accepted: February 22, 2023
      Received in revised form: February 7, 2023
      Received: November 22, 2022

      Publication stage

      In Press Journal Pre-Proof

      Footnotes

      Ethics Approval

      This study is in accordance with the Declaration of Helsinki and was approved by the Piracicaba Dental Ethical Committee, Registration number CAAE: 42235421.9.0000.5418.

      Author contribution

      All authors have made substantial contributions to the conception (ALDA, ARSS), draft and design (ALDA, ESCS, ISPF, CSS, TBS), and review (MAL, PAV, ATP, LPK, ACPLFC, ARSS) of the paper's final version. The authors agree to be accountable for all aspects of the work and ensures that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

      Statement of clinical relevance

      The interpretation and manual annotation of clinical images of oral leucoplakias can be very discrepant among oral medicine practitioners, affecting the quality of the final ground truth annotation that will drive image segmentation.

      Identification

      DOI: https://doi.org/10.1016/j.oooo.2023.02.018

      Copyright

      © 2023 Published by Elsevier Inc.

      ScienceDirect

      Access this article on ScienceDirect
      We use cookies to help provide and enhance our service and tailor content. To update your cookie settings, please visit the for this site.
      Copyright © 2023 Elsevier Inc. except certain content provided by third parties. The content on this site is intended for healthcare professionals.


      RELX