Medication-related osteonecrosis of the jaw: A cross-sectional survey assessing the quality of information on the Internet

Published:September 04, 2021DOI:


      The objective of this study was to evaluate the quality of information and readability of online content regarding medication-related osteonecrosis of the jaw (MRONJ).

      Study Design

      Three search engines were used to identify websites containing information regarding MRONJ. The content was assessed with 5 quality of information and readability tools. Statistical analyses were performed via GraphPad Software (GraphPad Software Inc., La Jolla, Calif, United States).


      The Health on the Net (HONcode) seal was present in 2 of the 21 websites that satisfied inclusion/exclusion criteria. The mean JAMA (Journal of the American Medical Association) benchmark satisfied per website was 1.619 out of 4 (standard deviation [SD] = 1.2; 95% confidence interval [CI], 1.07-2.17). The mean overall Quality Evaluation Scoring Tool score was 15.29 out of a maximum 28 (SD = 6.174; 95% CI, 12.48-18.1). The Quality Evaluation Scoring Tool scores of those websites that satisfied ≥3 JAMA criteria were significantly higher than those that did not (P = .01; 95% CI, −11.41 to −1.79). The readability levels of all content ranged from fairly difficult to very difficult.


      The information related to MRONJ on the Internet is unreliable and too difficult for the general public to read. Authors of online MRONJ information should consider use of quality of information and readability tools to ensure that reliable information is understood by those with poor health literacy skills.
      To read this article in full you will need to make a payment


        • Nicolatou-Galitis O
        • Schiødt M
        • Mendes RA
        • et al.
        Medication-related osteonecrosis of the jaw: definition and best practice for prevention, diagnosis, and treatment.
        Oral Surg Oral Med Oral Path Oral Radiol. 2019; 127: 117-135
        • Baron R
        • Ferrari S
        • Russell RG.
        Denosumab and bisphosphonates: different mechanisms of action and effects.
        Bone. 2011; 48: 677-692
        • Tanna N
        • Steel C
        • Stagnell S
        • Bailey E.
        Awareness of medication related osteonecrosis of the jaws (MRONJ) amongst general dental practitioners.
        Br Dent J. 2017; 222: 121-125
        • Yarom N
        • Shapiro CL
        • Peterson DE
        • et al.
        Medication-related osteonecrosis of the jaw: MASCC/ISOO/ASCO clinical practice guideline.
        J Clin Oncol. 2019; 37: 2270-2290
        • Ruggiero SL
        • Dodson TB
        • Fantasia J
        • et al.
        American Association of Oral and Maxillofacial Surgeons. American Association of Oral and Maxillofacial Surgeons position paper on medication-related osteonecrosis of the jaw—2014 update.
        J Oral Maxillofac Surg. 2014; 72: 1938-1956
        • Vandone AM
        • Donadio M
        • Mozzati M
        • et al.
        Impact of dental care in the prevention of bisphosphonate-associated osteonecrosis of the jaw: a single-center clinical experience.
        Ann Oncol. 2012; 23: 193-200
        • McInnes N
        • Haglund BJ.
        Readability of online health information: implications for health literacy.
        Inform Health Soc Care. 2011; 36: 173-189
        • Spring H.
        Health literacy and COVID-19.
        Health Info Libr J. 2020; 37: 171-172
        • Muscat DM
        • Shepherd HL
        • Nutbeam D
        • Trevena L
        • McCaffery KJ.
        Health literacy and shared decision-making: exploring the relationship to enable meaningful patient engagement in healthcare.
        J Gen Int Med. 2021; 36: 521-524
      1. National Centre for Education Statistics. The health literacy of America's adults: results from the 2003 National Assessment of Adult Literacy. Available at: Accessed 25 June 2021.

      2. Moser C. Improving literacy and numeracy: a fresh start. 1999. Available at: Accessed 25 June 2021.

        • Fitzsimmons PR
        • Michael B
        • Hulley JL
        • Scott GO.
        A readability assessment of online Parkinson's disease information.
        J R Coll Physicians Edinb. 2010; 40: 292-296
        • Ríordáin RN
        • McCreary C.
        Dental patients’ use of the Internet.
        Br Dent J. 2009; 207: 583-586
        • Gurzawska-Comis K
        • Becker K
        • Brunello G
        • Gurzawska A
        • Schwarz F.
        Recommendations for dental care during COVID-19 pandemic.
        J Clin Med. 2020; 9: 1833
        • Riordain RN
        • Hodgson T.
        Content and quality of website information on the treatment of oral ulcers.
        Br Dent J. 2014; 217: E15
        • Wiriyakijja P
        • Fedele S
        • Porter S
        • Ni Riordain R
        Web-based information on the treatment of oral leukoplakia-quality and readability.
        J Oral Pathol Med. 2016; 45: 617-620
        • Alsoghier A
        • Riordain RN
        • Fedele S
        • Porter S.
        Web-based information on oral dysplasia and precancer of the mouth—quality and readability.
        Oral Oncol. 2018; 82: 69-74
        • Riordain RN
        • McCreary C.
        Head and neck cancer information on the Internet: type, accuracy and content.
        Oral Oncol. 2009; 45: 675-677
        • Alnafea S
        • Fedele S
        • Porter S
        • Ni Riordain R
        Online information on the treatment of burning mouth syndrome: quality and readability.
        J Oral Facial Pain Headache. 2017; 31: 147-151
        • Lee KC
        • Berg ET
        • Jazayeri HE
        • et al.
        Online patient education materials for orthognathic surgery fail to meet readability and quality standards.
        J Oral Maxillofac Surg. 2019; 77 (180.e1-e8)
        • McGoldrick D
        • Kielty P
        • Cotter C.
        Quality of information about maxillofacial trauma on the Internet.
        Br J Oral Maxillofac Surg. 2017; 55: 141-144
        • Bizzi I
        • Ghezzi P
        • Paudyal P.
        Health information quality of websites on periodontology.
        J Clin Periodontol. 2017; 44: 308-314
        • Meade M
        • Dreyer C.
        Web-based information on orthodontic clear aligners: a qualitative and readability assessment.
        Aust Dent J. 2020; 65: 225-232
        • Schwarzbach HL
        • Mady LJ
        • Kaffenberger TM
        • et al.
        Quality and readability assessment of websites on human papillomavirus and oropharyngeal cancer.
        Laryngoscope. 2021; 131: 87-94
        • Meade MJ
        • Dreyer CW.
        Orthodontic temporary anchorage devices: a qualitative evaluation of Internet information available to the general public.
        Am J Orthod Dentofacial Orthop. 2020; 158: 612-620
      3. Health on the Net Foundation. HONcode section for medical professionals. Available at: Accessed June 25, 2021.

      4. Health on the Net. The certification process step by step. Available at: Accessed 25 June 2021.

        • Silberg WM
        • Lundberg GD
        • Musacchio RA.
        Assessing, controlling, and assuring the quality of medical information on the Internet: caveant lector et viewor—let the reader and viewer beware.
        JAMA. 1997; 277: 1244-1245
        • Robillard JM
        • Jun JH
        • Lai J-A
        • Feng TL.
        The QUEST for quality online health information: validation of a short quantitative tool.
        BMC Med Inform Decis Mak. 2018; 18: 87
        • Flesch R.
        A new readability yardstick.
        J Appl Psychol. 1948; 32: 221-233
        • McLaughlin GH.
        SMOG grading: a new readability formula.
        J Read. 1969; 12: 639-646
      5. EbizMBA. Top 15 most popular search engines. Available at: Accessed April 10, 2021.

        • Meade MJ
        • Dreyer CW.
        Evaluation of treatment risks and the quality of information contained within the websites of specialist orthodontists.
        Aust Orthod J. 2019; 35: 143-151
        • Griffiths SZ
        • Albana MF
        • Bianco LD
        • Pontes MC
        • Wu ES.
        Robotic-assisted total knee arthroplasty: an assessment of content, quality, and readability of available Internet resources.
        J Arthroplasty. 2021; 36: 946-952
        • Charnock D
        • Shepperd S
        • Needham G
        • Gann R.
        DISCERN: an instrument for judging the quality of written consumer health information on treatment choices.
        J Epidemiol Commun Health. 1999; 53: 105-111
        • Sivanadarajah N
        • El-Daly I
        • Mamarelis G
        • Sohail M
        • Bates P.
        Informed consent and the readability of the written consent form.
        Ann R Coll Surg Engl. 2017; 99: 645-649
        • U.S. Department of Health and Human Services
        Saying it Clearly.
        U.S. Government Printing Office, Washington, DC2000
        • Weiss BD.
        HealthLiteracy and PatientSafety: Help Patients Understand.
        American Medical Association Foundation, Chicago, IL2007 (Available at) (Accessed June 20, 2021)
        • Du H
        • Yang J
        • King RB
        • Yang L
        • Chi P
        COVID-19 increases online searches for emotional and health-related terms.
        Appl Psychol Health Well Being. 2020; 12: 1039-1053
        • Meade MJ
        • Dreyer CW.
        An assessment of the treatment information contained within the websites of direct-to-consumer orthodontic aligner providers.
        Aust Dent J. 2021; 66: 77-84