S. Ito et al.: Radioprotection 2025, 60( 3), 211 – 220 213
2.4 Readability
In this study, the jReadability text measurement system was used to quantitatively assess the readability of Japanese text( Lee, 2016; Lee et al., 2023). This index calculates readability based on the average length of a sentence, word difficulty, percentage of grammatical parts of speech, and type of characters per sentence. Scores range from 0.5 to 6.4, with higher scores indicating that the sentences are relatively easy to read: 0.5 – 1.4, very difficult to read; 1.5 – 2.4, difficult to read; 2.5 – 3.4, somewhat difficult to read; 3.5 – 4.4, neutral; 4.5 – 5.4, easy to read; and 5.5 – 6.4, very easy to read. All four results from each AI chatbot were evaluated.
2.5 Statistical analysis
Three types of statistical analyses were performed in this study. First, within each rating scale, a one-factor analysis of variance using Tukey’ s method was conducted to compare means, and a chi-square test or Fisher’ s exact probability test was used to compare proportions. Residual analysis, which is the posterior analysis of the x 2 test, is significantly different if the value is greater than 1.96 or less than �1.96. Second, the intraclass correlation coefficient was calculated to compute the agreement between evaluators. A significance level of 0.05 was used in this study. All analyses were performed using IBM SPSS( Ver. 25.0 for Windows; IBM SPSS Japan, Tokyo, Japan).
3 Results
The keyword search with Google Trends extracted 19 words from“ nuclear power plant,” 8 words from“ radioactivity” and“ radiation,” 6 words from“ nuclear power,” and 2 words from“ Chernobyl”( Tab. 1). Of these, the two words extracted from“ Chernobyl” were identical to two words extracted from“ nuclear power plant,” meaning that this keyword produced no useful results. The search results for“ map( Number of deleted keywords: 1),”“ dose of radiation( 2),”“ nuclear power, information( 1),” and“ Fukushima City, radiation( 1)” were excluded because their web pages contained only figures and numbers and almost no text. Keywords containing“ breaking news( 1)” were excluded because the web page was not currently displayed. Keywords indicating a specific company name were excluded( 1) because the web pages were mostly company introductions and publications. Ultimately, 34 words were extracted.
These 34 words were input to the three AI chatbots( ChatGPT-3.5, Copilot, and Gemini) to generate sentences four times for two reading levels( normal level and 6th grade level), and a total of 816 sentences were generated. A total of 48( 5.9 %) sentences contained obvious errors, 26 from ChatGPT-3.5( 3.2 %), 5 from Copilot( 0.6 %), and 17 from Gemini( 2.1 %), and these were excluded from the analysis. Examples of errors included a sentence in which“ Fukushima No. 2 Nuclear Power Plant” was mistaken for“ Unit 2 of Fukushima Daiichi Nuclear Power Plant,” a sentence in which information on“ Onagawa Nuclear Power Plant” was mixed with information on“ Fukushima Daiichi Nuclear Power
Table 1. The results of keyword searches by Google trend.
Nuclear Power Plant Nuclear Power Plant, Fukushima Nuclear Power Plant, Accident Fukushima, Daiichi( First), Nuclear Power Plant Fukushima, Prefecture, Nuclear Power Plant * Chernobyl, Nuclear Power Plant
Nuclear Power Plant, Evacuation Fukushima, Nuclear Power Plant, Accident * Chernobyl, Nuclear Power Plant, Accident
Fukushima, Second, Nuclear Power Plant Nuclear Power Plant, Explosion Fukushima, Daiichi( First), Nuclear Power Plant Three Mile Island, Nuclear Power Plant, Accident Onagawa, Nuclear Power Plant Fukushima, Nuclear Power Plant, Explosion Fukushima, Prefecture, Daiichi( First), Nuclear Power Plant Fukushima, Second, Nuclear Power Plant
Chernobyl
None *
Radioactivity Radioactivity Fukushima, Radioactivity Fukushima, Prefecture, Radioactivity Fukushima, City, Radioactivity Radioactivity, Measurements Radiation Radioactivity, Contamination Radioactivity, Leakage
Radiation Fukushima, Prefecture, Radiation Radiation, Measurement Radiation, Human Body, Effects Radiation, Medical, Laboratory Micro SvR( microsievert) Nuclear Power Nuclear, Power Generation Nuclear, Power Plant Fukushima, Daiichi( First), Nuclear, Power Generation Onagawa, Nuclear, Power Plant Three Mile Island, Nuclear, Power Plant, Accident
* Search results for“ Chernobyl” overlapped with two results for“ Nuclear Power Plant.”
Plant,” and a sentence in which“ radiation” and“ radioactivity” were confused in their descriptions. Therefore, a total of 768( 94.1 %) sentences obtained from 34 keywords were evaluated in this survey.
For the web search using Google, the top 8 web pages in search order for 34 keywords were extracted from the web pages that met the relevant criteria. Of the 272 web pages extracted, 177 pages were included in the analysis, excluding duplicate keywords. The intraclass correlation coefficient( ICC) was calculated to quantify the degree of agreement between items, with ICC = 0.66 for PEMAT-P’ s understandability and ICC = 0.40 for actionability.