S. Ito et al.: Radioprotection 2025, 60( 1), 109 – 119 111
The approximate reading time was calculated based on the amount of text in the material and the average Japanese reading speed of 653 words / minute( Kobayashi and Kawashima, 2018). In addition, a list of place names and addresses of evacuation centers was excluded from the jReadability evaluation. The reason for this was to prevent the difficulty rating of the sentences from becoming too high due to many difficult place names.
2.5 Credibility( JAMA benchmark criteria and GQS)
To evaluate the credibility of web pages, we utilized the Journal of the American Medical Association( JAMA) Benchmark Criteria( Silberg et al., 1997), which consist of authorship( clearly identifiable author and contributors with affiliations and relevant credentials present), attribution( references and sources clearly listed with any copyright information disclosed), currency( clearly identifiable posting date of any content as well as date of any revisions), and disclosure( website ownership clearly disclosed along with any sponsorship, advertising, underwriting, and financial support). Each criterion was assessed using a binary scale of 1 for“ met” and 0 for“ not met”. We also used the Global Quality Scale( GQS)( Bernard et al., 2007), which is a comprehensive indicator for assessing the quality of online materials. To stratify the quality of the materials, the GQS employs the following five-point rating system:( 1) poor quality, poor flow of the site, most information missing, not at all useful for patients;( 2) generally poor quality and poor flow, some information listed but many important topics missing, of very limited use to patients;( 3) moderate quality, suboptimal flow, some important information adequately discussed but other topics poorly discussed, somewhat useful for patients;( 4) good quality and generally good flow, most relevant information listed, but some topics not covered, useful for patients; and( 5) excellent quality and excellent flow, very useful for patients).
2.6 Statistical analysis
Three types of statistical analyses were performed in this study. First, within each rating scale, a one-factor analysis of variance using Tukey’ s method was conducted to compare means, a chi-square test or Fisher’ s exact probability test was used to compare proportions, and a Kruskal-Wallis test was used to compare non-normal distributions. Second, Spearman’ s correlation coefficient was calculated to examine the association between the year the sources were updated and each rating scale. Finally, the intraclass correlation coefficient was calculated to compute the agreement between evaluators. A significance level of 0.05 was used for these statistical analyses. In the residual analysis, a significant difference was considered to have occurred when the adjusted residual was greater than or less than 1.96. All analyses were performed using IBM SPSS( Ver. 25.0 for Windows; IBM SPSS Japan, Tokyo, Japan).
3 Results
In this study, 116 materials were included in the evaluation. The breakdown is as follows. First, of the 219 websites found in the keyword search, 160 were excluded based on exclusion criteria( 142 with no materials and 18 with duplicates), leaving 59 websites. Of these, 66 materials were extracted from the 59 websites because a single website contained multiple materials, e. g., one for the general public and one for elementary school students. Next, from the list of 35 Reconstruction Agency websites, 12 with different themes were excluded, leaving 23; from the list of 57 Nuclear Human Resource Development Network websites, 54 with different themes were excluded, leaving 3. Finally, 24 materials were added by manual searching. The intraclass correlation coefficient( ICC) was calculated to quantify the degree of agreement between items, with ICC = 0.52 for PEMAT-P’ s understandability and ICC = 0.78 for actionability.
The material attributes and the rating scale scores are listed in Table 1. The source of the material was the municipality in 54 cases( 46.6 %), the topic of the material was nuclear disaster prevention in 76 cases( 65.5 %), the target audience for the material was general adults in 102 cases( 87.9 %), and the year the material was last updated was 2023 in 21 cases( 18.1 %). The number of materials scoring 70 or higher on the PEMAT-P was 71( 61.2 %) for understandability and 57( 49.1 %) for actionability. Regarding the difficulty level of the texts by jReadability,“ Difficult” was the most frequent response with 82 responses( 70.7 %). We calculated the Spearman’ s correlation coefficient between the year the material was updated and each rating scale, and found that the year the material was updated correlated significantly with understandability( rho = 0.21, p = 0.046), GQS( rho = 0.33, p = 0.001), and total number of letters( rho = 0.32, p = 0.002) on the PEMAT-P. No significant differences were found among the other items. 3.1 Material themes
The themes of the materials were nuclear disaster prevention in 76 cases and basic knowledge of radiation in 40 cases( Tab. 2). More than 80 % of the nuclear disaster prevention materials described indoor and outdoor evacuation methods, means of information transmission, and the definition of a nuclear disaster. In contrast, less than 60 % of the materials described the criteria for evacuation and inspection at the time of evacuation. More than 80 % of the materials on basic knowledge of radiation explained how to measure radiation, food, and dose, and 45.0 % explained the effects of radiation on children.
3.2 Understandability and actionability
Regarding the results of group comparisons of the PEMAT- P’ s understandability and actionability scale scores, Table 3 and Figure 1 summarize the results. For the understandability scale score, in the comparison by source, municipalities( M = 76.7, SD = 13.9) and prefectures( M = 79.4, SD = 15.8) were significantly higher than Academic Organizations( M = 53.3, SD = 18.5). Additionally, in the comparison by source, municipalities( M = 75.2, SD = 23.2) and prefectures( M = 82.1, SD = 16.2) were significantly higher than the other groups. Furthermore, the percentage of materials scoring 70 or higher on the PEMAT-P is discussed. With respect to actionability, a larger proportion of municipalities( n = 34, 63.0 %) and prefectures( n = 21, 77.8 %) achieved a score of 70 or higher.