216 S. Ito et al.: Radioprotection 2025, 60( 3), 211 – 220
Table 4. PEMAT-P item scores among artificial intelligence chatbot and web search. Artificial Intelligence Chatbot Web Search
Normal level 6th Grade Level ChatGPT-3.5 Copilot Gemini ChatGPT-3.5 Copilot Gemini Google n % AR n % AR n % AR n % AR n % AR n % AR n % AR
Understandability
Content 1 The material makes its purpose completely evident 2 The material does not include information or content that distracts from its purpose
Word Choice & Style 3 The material uses common, everyday language 4 Medical terms are defined when they are used
Use of Numbers 5 Numbers appearing in the material are clear and easy to understand a
30 100.0 2.7 32 100.0 2.7 29 100.0 2.6 29 100.0 2.6 30 96.8 2.2 26 92.9 1.5 117 66.1 �8.0 30 100.0 1.2 32 100.0 1.2 29 100.0 1.2 29 100.0 1.2 28 90.3 �1.6 25 89.3 �1.8 168 94.9 �0.8
14 46.7 �1.0 20 62.5 0.9 19 65.5 1.2 25 86.2 3.5 24 77.4 2.6 27 96.4 4.6 67 37.9 �6.5 14 46.7 �0.9 16 50.0 �0.5 16 55.2 0.1 20 69.0 1.6 25 80.6 3.1 27 96.4 4.6 76 42.9 �4.4
0 0.0 – 2 20.0 �1.0 1 20.0 �0.7 1 100.0 1.4 5 35.7 0.1 1 50.0 0.5 42 34.7 0.4
6 The material does not expect the |
30 |
100.0 |
0.9 |
32 |
100.0 |
0.9 |
29 |
100.0 |
0.9 |
29 |
100.0 |
0.9 |
31 |
100.0 |
0.9 |
28 |
100.0 |
0.8 |
169 |
95.5 |
�2.9 |
user to perform calculations a |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Organization 7 The material breaks or“ chunks” information into short section 8 The material ' s sections have informative headers 9 The material presents information in a logical sequence 10 The material provides a summary Layout & Design
11 The material uses visual cues to draw attention to key points
Use of Visual Aids
12 The material uses visual aids whenever they could make content more easily understood
13 The material ' s visual aids reinforce rather than distract from the content
– – – – – – 66 56.9 0.0 – – – – – – 81 69.2 0.0
30 100.0 0.5 32 100.0 0.5 29 100.0 0.5 29 100.0 0.5 30 96.8 �1.5 28 100.0 0.5 175 98.9 �0.6 – – – – – – 18 15.4 0.0
5 16.7 �3.3 23 71.9 3.1 29 100.0 6.1 2 6.9 �4.4 25 80.6 4.1 28 100.0 6.0 51 28.8 �6.4
0 0.0 �3.5 0 0.0 �3.6 0 0.0 �3.5 0 0.0 �3.5 0 0.0 �3.6 0 0.0 �3.4 97 55.4 11.7
– – – – – – 110 77.5 –
– – – – – – 95 66.4 –