LEVEL 1 - REACTION
Level 1 may be best evaluated from two perspectives : the educator as well as the student ’ s perspective . Relative to conventional didactic presentations , attitudes toward virtual medical education are mixed . Benefits of virtual education included improvements in knowledge ( Geha & Dhaliwal , 2020 ) through promotion of active learning ( Dedeilia et al ., 2020 ), open access web-based resources ( Longhurst et al ., 2020 ), easy access to medical experts via social media ( Abi-Rafeh & Azzi , 2020 ; Hilburg et al ., 2020 ; Kaup et al ., 2020 ; Sleiwah et al ., 2020 ) and virtual mentoring ( Guadix et al ., 2020 ). Improved communication between instructors and students ( Geha & Dhaliwal , 2020 ; Lee et al ., 2020 ; Sleiwah et al ., 2020 ) which led to improved student engagement ( Marques da Silva , 2020 ) through Zoom as a virtual learning tool ( Hilburg et al ., 2020 ). The holistic approach supported students mentally ( Lee et al ., 2020 ), nurtured internal motivation ( Lee et al ., 2020 ; Soled et al ., 2020 ) and increased their resilience ( Kaup et al ., 2020 ; Sleiwah et al ., 2020 ) and students had better test performance ( Mohammed Sami Hamad et al ., 2020 ). Medical students identified virtual education flaws such as a reduced interaction between peers and tutors , tutors ' lack of understanding of individualized learning needs , and difficulties balancing practical skills and theoretical knowledge ( Kaur et al ., 2020 ). Some faculty felt that online teaching software was expensive and timeprohibitive ( Longhurst et al ., 2020 ; Machado et al ., 2020 ; Marques da Silva , 2020 ; Sahi et al ., 2020 ).
LEVEL 2 - LEARNING
As previously discussed , there is not enough outcome data available to measure learning via virtual teaching . Educators were challenged to design and deliver effective online learning instructional activities , while mostly lacking the pedagogical foundations needed ( Parchoma et al ., 2020 ) to teach online . Prior to the pandemic , preclinical teaching relied upon lectures and / or laboratory settings using active learning strategies , while clinical education relied on clinical rotations with a focus on hands-on teaching through direct patient interaction and a variety of health care practitioners including physicians , residents , nurses , and others . During the pandemic , clinical rotations were paused to ensure students ’ and patients ’ safety , and hence , virtual education emerged to maintain the continuity of medical education . With that said , becoming a physician is not only about acquiring clinical knowledge , understanding , and analyzing laboratory data , or graduating with distinction . There are elements of maturation , selfconfidence , evolving empathy and professionalism that are crucial to medical practice involving patient treatment . Such training relies on clinical experiences during the clinical years . The lack of hands-on experience and clinical exposure during COVID-19 eliminated the opportunity for students to ‘ learn by doing ’.
LEVEL 3 - BEHAVIOUR
At its core , medical education is based on demonstrating skill and competence in areas such as clinical application , leadership , teamwork , and communication . As discussed previously , the lack of practical ‘ hands on ’ experience in the learning phase may affect the practical application of core competencies . While online learning has likely encouraged students to become more self-directed learners ( Mukhtar et al ., 2020 ), a valuable attribute to be fostered in medical students in and of itself , there remains sparse data to support this assertion .
LEVEL 4 - RESULTS / LEVEL 5 - ROI
According to a sample cost analysis - online learning may result in significant cost-savings compared to traditional learning . From institutional perspectives , such savings are related to reduced instructor time , travel costs , and labor costs as well as reduced institutional infrastructure and its utilities . However , there is not yet sufficient data to measure ROI . Additionally , there is no data related to the effect of online learning on institutional metrics , including the number of publications or medical school rankings .
LEVEL 5 - SOCIETAL IMPACT
On a broader level , the impact of online learning on society , cannot yet be assessed as several years of observation and analysis will be necessary to draw meaningful conclusions . Future studies may focus on the effect and the quality of online education , taking specific criteria and factors into account , such as the conditions in a student ’ s home . For example , it is possible that having young children may affect the quality of that individual ’ s learning experience . The well-being of educators as well as of students and their families must be taken into consideration and assessed to establish a clear overall picture