Intérêt des outils pédagogiques d’aide à la perception spatiale dans l’enseignement des sciences de la santé : revue systématique

This is an Open Journal Systems article distributed under the terms of the Creative Commons Attribution License. (https://creativecommons.org/licenses/by-nc-nd/4.0) which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is cited.

Abstract

Background

The concept of spatial orientation is integral to health education. Students studying to be healthcare professionals use their visual intelligence to develop 3D mental models from 2D images, like X-rays, MRI, and CT scans, which exerts a heavy cognitive load on them. Innovative teaching tools and technologies are being developed to improve students’ learning experiences. However, the impact of these teaching modalities on spatial understanding is not often evaluated. This systematic review aims to investigate current literature to identify which teaching tools and techniques are intended to improve the 3D sense of students and how these tools impact learners’ spatial understanding.

Methods

The preferred reporting items for systematic reviews and meta-analysis (PRISMA) guidelines were followed for the systematic review. Four databases were searched with multiple search terms. The articles were screened based on inclusion and exclusion criteria and assessed for quality.

Results

Nineteen articles were eligible for our systematic review. Teaching tools focused on improving spatial concepts can be grouped into five categories. The review findings reveal that the experimental groups have performed equally well or significantly better in tests and tasks with access to the teaching tool than the control groups.

Conclusion

Our review investigated the current literature to identify and categorize teaching tools shown to improve spatial understanding in healthcare professionals. The teaching tools identified in our review showed improvement in measured, and perceived spatial intelligence. However, a wide variation exists among the teaching tools and assessment techniques. We also identified knowledge gaps and future research opportunities.

Résumé

Contexte

Le concept d’orientation spatiale fait partie intégrante de l’enseignement des professions de la santé. Les étudiants utilisent leur intelligence visuelle pour se représenter mentalement en 3D des images en 2D comme des radiographies, de l’IRM et des coupes tomodensitométriques, ce qui constitue une lourde charge cognitive. On développe actuellement des technologies et des outils pédagogiques innovants pour améliorer l’expérience d’apprentissage des étudiants. Cependant, l’impact de ces ressources pédagogiques sur la perception spatiale est rarement évalué. L’objectif de cette revue systématique de la littérature était de recenser les outils et techniques pédagogiques destinés à améliorer la perception 3D des apprenants et d’évaluer les effets de ces outils sur leur perception spatiale.

Méthodes

Suivant les lignes directrices PRISMA, nous avons consulté quatre bases de données avec des termes de recherche multiples, analysé les articles recensés en fonction de critères d’inclusion et d’exclusion, et évalué leur qualité.

Résultats

Dix-neuf articles correspondaient aux critères d’inclusion. Les outils pédagogiques axés sur l’amélioration de la perception spatiale peuvent être regroupés en cinq catégories. L’examen a révélé que les résultats obtenus par les groupes expérimentaux ayant utilisé l’outil pédagogique pour effectuer les tests et les tâches demandés sont aussi bons ou significativement meilleurs que les résultats obtenus par les groupes témoins.

Conclusion

Notre revue de la littérature visant à recenser et catégoriser les outils pédagogiques disponibles a montré que ces derniers améliorent la perception spatiale, notamment l’intelligence spatiale mesurée et perçue, des professionnels de la santé. Toutefois, il existe une grande variation entre les divers outils pédagogiques et techniques d’évaluation. Nous avons également relevé des lacunes dans nos connaissances et des pistes de recherche future.

Introduction

Spatial ability, also known as spatial intelligence, or visual intelligence, is defined as “the ability to generate, retain, retrieve, and transform well-structured visual images.” 1 Helping learners develop a mental model of an object and its interaction with its surrounding has always been the key to teaching many subjects that are foundational to health professional education, including anatomy and physiology. 2,3 Thus, spatial intelligence is an essential aspect of health professional education. The ability to build a mental model by orienting an object in one’s mind needs a conceptual arrangement of the elements of that object and the ability to determine the 3D orientation of the elements in relation to one’s body, 4 indicating the possibility of some people being better than others in this skill. Genes, hormones, gender, age, and environment can cause individual differences in spatial abilities, and this skill can be significantly improved by training. 4-7 Developing a clear understanding of the spatial relationship of the anatomical structure is a crucial part of medical, dental, and allied health professional education. 8,9 Students studying to be healthcare professionals use their visual intelligence to develop a three-dimensional mental model from two-dimensional images from tools including x-rays, magnetic resonance imaging (MRI), and computed tomography (CT). 9

Computer-based 3D models have more benefits than the traditional pedagogical approaches of teaching anatomy and can improve spatial understanding. 9 One study showed that when 3D models are presented to students at a fixed angle, there is a significant disadvantage to students with poor spatial abilities. 8 Another showed that when learners can control the rotation of the 3D computer model with a hand-held mouse, their spatial understanding is improved, suggesting that learner control as an essential key to successfully integrating complex spatial information. 10

Virtual and augmented reality (VR and AR) are two new technologies with the immense potential to improve learners’ spatial ability when used as a teaching tool. These technologies can either offer the experience of being present in the virtual world (VR) or incorporate a digital object in our natural world (AR). VR applications enable users to spatially visualize and interact with 3D objects in a virtual world, 11 a beneficial feature for medical, primarily surgical education. In recent years, VR, AR and other technology-infused teaching tools have been widely available in health professional education. However, the effectiveness of these teaching tools on different aspects of learning is not well documented, and thus no consensus exists regarding the advantages of these teaching tools. In this context, a careful evaluation of the current literature is needed to identify the evidence-based impact of these teaching tools on students’ learning.

From a theoretical perspective, teaching anatomy and physiology using conventional methods (e.g., 2D images, static bench-top models) places heavy cognitive loads on students to form mental reconstructions of complex structure and dynamic cellular events. 12,13 The Cognitive Load Theory suggests that learning becomes difficult when the brain’s limited resources are taxed due to the intrinsic and extrinsic loads of the subject matter. 14 An educator can play an active role in reducing the extrinsic load of the students. Many technology-infused teaching tools can provide additional visualization of the 3D anatomical structures, that instructors can use to help students reduce the extrinsic load of forming a mental model. Constructivist learning theory also supports the use of interactive and immersive tools. Constructivism states that learning occurs when the individual interacts with their environment. Integration of this interaction with learners’ existing knowledge allows them to construct new meaning and understanding. 15,16

Our systematic review aimed to investigate current literature to identify which tools or teaching pedagogy are designed to improve students understanding of 3D models and how these tools affect students’ spatial performance. Our specific research questions are as follows:

What tools are being used in health professions education improve spatial intelligence? How are the tools evaluated for the effectiveness of improving spatial intelligence? How effective are these tools in improving spatial intelligence?

Methods

We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines for our systematic review. 17 The Medical Education Research Study Quality Instrument (MERSQI) tool was used to assess the quality of the studies included in the review. 18

Search strategy

We searched four databases that were most relevant to our search questions: PubMed, CINAHL, Web of Science, and Education Research Complete. The search was performed in July 2021 and included all the articles published between 2011-2021. Because of the rapid evolution of VR and AR technologies, we limited our search to a 10-year bracket to capture the widespread application, and incorporation of VR, AR, and other high-tech teaching modalities in health professional education curriculum. We employed a free text strategy to search the databases using combinations of the following terms: “teaching tool,” “learning tool,” “spatial understanding,” “mental rotation,” “3D understanding.” The search parameters of the free text terms and the results of combining different search terms are listed in Table 1 .

Table 1

Details of search terms and search results

Search Parameters
Free text termPubMed search parameters
Teaching tool(“education”[MeSH Subheading] OR “education”[All Fields] OR “teaching”[All Fields] OR “teaching”[MeSH Terms] OR “teaches”[All Fields] OR “teach”[All Fields] OR “teachings”[All Fields] OR “teaching s”[All Fields]) AND “tool”[All Fields]
Learning tool(“learning”[MeSH Terms] OR “learning”[All Fields] OR “learn”[All Fields] OR “learned”[All Fields] OR “learning s”[All Fields] OR “learnings”[All Fields] OR “learns”[All Fields]) AND “tool”[All Fields]
Spatial understanding(“spatial”[All Fields] OR “spatialization”[All Fields] OR “spatializations”[All Fields] OR “spatialized”[All Fields] OR “spatially”[All Fields]) AND (“comprehension”[MeSH Terms] OR “comprehension”[All Fields] OR “understand”[All Fields] OR “understanding”[All Fields] OR “understands”[All Fields] OR “understandability”[All Fields] OR “understandable”[All Fields] OR “understandably”[All Fields] OR “understandings”[All Fields]
Mental rotation(“mental”[All Fields] OR “mentalities”[All Fields] OR “mentality”[All Fields] OR “mentalization”[MeSH Terms] OR “mentalization”[All Fields] OR “mentalizing”[All Fields] OR “mentalize”[All Fields] OR “mentalized”[All Fields] OR “mentally”[All Fields]) AND (“rotate”[All Fields] OR “rotated”[All Fields] OR “rotates”[All Fields] OR “rotating”[All Fields] OR “rotation”[MeSH Terms] OR “rotation”[All Fields] OR “rotations”[All Fields] OR “rotational”[All Fields] OR “rotator”[All Fields] OR “rotators”[All Fields])
3D understanding“3D”[All Fields] AND (“comprehension”[MeSH Terms] OR “comprehension”[All Fields] OR “understand”[All Fields] OR “understanding”[All Fields] OR “understands”[All Fields] OR “understandability”[All Fields] OR “understandable”[All Fields] OR “understandably”[All Fields] OR “understandings”[All Fields])
Search Results
Search TermsDatabaseNumber of PapersYear
Teaching tool AND Spatial understandingPubMed1012011-2021
Teaching tool AND Mental rotationPubMed252011-2021
Learning tool AND 3D understandingPubMed1092011-2021
Teaching tool AND Spatial understandingCINAHL102012-2020
Teaching tool AND Mental rotationCINAHL52011-2021
Learning tool AND 3D understandingCINAHL322011-2021
Teaching tool AND Spatial understandingWeb of Science1502011-2021
Teaching tool AND Mental rotationWeb of Science162011-2021
Learning tool AND 3D understandingWeb of Science2872011-2021
Teaching tool AND Spatial understandingEducation Research Complete452011-2021
Teaching tool AND Mental rotationEducation Research Complete152011-2021
Learning tool AND 3D understandingEducation Research Complete742011-2021
Total869

Inclusion and exclusion criteria

To answer our specific research question, we included studies that evaluated the spatial understanding of learners. According to our established inclusion-exclusion criteria ( Table 2 ), articles were screened in three steps: (i) Based on title only, (ii) Based on abstract, and (iii) Full article review. The study selection process is outlined in Figure 1 . Our final inclusion includes research articles that focused on assessing the effectiveness of a teaching tool, modality, or pedagogical approach designed to improve the spatial understanding of healthcare professional education. One author (NS) did the article screening (all three steps) and assessment of the study quality using the MERSQI tool. Two authors (NS and AC) sequentially contributed to the article review (Step iii). As Step iii was performed sequentially, no discrepancies occurred.

Table 2

Inclusion and exclusion criteria

InclusionExclusion
Language EnglishNon-English
Year of study Studies published between 2011-2021Studies published before 2011
Study focus Health professional educationNon- health professional education
Didactic/Surgery/SimulationTechnical drawing, mathematics, geography, physics, astronomy, engineering
Tools to improve educationTools to improve patient care, clinical practice.
Studies to assess spatial understanding of learnersWet lab studies, review articles
Only describing the teaching tool, without assessment.
Assessing engagement, academic performance, satisfaction.
Graduate and Undergraduate post-secondary educationK-12 education, graduate education
Study design AnyNone
Setting AnyNone
MERSQI score 5 or higherBelow 5

An external file that holds a picture, illustration, etc. Object name is CMEJ-14-070-g001.jpg

Flow diagram explaining the study selection process

Data extraction

We extracted data from the eligible studies focusing on study design, research participants, areas of interest, type of teaching tool used, assessment of the learners’ spatial understanding, and the study’s key outcomes. After data extraction from the literature to the chart (Appendix A, Table 4 ), the results relevant to the research questions were synthesized.

Table 3

Summary of teaching tools, learners, and the learners’ feedback

Teaching ToolsLearnersFindings
3D printed models
Models of acetabular fractures
(Awan et al., 2019) 23
Radiology residentsPost-test scores were higher for
students who received 3D models during lectures.
Models of congenital heart disease (Biglino et al., 2017) 27 Cardiac nurses80% of the students thought that 3D models helped in the appreciation of overall anatomy.
Model of the bladder
(Fan et al., 2019) 40
Surgery studentsStudents who used the model showed better accuracy in understanding the spatial structure of the bladder.
Model of Cleft lip and palate (CLP) (AlAi et al., 2018) 33 Surgery studentsThe use of 3D printed models significantly improved the mean test scores of students.
Model of synostosis
(Lane et al., 2020) 39
Surgery studentsThe survey showed that students using 3D printed models improved their understanding of the surgical approach.
3D reconstruction of 2D CT images
Pancreatic cancer patients
(Lin et al., 2020) 32
Surgery studentsStudents who learned with 3D images performed better in questions regarding tumor staging and surgery planning.
Patients from transabdominal pre-peritoneal repair surgery
(Wada et al.,2020) 31
Surgery studentsSurvey results showed that most students could understand the surgery anatomy by the 3D simulation extremely well.
CT of head and neck vascular anatomy
(Cui et al., 2017) 36
Medical studentsStudents who used the stereoscopic 3D models increased their ability to identify the head and neck vascular anatomy correctly.
CT from patients to generate head and neck anatomy
(Yao et al., 2014) 26
Surgery studentsThe survey suggests improvement in the perceived understanding of the anatomy after the 3D educational
module session.
CT from patients to generate head and neck anatomy
(Yeo et al., 2018) 28
Surgery studentsStudents with 3D models took a shorter time to complete the surgery plan.
VR Models
VR model of the human body
(Hu et al., 2020) 24
Ultrasonography studentsStudents who used the VR model showed significantly
higher scores in ultrasonography performance tests.
Interactive VR platform
(Kolla et al., 2020) 38
Medical studentsStudents responded in a survey that the VR platform improved their 3D understanding.
Second Life
(Morales-Vadillo et al., 2019) 35
Dentistry studentsStudents who used Second Life (SL) scored significantly higher in questions designed to test the spatial interpretation of the anatomical structure.
3D digital animation
Human anatomy
(Hoyek et al.,2014) 37
Undergraduate health science studentsThe students, who studied with 3D digital animation scored higher in questions that required spatial ability.
3D animated model of a pregnant horse (Gao et al., 200) 30 Veterinary studentsThe use of 3D animated model significantly increased the students’ final examination scores.
Others
3D tooth morphology quiz (TMQ) app
(Lone et al., 2019) 22
Dentistry StudentsNo difference in test scores between students who used, or did not use the app. Students with low MRT scores gradually performed better in tests after using the app.
Clay modeling
(Akle et al., 2018) 25
Neuroanatomy studentsQuiz scores of students who constructed the models were significantly higher than students who studied with traditional methods.
Digital surgical microscope to transmit a high-resolution 3D image
(Weiss et al., 2021) 29
Surgery students75% of the participants agreed that 3D visualization was more useful to identify anatomical structures.
3D Atlas of Human Embryology (Chekrouni et al., 2020) 34 Biomedical studentsThe class that used 3D Atlas showed significantly higher exam scores.

Assessment of study quality

We used the Medical Education Research Study Quality Instrument (MERSQI) tool to assess the quality of the studies included in our review. 18 MERSQI quantitatively evaluates quantitative studies in six domains: study design, sampling, type of data, validation of evaluation instrument, data analysis, outcomes measured. 18 This is a validated and widely used tool that scores an article between five (lowest quality) to 18 (highest quality). 19 The articles included in our review scored from seven to 15. The detail of the assessment of study quality is shown in Appendix A, Table 5 .

Results

Review flow

The initial search identified 869 articles from four databases (PubMed, CINAHL, Web of Science and Education Research Complete). The first screening removed 34 duplicates, keeping 835 articles for review by title only. Screening by title removed 669 articles that were either wet-lab research, unrelated to education, unrelated to health science, or involved in K-12 education. Screening by title retrieved 166 potential articles to be reviewed by abstract. Abstract inspection removed 117 pieces of literature primarily for not being related to health education or not assessing spatial understanding among learners. We had 49 articles selected for full-text evaluation, during which, 30 articles were removed primarily for not addressing our research questions, finally incorporating 19 studies in our systematic review. Details of the review flow are included in Figure 1 .

Features of the reviewed studies

Our review includes 19 research articles, 89% (n = 17) of which used quantitative research methods including, evaluating pre-test to post-test differences, scores in author-designed tests or formal academic examinations, or surveys based on Likert scales to collect data. Only 11% of our qualified articles (n = 2) followed a mixed-method study design, collecting both quantitative (test scores, survey) and qualitative (focus groups) data. Most of the studies (68 %, n = 13) were from medical schools, 10 % (n = 2) from dental schools, 5% each (n = 1) from nursing and veterinary schools. The remaining 10% (n = 2) were from other health professional programs (biomedical science and kinesiology). The 13 studies that concentrated on medical education focused on surgery (n = 8). A specific breakdown of areas of interest is shown in Figure 2 . As our selected articles used either quantitative or mixed-method studies, we used the MERSQI tool to assess the quality of our included articles. The mean MERSQI score of our studies was 11.9 (range: 7.0-15.0), which falls within the range identified as being acceptable. 13

An external file that holds a picture, illustration, etc. Object name is CMEJ-14-070-g002.jpg

Field of interest in literature included in the systematic review.

60% of the studies included in this review are focused on the learners from medical schools. The remaining includes dentistry, nursing, veterinary science, and other areas of health science education. Literature that broadly focused on medical education is further explored to specific filed of interest. Within medical education, 62% studies focused on learners from the division of surgery, and 15 % focused on anatomy. Other field of interest includes radiology, ultrasonography, and neuroanatomy.

Teaching tools and techniques used

Our systematic review identified teaching tools and pedagogical approaches designed to improve the spatial understanding in healthcare professional education. We have categorized these tools as i) 3D printed models, ii) 3D reconstruction of 2D CT images, iii) VR-based models, iv) 3D digital animations, and v) others that included tools that did not fit within any of our categories.

3D printed models. 3D printing is expected to revolutionize health care and health science education. 15 It is a manufacturing method in which any imaginable 3D shape can be created by depositing materials such as plastic or metal in layers. 20,21 To print a 3D object, users need to design and define the object’s structure in a computer-aided design (CAD) file. 20 Five studies included in our systematic review created 3D printed anatomical models for medical education (Appendix A, Table 4 , Table 5 ). 23,27,33,39,40

3D reconstruction of 2D CT images. Radiologists often find it challenging to diagnose based on axial CT images alone as they lack information on the third dimension (e.g., sagittal, and coronal dimensions). 41 The application of 3D computer rendering algorithms allowed the formation of images in the third dimension from data acquired in the axial plane, which is improved significantly by the recent development of multidetector CT scanners. 41 3D reconstructed CT images were used by five studies that are included in our review (Appendix A, Table 4 ; Table 5 ). 26,28,31,32,39 Wada et al. (2019), 31 for example, selected six patients who underwent laparoscopic transabdominal pre-peritoneal repair (TAPP) surgery. The preoperative CT images of those patients were used for 3D reconstruction and used as a teaching tool for surgery students. 31

Virtual Reality based models (VR models). Three studies included in our review used VR-based models as an educational tool. 24,35,38 A VR-enhanced ultrasonography training program was designed for medical students to teach ultrasonography using a virtual model of the human body. 24 For another human anatomy class of medical students, a VR-based teaching platform was created in which the students could interact with the virtual human model, identify anatomical structures on a computer-generated cadaveric specimen, and then draw these structures on a virtual skeleton using a 3D drawing tool. 38 The detail of the three VR-based teaching tools is found in Appendix A, Table 4 and Table 5 .

3D digital animation. Two studies used 3D digital animation to create a dynamic visualization platform of anatomical structure and physiological processes (Appendix A, Table 4 , and Table 5 ). 30,37 Hoyek et al. (2014) used 3D animations to teach human anatomy to undergraduate health science students. 37 Gao et al., 2020 studied the effectiveness of a 3D animated model of a pregnant horse to teach an obstetrics course to veterinary students. 30

Others. We identified some teaching tools that did not fit within the abovementioned categories. A 3D tooth morphology quiz is a computer application that allows students to study tooth morphology from multiple viewpoints and levels of magnification. Students can interact with the app and rotate the 3D tooth image to obtain a spatial understanding of the different surfaces of the tooth. 22

Akle et al. (2018) asked students to build a clay model of a specific anatomical structure of the brain. After the model building activity, students were asked to take a computer-based activity where they had to identify the brain’s structures in 2D images. 25

Weiss et al. (2020) used a digital surgical microscope to transmit a high-resolution 3D image as a teaching tool for the surgery students. The students observed the live surgical procedures with passive (polarized) 3D glasses in real-time. 29

The 3D Atlas of Human Embryology (3D Atlas) is a freely available online collection of interactive 3D computer files in Portable Document Format (PDF) representing developing human embryos. 34 These interactive digital 3D models of developing embryos can be rotated, manipulated in 3D space, and interactively viewed from all sides. 34

Assessment of spatial understanding

The studies included in our review evaluated learners’ spatial understanding in one of the following ways:

By evaluating test-scores, specially designed with questions that require 3D comprehension to answer By assessing the accuracy of a given task needing spatial intelligence to perform By correlating student performance and mental rotation test (MRT) By analyzing student perception

(i) By evaluating test-scores, specially designed with questions that require 3D comprehension to answer. Thirteen studies evaluated students’ test scores to evaluate their ability to answer the questions that require spatial intelligence. 22-26,32-37,39,40 To assess the teaching tool of interest, researchers compared pre-test and post-test scores or compared test scores between experimental and control groups (Appendix A, Table 4 ).

(ii) By assessing the accuracy of a given task needing spatial intelligence to perform. Two studies assessed teaching tools for surgery students by evaluating the accuracy and required time for surgery planning using that tool. 28, 40 Yeo et al. 2017 provided surgery students with 3D reconstructions of preoperative CT images. 28 Fan et al., 2019 assessed the accuracy of the surgical plan by groups of students who either used or did not use the 3D printed model as a teaching and learning tool. 40

(iii) By correlating student performance and mental rotation test (MRT). Two studies included in this review evaluated the correlation between learners’ Mental Rotations test (MRT) scores and their performance in tests. 22,36 All participants were allowed to take the MRT in one study, where male students scored significantly higher than female participants. In the first test (tooth morphology questions that require spatial ability to answer), male students outperformed female counterparts, indicating a positive correlation between the test performance and MRT scores. After using the computer application called the 3D tooth morphology quiz (TMQ) app for three weeks, the students took another test, where the difference between male and female participants disappeared. The authors concluded that the improved performance in female participants in assessments could result from training with the application (TMQ) and increased familiarity with the material. 22 Another study found that students with a low spatial ability (low score in MRT) improved their post-test scores to a level of students with a higher spatial ability (high score in MRT) after using the 3D model as a learning tool. 36

(iv) By analyzing student perception. Student perception of their 3D intelligence was another essential assessment tool used by several studies. Thirteen studies included in our review collected and analyzed student perception data either from surveys (quantitative data) or focus groups (qualitative data) (Appendix A, Table-4). 25-34,38-40

Effect of the teaching tools in spatial understanding

The research groups have taken several approaches to evaluate the effectiveness of the teaching tools in developing or improving learners’ spatial intelligence. Although the assessment tools varied widely among literature, the application of technology is shown to positively impact the overall spatial understanding of the students.

To assess the impact of 3D printed models, several evaluation methods were used, including the comparison between pre-test and post-test scores; 23 evaluation of the accuracy of a given task, 40 performance in test questions specially designed to assess learners spatial understanding, 33,39 and surveys to collect student perception of their learning. 40,39,33,27 Awan et al., 2019 23 reported that the post-test scores were higher for students who received 3D printed models during their lecture than the group that did not. Students who used 3D printed models as learning modality scored significantly higher 33 or similar 39 in tests compared to the control group, who did not have access to the 3D printed models. Fan et al. (2019) 40 divided surgery students into groups to limit their access to 3D printed models as an additional tool that can aid in surgical planning. The study found that the student group who could use both CT images and 3D printed models showed higher accuracy in their given task to plan the surgery. Survey results confirm that 3D models improved students’ visualization, understanding of spatial orientation, comprehension of anatomically complex structures, and overall learning experience. 27,33

Multiple studies in our review used computer-based 3D models reconstructed from the 2D CT images (Appendix A, Table 4 ). When used as a tool to aid in surgical planning, Yeo et al. (2017) and Lin et al. (2019) found that most surgery residents perceived that 3D models improved their confidence with the surgical plan. 28, 32 In a similar study by Wada et al. (2020), self-report survey results showed that reconstructed 3D simulations helped most surgery students to understand the surgery anatomy exceptionally well (40%) or very well (47%). 31 Yao et al. (2014) reported that the learners recognized that the addition of 3D imaging accelerated their understanding of sinus anatomy. 26 Students who learned with 3D images performed better in questions related to tumor staging and surgery planning 32 and increased their ability to correctly identify the head and neck vascular anatomy, 36 indicating an improvement in their 3D perception. Cui et al. (2017) 36 reported that 3D models improved the post-test scores of students with weaker spatial intelligence.

Questions designed to assess learners’ spatial understanding 35,24 and surveys to collect students’ perceptions of their 3D intelligence 38 were used to evaluate VR-based teaching tools. Hu et al. (2020) reported that the students in the experimental group who used a VR model of the human body scored significantly higher than the control group in ultrasonography performance tests, which requires spatial understanding. The finding of Hu and his colleagues indicates that the VR model of the human body facilitated 3D conceptualization and thus led to faster and better development of ultrasonographic competency. 24 Similarly, the dentistry students who used Second Life (SL), a VR-based interactive learning platform, scored significantly higher than the control group in questions designed to test the spatial interpretation of the anatomical structure. 35 When questioned about learners’ perception of their 3D understanding, Kolla et al. (2020) 38 found that 78.6% of the participants reported that VR was “much better” than lecture for learning 3D anatomical relationships.

Students who had access to 3D digital animations scored higher in questions requiring spatial ability. 37 Using a 3D animated model in a veterinary course significantly increased students’ final examination scores compared to the previous year’s cohort. A survey identified students’ agreement that the 3D animation tool improved their ability to understand the spatial orientation of the anatomical structure. 30

Lone et al., 2018 randomly divided students into two groups, one used the 3D tooth morphology quiz (TMQ) application, and the other group used extracted real teeth to study tooth morphology. The two groups showed no significant differences in their mean exam scores; however, a positive correlation was found between students’ performance on the tooth morphology quiz (TMQ), and scores on the final examination. In this study, male participants scored significantly higher than female participants in the mental rotation test (MRT) initially taken by the participants. Male participants also scored higher than the female counterparts in the first mock assessment, although this difference disappeared in later examinations. 22

The activity of making clay model improved the test scores of the students compared to the control group, who studied with traditional methods. Focus group discussions also revealed that the modeling activity reduced the students’ time spent studying the topic and increased their understanding of spatial relationships between structures in the brain. 25

Weiss et al., 2020 used a digital surgical microscope with the capability to transmit a high-resolution 3D image allowing students to observe the live surgical procedures with passive (polarized) 3D glasses in real-time. Survey results showed that 75% of the students agreed that 3D visualization of the surgical field was more helpful in identifying the anatomical topography and structures than 2D representation.

The application of the 3D Atlas, a collection of interactive 3D models of the human embryo, positively impacted students’ learning experience. The class that used the 3D Atlas showed significantly higher exam scores. Moreover, in the survey, 71% of the students agreed that the 3D atlas led to a better understanding of embryology. 34

A summary of the different groups of teaching tools, the learners, and the learner’s feedback is represented in Table 5 . Teaching tools from all the groups have positively impacted students’ learning and scores in tests requiring spatial understanding. Certain tools improve accuracy, confidence, and reduce surgery planning time for surgery students and residents. Survey and focus group data indicate student satisfaction and perceived improvement in spatial understanding.

Discussion

Our systematic review investigated current literature to identify teaching tools used in healthcare education to improve students’ spatial understanding. We grouped the educational tools into five major categories: 3D printed models; 3D reconstruction of 2D CT images; virtual reality-based models (VR models); 3D digital animation; and others. The wide variation among the teaching tools is evident, including the most cutting-edge VR models and simple techniques like clay modeling or animated videos. Technologies are rapidly being incorporated in health professional education. Our review identified the positive impact of more straightforward techniques on improving students’ spatial intelligence.

When the effectiveness of a teaching tool was evaluated by comparing between control and experimental group, studies reported that with access to the teaching tools, the experimental groups performed equally well or significantly better in tests and tasks than the control group. No negative impacts were reported. Although this improved or maintained academic performance is attractive, one should be cautious in drawing conclusions, as less than half (47.3%) of the included studies conducted randomized controlled trials. Many studies reporting positive impacts of a teaching tool conducted single-group post-test only or single-group cross-sectional studies ( Table 4 ). A small number of high-quality control groups and a wide variation of teaching tools limits recommendations for evidence-based teaching practice.

One of major weakness of the studies, included in this review is the absence of theory-driven application and evaluation of educational technologies. One of the purposes of educational theories is to predict learning outcomes and explain their underlying mechanisms. Our review reveals that theoretical considerations were omitted in most studies and future work must include clear and meaningful connections to educational principles and theories.

Most studies focused on measured outcomes like knowledge acquisition and skill development. Scores of specially designed tests or the accuracy of completing a task requiring spatial intelligence were measured to evaluate the effectiveness of a teaching tool. Several studies reported reaction level outcomes, reporting student perception of increased confidence, satisfaction, and 3D understanding after using these technologies. Only two studies in our systematic review correlated students’ performance with the mental rotation test (MRT). The teaching tools improved students’ performance in spatially complex questions. 22,36 The findings of Lone et al. (2018) 22 and Cui et al. (2017) 36 may be explained by Cognitive Load Theory. 14 Students with low spatial intelligence (low score in MRT) face difficulty forming 3D mental pictures from a given 2D image. The technology eased this difficulty, where students could rotate a computer-generated or 3D printed model, which reduced the extrinsic load of forming mental pictures for those students. Lone et al. (2018) also supported this idea by suggesting that the increasingly better performance of female participants (who had significantly lower MRT scores than the male students) could be a result of training and increased familiarity with the material. 22 Spatial intelligence is a skill that can be acquired and improved by practice. Improvement in test scores or perceived improvement in mental rotation skills does not always reflect the ability of a teaching tool to benefit students with a lower capacity for spatial rotation (low scores in MRT). This knowledge gap needs to be addressed in future research.

Spatial intelligence is an integral part of medical education and an essential surgical skill. The majority of the literature included in our review revolved around surgery ( Figure 2 ).

The most common surgical teaching tools included 3D reconstruction of 2D CT images and 3D printed models. 3D understanding is required by students from many other health professional disciplines like dentistry, veterinary medicine, and nursing. However, these groups are less represented and rarely included in studies. Future research should focus on evaluating the impact of these teaching tools on students from other health professional disciplines.

The quality of the included studies was evaluated using the MERSQI tool, which assesses an article based on study design, number of sample institutions, response rate, type of data, method of data analysis, outcome, and validity evidence for evaluation instrument ( Table 2 ). The average MERSQI score of the studies included in this review is 11.9, indicating good quality of the research and the legitimacy of the results reported. The quantitative analysis studies had a large sample size and thus reported statistically significant findings. However, none of the included studies had valid evidence for evaluation, suggesting the scope of further research in this area.

The teaching tools identified in our review show improvement in the learners’ spatial intelligence or perceived spatial intelligence. However, the wide variation among the assessment technique, study participants, and application areas makes direct comparison unrealistic between the teaching tools. Our findings also indicate that the application of teaching tools to improve spatial intelligence in medical education is a less explored area of educational research, and further studies are needed in this area.

Limitations

One major limitation of this study is that only one author (NS) did the article screening and assessment of the study quality using the MERSQI tool. However, two authors (NS and AC) contributed to the article review. We acknowledge that our findings are based on the few articles that met our inclusion criteria and are relevant to our research question. Readers must also be mindful that negative results may have been missed due to publication bias.

Conclusion

Our review identified teaching tools evaluated for their impact on spatial understanding in healthcare education. The application of technologies in creating models and modalities facilitates students’ learning. By creating visual, interactive models, these tools have reduced the cognitive load students experience when forming complex anatomical mental models. Thus, using these tools can free cognitive resources for the student to examine dynamic relationships between elements. Our study identifies and provides collective information on available teaching tools shown to improve spatial learning. Medical educators may find this information valuable for choosing an appropriate teaching tool for their students. We believe our review will also be a helpful guide for developing, evaluating, and incorporating these teaching technologies into existing curriculum to improve students’ spatial understanding.

Appendix A.

Table 4

Detail of the studies included in the systematic review

Author, Year, CountryResearch MethodResearch Aim / QuestionResearch ParticipantsAreaBrief Description
Of the study
Type of ToolAssessment to measure the visual-spatial abilityKey findings
Lone et al., 2019 22
Ireland
QuantitativeTo assess the effectiveness of the tooth morphology quiz (TMQ) as a teaching and learning tool.50 2 nd -year Bachelor of Dental Surgery (BDS) and 14 1st-year
diploma in dental hygiene (DDH) students, 5 incompletes (removed)
(n = 59)
Dentistry
Dental hygiene
The participants completed a mental
rotation test and a pre-study questionnaire at the beginning
of the study.
Participants were randomly
assigned into two study groups A and B. Group A first
accessed the TMQ app for three weeks to study tooth morphology. Students in Group B initially used extracted
teeth to study. At the
completion of the first block of three weeks, the participants
completed a mock examination (Mock 1).
After three-week, a post-test and a second mock
examination were conducted (Mock 2). Finally, the students completed the final
spot examination.
3D tooth morphology quiz (TMQ) app
(Computer-based)
Test scores
Mental Rotation Test
No significant differences are reported in mean exam scores between Groups A and B, but a comparison with the previous
cohort showed significantly improved results.
A positive correlation is found between student’s performance on the tooth morphology quiz (TMQ),
and scores on the final examination and mock assessment.
Male participants scored significantly higher than female
participants in the mental rotation test (MRT).
Male
participants also scored higher than the female counterparts in the first mock assessment, although this difference
disappeared in later assessments.
The increasingly better performance in female participants in mock 2 and final assessments could
be a result of training, and increased familiarity
with the material.
Awan et al, 2019 23
USA
QuantitativeExplored the benefit of 3D printed models in radiology resident training22 radiology residents
Medical school
Radiology22 radiology residents were randomly divided into two groups. Both groups received identical presentations.
Residents
in the experimental group received 3D printed models
with which to interact during the presentation, while the control group did not. Both groups received a
pre-test and a follow-up post-test three weeks later.
3D printed modelsTest questions
(Pre-test, post-test)
The post-test scores were significantly different between the experimental
and control groups.
The median of post-test scores was higher for
the group that received 3D models during their lecture compared to the group that did not.
Hu et al., 2020 24
Taiwan
QuantitativeTo assess the impact of VR anatomy instruction on the ultrasound
competency of novice learners
3 rd -year medical students
(n = 101)
UltrasonographyThe participants were randomly
divided into an experimental (VR) and a control group (traditional). Participants in the VR group used
VR as part of their training during the course. The participants in the control group took part in an ultrasound workshop
of a similar design. The VR anatomy component was replaced with a review session using a digital
atlas. At the end of the workshop, all participants were assessed using a standardized practical multi-station ultrasonography test.
3D VR model of human bodyTest QuestionsParticipants in the experimental group showed significantly
higher scores in ultrasonography performance tests than the control group. The experimental
group performed significantly better in six out of ten ultrasound tasks.
The authors suggest that the 3D VR model of the human body
facilitated the 3D conceptualization and thus led to faster and better development of ultrasonographic competency.
Akle et al, 2018 25
Colombia
MixedTo examine
how building a
3D-clay model affects learners’ understanding of periventricular
structures of the brain.
Undergraduate Medical students. (89 students in cohort
2013-II, and 62 students in cohort 2014-I).
(n =151)
NeuroanatomyIn the control group (2013 cohort),
the lecture was taught in one session using
2D images of the human brain and brain slabs in the
anatomy laboratory. After four days, students’ knowledge was assessed
with a 15-question quiz.
In the experimental group (2014 cohort), prior to the lecture, a
homework assignment, which asked students to
build a clay model of the periventricular
structures. After
the modeling activity, students were asked to take a
computer-based activity where they had to identify the
periventricular structures of the brain in 2D images. Students
took a knowledge quiz four days after
the lecture.
Both classes (2013 and 2014 cohorts)
were asked to complete a survey about their perception of their abilities and
difficulties in mentally rotating the structures.
Clay modelingQuiz
Student perception
(Focus group)
Quiz scores of students who constructed the models were
significantly higher than the control group, who studied with traditional methods.
Focus group discussion revealed that modeling activity reduced time spent studying the topic and
increased understanding of spatial relationships between structures in the brain.
The construction of 3D clay models in combination with autonomous learning
activities were a valuable and efficient learning tool in the anatomy course.
Yao et al, 2014 26
USA
QuantitativeTo validate the use of 3D reconstruction of computed tomography (CT) for the training of medical students4 th -year medical students
(n = 18)
Head
and neck surgery
Participants received
instruction of the sinus anatomy in two sessions, first through a 2D CT sinus scan review, followed by an educational module of the 3D reconstruction.
After each session, participants rated their knowledge of the sinus and adjacent structures on
a self-assessment questionnaire.
3D reconstruction of CT imagesSurvey
Test score
Student perception
Students’ self-Assessment of knowledge suggest
significant improvement in the perceived understanding of the anatomy after the 3D educational
module session.
Every student recognized that the addition of 3D imaging
accelerated their education of sinus anatomy.
Biglino et al., 2017 27
UK
MixedTo assess the feasibility of using 3D models of congenital heart disease (CHD) during a
training course for cardiac nurses
Cardiac nurses
(n = 100)).
Cardiology
Nursing
The 3D models were displayed on a table outside the lecture room, participants
were encouraged to access them throughout the five-day course.
At the end of the course, participants were asked to complete a
short questionnaire.
3D printed modelsSurvey
Student perception
Students found that 3D models helped in the appreciation of overall anatomy (86%),
spatial orientation (70%), and anatomical complexity after treatment (66%).
There was no statistically significant difference between adult and pediatric nurses’ responses.
Yeo et al, 2018 28
Canada
QuantitativeTo determine if 3D reconstruction of preoperative CT/MR images helps resident-level trainees in making appropriate surgery plans.14 Senior level surgical residents
Medical school
Surgery
Liver resection surgery
10 preoperative patients’ CT/MR images were selected and divided into either 2D or 3D groups. The 2D group consisted of raw CT/MR images without the radiologist’s report. The 3D group had digital reconstructions that can be rotated in all orientations to view the tumor(s).
The Residents were asked to evaluate the images from the 10 cases and write down the optimal surgical approach for each.
Their surgical plan and time (seconds) needed for planning was recorded. The residents completed a pre-study and post- study questionnaire regarding their level of training.
3D reconstruction of preoperative CT/MR imagesSurvey
Student perception
The resident from the 3D group took a shorter time to complete the surgery plan.
13 out of 14 residents found the 3D model was easier to use than the 2D.
Most residents recognized that the 3D model improved their confidence with the surgical plan.
Weiss et al, 2021 29
Germany
QuantitativeTo determine whether otorhinolaryngology trainees
gain additional comprehension of the anatomical structures and the surgical site when 3D visualization is used.
Medical students
(n = 112)
SurgeryTrainees observed the live surgical procedures
with passive (polarized) 3D glasses in real-time.
At the end of each course, participants were asked to complete
a questionnaire that included six questions concerning
the comprehension of the anatomy and the surgical
steps compared to 2D visualization.
Digital surgical microscope
with the capability
to transmit a high-resolution 3D image
Survey
Student perception
75% of the participants fully
agreed to the statement that 3D visualization
of the surgical field was more useful to identify the anatomical
topography and structures compared to 2D representation.
Gao et al, 2020 30
China
QuantitativeTo study the effectiveness of 3D animated model of a pregnant horse as a teaching tool for veterinary obstetrics course.Total 885 Veterinary students.
Exp group: 3 rd year, 300
Control group:
275 4 th year and 310 5 th year students.
(n =885)
VeterinaryAll students
received lectures with PowerPoint presentation. Only 3 rd year students studied with 3D animated model. The students from 4 th and 5 th year received traditional teaching. By
the end of term, all students were invited to participate in an
anonymous survey.
Final examination results were compared.
3D animation toolSurvey
Student perception
The use of 3D animated model significantly increased the 3rd year students’ final examination scores
compared to the 4th- and 5th- year students.
Students believed the 3D animation tool improved their ability to understand the presentation position and posture of the fetus.
Wada et al, 2020 31
Japan
QuantitativeTo investigate the usefulness of the 3D imaging technique in laparoscopic TAPP as an educational tool for medical studentsMedical students
(n = 30)
SurgeryMost students
previously studied the surgical anatomy from the textbook, and preoperative CT images. In this study, students were provided with the 3D reconstruction of preoperative computed tomography (CT) images from 6 patients who underwent laparoscopic transabdominal pre-peritoneal repair (TAPP) for inguinal hernia.
Students were asked to complete a survey.
3D reconstruction of preoperative imagesSurvey
Student perception
Survey results showed that most students could understand the surgery
anatomy by the 3D simulation extremely well (40%) or very well (47%) and agreed on the usefulness of this procedure for
learning anatomy.
Lin et al., 2020 32
China
QuantitativeTo assess the effectiveness of 3D visualized model as a teaching tool in
tumor evaluation and surgery planning.
Medical students
(n = 88)
SurgerySurgical residents were randomly divided into two groups (computed tomography, CT) group and 3D group).
The groups
learned a sample case either on 3D reconstruction
visualization tables or CT images. At the end of the course, both groups completed the same test consisting of two pancreatic cases with CT images
as well as questionnaires.
3D reconstructions, created by 3D multitouch
visualization table (MVT).
Imaging test and Questions
Student perception
Students who learned with 3D image performed better in certain questions. For example, the mean scores for questions regarding
tumor staging and surgery planning, were consistently and significantly higher in the 3D group.
Participants in 3D
group agreed that 3D technology was more beneficial in understanding and making surgery planning.
AlAi et al, 2018 33
UK
Kuwait
QuantitativeTo investigate the use of 3D-printed models in educational seminars compared to the traditional approach.Medical students
(n = 67)
Seminar on Cleft Lip and Palate
(Department of Surgery)
Participants were randomized into 2 groups. Group 1 (control group) attended a seminar with a PowerPoint presentation. Group 2 (Experimental group) attended a seminar with the same PowerPoint presentation, but with a physical demonstration using 3D-printed models.
Knowledge was compared between the groups using a multiple-choice question test before and after the teaching intervention. A survey was done to collect student perception.
3D printed modelsTest Questions
Student perception
Use of 3D printed models as an additional teaching tool significantly improved the mean test scores
Survey results showed that students felt
the 3D-printed models significantly improved their learning
experiences and visualizations.
Chekrouni et al, 2020 34
Netherlands
QuantitativeTo evaluate if the use of the 3D atlas as an additional teaching tool can improve students’ learning experiences.1st-year biomedical students (n = 91)EmbryologyThe 3D atlas was introduced and integrated in lectures and practical classes of an existing embryology course.
The cohort included both new students and repeaters. The test scores were compared between this class and the previous cohort, who did not learn with 3D Atlas.
3D Atlas of Human Embryology (3D Atlas)Test scores,
Survey
Student perception
The class that used 3D Atlas showed significantly higher exam scores for both new students and repeaters.
In the survey, 71% of the students agreed that the 3D atlas led to a better understanding of embryology.
Morales-Vadillo et al, 2019 35
Peru
QuantitativeTo
compare the effectiveness of using Second Life (SL) to the traditional
teaching methods
3rd year Dentistry students
(n = 62)
DentistryStudents were divided into control and experimental group. Students in the experimental group received training on how to use Second Life (SL) software. The control group did not have SL experience. Pre-test and post-test scores were compared between groups.Second Life (SL)
Virtual Reality based tool
Test scores,
Questions designed to test spatial interpretation of anatomical structure.
Students who used Second Life (SL) scored significantly higher than the control group in questions designed to test the spatial interpretation of the anatomical structure.
Cui et al, 2017 36
USA
QuantitativeTo explore if 3D stereoscopic
models created from computed tomographic angiography (CTA) data were effective
teaching tools for the head and neck vascular anatomy.
1st year medical students (n = 39)Head and neck vascular anatomyStudents were randomized
into either a 2D or a 3D learning session. 2D group used 2D images, snapshots and radiographic images.
Students in the 3D learning session learned using 3D stereoscopic models, which they could manipulate and rotate to any angles for visualizations.
Students from both groups were assessed using pre and post knowledge test, mental rotation test and satisfaction survey.
3D stereoscopic modelTest scores,
Mental Rotation Test
Survey
Students who used the stereoscopic
3D models increased their ability to correctly
identify the head and neck vascular anatomy.
Students with low spatial
ability (low score in mental rotation test), use of 3D models in the 3D learning sessions improved their post test scores to a level of students who had
high-spatial ability. This result indicates that the
use of 3D stereoscopic models can be valuable to students with low spatial
abilities.
Hoyek et al, 2014 37
France
QuantitativeTo evaluate the
effectiveness of 3D digital animation as a teaching tool by comparing two groups from two different academic years.
Undergraduate students of kinesiology
(n = 391)
Human anatomyThe study included two groups of students from two different years. The 2D group got lessons with power point embedded 2D images. The experimental 3D group received lesson with power point and 3D digital animation. Formal exam scores were compared between groups3D digital animationTest scoresThe students from 3D group, who studied with 3D digital animation scored higher in questions that required spatial ability.
Kolla et al, 2020 38
USA
QuantitativeTo assess the usefulness of VR in teaching anatomy.1st year medical students
(n = 28)
Human anatomyParticipants took part in a VR
anatomy training exercise during the 2nd and 3rd weeks
of a 4-week anatomy course on the musculoskeletal system. At the end, students were asked to complete a survey.
Virtual Reality (VR) anatomy platformSurvey
Student perception
In response to question related to 3D understanding, (78.6%) of the participants reported
VR to be “much better” and (17.9%) of the participants
reported VR to be “somewhat better” than lecture for learning
3D anatomical relationships.
Lane et al, 2020 39
USA
QuantitativeTo investigate the educational values of 3D printed craniofacial pathology models.2nd year medical students
(n = 44)
SurgeryStudents were randomly assigned to a control
group or model group. Both groups received power point presentations. Besides power point, the experimental model group was
also provided with 3D-printed models
created using patient-specific preoperative
computed tomography data. A survey using the Likert scale
evaluated participants’ learning experience. Pre- and post-module
scores on a quiz were recorded.
3D printed modelsTest score
Survey
Student perception
The survey showed that students in the experimental group using 3D printed model scored higher in understanding of the anatomy
and visualization of the pathology,
gaining an improved
understanding of surgical approach.
The
mean pre- and post-module quiz scores between groups were similar.

Table 5

Quality assessment using MERSQI

MERSQI Score breakdownTotal MERSQI Score
Author, YearStudy designSampling
institution
Response
rate: ≥ 75%
Type of
Data
Validity evidence for evaluation instrument scoresData analysis sophisticationData analysis appropriateOutcome
Lone et al., 2019 22 Randomized cross-over trial (3)1 (0.5)(1.5)Objective (3)NABeyond descriptive analysis (2)Appropriate for study: 1Knowledge, skills (1.5)12.5
Awan et al., 2019 23 Randomized controlled trial (3)1 (0.5)(1.5)Objective (3)NABeyond descriptive analysis (2)Appropriate for study: 1Knowledge, skills (1.5)12.5
Hu et al., 2020 24 Randomized controlled trial (3)1 (0.5)(1.5)Objective (3)NABeyond descriptive analysis (2)Appropriate for study: 1Knowledge, skills (1.5)12.5
Akle et al., 2018 25 Nonrandomized, 2 groups (2)1 (0.5)(1.5)Objective and assessment by participants (4)NABeyond descriptive analysis (2)Appropriate for study: 1Knowledge, skills, satisfaction (2.5)13.5
Yao et al., 2014 26 Single group, post-test only (1)1 (0.5)(1.5)Objective and assessment by participants (4)NABeyond descriptive analysis (2)Appropriate for study: 1Knowledge, skills, satisfaction (2.5)12.5
Biglino et al., 2017 27 Single group, cross-sectional (1)1 (0.5)(1.5)Assessment by study participants (1)NABeyond descriptive analysis (2)Appropriate for study: 1Perception, satisfaction (1)8
Yeo et al., 2018 28 Nonrandomized, cohort study (2)1 (0.5)(1.5)Objective and assessment by participants (4)NABeyond descriptive analysis (2)Appropriate for study: 1Knowledge, perception (2.5)13.5
Weiss et al., 2021 29 Single group, post-test only (1)1 (0.5)(1.5)Assessment by study participants (1)NADescriptive analysis only (1)Appropriate for study: 1Perception, satisfaction (1)7
Gao et al., 2020 30 Nonrandomized, 2 groups (2)1 (0.5)(1.5)Assessment by study participants (1)NABeyond descriptive analysis (2)Appropriate for study: 1Perception, satisfaction (1)9
Wada et al., 2020 31 Single group, cross-sectional (1)1 (0.5)(1.5)Assessment by study participants (1)NADescriptive analysis only (1)Appropriate for study: 1Perception, satisfaction (1)7
Lin et al., 2020 32 Randomized, controlled (3)1 (0.5)(1.5)Objective and assessment by participants (4)NABeyond descriptive analysis (2)Appropriate for study: 1Knowledge, perception (2.5)14.5
AlAi et al., 2018 33 Randomized, controlled (3)2 (1)(1.5)Objective and assessment by participants (4)NABeyond descriptive analysis (2)Appropriate for study: 1Knowledge, perception (2.5)15
Chekrouni et al., 2020 34 Nonrandomized, 2 groups (2)1 (0.5)(1.5)Objective and assessment by participants (4)NABeyond descriptive analysis (2)Appropriate for study: 1Knowledge, perception (2.5)13.5
Morales-Vadillo et al., 2019 35 Randomized controlled (3)1 (0.5)(1.5)Objective (3)NABeyond descriptive analysis (2)Appropriate for study: 1Knowledge, skills (1.5)12.5
Cui et al., 2017 36 Randomized, controlled (3)1 (0.5)(1.5)Objective and assessment by participants (4)NABeyond descriptive analysis (2)Appropriate for study: 1Knowledge, perception (2.5)14.5
Hoyek et al., 2014 37 Nonrandomized, 2 groups (2)1 (0.5)(1.5)Objective and assessment by participants (4)NABeyond descriptive analysis (2)Appropriate for study: 1Knowledge, perception (2.5)13.5
Kolla et al., 2020 38 Single group, cross-sectional (1)1 (0.5)(1.5)Assessment by study participants (1)NADescriptive analysis only (1)Appropriate for study: 1Perception, satisfaction (1)7
Lane et al., 2020 39 Randomized, controlled (3)1 (0.5)(1.5)Objective and assessment by participants (4)NABeyond descriptive analysis (2)Appropriate for study: 1Knowledge, perception (2.5)14.5
Fan et al., 2019 40 Randomized, controlled (3)1 (0.5)(1.5)Objective and assessment by participants (4)NABeyond descriptive analysis (2)Appropriate for study: 1Knowledge, perception (2.5)14.5

Funding Statement

Funding: No funding.