ABSTRACT
Objective:
This study aimed to evaluate the quality of laparoscopic myomectomy videos on YouTube and WebSurg.
Material and Methods:
We searched using the keyword “laparoscopic myomectomy” on WebSurg and selected surgical interventions in the gynecology section. Eleven videos on WebSurg were enrolled. We selected the 22 most-relevant videos on YouTube to create a comparison group, with a ratio of 1:2. Sound in videos, number of subscribers, views, likes, and comments, number of days since videos were uploaded and durations of videos were recorded. View/day, like/view, like/subscriber, and view/subscriber ratios were calculated. The videos were evaluated with usefulness score (US), global quality scoring (GQS), modified discern score (mDS) and laparoscopic surgery video educational guidelines (LAP-VEGaS).
Results:
The view/day ratio was lower in WebSurg compared to YouTube [1.3 (1.9) vs. 7.5 (30.6), respectively; p=0.039]. No difference was found between WebSurg and YouTube in terms of US, GQS and mDS. On LAP-VEGaS assessment, WebSurg was found to be superior to YouTube in terms of intraoperative findings [2 (1-2) vs. 1 (0-2), p=0.001], additional materials [1 (0-2) vs. 1 (0-1), p=0.041], audio/written commentary [2 (2-2) vs. 2 (0-2), p=0.037], image quality [2 (2-2) vs. 2 (0-2), p=0.023], questions and total score [12 (11-13) vs. 10.5 (4-13), p=0.006]. The proportion of high-quality video was higher in WebSurg compared to YouTube, when the cut-off value of total score of 11 or 12 was used as 10 (100%) vs. 10 (50%), p=0.011 and 9 (90%) vs. 5 (25%), p=0.001, respectively.
Conclusion:
WebSurg was better compared to YouTube in terms of quality of laparoscopic myomectomy videos.
Introduction
Uterine fibroids are the most common benign gynecological tumors and almost three-quarters of women of reproductive age have fibroids (1). Although there are currently medical and radiological treatment options, surgery is still the most common treatment modality in the presence of appropriate indications. With the development of endoscopic surgery, laparoscopy has been frequently preferred for myomectomy. We speculate that although laparoscopic myomectomy is currently thought of as advanced gynecological surgery, it will probably become a gold-standard in the future with the increase in surgical experience.
During the coronavirus disease-2019 (COVID-19) pandemic, admission of patients was prohibited, except for emergencies, in most hospitals. Many elective surgeries had to be postponed and consequently residents’ opportunities to see and perform surgery was limited. There were studies in the literature showing that residents were not satisfied with education during the pandemic (2). Distance learning methods were introduced worldwide as a solution and many institutions continued their education online. As a result, the COVID-19 pandemic has shown us the importance and power of distance education (3).
YouTube is the most popular video sharing website with over one billion hours of content watched every day by people all around the world (4). Besides music, movies or reality shows, YouTube has a large amount of medical content and is an almost unlimited resource for both healthcare seekers and providers. However, YouTube does not have a standardized peer-review process, which may cause some problems. It is known that a lot of information obtained from the internet is not correct (5). Due to its widespread use, misinformation can be transferred to many people via YouTube. In medicine, this false information transfer has the potential to cause considerable harm.
WebSurg, which was founded in France, is an online university of Research Institute Against Cancers of the Digestive System (IRCAD) (6). WebSurg is a peer-reviewed, distance-learning platform that is freely available and provides information on the latest developments in endoscopic surgery. It would be expected that WebSurg would have more accurate medical information compared to YouTube, as it has content produced and evaluated by a professional team.
YouTube and WebSurg platforms have become popular sources for surgery-related information due to the ease of access to the internet and the belief that audio-visual media enhance the learning process. There are published studies comparing videos on YouTube and WebSurg platforms. In a study evaluating laparoscopic hysterectomy videos, WebSurg was found to be superior to YouTube (7). Similarly, WebSurg was found to be a better platform for laparoscopic gastrectomy videos in terms of educational quality (8). However, in another study on laparoscopic adrenalectomy, YouTube was found to be as useful as WebSurg (9). Therefore, there is no consensus about which of these two platforms is better when used for online learning.
In the literature, there is a notable gap in the existing evidence concerning direct comparison of laparoscopic myomectomy videos on platforms such as WebSurg and YouTube. Furthermore, it has been observed that the assessment of these videos is commonly conducted using a limited set of evaluation scales within published articles. This limitation in the range of assessment tools hinders a comprehensive understanding of the instructional value of these videos for surgical procedures. Addressing these gaps through comparative analysis and diversification of assessment methods could potentially enhance the educational efficacy of these resources and contribute to improved surgical outcomes. Therefore, the aim of the present study was to evaluate and compare the quality of laparoscopic myomectomy videos on YouTube and WebSurg using four different scales.
Material and Methods
In this comparative study, the keyword “laparoscopic myomectomy” was used to search on the WebSurg platform on November 13, 2022 and surgical interventions in the gynecology section were selected, yielding 11 videos from the WebSurg platform. A search with the same keywords was also done on YouTube. We selected the 22 most-relevant videos on YouTube to create a comparison group with a ratio of 1:2. Only surgical videos uploaded by surgeons were included in the study from both platforms. As we used publicly-available data, the ethical approval was not needed.
Two authors (S.C., F.A.) who were senior surgeons with experience in laparoscopy reviewed all videos. Sound in videos, number of subscribers, views, likes, and comments, number of days since videos were uploaded and durations of videos were recorded. The following ratios were calculated: view/day; like/view; like/subscriber; and view/subscriber.
The included videos were categorized into before and after 2019, because 10 steps to be followed in laparoscopic myomectomy were described by Fava et al. (10) in 2019. These steps are surgical preparation, ergonomics/material, preventive hemostasis, hysterotomy, enucleation, bipolar hemostasis, control of missing fibroids, suturing, morcellation/extraction and prevention of adhesions. All included videos were evaluated for the sequential implementation of these steps.
All videos were evaluated and scored by S.C. and F.A. When there was a disagreement, a common score was decided after discussion. The videos were evaluated with usefulness score (US), global quality scoring (GQS), modified discern score (mDS) and laparoscopic surgery video educational guidelines (LAP-VEGaS) video assessment tool. In US, the videos were given a score between 0 and 2 in terms of the presentation of cause, symptom, diagnosis, treatment, recovery and the final score ranged from 0 to 10 (11). The GQS is a five-point scale used to evaluate the overall quality of videos (12). Similarly, videos were scored as 0 (not mentioned) or 1 (mentioned) in terms of clarity, reliability, bias, referencing and uncertainty of content in mDS (13). In both scoring systems, the total score ranged from 0 to 5, and a higher score was associated with better quality. In LAP-VEGaS, there were nine questions about authors/institution information, presentation of the case, position of patient, ports, extraction site and team standardized step by step fashion, intraoperative findings, outcomes of the procedure, additional materials, audio/written commentary and image quality with scores ranging from 0 to 2 for each question (14). In the performance analysis of LAP-VEGaS video assessment tool to determine a quality video, it was suggested that a total score of 11 or higher can be used as a threshold with a sensitivity of 94% and specificity of 73%, while a total score of 12 or higher with a sensitivity of 84% and a specificity of 84%. In our study, total LAP-VEGaS scores were analyzed according to both thresholds.
Results
The characteristics of laparoscopic myomectomy videos on WebSurg and YouTube platforms are shown in Table 1. While 80% of WebSurg videos were shared before 2019, only 20% of YouTube videos were published before 2019 (p=0.004). Compared to WebSurg, YouTube videos had more comments [0 (1) vs. 17 (31), respectively; p=0.001]. WebSurg videos were older compared to YouTube videos [4044 (1912.8) vs. 828.5 (970), respectively; p=0.001]. The view/day ratio was lower on WebSurg compared to YouTube [1.3 (1.9) vs. 7.5 (30.6), respectively; p=0.039].
The comparison of US, GQS, mDS and LAP-VEGaS scores of laparoscopic myomectomy videos on WebSurg and YouTube is summarized in Table 2. No difference was found between WebSurg and YouTube in terms of US, GQS and mDS. On LAP-VEGaS assessment, WebSurg was found to be superior to YouTube in terms of intraoperative findings [2 (1-2) vs. 1 (0-2), p=0.001], additional materials [1 (0-2) vs. 1 (0-1), p=0.041], audio/written commentary [2 (2-2) vs. 2 (0-2), p=0.037], questions of image quality [2 (2-2) vs. 2 (0-2), p=0.023] and total score [12 (11-13) vs. 10.5 (4-13), p=0.006]. In addition, the proportion of high-quality video was higher in WebSurg compared to YouTube when the cut-off value of total score of 11 or 12 was used: 10 (100%) vs. 10 (50%), p=0.011 and 9 (90%) vs. 5 (25%), p=0.001, respectively.
Discussion
Our results showed that laparoscopic myomectomy videos on WebSurg were superior to videos on YouTube in terms of intraoperative findings, additional materials, commentary, questions of image quality and total score on LAP-VEGaS assessment. In addition, the percentage of high-quality video was higher in WebSurg compared to YouTube when different cut-off values were used. YouTube has been shown to be a more popular platform compared to WebSurg because the view/day rate of YouTube videos was almost 5.5 times higher than videos on WebSurg.
The COVID-19 pandemic changed many people’s behavior, including much more widespread use of online learning techniques. Accessing information through online platforms in order to ensure the continuity of training has been one of the biggest changes (15). Due to the ease of access, online platforms have changed the way surgical education was delivered during the pandemic. In addition, seeing important anatomical structures better, showing the important stages of the surgery by slowing down, and being able to watch the videos again when desired are additional advantages of online learning (8,11). In some studies it was shown that distance learning was at least as effective as traditional methods (16). It is obvious that online education is no substitute for training that requires hands-on practice. For this reason, it would be appropriate to use distance education together with classical education methods.
YouTube is the biggest video sharing website in the world. This platform has become a potential source to share health-related contents because it has billions of visitors every day. This situation has also attracted the attention of researchers. Different medical disciplines, such as urology, ophthalmology, orthopedics, endocrinology and radiology have been examining the role of YouTube as a source of medical information(17,18,19,20,21). Another of these disciplines is gynecology. Kaya et al. (11) reported that approximately 20% of the endometrioma surgery videos on YouTube were useful, although view ratio was high. In a study by Lee et al. (22), half of the hysterectomy videos on YouTube were found to be of low quality. In the literature, there is only one study evaluating uterine fibroids and myomectomy related videos shared online (23). This study found that the quality of YouTube videos, especially those not shared by health professionals, was low. In another study aimed at assessing the reliability and quality of YouTube videos discussing ovarian cysts, researchers identified 50 relevant videos and evaluated them using the discern score and GQS (24). The videos were divided into three categories based on scores: misleading/poor quality (54%), medium quality (18%), and useful/good quality (28%). Overall, the study highlights that YouTube videos related to ovarian cysts tend to be of low quality. Notably, videos produced by non-medical professionals attracted more attention despite their lower quality compared to those created by medical professionals. In the present study, at least half of YouTube videos were evaluated as low quality, regardless of the thresholds used in LAP-VEGaS video assessment tool. Therefore, it may be suggested that YouTube videos are still not sufficient in terms of surgical content. The lack of peer review process may lead to this situation amongst videos on YouTube.
WebSurg is a professional video sharing platform from IRCAD and allows surgery-related content produced by professionals to be shared with healthcare providers. It is reasonable to assume that WebSurg would have more accurate contents than YouTube. In a study by Anand et al. (25), all thoracoscopic lobectomy videos on WebSurg had high quality, however only three of ten of the most-viewed videos on YouTube had sufficient quality. In another study by Yuksel and Çulcu (8), laparoscopic gastrectomy videos on WebSurg were evaluated as superior to videos on YouTube. In the first WebSurg and YouTube comparison made in the field of gynecology, treatment score of US and position of patient, standardized step by step fashion, intraoperative findings, commentary and total scores of LAP-VEGaS were found to be better for laparoscopic hysterectomy videos on WebSurg (7). Similarly, the present study revealed that WebSurg was superior to YouTube, in terms of both total score and proportion of good quality videos when assessed by the LAP-VEGaS video assessment tool. In contrast, some studies have shown that YouTube was as good as WebSurg (9). In a study, the goal was to compare the quality, educational value, and source accuracy of laparoscopic adrenalectomy videos on two online platforms. The researchers selected the most viewed videos from YouTube using the keyword “laparoscopic adrenalectomy.” Novel scoring systems were employed to assess data quality, educational value, source accuracy, and technical quality. The study concluded that while WebSurg videos, often provided by academicians and subject to professional review, fell short of expected quality, selectively chosen YouTube content on laparoscopic adrenalectomy was nearly as accurate as the WebSurg content. However in this study, an unvalidated scale was used to evaluate the videos on both platforms. In the present study, the videos were evaluated with commonly used tools: US, GQS, mDS and LAP-VEGaS video assessment tool. These scales were carefully selected to address this limitation. US, GQS, and mDS are commonly used scales in the literature (11,26,27,28,29) while LAP-VEGaS, is not only more comprehensive than the others tools but has also undergone a validation study (14). In the present study the LAP-VEGaS video assessment tool was probably more successful in detecting the difference between the groups of videos than other simple scoring systems, probably due to its detailed questions. Therefore, it can be suggested that appropriate and validated scales should be used to determine the difference between videos hosted on the two platforms.
Conclusion
We found that WebSurg was superior to YouTube in terms of quality of laparoscopic myomectomy videos. This may have been due to the peer review process applied to videos on WebSurg. Creating a medical sub-category, supervised by health professionals, on YouTube may improve the quality and utility of medical content sharing on this extremely popular platform.
Statistical analysis
Statistical analysis was performed using SPSS, version 25 (IBM Corp., Armonk, NY, USA). The Kolmogorov-Smirnov and Shapiro-Wilk were used to test the normality of the distributions. None of the variables were normally distributed. Ordinal and continuous variables were compared using the Mann-Whitney U test. Pearson’s chi-square test or Fisher’s exact test were used to compare categorical variables. Median (interquartile range) for continuous values, median (minimum-maximum) for ordinal values and number (percentage) for categorical values are given. A p<0.05 was considered statistically significant.
Study limitations
There are some strengths and limitations in our study. This is the first study to compare laparoscopic myomectomy videos on the WebSurg and YouTube platforms. This is the main strength, as it addresses a novel question and contributes to the understanding of video content across these two major educational platforms. The use of four distinct scoring systems, which are commonly used and/or validated, further enhanced the objectivity and comprehensiveness of our assessment process.
However, as with any study, there are certain limitations that warrant consideration. Firstly, the limitation in the availability of laparoscopic myomectomy videos on the WebSurg platform posed a challenge in terms of sample size. This situation could potentially influence the diversity and representation of videos in our analysis, impacting the generalizability of our findings. Additionally, the undisclosed algorithm behind the “most relevant” filter employed by YouTube for video retrieval introduces an element of uncertainty. This opacity in the search mechanism might have inadvertently influenced the selection and inclusion of videos in our study, introducing an inherent bias that we acknowledge.