 |
|
||||
|
|||||
| About | Latest Articles | Back-Issues | Articles
|
Search | Instructions | Online Submission | Subscribe | Etcetera | Contact |
| |||||||||||||||||||||
|
A comparison of the quality of the information available on the internet on interventional radiology, vascular surgery, and cardiology A Alsafi1, G Kaya2, H Patel3, MS Hamady11 Department of Interventional Radiology, Imperial College, St. Mary's Hospital, Praed Street, London W2 1NY, United Kingdom 2 Department of Medicine, The Hillingdon Hospital, Pield Heath Road, Uxbridge, Middlesex UB8 3NN, United Kingdom 3 Department of Surgery, Charing Cross Hospital, Fulham Palace Road, Hammersmith, London W6 8RF, United Kingdom
Correspondence Address: Source of Support: None, Conflict of Interest: None DOI: 10.4103/0022-3859.109509
Context and Aims: Internet use is rapidly expanding and increasingly plays a substantial role in patient education. We sought to evaluate and compare the quality of information available to patients online on three closely linked specialties: Interventional radiology (IR), cardiology, and vascular surgery. Materials and Methods: We searched the leading three search engines for the terms: "Interventional Radiology", "cardiology," and "vascular surgery," collating the top 50 hits from each search. After excluding duplicates and irrelevant sites, 43, 25, and 36 sites remained, respectively. Sites were analyzed using the LIDA instrument (an online tool for assessing health-related websites) and Fleisch Reading Ease Scores (FRES) were compared across the different search terms and correlated with the country of origin and certification by the Health on the Net (HON) Foundation. Results: There was no significant difference ( P>0.05) in the total LIDA, accessibility, usability or reliability scores between the three specialties. HONCode certification was associated with higher LIDA (83.1±1.6 vs. 71.53±0.8 ( P<0.0001)), reliability (75.7±3.6 vs. 49.0±1.6 ( P<0.0001)) and FRES (37.4±4.0 vs. 29.7±1.4 ( P=0.0441)). Conclusion: Websites are generally well designed and easy to use; the majority however, lacks currency and reliability. Despite similarity in quality of online information, there is a disparity in knowledge of IR; this may be due to low web-traffic figures of IR sites. Wikipedia's user-generated content, ranks highly in major search engines, as such; this could serve as means of disseminating reliable health information to patients. Keywords: Cardiology, internet reliability, interventional radiology, patient education, vascular surgery
Half of the adults in UK and up to 75% of the adults in USA accessed the Internet to source healthcare-related information in 2011, [1] with 4.5% of global Internet searches being health-related. [2] It is not uncommon to 'Google' a symptom or illness before or after visiting one's doctor. [3],[4] Patients often take the information gathered online to consultations and request a test or treatment based on these. More worryingly however, patients often use the Internet to diagnose or treat themselves and decide whether to seek medical attention. This may have devastating consequences, [4],[5],[6] particularly as the quality of this information is immensely variable. [4],[7],[8],[9] Although some websites maintain high standards, most of them are unreliable, sometimes misleading and potentially harmful. [4],[7],[8],[9],[10] As such it is important for clinicians to be aware of the quality of information available on the Internet and direct their patients toward reliable resources. Using a search engine is the most popular method of obtaining information online, [11] with Google, [12] Yahoo, [13] and Bing [14] being the market leaders in search engine technology, together fielding more than 93% of worldwide searches. [15] The influence of these search engines continues to grow with an increasing number of UK households having an Internet connection (75%), doubling of wireless hotspot use in 2011 and the expansion of Internet access from mobile devices. [16] Various approaches for assessing the quality of healthcare information on the Internet exist. [17] The lack of an industry standard however, poses many challenges to patients and health-professionals. The variety of websites aimed at different audiences makes the development of a single tool unrealistic and perhaps undesirable, but they do serve as a mean of standardizing assessment, if sites are to be compared. Previous studies used the LIDA assessment tool, [18] demonstrating that health-related websites are usually well designed and easy to use, albeit unreliable. The highest quality sites often did not rank highly in the most popular search engines, which is concerning. [7],[8] To ensure high quality, a variety of accreditation systems have been developed, where accreditation is awarded to websites adhering to pre-set standards. The Health on the Net (HON) Foundation has developed one such accreditation system: HONCode. [19] What is new in this study? To date, all studies using the LIDA instrument have searched the World Wide Web for a procedure or disease process as surrogates for a specialty. [7],[8],[9],[10],[20],[21] In this study, we have searched directly for information on specific specialties. Furthermore, we are the first to compare the information available on the Internet on different specialties, on the premise that the more established specialties such as cardiology would have better designed, more reliable websites compared with vascular surgery and interventional radiology (IR).
On 3 rd December 2011, we used Google, Yahoo, and Bing, to search for the term "Interventional Radiology" We used the native search engines without modification. The top 50 generated web-links from each search engine were collated yielding a total of 150. We elected to analyze the first 50 hits generated by each search engine, closely mirroring the behavior of Internet users: Less than 10% view sites beyond the first 10, less than 2% view sites beyond the first 30 and a very small minority go beyond the first 50 results, according to leaked search data from AOL in 2006. [22] We excluded links to scientific journals, duplicate websites, websites requiring registration, links to videos/documents, directories and websites explicitly not aimed at patients. The remaining websites were then analyzed. The above process was repeated using the search terms "cardiology" and "vascular surgery". Each site was assessed using the LIDA instrument and Fleisch Reading Ease Score (FRES). Information about the country of the website, type of organization and HON certification were also noted. LIDA instrument The LIDA instrument by Minervation, [23] is a validated tool for assessing websites providing health-related information. [18] It is based on 41 questions in three categories namely: Accessibility, usability, and reliability. Accessibility This section combines an automated online score based on the websites' compliance with legislative accessibility requirements, correct rendering in common web-browsers and the need to register to view the site. [24] Usability Ease of locating sought information by means of navigation or use of the website's built-in search engine (if available) is assessed. Clear websites with seamless navigation and a good search facility score highest. [25] Reliability Websites are assessed on clarity and transparency of their content production, conflict of interests and expertise of the authors. The information must also be critically appraised, referenced appropriately and reviewed by subject experts to score a maximum of 51. [26] Flesch reading ease score The FRES is a formula based text-assessment tool used to calculate a score out of 100 based on average sentence length (ASL) and average syllables per word (ASW): FRES=206.835 − (1.015 × ASL) − (84.6 × ASW) The lower the score, the harder it is to read the text, with a score of 60-70 said to be standard. [27] The main body of text from each site was stripped of tags and formatting and transferred into Microsoft Word (Microsoft® Word for Mac 2011Microsoft, Redmond, WA, USA), where a FRES was obtained from the spelling and grammar tool. Health on the net foundation certification The HON Foundation is a Swiss-based not-for-profit organization that provides a portal to health-related information online. [19] The HON Foundation has developed a code of conduct (HONCode) to standardize the reliability of health-related information available on the World Wide Web. HONCode certification is free and implies the site adheres by HONCode standards. It is not however, an assessment of the quality of information. Statistical analysis FRES, total and component LIDA scores, were exported into GraphPad Prism 5.0 for Mac OS X (GraphPad Software Inc., San Diego, CA, USA) and compared using unpaired, two-tailed Student t-test. P<0.05 was considered significant.
The top 50 results from each search engine were collated yielding 150 web-links for each search term. After excluding duplicates and otherwise irrelevant sites, 43, 25, and 36 web-links remained for IR, cardiology, and vascular surgery searches, respectively. Of note, most of the web-links were from the USA [Table 1].
Out of 104 analyzed sites, only five reached the gold standard LIDA score of 90%, two from IR and cardiology and one from vascular surgery. Only three websites, one from each of the assessed specialties had a FRES score greater than 60, although none of the analyzed sites were explicitly or solely aimed at healthcare professionals [Table 2].
Comparing the FRES, LIDA, and LIDA component scores, we demonstrated no significant difference across the three searched specialties [Figure 1] and [Table 3]. Interestingly, we demonstrated no significant difference between the average scores of sites from the USA and the aggregate average of sites from other countries. Previous studies demonstrated that UK sites had higher scores; however our searches retrieved few sites from the UK, making a UK versus USA comparison of little value [Table 4].
Accessibility The mean accessibility scores (raw scores in brackets) were 80%, 82% and 84% for IR, cardiology, and vascular surgery, respectively with all sites displaying correctly in the most popular web-browsers, namely: Internet Explorer, Safari, and Mozilla Firefox [Figure 1]. Usability Most sites were easy to use with mean scores of 83%, 82% and 79% for IR, cardiology, and vascular surgery, respectively. Most analyzed websites had good browsing and search facilities, where users can find the information they seek with relative ease [Figure 1]. Reliability This section attracted the lowest scores, with means of: 55%, [27] 56%, [27] and 53% [27] for IR, cardiology, and vascular surgery, respectively. Few sites were appropriately updated. Content production methodology was unclear and authors and their credentials were seldom mentioned. Of the few sites with referenced content, few had it critically appraised or reviewed by subject experts. Furthermore, the sources of funding and conflict of interests were rarely declared [Figure 1]. Fleisch reading ease scores The FRES scores were indeed low with means of 30%, 35%, and 29%, respectively, with most websites using language that is too complex for the layperson [Figure 1]. Health on the net code As only few websites were HONCode certified; we collated these across the three specialties (16 sites) and compared them with the remaining 88 non-HONCode certified websites. LIDA scores were significantly higher for HONCode certified sites (83.1±1.6 vs. 71.53±0.8, P<0.0001), the bulk of the difference being due to higher reliability scores (75.7±3.6 vs. 49.0±1.6, P<0.0001). HONCode certification was associated with significantly higher usability (87.0±1.9 vs. 81.5±0.9, P=0.0162), but not accessibility scores (85.9±1.5 vs. 81.6±1.1, P=0.953) [Figure 2] and [Table 5].
Perhaps unsurprisingly, HONCode sites were also significantly easier to read with an average FRES of 37.4±4.0 compared with 29.7±1.4 ( P=0.0441) for non-HONCode sites [Figure 2] and [Table 5]. Most sites were deemed too difficult to read by the layperson, irrespective of HONCode certification.
Health systems vary greatly throughout the world. Some allow patients to self-refer to specialist care, while others employ a gatekeeper system, whereby a general practitioner is the patient's first point of access, who then refers them to secondary care if clinically indicated, as is the case in the UK. [28],[29],[30],[31] Whether patients are referred by a primary care physician or are considering self-referral to a specialist, they may search online for a disease or symptom and may also want to learn about the specialty they are being referred to. Obtaining reliable information is invaluable particularly as misconceptions among patients and indeed among clinicians regarding the different medical and surgical specialties are rife, [32],[33],[34],[35],[36] not least due to increasing sub-specializations and overlap of different fields. In this study, we compared the quality of information available on the Internet on IR, cardiology, and vascular surgery. In order to make our comparison fair, we searched directly for the specialties in question rather than a surrogate disease process or procedure, which may not be exclusive to a specific clinical specialty (e.g., "angioplasty"). We demonstrated no significant difference in LIDA or FRES of the search results for "interventional radiology" "cardiology," and "vascular surgery." Sites across all three searches were overall well designed, accessible, functional and easy to use albeit unreliable, poorly referenced, and seldom declared conflicts of interest or funding sources. As in previous studies, we have demonstrated that HONCode certified sites have higher LIDA scores. This difference was primarily due to these sites being more reliable than their non-HONCode counterparts. HONCode certified sites were highly represented in the top five results from each specialty. While HONCode certification was associated with higher FRE scores, these sites remained largely inaccessible for the layperson. Of note, Wikipedia, a free encyclopedia, that is continually updated and governed by its users, was one of the top five scoring websites for all three searches and placed first in all three search engine results. Scorer bias may have led to slightly inflated LIDA scores, as Wikipedia is increasingly perceived, by many, as trust-worthy and reliable. Wikipedia nevertheless, is a not-for-profit organization that is solely dependent on its users' intellectual contributions and monetary donations, rendering it a perfect forum for experts in their field to share their knowledge and make their niche fields more accessible to the public. Furthermore, continual input from different users ensures Wikipedia's currency, while site 'senior' user input ensures the reliability of user-generated content. The relationship between Google and Wikipedia has been recently brought into question, [37] however, Wikipedia ranks highly in other search engines and its content proved to be accurate, reliable and well referenced. A survey in 2002 demonstrated that the public has a poor understanding of IR compared with other specialties. [38] This may be, at least partly, due to clinicians' poor knowledge of IR. [36] In 2011, Wikipedia's cardiology article attracted 23,118±844 monthly visits, which is more than double those of the IR (10,263±250) and vascular surgery articles (10,940±301). [39] As these articles are the highest ranking in all three searches, they likely generate over 40% of web-traffic from their respective searches. This could explain the poor perceived knowledge of IR, as although there is good quality information online, few people access it. Interventional radiology, a term coined in 1967 by Alexander Margulis, [40] is a subspecialty of radiology that was pioneered by Charles Dotter, who first introduced the idea of using angiographic catheters for more than diagnostic procedures and henceforth performed the first angioplasty. [41],[42],[43] IR is concerned with preforming image-guided diagnostic and therapeutic procedures, under ultrasound, fluoroscopy, computed tomography or more recently magnetic resonance imaging guidance. During the last 30 years, interventional radiologists have been responsible for much of the medical innovation and development of minimally invasive procedures that are commonplace today. [43] Interventional radiologists treat a wide variety of conditions including peripheral artery disease, aortic aneurysms, vascular malformations, tumor complications, hepato-biliary obstruction, uterine fibroids, and acute cerebral arterial occlusions to name a few. Cardiology is one of the more widely recognized and established medical specialties. Alongside major drug discoveries, the biggest milestone in modern cardiology is arguably the development of coronary artery angiography and stenting. [42] Vascular surgery evolved from general and cardiac surgery, with arterial and venous repair techniques developed at the turn of the nineteenth century remaining in use today. [44] Perhaps the biggest recent advance in vascular surgery was the development of minimally invasive endoluminal procedures. The boundaries of vascular surgery and IR are becoming increasingly blurred with a proposed common training pathway for senior IR and vascular surgery trainees in the UK. [45] We postulated that the more established specialties would have better designed more reliable sites. Although sites scored similarly across the searched specialties, the IR search resulted in more unique web-hits compared with cardiology and vascular surgery (66 vs. 49 vs. 41). Traditionally IR has been a tertiary referral specialty, but increasingly there is a move for patients to self-refer, particularly for procedures such as uterine artery embolization; which may explain the greater number of sites aimed at patients. Furthermore, we excluded 18 academic web-links from the cardiology results compared with three from vascular surgery and four from IR. A PubMed search, on 6 th May 2012, revealed 147,198 PubMed-listed cardiology articles compared with 25,740 vascular surgery and 9255 IR articles, which reflects the general search engines' results in this study. Study limitations Various online tools can be used to calculate FRES. Depending on the software used, the scores may vary. This study used Microsoft Word due its popularity and widespread availability. Also FRES does not take into account factors that affect comprehension such as writing style and explanation of medical jargon. [46] Although widely accepted for assessment of health-related websites, the LIDA tool remains largely subjective; however, a single user scored all search results, thereby reducing bias while comparing the three searches. The authors do accept that Internet users will have different perceptions of the same site depending on their level of education among other factors. The Internet is rapidly changing and dynamic, with some sites updating their content, sometimes daily; this study represents a snapshot of the information available on day we performed the search. We believe that this parallels Internet user's search pattern, as the average user tends to research a subject of interest acutely to answer their question, rather than pursue a protracted search over several days. We repeated the original search on 4 th October 2012 to assess changes in search engine ranking and demonstrated that less than 50% of the top 50 results remained so ten months later. In fact, fewer than 10% of the retrieved websites retained their original search engine ranking. Interestingly, however, Wikipedia remains the top ranking website across the three used search engines for the three searched terms [Table 6].
Recommendations
[Figure 1], [Figure 2]
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5], [Table 6]
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||
