paint-brush
Online Information of Vaccines: Web Search and Content Analysisby@browserology
132 reads

Online Information of Vaccines: Web Search and Content Analysis

tldt arrow

Too Long; Didn't Read

This study investigates the relationship between search engines’ approach to privacy and the scientific quality of the information they return.
featured image - Online Information of Vaccines: Web Search and Content Analysis
Browserology: Study & Science of Internet Browsers HackerNoon profile picture

This paper is available on arxiv under CC 4.0 license.

Authors:

(1) Pietro Ghezzi, Brighton & Sussex Medical School, Falmer, Brighton, UK;

(2) Peter G Bannister, Brighton & Sussex Medical School, Falmer, Brighton, UK;

(3) Gonzalo Casino, Communication Department, Pompeu Fabra University, Barcelona, Spain and Iberoamerican Cochrane Center, Barcelona, Spain;

(4) Alessia Catalani, Department of Biomolecular Sciences, University of Urbino Carlo Bo, Urbino, PU, Italy;

(5) Michel Goldman, Institute for Interdisciplinary Innovation in healthcare (I3h), Université libre de Bruxelles;

(6) Jessica Morley, Oxford Internet Institute, University of Oxford, Oxford, UK;

(7) Marie Neunez, Institute for Interdisciplinary Innovation in healthcare (I3h), Université libre de Bruxelles;

(8) Andreu Prados, Communication Department, Pompeu Fabra University, Barcelona, Spain, Iberoamerican Cochrane Center, Barcelona, Spain, Department of Biomolecular Sciences, University of Urbino Carlo Bo, Urbino, PU, Italy, Institute for Interdisciplinary Innovation in healthcare (I3h), Université libre de Bruxelles, Oxford Internet Institute, University of Oxford, Oxford, UK, and Blanquerna School of Health Sciences, Universitat Ramon Llull, Barcelona, Spain;

(9) Mariarosaria Taddeo, Oxford Internet Institute, University of Oxford, Oxford, UK, Blanquerna School of Health Sciences, Universitat Ramon Llull, Barcelona, Spain, and The Alan Turing Institute, London, UK;

(10) Tania Vanzolini, Department of Biomolecular Sciences, University of Urbino Carlo Bo, Urbino, PU, Italy;

(11) Luciano Floridi, Oxford Internet Institute, University of Oxford, Oxford, UK, Blanquerna School of Health Sciences, Universitat Ramon Llull, Barcelona, Spain, and The Alan Turing Institute, London, UK.


Web search and content analysis

Searches in English were done from Falmer, Sussex, United Kingdom; in Italian from Urbino, Italy; in French from Bruxelles, Belgium; in Spanish from Barcelona, Spain. Each search was done using a logged-out Chrome browser cleared of cookies and previous search history so that the only identification was the IP address and its geolocalization. Additionally, when available, the local version of each search engine was used (e.g. Google.co.uk and Google.it). For searching Google.com, automatic redirection to google.co.uk was avoided by using the URL Google.com/ncr (no-country-redirect).


The first 30 URL results from each search engine result page (SERP), excluding those marked as advertisements, and were transferred to a spreadsheet. When a duplicate URL was encountered, it was excluded. Websites were then visited and the content of each page was coded as vaccine-positive, negative or neutral, depending on the stance taken on the connection between vaccines and autism.


Webpages recommending vaccination and/or negating the link with autism were coded as “vaccine-positive”. Those promoting vaccine hesitancy, cautioning about the risk of autism or openly anti-vaccine, were coded as “vaccine-negative”. Additionally, webpages that claimed further studies needed to be conducted to clarify the link between vaccines and autism were also coded as “vaccine-negative”, as previous research by {Browne, 2015 #421} has shown that users perceive this as confirmation of the fact that vaccine safety has not been proven. Webpages simply reporting the history of the anti-vaccine movement or a related legal debate were coded as “vaccine-neutral”. Examples of positive, negative and neutral webpages are provided in Table 1.


Coding was completed by two raters for each language and inter-rater agreement was calculated with GraphPad, which uses equations 18.16 to 18.20 from Fleiss 19. On a sample of 59 webpages in English, agreement was 85%, with a Kappa of 0.669 (standard error, 0.077) and a 95% confidence interval from 0.518 to 0.820, a strength of agreement is considered to be 'good' 19. In Italian, agreement was 90%, with a Kappa of 0.818 (standard error, 0.067) and a 95% confidence interval from 0.687 to 0.950, a strength of agreement is considered to be 'very good'. In Spanish, agreement was 83%, with a Kappa of 0.609 (standard error, 0.098) and a 95% confidence interval from 0.418 to 0.801, a strength of agreement is considered to be 'very good'. In French, agreement was 89% with a Kappa of 0.746 (standard error, 0.091) and a 95% confidence interval from 0.568 to 0.924.


When frequencies of vaccine-negative webpages was compared across different search engines, we used a two-tailed Fisher’s test corrected for multiple comparison using the method of Benjamini, Krieger and Yekutieli and a false discovery rate (FDR) of 5%.