Ethical Considerations When Approaching Scientific Content

Understanding some basic neuroscience can be highly liberating as it can undo some of our apparently irreversible limitations or failings. I wrote in more detail about some topics that proved life-changing for me here: A Practical Framework to Create and Break Habits, Mastering a Crucial Skill for Adaptation: Learning How to Learn, or Proven Strategies against Procrastination.

As you can imagine, much of this content is based on scientific research. Reading studies about neuroscience, procrastination, productivity, etc., is where the waters become muddier because scientific research often has an agenda that can lead to the “file drawer problem” or publication bias. This bias states that research results that don’t validate the researchers’ hypothesis tend to end up in the file drawers. In contrast, research that results in findings according to researchers’ hypothesis is more likely to be submitted to publication in scientific journals. And so, “the file drawer problem” leads to biased published data or fraudulent research with fake data to match researchers’ hypotheses.

The Science Integrity Digest website has a compelling series about research misconduct about plagiarism, falsification (changing data such as removing outliers from measurements, showing overlapping images to represent different experiments, adjusting brightness for one sample but not the other samples in a picture, etc.), and fabrication (making up data by reporting experiments that never happened or patients that never existed).

Personally, I was stupefied when I discovered that one of my favourite books, Why We Sleep by Matthew Walker, was riddled with scientific misrepresentations. Alexey Guzey wrote an impressive article which tackles the book’s inaccuracies (such as shorter sleep does not imply a shorter life span, sleep deprivation might be, in fact, beneficial as a depression therapy, sleeping less than 6 hours a night does not double the risk of cancer, the World Health Organization never declared a sleep loss epidemic, and so on).

Guzey also writes about Walker’s cherry-picking evidence who cut a part of a research graph that contradicted Walker’s claim (“a perfect linear relationship: the less sleep that you have, higher your injury risk”), except that the five hours of sleep column from the original research showed in fact, a lower likelihood of injury than the six hours of sleep column.

I am one of those people that took Walker’s claims at face value (“[t]wo-thirds of adults throughout all developed nations fail to obtain the recommended eight hours of nightly sleep”) and forced myself to sleep for eight hours.

As per Wikipedia:

Walker failed to disclose that numerous meta-analyses involving over 4 million adults found the lowest mortality was associated with 7 hours of sleep, and that the increased risk of death associated with sleeping more than 7 hours was significantly greater than the risk of sleeping less than 7 hours as defined by a J-shaped curve.

Psychologist Stuart J. Ritchie criticised Walker’s approach in his book.

Walker could have written a far more cautious book that limited itself to just what the data shows, but perhaps such a book wouldn’t have sold so many copies or been hailed as an intervention that ‘should change science and medicine’.

Of course, sleep and sleeping well are crucial for maintaining a healthy body and mind but writing an anxiety-inducing book about sleep could do more harm than good. Walker published a response to these critics here.

The [“replication crisis”](https://en.wikipedia.org/wiki/Replication_crisis#:~:text=The%20replication%20crisis%20(also%20called,difficult%20or%20impossible%20to%20reproduce) is another problem to consider when reading scientific research, aside from faking or misleading data. This issue means that new shiny discoveries may or may not be able to be reproduced by other independent researchers, either because of the initial experiments’ design or lack of funds. It might be much easier to obtain funds for studies when a research team is the first to discover something, and it is more challenging to get a research budget to reproduce a previous finding.

The Replicability-Index index website published an article about the replication issue regarding a chapter from another famous book, Daniel Kahneman’s Thinking, Fast and Slow (the article concluded that the research Kahneman used in his book didn’t replicate). Kahneman accepted the conclusions of this article in a comment:

I accept the basic conclusions of this blog.

…What the blog gets absolutely right is that I placed too much faith in underpowered studies. As pointed out in the blog, and earlier by Andrew Gelman, there is a special irony in my mistake because the first paper that Amos Tversky and I published was about the belief in the “law of small numbers,” which allows researchers to trust the results of underpowered studies with unreasonably small samples.

Is it any wonder that most of us, laypeople, won’t research the science behind the articles or books we read, as it is too much to discern: how recent was that particular study (last month, five years ago)? How many citations does that article have? How was the research conducted (was it a large enough sample, did it include or exclude specific populations)? Did we find any biases, logical fallacies or other inconsistencies in the text? Did we investigate the resources listed? Did we look into conflicts of interests between author, publisher or funder? Did we search for evidence that refutes the claims we read?

And so, we are at the mercy of algorithm manipulated feeds as they comply with our implicit biases, especially confirmation bias, creating and serving a self-reinforcing loop with articles, books or comments that support what we already believe. Consider the last two years, with the controversial claims about miraculous new cures against COVID-19 and the highly problematic tendencies of our society towards fragmentation and tribalism when we build a narration of “us versus them”.

Or we are at the mercy of science book authors and editors and hope they got their facts straight.

From Guzey’s article about Why We Sleep:

… imagine that a 20-year-old who naturally needs to sleep for 7 hours a night, reads Why We Sleep, gets scared, and decides to spend the full 8 hours in bed every day. Then, assuming that they live until 75, they will waste more than 20,000 hours or more than 2 years of their life, with uncertain long-term side-effects.

And some comments left to this article:

Hello Alexey

I wanted to drop you a line to thank you for all the time and effort involved in debunking Matthew Walker’s book. As someone who works with individuals with insomnia on a daily basis, I know from firsthand experience the harm that Walker’s book is causing.

I have many stories of people who slept well on less than eight hours of sleep, read Walker’s book, tried to get more sleep and this led to more time awake, frustration, worry, sleep-related anxiety, and insomnia.

Dear Alexey,

Your essay on Why we sleep – I can’t thank you enough. I’m a sleep doctor in Oregon and have seen many many patients who have developed severe sleep anxiety and insomnia. Two friends in the sleep field and myself weekly have talked about people that slept well until reading this book.

The follow-up article to the Replication Index article about Fast and Slow remarks that:

Readers of “Thinking: Fast and Slow” should read the book as a subjective account by an eminent psychologist, rather than an objective summary of scientific evidence. Moreover, ten years have passed and if Kahneman wrote a second edition, it would be very different from the first one. Chapters 3 and 4 would probably just be scrubbed from the book. But that is science. It does make progress, even if progress is often painfully slow in the softer sciences.

Falling on the other extreme, full-fledge dismissal of anything that comes from scientific research might lead to tragedy.

Doubt is our product since it is the best means of competing with the ‘body of fact’ that exists in the minds of the general public. It is also the means of establishing a controversy.

This quote comes from a tobacco industry executive in 1969 about how cigarette companies should deal with the ‘body of fact’ (the link between smoking and diseases). As the pile of evidence against tobacco grew and grew, convincing smokers that cigarettes were safe would have been a lost bet. But what if you muddle up the research that showed that smoking was dangerous, create doubt about study results, call for more research or fund your own research until smokers give up and say: “I don’t know what to believe now”? Then, you still have a viable product (tobacco) that sells.

Assessing information brings us to the Goldilocks problem: neither too much astounding credulity nor too much toxic cynicism, but trying to converge to a zone where healthy scepticism and curiosity are just right because it makes us face the possibility that we might be mistaken.

Doubt is a powerful weapon that can strike either way. We need to use it wisely when we come across all sorts of content. Including this article.

Resources I follow to stay up to date on current development in science in general and neuroscience in particular:

Barbara Oakley’s newsletter

Lisa Feldman Barrett (twitter link, non-twitter link)

Neuroskeptic (twitter link, non-twitter link)

Stanislas Dehaene (twitter link, non-twitter link)

Neuroscience News

Scientific American 

Quanta Magazine

Previously published at https://www.roxanamurariu.com/dilemmas-in-approaching-scientific-content/