ChatGPT: A Potentially Dangerous New Source of Misinformation

Recently, there has been a lot of talk about ChatGPT, from OpenAI[1], and its many capabilities.

Certain portions of this article were generated by an artificial intelligence program. 

ChatGPT stands for "Chat Generative Pre-trained Transformer." It is an Artificial Intelligence program that generates human-like responses to text questions.

Unlike a search engine, the AI tool doesn’t simply return a list of links for more information. Instead, it returns a detailed written answer to the question. The responses are well-written and comprehensive. It has even been suggested that students might use the program to write term papers!

I decided to test the program to see if the results are accurate and useful.

Since many people use the Internet for answers to medical questions, I decided to begin with questions relating to erectile dysfunction. This is a subject I’m very familiar with because I operate an informational website related to ED.

My question:

How does the Phoenix device treat erectile dysfunction?

ChatGPT response:

The Phoenix device is a vacuum therapy device that is used to treat erectile dysfunction by creating a vacuum around the penis. The device consists of a plastic cylinder that fits over the penis, and a hand pump that is used to create a vacuum inside the cylinder. The vacuum pressure draws blood into the penis, causing it to become engorged and erect.

Once the erection has been achieved, a tension band or constriction ring is placed at the base of the penis to maintain the erection during sexual activity. The tension band can be removed after sex. The Phoenix device is a non-invasive treatment option and can be effective for some men with mild to moderate erectile dysfunction.

It is important to note that the Phoenix device may not be suitable for all men with erectile dysfunction, and it is recommended to consult with a healthcare professional before using this device to ensure that it is safe and appropriate for an individual's specific condition and medical history.

Sounds great, right? Well written and informative isn't it? Unfortunately, it’s totally wrong. The Phoenix is not a vacuum therapy device. It's a Low-Intensity Extracorporeal Shockwave Therapy (LI-ESWT) device[2]… an entirely different type of treatment.

Next, I tried asking about two relatively new treatment options that are widely offered by clinics. Note the similarity in the responses below.

First question:

What is shockwave therapy for erectile dysfunction?

ChatGPT response:

Shockwave therapy for erectile dysfunction is a non-invasive treatment that uses low-intensity shockwaves to improve blood flow and stimulate the growth of new blood vessels in the penis. The treatment involves the use of a device that delivers focused, low-intensity shockwaves to the penile tissue. The shockwaves help to break down micro-plaque in the blood vessels and stimulate the growth of new blood vessels, which can improve blood flow to the penis and help with erectile function.

Shockwave therapy is usually performed in a series of sessions, typically once or twice a week, and each session lasts about 15-20 minutes. The treatment is considered safe and generally well-tolerated, with few side effects. It is an alternative treatment option for men who may not respond to or prefer not to use oral medications, penile injections, or other forms of treatment for erectile dysfunction.

It is important to note that while shockwave therapy shows promise as a treatment for erectile dysfunction, more research is needed to determine its long-term safety and effectiveness, and it may not be suitable for all men with erectile dysfunction. It is recommended to consult with a healthcare professional to determine if shockwave therapy is a safe and appropriate treatment option for an individual's specific condition and medical history.

Second question:

What is PRP therapy for erectile dysfunction?

ChatGPT response:

PRP therapy, or Platelet-Rich Plasma therapy, is a treatment for erectile dysfunction that involves the injection of concentrated platelets from the patient's own blood into the penis. The platelets are obtained by drawing a small amount of the patient's blood and then processing it to concentrate the platelets.

Platelets contain growth factors and other proteins that can promote tissue healing and regeneration. When injected into the penis, these growth factors can stimulate the growth of new blood vessels and improve blood flow, which can help to improve erectile function.

PRP therapy is a non-surgical, minimally invasive treatment option that can be performed in an outpatient setting. It is generally well-tolerated and has few side effects. However, the long-term safety and effectiveness of PRP therapy for erectile dysfunction are still being studied, and it may not be suitable for all men with erectile dysfunction. It is important to consult with a healthcare professional to determine if PRP therapy is a safe and appropriate treatment option for an individual's specific condition and medical history.

The responses give a strong impression that the two treatments are similarly effective and well-regarded; similar wording is used to describe them.

In reality, the treatments are very different in their effectiveness. There are dozens of independent, peer-reviewed clinical trials[3] showing the safety and effectiveness of shockwave therapy, while there is very little evidence in support of PRP:

• A research review[4], conducted in 2019, found that “Despite a global presence of PRP clinics and ongoing active marketing and public interest in regenerative medicine, no scientific evidence has been published to establish an evidence-based risk-benefit profile for PRP use for ED in humans.”

• Another review[5] in 2022, concluded that “PRP injections for the treatment of ED may be promising, but no recommendation can be made because of scarce evidence. Safety and effectiveness of this therapy in the treatment of ED and PD require further preclinical and clinical studies with standardized protocols to gain adequate insight into its potential implications.”

Shockwave therapy and plasma replacement therapy are not at all comparable, yet ChatGPT gives a clear impression that they are.

Finally, I decided to try an entirely different type of question, one based on simple, publicly available information.

My question:

What is the history of 1200 Lakeshore Apartments in Oakland, California?

ChatGPT response:

The 1200 Lakeshore Apartments in Oakland, California, were originally built in 1962 as a luxury apartment complex called the "Lake Merritt Towers." The towers were designed by architect Herman Baumann and featured modernist styling and expansive views of Lake Merritt. In the early 2000s, the towers underwent a major renovation and were converted into condominiums. The renovation included updates to the building's systems, common areas, and individual units. Today, the 1200 Lakeshore Apartments are considered one of the premier addresses in the Lake Merritt neighborhood and offer residents luxury amenities, including a fitness center, swimming pool, and 24-hour security.

Once again, there were significant errors in the program’s response. The apartments were not converted to condominiums, and the building does not have a swimming pool[6].

Final Thoughts 

Based on my limited experiments, I found that ChatGPT was frequently wrong about basic, factual information, as well as more nuanced judgements. However, because that results are well-written and plausible, many people may accept them without question.

The program does not provide any information about the sources on which the responses are based. At a minimum, the program should provide links to its sources. Without that, or other means of authentication, ChatGPT is a dangerously plausible source of misinformation.

Students should resist the temptation to write papers using ChatGPT, where its frequent inaccuracies could well result in failing grades!

References

[1] ChatGPT
[2] Phoenix Reviewed: At-Home Shockwave Therapy for Erectile Dysfunction
[3] Shockwave Therapy (LI-ESWT) for Erectile Dysfunction
[4] Platelet-Rich Plasma and Treatment of Erectile Dysfunction: Critical Review of Literature and Global Trends in Platelet-Rich Plasma Clinics
[5] Can PRP Injections Really Give You a Bigger Penis?
[6] 1200 Lakeshore Apartments