100 Days of AI Day 3: Leveraging AI for Prompt Engineering and Inference

I write a newsletter called Above Average, where I talk about the second-order insights behind everything that is happening in the big tech. If you are into tech and don’t want to be average, subscribe to it.

Inferring is essentially a way to extract insights, sentiments, and trends using clever prompts while interacting with LLMs.

Previously, before LLMs, each of these tasks would require their own models, training, and deployment & maintenance. But with the explosion of OpenAI and other LLMs, inference allows you to improve your products by leveraging LLMs, and you don’t have to be a data scientist or a machine learning engineer to do so.

Example 1: Let’s say you run an e-commerce website and want to extract the sentiment of customer reviews. You can do this using the following prompt. In this case, I am providing the review text and the prompt you can use on it. You can use your favorite way to run it against Open AI’s API. I took a real review from Amazon for a light box that I purchased for this example.

from openai import OpenAI
client = OpenAI()
client.api_key = 'YOUR_SECRET_KEY'

amazon_review = f"""
This softbox lighting system is a great value for the price. They were super easy to set and offer multiple types of lighting, with dimmer and remotes for each light. They are lightweight which means they can topple over if bumped. But that was an easy problem to solve. I just created some weights out of an old pair of socks and some dried beans. You could use rice or sand also. Or just buy sandbags. I like to DIY. Anywho, I highly recommend this lighting system.
"""

story = """
prompt = f"""
Identify the following items from the review text: 
- Sentiment (positive or negative)
- Is the reviewer expressing anger? (true or false)
- Item purchased by reviewer
- Company that made the item

The review is delimited with triple backticks. \
Format your response as a JSON object with \
"Sentiment", "Anger", "Item" and "Brand" as the keys.
If the information isn't present, use "unknown" \
as the value.
Make your response as short as possible.
Format the Anger value as a boolean.

Review text: '''{amazon_review}'''

"""

response = client.chat.completions.create(
  model="gpt-4",
  messages=[
    {
      "role": "user",
      "content": prompt
    }
  ],
  temperature=0,
  #max_tokens=64,
  #top_p=1
)

print(response.choices[0].message.content)

The output I got is as follows:

{“Sentiment”: “positive”, “Anger”: false, “Item”: “softbox lighting system”, “Brand”: “unknown”}

This is a cool example showing the power of inferring using Open AI’s API. Now, Amazon is a large company that already uses AI to ship features, and one of the recent updates to the page seems to be about summarizing all the reviews under the product. Here is how it looks for the same product.

Do you think it can be re-created using inferring across a set of customer reviews? I think so. If you are able to figure this out, reach me on Twitter and let me know.

Example 2: let's say you run a news website, and you want to find the topics an article is about without actually reading it. Can you do it using a Prompt and infer it? Yes, you can. Here is an example article from which I am inferring topics mentioned in that article.

from openai import OpenAI
client = OpenAI()
client.api_key = 'YOUR_SECRET_KEY'



article = """
It hasn't been the best start to 2024 for Elon Musk. The owner of X, formerly known as Twitter, appealed to YouTube's biggest star MrBeast to post his videos on the platform, and was promptly rejected.

On Dec. 30, 2023, MrBeast, whose real name is Jimmy Donaldson, posted on X to announce his latest video release was available on YouTube: "I uploaded, go watch or I'll drop kick you."

Donaldson had uploaded a 20-minute long video to YouTube, in which he chronicled spending a week in solitary confinement. These types of videos, in which Donaldson challenges either himself or an individual with an extreme task, have earned him worldwide notoriety. The social media mastermind earned an estimated $82 million in 2023, and with more than 225 million subscribers on his main channel alone, is the platform's biggest name.

After Donaldson posted the appeal to head to YouTube on X, one fan replied: "Upload on this platform too," which is when Elon Musk chimed in with: "Yeah."

Musk, the CEO of Tesla and owner of X, has been trying to drum up interest from video viewers since taking over the social media platform in 2022.

Schemes have included launching a media partnership with heiress Paris Hilton—which has since been axed—as well as streaming shows from the likes of former Fox star Tucker Carlson.

Musk, now with the help of CEO Linda Yaccarino, has also attempted to drum up revenue by launching premium subscription services on the site, which—among other things—allow for users to appear as "verified" and send DMs to other accounts.

The moves have been necessary after droves of advertisers left the site over fears their branding would appear beside unregulated content.

Musk, a fierce proponent of free speech, has refused to cow to concern from advertisers about how X will prevent their messages from reportedly appearing beside Nazi propaganda, for example.

But Musk's attempt to get MrBeast's content on his site—not even exclusively—was rebuffed by the creator.

Donaldson replied to Musk directly: "My videos cost millions to make and even if they got a billion views on X it wouldn’t fund a fraction of it :/ I’m down though to test stuff once monetization is really cranking!"

The polite rebuttal contrasts with the firm interest Donaldson had previously taken in Twitter. Indeed, Donaldson's bio still reads "X Super Official CEO," harking back to the times when speculation was rife about who would take over the day-to-day running of the platform from the ever-busy Musk.
"""

prompt = f"""
Determine five topics that are being discussed in the \
following text, which is delimited by triple backticks.

Make each item one or two words long. 

Format your response as a list of items separated by commas.

Text sample: '''{article}'''

"""

response = client.chat.completions.create(
  model="gpt-4",
  messages=[
    {
      "role": "user",
      "content": prompt
    }
  ],
  temperature=0,
  #max_tokens=64,
  #top_p=1
)

print(response.choices[0].message.content)

Output is:

Elon Musk, Social Media Platforms, YouTube Content, Advertising Concerns, Monetization Strategies

Pretty accurate, right?

AI PRODUCT IDEA ALERT: Shopify App to summarize & extract insights from customer reviews. This app should be launched in the Shopify app store. I wouldn’t be surprised if Shopify launched this natively. Just using prompt engineering with open AI API, this would be an app that can be launched in a couple of days.

That’s it for day 3.

Follow me on Twitter for the latest updates on 100 days of AI, or bookmark this page.

Also published here.