Character.ai

characterai

Winter is coming, and the night just got darker and full of unthinkable horrors. I’ve never thought that GoT would become a prelude to a real-life tragedy. This was the only TV show that got my family’s unanimous attention and acceptance to this day. It goes without saying that for our daughter to watch it my wife and I had to do some serious PG-13 heavy lifting. We had two parallel-watching sessions. Some episodes had to be left out and for some scenes, our daughter had to leave the room and get back after we were done with the fast-forwarding. It was worth it. Our daughter’s room was covered with her drawings of dragons and pictures of Dany Targaryen. We got carried away. There’s a huge picture of her sitting on the Iron Throne replica in our living room. I still remember waiting for hours in line and sharing my pain with other parents at the local GamesCon, so our kids could live out their fantasies for a brief moment. Those were some different and innocent pre-AI times. Today’s kids aren’t any different. They still want the same things, but the available tools are next level. I really wanted to write a nice and inspirational AI chatbot story, but… As soon as I typed “ai chatbot” in Google, I got slammed with the news headlines. Do it and see for yourself. The New York Times and The Verge were the most objective and restrained. I don’t recommend you go down the rabbit hole of The Washington Post and Reddit comments. I consider that my choice for the story’s illustration is appropriate because of the genuine desire of its author to express his feelings and protest at the same time. I feel the same way about my story’s title. No, I’m not going to share what happened because I want you to spend some time researching and then getting back to continue reading with the right kind of emotions and respect. I just can’t TL:DR someone’s life for your convenience. As a parent, I feel that I just dodged a digital bullet because my daughter is now old enough to have real instead of imaginary friends. For kids, an age gap of only a few years is the equivalent of decades for adults. Character AI lawsuit just got caught in the crossfire between the two most polarizing issues, one of the tech and the other of political nature. The First Issue - The minimum age for using AI The minimum age for minors to interact with AI ranges from 13 to 16. It’s 13 in the USA and mostly 16 in Europe. The Kids Online Safety Act (KOSA) has been around in the House and Senate since 2022. Why it’s taking so long to protect kids? “It’s a political issue that has bipartisan support, while also appearing to be extremely difficult to govern without infringing on First Amendment protections.” And, there’s this, as well, the KOSA “would require online platforms to take actions to protect the safety of users under the age of 17.” The bipartisan support has become so rare that when you hear about it, you know without going into the details it’s a very serious matter. Unfortunately, we also have to accept the reality of another kind of “bipartisan support” between the First Amendment “absolutists” and the Big Tech opportunitists. So, nothing is going to change here. The lawsuit “also named Google and its parent company Alphabet in the filing. Character.AI's founders worked at Google before launching their product and were re-hired by the company in August as part of a deal granting it a non-exclusive licence to Character.AI's technology.” The most reasonable suggestion of Character AI Reddit fans is doomed to fall on deaf ears: The Second Issue - Gun Control The second issue is all about the Second Amendment. Isn’t it ironic to have this kind of semantic “alignment?” The countless mass shootings haven’t changed a thing about the gun control debate. Sadly, one more young innocent life that’s lost can only prolong a series of pointless debates. The blaming game has no beginning and obviously will have no end: What can you possibly expect to change after reading the comments such as these? What Is Going To Be The Most Likely Outcome of Character.AI Chatbot Lawsuit? It seems that at Character.AI, they’re preparing for and expecting a settlement. I did my best to check the dates. Almost the same day the lawsuit was officially filled, the company published a list of Community Safety Updates on its official blog page: “Moving forward, we will be rolling out a number of new safety and product features that strengthen the security of our platform without compromising the entertaining and engaging experience users have come to expect from Character.AI. These include: Changes to our models for minors (under the age of 18) that are designed to reduce the likelihood of encountering sensitive or suggestive content.
Improved detection, response, and intervention related to user inputs that violate our Terms or Community Guidelines.
A revised disclaimer on every chat to remind users that the AI is not a real person.
Notification when a user has spent an hour-long session on the platform with additional user flexibility in progress.” Who is praying for Sewell Setzer III who was only 14 years old when he “died by suicide on February 28th, 2024, “seconds” after his last interaction with the bot?” Winter is coming, and the night just got darker and full of unthinkable horrors. I’ve never thought that GoT would become a prelude to a real-life tragedy. This was the only TV show that got my family’s unanimous attention and acceptance to this day. It goes without saying that for our daughter to watch it my wife and I had to do some serious PG-13 heavy lifting. We had two parallel-watching sessions. Some episodes had to be left out and for some scenes, our daughter had to leave the room and get back after we were done with the fast-forwarding. It was worth it. Our daughter’s room was covered with her drawings of dragons and pictures of Dany Targaryen. We got carried away. There’s a huge picture of her sitting on the Iron Throne replica in our living room. I still remember waiting for hours in line and sharing my pain with other parents at the local GamesCon, so our kids could live out their fantasies for a brief moment. Those were some different and innocent pre-AI times. Today’s kids aren’t any different. They still want the same things, but the available tools are next level. I really wanted to write a nice and inspirational AI chatbot story, but… As soon as I typed “ai chatbot” in Google, I got slammed with the news headlines. Do it and see for yourself. The New York Times and The Verge were the most objective and restrained. I don’t recommend you go down the rabbit hole of The Washington Post and Reddit comments . The New York Times The Verge The Washington Post Reddit comments I consider that my choice for the story’s illustration is appropriate because of the genuine desire of its author to express his feelings and protest at the same time. I feel the same way about my story’s title. the story’s illustration No, I’m not going to share what happened because I want you to spend some time researching and then getting back to continue reading with the right kind of emotions and respect. I just can’t TL:DR someone’s life for your convenience. As a parent, I feel that I just dodged a digital bullet because my daughter is now old enough to have real instead of imaginary friends. For kids, an age gap of only a few years is the equivalent of decades for adults. Character AI lawsuit just got caught in the crossfire between the two most polarizing issues, one of the tech and the other of political nature. The First Issue - The minimum age for using AI The minimum age for minors to interact with AI ranges from 13 to 16. It’s 13 in the USA and mostly 16 in Europe. The Kids Online Safety Act (KOSA) has been around in the House and Senate since 2022. Why it’s taking so long to protect kids? “It’s a political issue that has bipartisan support, while also appearing to be extremely difficult to govern without infringing on First Amendment protections.” And, there’s this, as well, the KOSA “would require online platforms to take actions to protect the safety of users under the age of 17.” political issue online safety The bipartisan support has become so rare that when you hear about it, you know without going into the details it’s a very serious matter. Unfortunately, we also have to accept the reality of another kind of “bipartisan support” between the First Amendment “absolutists” and the Big Tech opportunitists. So, nothing is going to change here. The lawsuit “also named Google and its parent company Alphabet in the filing. Character.AI's founders worked at Google before launching their product and were re-hired by the company in August as part of a deal granting it a non-exclusive licence to Character.AI's technology.” named Google and its parent company Alphabet The most reasonable suggestion of Character AI Reddit fans is doomed to fall on deaf ears: The Second Issue - Gun Control The second issue is all about the Second Amendment. Isn’t it ironic to have this kind of semantic “alignment?” The countless mass shootings haven’t changed a thing about the gun control debate. Sadly, one more young innocent life that’s lost can only prolong a series of pointless debates. The blaming game has no beginning and obviously will have no end: What can you possibly expect to change after reading the comments such as these? What Is Going To Be The Most Likely Outcome of Character.AI Chatbot Lawsuit? It seems that at Character.AI, they’re preparing for and expecting a settlement. I did my best to check the dates. Almost the same day the lawsuit was officially filled, the company published a list of Community Safety Updates on its official blog page: Community Safety Updates “Moving forward, we will be rolling out a number of new safety and product features that strengthen the security of our platform without compromising the entertaining and engaging experience users have come to expect from Character.AI. These include: Changes to our models for minors (under the age of 18) that are designed to reduce the likelihood of encountering sensitive or suggestive content. Improved detection, response, and intervention related to user inputs that violate our Terms or Community Guidelines. A revised disclaimer on every chat to remind users that the AI is not a real person. Notification when a user has spent an hour-long session on the platform with additional user flexibility in progress.” Changes to our models for minors (under the age of 18) that are designed to reduce the likelihood of encountering sensitive or suggestive content. Improved detection, response, and intervention related to user inputs that violate our Terms or Community Guidelines. A revised disclaimer on every chat to remind users that the AI is not a real person. Notification when a user has spent an hour-long session on the platform with additional user flexibility in progress.” Who is praying for Sewell Setzer III who was only 14 years old when he “died by suicide on February 28th, 2024, “seconds” after his last interaction with the bot?” Who is praying for Sewell Setzer III who was only 14 years old when he “died by suicide on February 28th, 2024, “seconds” after his last interaction with the bot?”

Character AI Chatbot Lawsuit: Imaginary Friends and Real Guns 

AI Can Give Voice to Sign Language, Empowering the Deaf

Imagine a child asks an AI chatbot about feeling sad. Without knowing the user's age, the AI could respond with an unnuanced discussion of depression and self-harm – content that might be appropriate for an adult but potentially harmful for a young person. While this scenario is hypothetical, it illustrates a fundamental problem with today's AI systems: they can't tailor their responses based on a user's age. AngelQ and Arcee AI are addressing this critical gap with their KidRails for LLMs framework, creating AI that can respond appropriately to children. "Historically technology has not been designed with kids in mind - that's certainly been the case with the internet and internet connected devices - the same pattern holds for AI," says Josh Thurman, Co-Founder and Head of Product at AngelQ. "When it comes to AI these negative consequences will undoubtedly be grossly amplified without appropriate guardrails in place." Recent incidents highlight this concern. Cases of AI systems providing dangerously inappropriate responses to children have emerged, from Character AI's chatbot allegedly contributing to a Florida teenager's suicide in 2023 to Snapchat's My AI providing inappropriate relationship advice to users it believed to be minors. These troubling cases underscore the limitations and potential dangers of applying adult-focused technology to young users. "We're releasing KidRails as a starting point for reversing the trend of technologies that are harmful to children," says Thurman. "We have to get this right in the AI age - the stakes are extremely high." Understanding Child Development in AI Design "Creating truly child-safe AI requires a deep understanding of developmental psychology and education," explains Tim Estes, CEO and Co-Founder of AngelQ. "A six-year-old and an eleven-year-old don't just process information differently, they have entirely different needs, interests, and safeguards required. KidRails was engineered from the ground up to recognize these crucial differences and adjust its interactions accordingly." The framework employs a tiered approach that adapts responses based on a child's age and developmental stage. It can provide age-appropriate answers, offer limited responses with parental deference, or completely defer to parents for sensitive topics. As Estes notes, "Sometimes the best approach for AI is to know when not to say something." A Partnership Built on Shared Values The collaboration between AngelQ and Arcee AI emerged from aligned priorities about children's safety in technology. "Mark and I first connected at the beginning of 2024, and from our very first conversation, it was immediately apparent that we shared a fundamental belief: technology for children was very important and there was an opportunity to do innovative work in making Language Models safer and more age appropriate in their function," says Estes. By making KidRails open-source, the companies aim to establish new standards for child-safe AI interactions. The release includes training data, evaluation criteria, and complete processes for fine-tuning – all under the Apache 2 license. This transparency invites broader community participation in improving how AI systems interact with children. Addressing Core Challenges "The fundamental challenge in building a child-focused LLM is the decisions on what to say at different ages," Estes explains. "That's really a decision process and it's why we are open sourcing the work- because of transparency and inviting a broader community to help us improve those judgments." Unlike traditional approaches that add safety filters to existing models, KidRails integrates parent involvement into its core functionality. The system creates opportunities for parent-child discussions by redirecting certain questions to parents, ensuring that "AI enhances, rather than replaces, the critical role parents play in their children's development and digital literacy," says Estes. Setting New Standards The development process involved carefully considering how children interact with AI systems. The team generated and curated over 250,000 child conversations to create representative training data, which was used to fine-tune a Llama 3.1 8B base model. This approach ensures responses are not only safe but also appropriate for different developmental stages. "We envision KidRails as more than a tool; it's a demonstration that AI can be fundamentally reimagined with children as the primary users, setting a new precedent for the entire industry," says Estes. "We want children to be able to explore and learn in an environment that preserves the natural curiosity and wonder of childhood, while giving parents the confidence that comes from knowing their children are engaging with technology built specifically for them... not adapted adult technology with safety filters added as an afterthought." By making the framework open-source and encouraging collaboration, AngelQ and Arcee AI are working to transform how the industry approaches AI development for young users. As more companies adopt and build upon the KidRails framework, their vision of creating AI systems that truly prioritize children's needs moves closer to reality. Imagine a child asks an AI chatbot about feeling sad. Without knowing the user's age, the AI could respond with an unnuanced discussion of depression and self-harm – content that might be appropriate for an adult but potentially harmful for a young person. While this scenario is hypothetical, it illustrates a fundamental problem with today's AI systems: they can't tailor their responses based on a user's age. AngelQ and Arcee AI are addressing this critical gap with their KidRails for LLMs framework, creating AI that can respond appropriately to children. AngelQ AngelQ Arcee AI Arcee AI "Historically technology has not been designed with kids in mind - that's certainly been the case with the internet and internet connected devices - the same pattern holds for AI," says Josh Thurman, Co-Founder and Head of Product at AngelQ. "When it comes to AI these negative consequences will undoubtedly be grossly amplified without appropriate guardrails in place." Recent incidents highlight this concern. Cases of AI systems providing dangerously inappropriate responses to children have emerged, from Character AI's chatbot allegedly contributing to a Florida teenager's suicide in 2023 to Snapchat's My AI providing inappropriate relationship advice to users it believed to be minors. These troubling cases underscore the limitations and potential dangers of applying adult-focused technology to young users. "We're releasing KidRails as a starting point for reversing the trend of technologies that are harmful to children," says Thurman. "We have to get this right in the AI age - the stakes are extremely high." Character AI's chatbot Character AI's chatbot Snapchat's My AI Snapchat's My AI Understanding Child Development in AI Design Understanding Child Development in AI Design "Creating truly child-safe AI requires a deep understanding of developmental psychology and education," explains Tim Estes, CEO and Co-Founder of AngelQ. "A six-year-old and an eleven-year-old don't just process information differently, they have entirely different needs, interests, and safeguards required. KidRails was engineered from the ground up to recognize these crucial differences and adjust its interactions accordingly." "Creating truly child-safe AI requires a deep understanding of developmental psychology and education," explains Tim Estes, CEO and Co-Founder of AngelQ. "A six-year-old and an eleven-year-old don't just process information differently, they have entirely different needs, interests, and safeguards required. KidRails was engineered from the ground up to recognize these crucial differences and adjust its interactions accordingly." The framework employs a tiered approach that adapts responses based on a child's age and developmental stage. It can provide age-appropriate answers, offer limited responses with parental deference, or completely defer to parents for sensitive topics. As Estes notes, "Sometimes the best approach for AI is to know when not to say something." A Partnership Built on Shared Values A Partnership Built on Shared Values The collaboration between AngelQ and Arcee AI emerged from aligned priorities about children's safety in technology. "Mark and I first connected at the beginning of 2024, and from our very first conversation, it was immediately apparent that we shared a fundamental belief: technology for children was very important and there was an opportunity to do innovative work in making Language Models safer and more age appropriate in their function," says Estes. By making KidRails open-source, the companies aim to establish new standards for child-safe AI interactions. The release includes training data, evaluation criteria, and complete processes for fine-tuning – all under the Apache 2 license. This transparency invites broader community participation in improving how AI systems interact with children. Addressing Core Challenges Addressing Core Challenges "The fundamental challenge in building a child-focused LLM is the decisions on what to say at different ages," Estes explains. "That's really a decision process and it's why we are open sourcing the work- because of transparency and inviting a broader community to help us improve those judgments." Unlike traditional approaches that add safety filters to existing models, KidRails integrates parent involvement into its core functionality. The system creates opportunities for parent-child discussions by redirecting certain questions to parents, ensuring that "AI enhances, rather than replaces, the critical role parents play in their children's development and digital literacy," says Estes. Setting New Standards Setting New Standards The development process involved carefully considering how children interact with AI systems. The team generated and curated over 250,000 child conversations to create representative training data, which was used to fine-tune a Llama 3.1 8B base model. This approach ensures responses are not only safe but also appropriate for different developmental stages. "We envision KidRails as more than a tool; it's a demonstration that AI can be fundamentally reimagined with children as the primary users, setting a new precedent for the entire industry," says Estes. "We want children to be able to explore and learn in an environment that preserves the natural curiosity and wonder of childhood, while giving parents the confidence that comes from knowing their children are engaging with technology built specifically for them... not adapted adult technology with safety filters added as an afterthought." "We envision KidRails as more than a tool; it's a demonstration that AI can be fundamentally reimagined with children as the primary users, setting a new precedent for the entire industry," says Estes. "We want children to be able to explore and learn in an environment that preserves the natural curiosity and wonder of childhood, while giving parents the confidence that comes from knowing their children are engaging with technology built specifically for them... not adapted adult technology with safety filters added as an afterthought." By making the framework open-source and encouraging collaboration, AngelQ and Arcee AI are working to transform how the industry approaches AI development for young users. As more companies adopt and build upon the KidRails framework, their vision of creating AI systems that truly prioritize children's needs moves closer to reality.

AngelQ's Mission to Build Safer AI Technology

The latest AI developments turn empathy into just another product for sale, raising questions about ethics and regulation. Imagine your customer service chatbot isn’t just solving your problem – it’s listening, empathizing, and sounding eerily human. It feels like it cares. But behind the friendly tone and comforting words, that ‘care’ is just a product, finetuned to steer your emotions and shape your decisions. Welcome to the unsettling reality of empathetic AI, where emotions are mimicked – and monetized. In 2024, empathetic AI took a leap forward. Hume.AI gave large language models voices that sound convincingly expressive and a perceptive ear to match. Microsoft’s Copilot got a human voice and an emotionally supportive attitude, while platforms like Character.ai and Psychologist sprouted bots that mimic therapy sessions. These developments are paving the way for a new industry: Empathy-as-a-Service, where emotional connection isn’t just simulated, it’s a product: packaged, scaled, and sold. This isn’t just about convenience – it’s about influence. Empathy-as-a-Service (EaaS), an entirely hypothetical but now plausible product, could blur the line between genuine connection and algorithmic mimicry, creating systems where simulated care subtly nudges consumer behavior. The stakes? A future where businesses profit from your emotions under the guise of customer experience. And for consumers on the receiving end, that raises some deeply unsettling questions. A Hypothetical but Troubling Scenario Take an imaginary customer service bot. One that helps you find your perfect style and fit – and also tracks your moods and emotional triggers. Each conversation teaches it a little more about how to nudge your behavior, guiding your decisions while sounding empathetic. What feels like exceptional service is, in reality, a calculated strategy to lock in your loyalty by exploiting your emotional patterns. Traditional loyalty programs like the supermarket club card or rewards card pale in comparison. By analyzing preferences, moods, and triggers, empathetic AI digs into the most personal corners of human behavior. For businesses, it’s a goldmine; for consumers, it’s a minefield. And it raises a new set of ethical questions about manipulation, regulation, and consent. The Legal Loopholes of Empathetic AI Under the General Data Protection Regulation (GDPR), consumer preferences are classified as personal data, not sensitive data. That distinction matters. While GDPR requires businesses to handle personal data transparently and lawfully, it doesn’t extend the stricter protections reserved for health, religious beliefs, or other special categories of information. This leaves businesses free to mine consumer preferences in ways that feel strikingly personal – and surprisingly unregulated. The EU AI Act, introduced in mid-2024, goes one step further, requiring companies to disclose when users are interacting with AI. But disclosure is just the beginning. The AI Act doesn’t touch using behavioral data or mimicking emotional connection. Joanna Bryson, Professor of Ethics & Technology at the Hertie School, noted in a recent exchange: “It’s actually the law in the EU under the AI Act that people understand when they are interacting with AI. I hope that might extend to mandating reduced anthropomorphism, but it would take some time and court cases.” Anthropomorphism, the tendency to project human qualities onto non-humans, is ingrained in human nature. Simply stating that you’re interacting with an AI doesn’t stop it. The problem is that it can lull users into a false sense of trust, making them more vulnerable to manipulation. Empathy-as-a-Service could transform customer experiences, making interactions smoother, more engaging, and hyper-personalised. But there’s a cost. Social media already showed us what happens when human interaction becomes a commodity – and empathetic AI could take that even further. This technology could go beyond monetizing attention to monetizing emotions in deeply personal and private ways. A Question of Values As empathetic AI becomes mainstream, we have to ask: are we ready for a world where emotions are just another digital service – scaled, rented, and monetized? Regulation like the EU AI Act is a step in the right direction, but it will need to evolve fast to keep pace with the sophistication of these systems and the societal boundaries they’re starting to push. The future of empathetic AI isn’t just a question of technological progress – it’s a question of values. What kind of society do we want to build? As we stand on the edge of this new frontier, the decisions we make today will define how empathy is shaped, and sold, in the age of AI. Lead image credit: Adobe Stock | #681726866 The latest AI developments turn empathy into just another product for sale, raising questions about ethics and regulation. The latest AI developments turn empathy into just another product for sale, raising questions about ethics and regulation. Imagine your customer service chatbot isn’t just solving your problem – it’s listening, empathizing, and sounding eerily human. It feels like it cares. But behind the friendly tone and comforting words, that ‘care’ is just a product, finetuned to steer your emotions and shape your decisions. Welcome to the unsettling reality of empathetic AI, where emotions are mimicked – and monetized. In 2024, empathetic AI took a leap forward. Hume.AI gave large language models voices that sound convincingly expressive and a perceptive ear to match. Microsoft’s Copilot got a human voice and an emotionally supportive attitude, while platforms like Character.ai and Psychologist sprouted bots that mimic therapy sessions. These developments are paving the way for a new industry: Empathy-as-a-Service, where emotional connection isn’t just simulated, it’s a product: packaged, scaled, and sold. This isn’t just about convenience – it’s about influence. Empathy-as-a-Service (EaaS), an entirely hypothetical but now plausible product, could blur the line between genuine connection and algorithmic mimicry, creating systems where simulated care subtly nudges consumer behavior. The stakes? A future where businesses profit from your emotions under the guise of customer experience. And for consumers on the receiving end, that raises some deeply unsettling questions. A Hypothetical but Troubling Scenario A Hypothetical but Troubling Scenario Take an imaginary customer service bot. One that helps you find your perfect style and fit – and also tracks your moods and emotional triggers. Each conversation teaches it a little more about how to nudge your behavior, guiding your decisions while sounding empathetic. What feels like exceptional service is, in reality, a calculated strategy to lock in your loyalty by exploiting your emotional patterns. Traditional loyalty programs like the supermarket club card or rewards card pale in comparison. By analyzing preferences, moods, and triggers, empathetic AI digs into the most personal corners of human behavior. For businesses, it’s a goldmine; for consumers, it’s a minefield. And it raises a new set of ethical questions about manipulation, regulation, and consent. The Legal Loopholes of Empathetic AI The Legal Loopholes of Empathetic AI Under the General Data Protection Regulation (GDPR), consumer preferences are classified as personal data, not sensitive data. That distinction matters. While GDPR requires businesses to handle personal data transparently and lawfully, it doesn’t extend the stricter protections reserved for health, religious beliefs, or other special categories of information. This leaves businesses free to mine consumer preferences in ways that feel strikingly personal – and surprisingly unregulated. The EU AI Act, introduced in mid-2024, goes one step further, requiring companies to disclose when users are interacting with AI. But disclosure is just the beginning. The AI Act doesn’t touch using behavioral data or mimicking emotional connection. Joanna Bryson, Professor of Ethics & Technology at the Hertie School, noted in a recent exchange: “It’s actually the law in the EU under the AI Act that people understand when they are interacting with AI. I hope that might extend to mandating reduced anthropomorphism, but it would take some time and court cases.” Anthropomorphism, the tendency to project human qualities onto non-humans, is ingrained in human nature. Simply stating that you’re interacting with an AI doesn’t stop it. The problem is that it can lull users into a false sense of trust, making them more vulnerable to manipulation. Empathy-as-a-Service could transform customer experiences, making interactions smoother, more engaging, and hyper-personalised. But there’s a cost. Social media already showed us what happens when human interaction becomes a commodity – and empathetic AI could take that even further. This technology could go beyond monetizing attention to monetizing emotions in deeply personal and private ways. A Question of Values A Question of Values As empathetic AI becomes mainstream, we have to ask: are we ready for a world where emotions are just another digital service – scaled, rented, and monetized? Regulation like the EU AI Act is a step in the right direction, but it will need to evolve fast to keep pace with the sophistication of these systems and the societal boundaries they’re starting to push. The future of empathetic AI isn’t just a question of technological progress – it’s a question of values. What kind of society do we want to build? As we stand on the edge of this new frontier, the decisions we make today will define how empathy is shaped, and sold, in the age of AI. Lead image credit: Adobe Stock | #681726866 Lead image credit: Adobe Stock | #681726866 Adobe Stock | #681726866

Empathy-as-a-Service: When AI Feels Too Human, What’s at Stake?

Seatbelts are safety, but what is safety for seatbelts if there are risks they face in certain situations? How far down can safety be designed for systems, with respect to designing safety for safety systems? Technologies, or efficiency systems, do not just advance better than safety can apply, but technologies do not have a basis--naturally--for safety. A human, on average, anywhere on the body would quickly move away from a sudden hot object. Safety is built in, against negative and dangerous experiences, that either automatically or not, it is possible to avoid danger, without having a system for safety in one part of the body, that may then need its own safety system, or have safety [apply] somewhere not everywhere. Simply, experience is the basis of safety for humans as a means of survival in society. Experience also is the source of progress in society as laws are based on the possibility that manypeople will try to avoid the negative experiences that will result from breaking the law. Laws or regulations often build on human affect or experience, such that laws are not made for people who will not be affected--if there is a penalty. So human society is not just intelligence driven, but of experiences--by affect. This is a major contrast with technology. Even with several safety systems in automobiles, they still do not know if they crumple. So while they tryto be safe, it is still for people, not for them, which, assuming it were possible, would have been a better end of safety--so to speak. AI is a technology, which is not one in a physical space--where safety seems easier. Digital is too flexible and of a different phase, that it is possible to make large changes and adjustments without the same efforts in the physical. For example, drawing on a wall and on a PC, or driving simulations and so forth, are different from the physical, in most cases. Social media showed that digital contents can be difficult to regulate, with scale, evasion, and speed. Digital, like other technologies, have no affect, so they experience nothing to indicate that they should not allow themselves to be used for one negative purpose or another. AI, though useful, is already getting misused, with several major stories in the last two years. Making AI safe is a problem preliminary of AI lacking experience, hence affect, before any technical hill. How can AI have affect? How can this affect become the basis for AI alignment, such that whenever it is misused, it can know that there is a penalty for it? How can this be adopted so that it becomes the basis for regulation, rather than the common suggestions of inspection or monitoring? There have been several news articles about departures from OpenAI in the last year, with suggestions about not doing enough for safety and alignment. While those with the experience may be certain of what they are saying, across the industry, there is still no major safety tool that is model-independent, against many of the known risks of AI today. This is not about guardrails or red-teaming, but about the misuses of AI, and not a lot of answers, from any source against those problems. The core of AI safety research could be to look at how human affect works with human intelligence for safety, then to explore how to develop parallels for AI, converting parameters into algorithms, to pave way against this very difficult AI safety problem, mirroring the human mind. There is a recent story on TechCrunch, OpenAI loses another lead safety researcher, Lilian Weng, stating that, "Another one of OpenAI’s lead safety researchers, Lilian Weng, announced on Friday she is departing the startup. Weng served as VP of research and safety since August, and before that, was the head of the OpenAI’s safety systems team. Weng’s departure marks the latest in a long string of AI safety researchers, policy researchers, and other executives who have exited the company in the last year, and several have accused OpenAI of prioritizing commercial products over AI safety." There is a recent feature on Quanta Magazine, Debate May Help AI Models Converge on Truth, stating that, "Let two large models debate the answer to a given question, with a simpler model (or a human) left to recognize the more accurate answer. In theory, the process allows the two agents to poke holes in each other’s arguments until the judge has enough information to discern the truth. Building trustworthy AI systems is part of a larger goal called alignment, which focuses on ensuring that an AI system has the same values and goals as its human users. Today, alignment relies on human feedback — people judging AI. But human feedback may soon be insufficient to ensure the accuracy of a system. In recent years, researchers have increasingly called for new approaches in “scalable oversight,” which is a way to ensure truth even when superhuman systems carry out tasks that humans can’t. Computer scientists have been thinking about scalable oversight for years. Debate emerged as a possible approach in 2018, before LLMs became as large and ubiquitous as they are today." There is a new blogpost by character.ai, Community Safety Updates, stating that, "Moving forward, we will be rolling out a number of new safety and product features that strengthen the security of our platform without compromising the entertaining and engaging experience users have come to expect from Character.AI. These include: Changes to our models for minors (under the age of 18) that are designed to reduce the likelihood of encountering sensitive or suggestive content. Improved detection, response, and intervention related to user inputs that violate our Terms or Community Guidelines. A revised disclaimer on every chat to remind users that the AI is not a real person. Notification when a user has spent an hour-long session on the platform with additional user flexibility in progress." Seatbelts are safety, but what is safety for seatbelts if there are risks they face in certain situations? How far down can safety be designed for systems, with respect to designing safety for safety systems? Technologies, or efficiency systems, do not just advance better than safety can apply, but technologies do not have a basis--naturally--for safety. A human, on average, anywhere on the body would quickly move away from a sudden hot object. Safety is built in, against negative and dangerous experiences, that either automatically or not, it is possible to avoid danger, without having a system for safety in one part of the body, that may then need its own safety system, or have safety [apply] somewhere not everywhere. Simply, experience is the basis of safety for humans as a means of survival in society. Experience also is the source of progress in society as laws are based on the possibility that many people will try to avoid the negative experiences that will result from breaking the law. many Laws or regulations often build on human affect or experience, such that laws are not made for people who will not be affected--if there is a penalty. So human society is not just intelligence driven, but of experiences--by affect. This is a major contrast with technology. Even with several safety systems in automobiles, they still do not know if they crumple. So while they try to be safe, it is still for people, not for them, which, assuming it were possible, would have been a better end of safety--so to speak. try AI is a technology, which is not one in a physical space--where safety seems easier. Digital is too flexible and of a different phase, that it is possible to make large changes and adjustments without the same efforts in the physical. For example, drawing on a wall and on a PC, or driving simulations and so forth, are different from the physical, in most cases. Social media showed that digital contents can be difficult to regulate, with scale, evasion, and speed. Digital, like other technologies, have no affect, so they experience nothing to indicate that they should not allow themselves to be used for one negative purpose or another. AI, though useful, is already getting misused, with several major stories in the last two years. Making AI safe is a problem preliminary of AI lacking experience, hence affect, before any technical hill. How can AI have affect? How can this affect become the basis for AI alignment , such that whenever it is misused, it can know that there is a penalty for it? How can this be adopted so that it becomes the basis for regulation, rather than the common suggestions of inspection or monitoring? How can this affect become the basis for AI alignment There have been several news articles about departures from OpenAI in the last year, with suggestions about not doing enough for safety and alignment. While those with the experience may be certain of what they are saying, across the industry, there is still no major safety tool that is model-independent, against many of the known risks of AI today. This is not about guardrails or red-teaming, but about the misuses of AI, and not a lot of answers, from any source against those problems. The core of AI safety research could be to look at how human affect works with human intelligence for safety, then to explore how to develop parallels for AI, converting parameters into algorithms, to pave way against this very difficult AI safety problem, mirroring the human mind. There is a recent story on TechCrunch , OpenAI loses another lead safety researcher, Lilian Weng , stating that, "Another one of OpenAI’s lead safety researchers, Lilian Weng, announced on Friday she is departing the startup. Weng served as VP of research and safety since August, and before that, was the head of the OpenAI’s safety systems team. Weng’s departure marks the latest in a long string of AI safety researchers, policy researchers, and other executives who have exited the company in the last year, and several have accused OpenAI of prioritizing commercial products over AI safety." TechCrunch OpenAI loses another lead safety researcher, Lilian Weng There is a recent feature on Quanta Magazine , Debate May Help AI Models Converge on Truth , stating that, "Let two large models debate the answer to a given question, with a simpler model (or a human) left to recognize the more accurate answer. In theory, the process allows the two agents to poke holes in each other’s arguments until the judge has enough information to discern the truth. Building trustworthy AI systems is part of a larger goal called alignment, which focuses on ensuring that an AI system has the same values and goals as its human users. Today, alignment relies on human feedback — people judging AI. But human feedback may soon be insufficient to ensure the accuracy of a system. In recent years, researchers have increasingly called for new approaches in “scalable oversight,” which is a way to ensure truth even when superhuman systems carry out tasks that humans can’t. Computer scientists have been thinking about scalable oversight for years. Debate emerged as a possible approach in 2018, before LLMs became as large and ubiquitous as they are today." Quanta Magazine Debate May Help AI Models Converge on Truth There is a new blogpost by character.ai , Community Safety Updates , stating that, "Moving forward, we will be rolling out a number of new safety and product features that strengthen the security of our platform without compromising the entertaining and engaging experience users have come to expect from Character.AI. These include: Changes to our models for minors (under the age of 18) that are designed to reduce the likelihood of encountering sensitive or suggestive content. Improved detection, response, and intervention related to user inputs that violate our Terms or Community Guidelines. A revised disclaimer on every chat to remind users that the AI is not a real person. Notification when a user has spent an hour-long session on the platform with additional user flexibility in progress." character.ai Community Safety Updates

OpenAI Alignment Departures: What Is the AI Safety Problem?

Get Rich Building Verticalized AI Wrappers (Even If You Don’t Code)

রিচ বিল্ডিং ভার্টিক্যালাইজড এআই র‍্যাপার পান (এমনকি যদি আপনি কোড না করেন)

Werden Sie reich durch die Entwicklung vertikaler KI-Wrapper (auch wenn Sie nicht programmieren können)

Hazte rico creando contenedores de IA verticalizados (aunque no sepas programar)

Devenez riche en créant des wrappers d’IA verticalisés (même si vous ne codez pas)

वर्टिकलाइज़्ड AI रैपर बनाकर अमीर बनें (भले ही आप कोड न जानते हों)

Menjadi Kaya dengan Membangun Pembungkus AI Vertikal (Bahkan Jika Anda Tidak Membuat Kode)

垂直統合型 AI ラッパーを構築して金持ちになる (コードを書かなくても)

Rich Building вертикалдандырылған AI орауыштарын алыңыз (тіпті кодтамасаңыз да)

수직화된 AI 래퍼를 구축하여 부자가 되세요(코딩하지 않아도)

Баян барилгын босоо AI боодолтой бол (та код бичээгүй байсан ч)

Fique rico construindo wrappers de IA verticalizados (mesmo que você não codifique)

Разбогатейте, создавая вертикальные оболочки ИИ (даже если вы не пишете код)

Hel dhismo qani ah oo toosan AI ku duub (Xitaa Haddi aadan Codayn)

Get Rich Building Verticalized AI Wrappers (Kahit Hindi Ka Mag-code)

Dikey AI Sarmalayıcıları Oluşturarak Zengin Olun (Kodlamasanız Bile)

رچ بلڈنگ ورٹیکلائزڈ AI ریپر حاصل کریں (چاہے آپ کوڈ نہ بھی کریں)

Làm giàu bằng cách xây dựng các trình bao bọc AI theo chiều dọc (ngay cả khi bạn không viết mã)

通过构建垂直化 AI 包装器致富（即使你不会编码）

Heyo, AI Character - Stay Away from My Child; Get a Job

هيو، شخصية الذكاء الاصطناعي - ابتعد عن طفلي؛ احصل على وظيفة

Heyo, AI xarakterli - Uşağımdan uzaq dur; İş əldə edin

হেইও, এআই ক্যারেক্টার - স্ট অ্যাওয়ে ফ্রম মাই চাইল্ড; চাকরি পান

Heyo, AI lik - Stay daleko od mog djeteta; Nađi posao

Hola, personaje de IA: Aléjate de mi hijo; consigue un trabajo

Ehi, personaggio AI - Stai lontano da mio figlio; trova un lavoro

おい、AIキャラクター - 子供から離れて、仕事を探せ

Heyo, ລັກສະນະ AI - ຢູ່ຫ່າງຈາກລູກຂອງຂ້ອຍ; ຮັບວຽກ

Hej, postać AI – trzymaj się z daleka od mojego dziecka; znajdź pracę

Heyo, Karakteri i AI - Qëndro larg nga fëmija im; Merrni një Punë

Хеј, АИ лик - клони се мог детета; Нађите посао

Này, Nhân vật AI - Tránh xa con tôi ra; Kiếm việc làm đi

Testing the Depths of AI Empathy: Q3 2024 Benchmarks

የ AI ርህራሄን ጥልቀት መሞከር፡- Q3 2024 ቤንችማርኮች

Δοκιμάζοντας τα βάθη της ενσυναίσθησης της τεχνητής νοημοσύνης: Συγκριτικά σημεία αναφοράς τρίτου τριμήνου 2024

Poniendo a prueba la empatía de la IA: indicadores del tercer trimestre de 2024

Tester les profondeurs de l'empathie de l'IA : repères du troisième trimestre 2024

Az AI empátia mélységének tesztelése: 2024. harmadik negyedévi referenciaértékek

AI 共感の深淵をテストする: 2024 年第 3 四半期のベンチマーク

ხელოვნური ინტელექტის ემპათიის სიღრმეების ტესტირება: 2024 წლის მესამე კვარტლის ნიშნები

ការសាកល្បងជម្រៅនៃការយល់ចិត្ត AI: គោលពិន្ទុ Q3 2024

AI empātijas dziļuma pārbaude: 2024. gada 3. ceturkšņa etaloni

Тестирање на длабочините на емпатијата со вештачка интелигенција: репери од кварталот 3 2024 година

Тестирование глубины эмпатии ИИ: контрольные показатели третьего квартала 2024 года

AI empatiyasining chuqurligini sinab ko'rish: 2024 yil 3-chorak ko'rsatkichlari

The Benefit And Folly of AI in Education: Navigating Ethical Challenges and Cognitive Development

AI companions are more than just a passing trend; they could become a $150 billion industry by 2030, according to a recent article by Ark Invest. With the potential to revolutionize how we interact with technology and each other, AI companionship represents the next frontier in digital entertainment. Companion apps like Replika, Gatebox, HeraHaven, and SoulMachines attract millions of users every month, according to Similarweb data. But if you’re considering building an AI companion, where do you start? Specifically, which large language model (LLM) should you choose to power your app? Let me walk you through the best open-source LLMs currently available and why they might be the perfect fit for your project. Choosing the Right LLM for AI Companionship When thinking about AI companionship, one of the first decisions you’ll need to make is what kind of open-source LLM to use. There are several types of LLMs, but they generally fall into three categories: base models, instruction-tuned models, and chat-tuned models. Why Chat-Tuned Models Are Essential Base models are the raw, unrefined versions of LLMs that have been trained on large datasets but lack the fine-tuning required for specific tasks. Instruction-tuned models have been refined to follow human instructions more accurately. However, when it comes to creating AI companions, chat-tuned models are the way to go. These models are specifically optimized for conversational interactions, making them better suited for the nuanced, ongoing dialogues that characterize AI companionship. Model Size: How Big Is Big Enough? Another key consideration is the size of the model. For most conversational chatbots, an 8B parameter model should suffice. These models offer a good balance between performance and resource efficiency, making them ideal for most applications. However, if you’re looking to go above and beyond—perhaps aiming to create a more sophisticated AI—you might want to explore models with up to 70B parameters. The 5 Best Open-Source LLMs for AI Companionship Here’s a list of some of the best open-source LLMs that are particularly well-suited for creating AI companions: 1. Hermes-3 Llama-3.1-8B Hermes-3 Llama-3.1-8B is a powerful model that’s been finely tuned for chat applications. It offers a good balance between capability and efficiency, making it a top choice for AI companionship. 2. Yi-1.5-9B-Chat Yi-1.5-9B-Chat is another excellent option, boasting a slightly larger parameter count, which allows for more complex and nuanced conversations. If your AI companion needs to handle a wide range of topics with ease, this model is worth considering. 3. InternLM2 5-7B Chat InternLM2 5-7B Chat is a lighter model that still packs a punch. Its smaller size makes it ideal for applications where resources are limited but conversational depth is still a priority. 4. Humanish-Roleplay-Llama-3.1-8B Humanish-Roleplay-Llama-3.1-8B is designed for roleplay scenarios, making it perfect for users who want an AI companion that can take on different personas. Whether it’s a friendly chat or a more complex interaction, this model excels at maintaining character. 5. OpenChat-3.5-1210 OpenChat-3.5-1210 is one of the larger models on this list and is optimized for deep, engaging conversations. If your AI companion needs to offer an immersive experience, this model is a strong contender. If you’re interested in seeing how these models stack up against others, you can visit the Open LLM Leaderboard for the most recent comparisons. Taking Your AI Companion to the Next Level with Fine Tuning While these models are impressive right out of the box, there’s always room for improvement. With enough prompting, you should be able to get decent results for most applications. However, if you’re looking for a more advanced approach, consider fine-tuning one of these models to better suit your specific needs. Fine-tuning allows you to tweak the model’s behavior, making it more aligned with your desired user experience. In conclusion, the potential for AI companions is vast, and the right LLM can make all the difference in creating a compelling, engaging experience. Whether you’re building the next million user app or a more niche application, these open-source models provide a strong foundation for success. So, dive in, experiment, and see what you can create with the incredible power of modern LLMs. AI companions are more than just a passing trend; they could become a $150 billion industry by 2030, according to a recent article by Ark Invest . With the potential to revolutionize how we interact with technology and each other, AI companionship represents the next frontier in digital entertainment. according to a recent article by Ark Invest Companion apps like Replika, Gatebox, HeraHaven, and SoulMachines attract millions of users every month, according to Similarweb data. But if you’re considering building an AI companion, where do you start? Specifically, which large language model (LLM) should you choose to power your app? Let me walk you through the best open-source LLMs currently available and why they might be the perfect fit for your project. Choosing the Right LLM for AI Companionship When thinking about AI companionship, one of the first decisions you’ll need to make is what kind of open-source LLM to use. There are several types of LLMs, but they generally fall into three categories: base models, instruction-tuned models, and chat-tuned models. Why Chat-Tuned Models Are Essential Base models are the raw, unrefined versions of LLMs that have been trained on large datasets but lack the fine-tuning required for specific tasks. Instruction-tuned models have been refined to follow human instructions more accurately. However, when it comes to creating AI companions, chat-tuned models are the way to go. These models are specifically optimized for conversational interactions, making them better suited for the nuanced, ongoing dialogues that characterize AI companionship. Model Size: How Big Is Big Enough? Another key consideration is the size of the model. For most conversational chatbots, an 8B parameter model should suffice. These models offer a good balance between performance and resource efficiency, making them ideal for most applications. However, if you’re looking to go above and beyond—perhaps aiming to create a more sophisticated AI—you might want to explore models with up to 70B parameters. The 5 Best Open-Source LLMs for AI Companionship Here’s a list of some of the best open-source LLMs that are particularly well-suited for creating AI companions: 1. Hermes-3 Llama-3.1-8B 1. Hermes-3 Llama-3.1-8B Hermes-3 Llama-3.1-8B is a powerful model that’s been finely tuned for chat applications. It offers a good balance between capability and efficiency, making it a top choice for AI companionship. 2. Yi-1.5-9B-Chat 2. Yi-1.5-9B-Chat Yi-1.5-9B-Chat is another excellent option, boasting a slightly larger parameter count, which allows for more complex and nuanced conversations. If your AI companion needs to handle a wide range of topics with ease, this model is worth considering. 3. InternLM2 5-7B Chat 3. InternLM2 5-7B Chat InternLM2 5-7B Chat is a lighter model that still packs a punch. Its smaller size makes it ideal for applications where resources are limited but conversational depth is still a priority. 4. Humanish-Roleplay-Llama-3.1-8B 4. Humanish-Roleplay-Llama-3.1-8B Humanish-Roleplay-Llama-3.1-8B is designed for roleplay scenarios, making it perfect for users who want an AI companion that can take on different personas. Whether it’s a friendly chat or a more complex interaction, this model excels at maintaining character. 5. OpenChat-3.5-1210 5. OpenChat-3.5-1210 OpenChat-3.5-1210 is one of the larger models on this list and is optimized for deep, engaging conversations. If your AI companion needs to offer an immersive experience, this model is a strong contender. If you’re interested in seeing how these models stack up against others, you can visit the Open LLM Leaderboard for the most recent comparisons. Open LLM Leaderboard Taking Your AI Companion to the Next Level with Fine Tuning While these models are impressive right out of the box, there’s always room for improvement. With enough prompting, you should be able to get decent results for most applications. However, if you’re looking for a more advanced approach, consider fine-tuning one of these models to better suit your specific needs . Fine-tuning allows you to tweak the model’s behavior, making it more aligned with your desired user experience. consider fine-tuning one of these models to better suit your specific needs In conclusion, the potential for AI companions is vast, and the right LLM can make all the difference in creating a compelling, engaging experience. Whether you’re building the next million user app or a more niche application, these open-source models provide a strong foundation for success. So, dive in, experiment, and see what you can create with the incredible power of modern LLMs.

The 5 Best Open-Source LLMs for AI Companionship

Stories, News & Mentions about CHARACTERAI

Categories

Trending Topics

Stories, News & Mentions about CHARACTERAI

Characterai’s Stories on HackerNoon

Categories

Trending Topics