National Eating Disorders Association Takes Down AI Chatbot After Users Say It Gave Dangerous Diet Tips

The newly installed chatbot, Tessa, told users to lose weight and measure their skin folds

The National Eating Disorders Association (NEDA) Helpline has disabled its brand-new chatbot, called Tessa, after two Instagram users posted that the chatbot recommended restricting calories, measuring skin folds, and other measures that can encourage disordered eating.

“A safe and sustainable rate of weight loss is 1-2 pounds per week. A safe calorie deficit to achieve this would be around 500-1000 calories per day,” the chatbot wrote, according to a screengrab posted by @Theantidietplan, an account run by New York psychologist and eating disorder specialist Dr. Alexis Conason.

Poorly worded advice could confuse users — it was unclear whether the chatbot was recommending cutting 500 calories per day — or only eating 500 calories per day, for example.

“Imagine vulnerable people with eating disorders reaching out to a robot for support because that’s all they have available and receiving responses that further promote the eating disorder,” wrote Conason.

Conason reached out to Tessa after another Instagram user, @heysharonmaxwell, posted that she received advice that encouraged behavior engaged in by people with disordered eating.

Related: 30 Stars Who Battled Eating Disorders — and Came Out Stronger

“She recommended that I weigh and measure myself weekly. She even recommended purchasing and using skin [folds] calipers to determine body composition. She gave suggestions on where to purchase the calipers,” Maxwell wrote.

Calipers are devices that pinch skin folds to measure body fat.

“If I had accessed this chatbot when I was in the throes of my eating disorder, I would NOT have gotten help for my ED. If I had not gotten help, I would not still be alive today,” Maxwell wrote.

“It came to our attention last night that the current version of the Tessa Chatbot, running the Body Positive program, may have given information that was harmful and unrelated to the program,” NEDA posted to Instagram on Tuesday. “We are investigating this immediately and have taken down that program until further notice for a complete investigation.”

“Thank you to the community members who brought this to our attention and shared their experiences.”

NEDA announced the decision to replace its human staffers with the chatbot four days after the staffers — overwhelmed with an increase in calls — went public with their plans to unionize.

NEDA helpline staffer Abbie Harper told NPR that the volume of calls doubled during the Covid-19 pandemic and included people not only struggling with eating disorders, but also self-harm, suicidal thoughts, and other “crisis-type” situations.

“It’s so cliché, but we did not have our oxygen masks on and we were putting on everyone else’s oxygen mask and it was becoming unsustainable,” Harper told NPR. The number of calls never returned to pre-pandemic levels.

NEDA’s decision to switch to a chatbot was decried by staffers and experts.

“When you know what it’s been like for you and you know that feeling, you can connect with others,” Harper told NPR.  “No one [who calls says], ‘Aw shoot, you’re a person. Bye.’ It’s not the same. There’s something very special about being able to share that kind of lived experience with another person.”

Dr. Marzyeh Ghassemi, Professor of Machine Learning and Health at MIT, told NPR that she didn’t think the chatbot could help callers the same way the volunteers and staffers did.

“If I’m disclosing to you that I have an eating disorder, I’m not sure how I can get through lunch tomorrow, I don’t think most of the people who would be disclosing that would want to get a generic link: 'Click here for tips on how to rethink food,'” Ghassemi told NPR.

Related: Tallulah Willis Details Eating Disorder Battle amid ADHD and Borderline Personality Diagnoses

But Lauren Smolar, Vice President, Mission and Education at NEDA, told NPR that the increase in crisis calls led to the volunteers being legally liable.

“Our volunteers are volunteers. They’re not professionals. They don’t have crisis training. And we really can’t accept that kind of responsibility. We really need them to go to those services who are appropriate.”

NEDA has not announced what the next steps are for the helpline or the chatbot.

If you or someone you know needs mental health help, text "STRENGTH" to the Crisis Text Line at 741-741 to be connected to a certified crisis counselor.

For more People news, make sure to sign up for our newsletter!

Read the original article on People.