Faulty AI that told people to break the law defended by New York mayor

Benedict Collins
·2 min read
AI.
AI.
  • Oops!
    Something went wrong.
    Please try again later.

A AI chatbot recently introduced to the city of New York as a way for business owners to get 24/7 access to "actionable and trusted information” has been defended by Mayor Eric Adams after it was called out for encouraging entrepreneurs to break the law.

The MyCity AI chatbot uses Microsoft’s Azure AI large language models, but is plagued by the same issues that most AI models suffer from, including giving wrong and misleading information.

In response to the criticisms, Mayor Adams stated that the AI tool is “wrong in some areas, and we've got to fix it,” further defending the chatbot saying that, “Any time you use technology, you need to put it into the real environment to iron out the kinks.”

Break the law, it's perfectly legal

The chatbot was previously reported as telling landlords that they were free to discriminate based on income, and that business owners could take workers tips as their own - despite both of these practices being illegal in New York.

Subsequent probes of the chatbot’s responses on Thursday revealed that it was still giving out incorrect information, with Reuters reporting that the AI had said that stores no longer had to accept cash as a method of payment, despite the fact that this violates New York law.

The director of the NYC Hospitality Alliance, Andrew Rigie, said “I commend the city for trying to use AI to help businesses, but it needs to work. “If when I ask a question and then I have to go back to my lawyers to know whether or not the answer is correct, it defeats the purpose.”

The MyCity chatbot has since been updated to state that the AI’s responses “may sometimes be inaccurate or incomplete” and that the answers given in response to queries should not be used as legal or professional advice.

More from TechRadar Pro