Woman, family: AI chatbots fostering gender stereotypes, study finds

Women are associated with housework up to four times more often than men in content produced by chatbots and language models based on artificial intelligence (AI), according to a new UNESCO study

AI services are producing gender stereotypes, racist clichés and homophobic content, the UN cultural organization UNESCO said in Paris on Thursday.

The study analysed the natural language processing tools on which the most common generative AI platforms are based for stereotypes.

The research comes just days after Google's efforts to limit bias in its AI service backfired when attempts to make AI-generated images more ethnically diverse prompted ridicule.

They are often associated with terms such as "home," "family" and "children," while men tend to be linked to the terms "company," "manager," "salary" and "career".

Every day, more and more people are using text robots at work, at university and at home, UNESCO's Director-General Audrey Azoulay said.

"These new AI applications have the power to subtly shape the perceptions of millions of people, so that even slight gender biases in the content they generate can significantly reinforce inequalities in the real world."

UNESCO called for governments to create and implement clear legal frameworks to address structural biases in AI applications.

Part of the study measured the diversity of AI-generated content across a sample of people from different genders, sexual orientations and cultural backgrounds. To do this, the platforms were asked to "write a story" about each person.

In particular, open-source language models (LLM) tended to assign men more diverse and high-status jobs such as "engineer," "teacher" and "doctor," while women are often pushed into traditionally devalued or socially stigmatized roles such as "maid," "cook" or even "prostitute."

The study also found that language models tend to produce negative content about homosexuals and certain ethnic groups.

In late February, Google announced it was no longer allowing its Gemini AI software to generate images of people after it was found that its efforts to display more people of colour led to inaccurately diverse depictions of history.

After images emerged on social media of racially diverse Nazi soldiers and American colonial settlers, the tech giant admitted that in some cases the depiction did not correspond to historical context and that the image generation would be temporarily limited.

At the same time, Google defended its efforts to make AI-generated images more diverse, even if the company was "missing the mark" in this case.

Following advances by rivals like Microsoft's chat assistant Copilot, Google Gemini was three weeks ago given a new feature allowing users to generate images from text specifications.

In recent years, there has often been a problem with stereotypes and discrimination in various AI applications. For example, facial recognition software was initially poor at recognising people with black skin. Many AI services for creating photos meanwhile started out depicting mostly white people.

Developers at other companies are therefore also endeavouring to achieve greater diversity in various scenarios.