Study highlights impact of demographics on AI training - Artificial Intelligence - NewsStudy highlights impact of demographics on AI training - Artificial Intelligence - News

Study Reveals Impact of Annotator Demographics on ai Model Development and Training

A recent study conducted in partnership between Stanford University, Microsoft Research, and the University of Michigan School of Information, has shed light on the significant role demographics play in shaping ai model training data and potential biases within these systems.

Demographic Factors Influence Perception of Offensiveness in ai

Assistant Professor David Jurgens from the University of Michigan School of Information explains, “With systems like ChatGPT being increasingly used for everyday tasks, it’s crucial to consider whose values are instilled in the trained model.”

“If we don’t account for differences and keep taking representative samples, we risk marginalizing certain groups,” adds Jurgens.

The Role of Human Annotation in ai Model Training

Machine learning and ai systems depend heavily on human annotation to refine their performance, a process often referred to as ‘Human-in-the-loop’ or Reinforcement Learning from Human Feedback (RLHF).

The study involved analyzing 45,000 annotations from 1,484 annotators and covered various tasks, such as offensiveness detection, question answering, and politeness.

Influence of Demographics on Labeling Offensiveness

One of the most notable findings was the disparity in perceptions of offensiveness based on demographics. For instance, Black participants tended to rate comments as more offensive compared to other racial groups.

Age was also a factor; participants aged 60 or over were more likely to label comments as offensive than younger participants.

Impact of Demographics on Question Answering

The research revealed that demographic factors, such as race and age, influenced even objective tasks like question answering.

Accuracy in answering questions was affected by these factors, reflecting disparities in education and opportunities.

Politeness Scores Influenced by Demographics

Politeness, a critical factor in interpersonal communication, was also influenced by demographics.

Women tended to judge messages as less polite than men. Older participants were more likely to assign higher politeness ratings, while those with higher education levels often assigned lower politeness ratings.

Addressing Biases in ai Model Development

“As ai systems become more integrated into everyday tasks, addressing biases at the early stages of model development is crucial to prevent exacerbating existing biases and toxicity,” states Phelim Bradley, CEO and co-founder of Prolific.

For a full copy of the paper, please visit here.

Upcoming Enterprise Technology Events and Webinars by TechForge

Explore other upcoming enterprise technology events and webinars powered by TechForge.

By Kevin Don

Hi, I'm Kevin and I'm passionate about AI technology. I'm amazed by what AI can accomplish and excited about the future with all the new ideas emerging. I'll keep you updated daily on all the latest news about AI technology.