Bias in training data on display in weird way

Reddit r/artificial / 4/28/2026

💬 OpinionSignals & Early TrendsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The author used the same exact prompt to two different video-generating AI models to produce a ’90s toy commercial with boys and girls of different races in Halloween costumes, but both outputs contained zero girls.
In both results, the characters were consistently assigned by race in a way that the author did not anticipate (e.g., a Black boy as a pirate, an East Asian boy as a ninja, and a white boy as a spy).
The author observes that the models’ consistent pattern suggests bias learned from training data rather than reflecting the user’s prompt.
The post frames this as a surprising but enlightening demonstration of how model outputs can reveal underlying dataset biases.
The author ends by suggesting others may find the example informative, highlighting the practical need to test AI systems with controlled prompts.

So i was working on this Tabletop roleplaying game project and for my own amusement I told two different video generating ai models to generate
"a '90s toy commercial featuring boys and girls of different races in halloween costumes saying "I've got the urge to be a pirate" "ive got the urge to be a ninja!" or spy or whatever they are dressed as"
thats it thats the exact prompt, and both of them gave me very different products but both had zero girls, and in both the pirate was a black boy, the ninja an east asian boy, and the spy a white boy. Makes perfect sense in hindsight but I really didn't see it coming and most surprising (for me) is the black child as pirate. Kind of arbitrary but must be reflecting something in the data. Anyway, i found that kinda enlightening, maybe you will too, bye.

submitted by /u/Immediate_Tooth4437
[link] [comments]