Annie Xiong ’25, German

Annie Xiong ’25

Major

German

Project Title

Norm Surfacing in Television News and LLMs

Minor(s): Environmental Studies, History

I investigated the prevalence of normative language in the television news and the ability for the large language model ChatGPT to identify normative language. Alongside a team of expert coders, I worked to code normative statements in news transcripts from CNN, MSNBC and Fox News and these same transcripts were coded by ChatGPT. Throughout the internship, I received training on social norms to become an expert coder and sorted through over 3,000 news segments. Comparing our expert coding to ChatGPT’s results, I found that ChatGPT has an over 50% accuracy when identifying norms, meaning that it could be a relatively good tool for screening out non-normative text. ChatGPT also had the best accuracy for identifying descriptive norms, which are norms that describe social behaviors and beliefs. Future research topics could include looking at different features of norms, such as text length or topic, to identify other predictors of ChatGPT accuracy.