A New Classifier Profiles a Political Image by the Identities That Divide Over It
The same photograph of a protest can strike one viewer as dangerous unrest and another as legitimate dissent, and which way a person reads it tends to track their politics. In a new paper, CDS Faculty Fellow Elena Sirotkina built a classifier that predicts, from an image alone, how that divide will fall.
The paper, “Unpacking the Eye of the Beholder: Social Location, Identity, and the Moving Target of Political Perspectives,” takes aim at a habit baked into most computational analysis of political media. Tools that score a tweet’s sentiment or a photo’s framing usually return one number per item, on the assumption that any careful observer would land in roughly the same place. Sirotkina argues that the assumption falls apart the moment the content touches politics.
“Computational tools for analyzing political content typically assign a single sentiment label to each text or image and assume reasonable observers would converge on it,” Sirotkina said. “That’s both theoretically and empirically wrong. Political science has shown for decades that any content touching political or social identity produces structured disagreement, because interpretation passes through the identity of the observer.”
Her classifier, the Perspectivist Visual Political Sentiment model, or PVPS, is built to keep that disagreement rather than average it away. It learned from roughly 82,000 ratings of 1,264 political images, supplied by 5,575 U.S. adults who each scored images on a seven-point scale and shared demographic information that included party, ideology, age, gender, education, income, and ethnicity. Instead of one score per image, PVPS returns a profile: which audiences rate the image more favorably, along which fault lines, and with what confidence.
“Think of PVPS as a way to run a survey on autopilot,” Sirotkina said. “You show it a political image and it estimates the perspectives you would get from a young Republican, an older Democrat, or a liberal woman, and other audience segments political scientists care about.”
Not every dividing line shows up in the pixels. The model read partisan splits most cleanly, predicting which side a group fell on correctly about 69% of the time for party and 79% for ideology, against a coin-flip baseline of 50%. Purely demographic gaps like gender and age proved fainter, and recovering them at all took a second modeling stage. Political identity, it turns out, leaves a sharper visual signature than demographics do.
The payoff comes when PVPS is pointed at existing research. Sirotkina ran it over the UCLA Protest Image Dataset, where earlier work by Won, Steinert-Threlkeld, and Joo had tied perceived violence to visual features like fire and the presence of police. Those features still predicted violence. But PVPS added a second layer.
“Object recognition can tell you that an image contains fire, riot police, or armed crowds, and research has shown those are the features that make people rate a scene as more violent,” Sirotkina said. “PVPS adds the second half of the picture, which is who in the audience responds to those images more favorably.”
The same approach reshaped a study of why Black Lives Matter images spread online. Enthusiasm drove sharing, the original analysis found. Running the images through PVPS showed that the motivation to share differed by audience: enthusiasm mobilized people around images favorable to Democratic women, while fear and disgust drove sharing of images favorable to Republican men.
Sirotkina sees the tool as a way to ask questions that used to require fielding a fresh survey for every image. With PVPS, a researcher can take millions of images moving through social feeds and news coverage and estimate how each would land across the electorate. That opens the door to testing whether platforms like Instagram or TikTok amplify the images most likely to divide audiences, and how visual misinformation travels through different political and social groups.
By Stephen Thomas
