AI crab Jesus and the problem with Facebook spam
Plus fair training, AI cookbooks and more AI Jesus
I set out this week to do a deep dive on AI-generated content in politics groups on Facebook. It seemed a relatively straightforward piece of research, finding odd Boomer misinformation trends on Facebook. It was something I did all the time in the immediate aftermath of the 2016 US presidential elections, minus the AI element.
I figured generative AI would prove a welcome replacement to the badly formatted shitposting so beloved of Boomers over the past two decades. But no, the early groups I joined (all the crazy stuff is in groups, often private, that don’t show on the soon soon-to-be sunsetted Meta-monitoring tool CrowdTangle) stuck with the tried and true mix of recycled memes and links to dodgy websites. There is some dodgy stuff enabled by AI content, BBC recently had a strong report on Trump supporters targeting black voters with faked AI images, but it’s not as prevalent as I expected.
Instead, I found lots of posts like this:
AI-generated people with captions that request praise. Marvel at my (AI, fake) son’s beautiful (AI, fake) painting! Rate my (AI, fake) granny’s beautiful (AI, fake) crochet work! Mark my (an AI, fake man) first attempt at making a delicious (and AI, and fake) pizza!
It’s odd and a little funny until you look at the engagement. Over seven thousand comments and 239 shares on an obvious AI image of a teenager with a cake (cake’s are a popular gimmick). Most of the comments are positive, pelnty of simple “very nice job” or “beautiful, well done”. If a scammer needed a list of people who were susceptible to an online scam they could start with the 7,000 commenters below an AI cake image.
In fact, very meta on Meta, here is a scammer commenting below one of these AI scam posts:
On a cursory search, I found lots of pages with over 100,000 likes. Some link to generic web pages, others include contact information for ‘advertising’. The spamming isn’t new, Facebook has always felt a lot more spammy than other platforms, but the generative AI content is. And the level of apparent automation in the process sometimes leads to just plain insane posts, this is my favourite:
But, four thousand comments! That’s a lot of engagement. There’s a deeper dive on these types of pages from researchers at Standford Internet Observatory here (PDF).
So what’s the point of all this? Well Chris Alsikkan, who outlined the trend with his great recent X thread, put it best:
If some of these scammy pages do suddenly pivot to political content in the coming months it will get press attention. Unlike 2016, most newsrooms now have a decent internet culture reporter, or at least some reporters who understand maintaining an internet culture beat. And 2016 spawned an industry of researchers and former journalists focused on misinformation and disinformation trends. It’s good we have eyes on it, of course.
But much of the reaction will focus more on AI doomism and less on how we’ve built an internet where people can be relied upon to complement fake food products created by fake strangers. It’s simultaneously wholesome and incredibly poignant, all these people being very nice and supportive towards illusions.
Yesterday Ryan Broderick wrote, “I do think it’s probably time to accept that we have reached the limitations of the social web as it’s currently constructed.” Scrolling past a selection of popular crocheting AI grannies today, I’m inclined to agree.
Small Bits #1: Fairly trained AI companies
Fairly Trained is the brainchild of creator advocate, and ex-Stability AI employee, Ed Newton-Rex and his non-profit does exactly what it says, certifying AI companies that use ethically sourced training data. They have recently announced five new AI models that are fairly trained certified, should you wish to check out some ethically trained AI models. You can read more about it here.
Small Bits #2: The AI Cookbook
There is no shortage of AI-generated books on Amazon, but The Complete Crock Pot Cookbook For Beginners seems to have suckered in a lot of customers. The original Amazon page, complete with suspiciously positive reviews, has since been taken down but you can read some of the red flag relating to the book in a viral X thread here.
Small Bits #3: The AI Jesus
I found this after writing about AI crab Jesus (above) but yeah, “made this with my own hands! Thanks to everyone who appreciates this” is going to caption a lot of AI Jesus content over the coming year.