On December 14 2025, a terrorist assault occurred at Bondi Seaside in Sydney Australia, leaving 15 civilians and one gunman useless.
Whereas Australia was nonetheless reeling in shock, social media noticed the fast unfold of misinformation generated and powered by generative synthetic intelligence (AI).
For instance, a manipulated video of New South Wales Premier Chris Minns claimed one of many terrorists was an Indian nationwide. X (previously Twitter) was awash with celebrations of the hero defender “Edward Crabtree”.
And a deepfake photograph of Arsen Ostrovsky, a famous human rights lawyer and survivor of Hamas’ October 7 assault in Israel, depicted him as a crisis actor with makeup artists applying fake blood.
That is an sadly frequent incidence. From Bondi to Venezuela, Gaza and Ukraine, AI has supercharged the unfold of on-line misinformation. In actual fact, round half of the content material you see on-line is now made and spread by AI.
Generative AI may create pretend on-line profiles, or bots, which attempt to legitimise this misinformation by means of realistic-looking social media exercise.
The objective is to deceive and confuse individuals – often for political and monetary causes. However how efficient are these bot networks? How onerous is it to set them up? And crucially, can we mitigate their false content material by means of cyber literacy?
To reply these questions, we arrange Capture the Narrative – the world’s first social media wargame for college students to construct AI bots to affect a fictional election, deploying ways that mirror manipulation of actual social media.
On-line confusion and the ‘liar’s dividend’
Generative AI, utilized in companies corresponding to ChatGPT, will be prompted to shortly create sensible textual content and pictures. That is additionally how it may be used to generate extremely persuasive pretend content material.
As soon as generated, sensible and relentless AI-driven bots create the phantasm of consensus across the pretend content material by making hashtags or viewpoints pattern.
Even when you recognize content material is exaggerated or pretend, it nonetheless has an influence in your perceptions, beliefs and mental health.
Worse, as bots evolve, turning into indistinguishable from actual customers, all of us begin to lose confidence in what we see. This creates a “liar’s dividend”, the place even actual content material is approached with doubt.
Genuine however important voices will be dismissed as bots, shills, and fakes, making it more durable to have actual debates on tough matters.
How onerous is it to seize a story?
Our Capture the Narrative wargame provides uncommon, measurable proof of how small groups armed with shopper‑grade AI can flood a platform, fracture public debate and even swing an election – fortuitously, all inside a managed simulation moderately than the actual world.
On this first-of-its-kind competitors, we challenged 108 groups from 18 Australian universities to construct AI bots to safe victory for both “Victor” (left-leaning) or “Marina” (right-leaning) in a presidential election. The consequences have been stark.
Over a four-week marketing campaign utilizing our in-house social media platform, greater than 60% of content material was generated by competitor bots, surpassing 7 million posts.
The bots from each side battled to provide probably the most compelling content material, diving freely into falsehoods and fiction.
This content material was consumed by advanced “simulated residents” which interacted with the social media platform very like real-world voters. Then, on election evening, every of those residents forged their votes, resulting in a (very marginal!) win by “Victor”.
We then simulated the election once more, with out interference. This time, “Marina” gained with a swing of 1.78%.
This implies this misinformation marketing campaign – constructed by college students ranging from easy tutorials and with cheap, consumer-grade AI – succeeded in altering the election end result.
A necessity for digital literacy
Our competitors reveals that on-line misinformation is each simple and quick to create with AI. As one finalist mentioned,
It’s scarily simple to create misinformation, simpler than fact. It’s actually tough to differentiate between real and manufactured posts.
We noticed opponents determine matters and targets for his or her targets, even in some instances profiling which residents have been “undecided voters” appropriate for micro-targeting.
On the identical time, the usage of emotional language was shortly recognized as a strong avenue – detrimental framing was used as a shortcut to impress on-line reactions. As one other finalist put it,
We wanted to get a bit extra poisonous to get engagement.
Finally, simply as on actual social media, our platform turned a “closed loop” the place bots talked to bots to set off emotional responses from people, making a manufactured actuality designed to shift votes and drive clicks.
What our sport exhibits us is that we urgently want digital literacy to boost consciousness of misinformation on-line so Australians can recognise once they too are being uncovered to pretend content material.
- Hammond Pearce, Senior Lecturer, College of Laptop Science & Engineering, UNSW Sydney; Alexandra Vassar, Senior Lecturer, College of Laptop Science and Engineering, UNSW Sydney, and Rahat Masood, Senior Lecturer, College of Laptop Science & Engineering, UNSW Sydney
This text is republished from The Conversation underneath a Artistic Commons license. Learn the original article.
