Commentary: Generative AI is enabling new forms of gendered violence online — act now to protect women and girls in Singapore
As International Women's Day approaches, we celebrate the achievements and progress of women worldwide. Despite these advancements, a new shadow looms, cast by the negative applications of generative artificial intelligence (AI) — one which disproportionately affects women and girls.

This audio is AI-generated.
As International Women's Day approaches, we celebrate the achievements and progress of women worldwide. Despite these advancements, a new shadow looms, cast by the negative applications of generative artificial intelligence (AI) — one which disproportionately affects women and girls.
These new dangers and risks posed by the misuse of generative AI call for a concerted, multifaceted approach to protect women and girls in digital spaces.
GENDERED HARMS FROM GENERATIVE AI
The rise of generative AI facilitates the creation of deepfake pornography and manipulated images targeting women, exploiting their images and voices without consent. This not only violates their individual privacy and autonomy, but also perpetuates harmful gender stereotypes and contributes to the objectification of women on a wider scale.
In late January, explicit AI-generated images of pop icon Taylor Swift went viral on social media platforms and messaging apps. The images first emerged from an ongoing campaign that began last year on fringe platforms to produce sexually explicit images of famous women.
This incident brought to light the deliberate efforts of bad actors to exploit new technologies, sparking greater awareness of the immense concern non-consensual deepfake porn poses not just to celebrities like Swift, but women everywhere.
Despite its reputation for cyber vigilance, Singapore is not immune from tech-enabled sex scandals and dangers.
One example is SG Nasi Lemak, a chat group on Telegram that hosted and transmitted obscene photos and videos of women and underaged girls in Singapore. The content was often shared without their knowledge and consent.
Some netizens also set up Tumblr blogs to reshare photos of female students with sexually explicit captions, highlighting the ease with which digital platforms can be weaponised against women.
These scandals underscore a disturbing trend where digital spaces, instead of being safe havens for expression and connectivity, are increasingly becoming battlegrounds for gender-based violence.
WHY DOES THIS MATTER?
The malicious use of generative AI can have devastating effects on the lives and reputations of the victims.
In a society like Singapore’s, where social norms and reputational concerns hold significant sway, being falsely depicted in sexually compromising or exploitative situations can lead to severe consequences, including ostracisation, job loss, and psychological trauma.
Victims of AI-generated fabrications such as these may find themselves grappling with long-term repercussions of having their identities maladapted and misrepresented in digital spaces.

Beyond that, the continued creation and spread of fabricated images that sexualise women and girls — often in violent ways — also perpetuate harmful stereotypes and reinforce gender-based discrimination.
These are acute concerns for Singaporean youths, according to the Safeguarding Online Spaces (SOS) Study released on Feb 8 by the non-profit SG Her Empowerment and the youth organisation Global Shapers Community.
The study found that female youth were significantly more worried than their male peers about image-based sexual abuse, with 57 per cent of female youths expressing concern compared to 39 per cent of male youths.
The top three harms of AI-altered or generated images that Singaporean youths, both male and female, are concerned about also reflect shared anxieties about generative AI abuses: Misrepresentation of faces and bodies from the use of filters (55 per cent), sexualisation and objectification of girls and women (54 per cent), creation and sharing of AI-altered or generated images of a person without their consent (52 per cent).
POTENTIAL SOLUTIONS
Our focus group discussions jointly conducted by Global Shapers and SG Her Empowerment found that many youths saw the significance of individual action in flagging offensive content online.
When individuals report harmful, or inappropriate content, they help platforms identify and take action against content that violates their policies as well as improve their content moderation systems and policies.
On a larger scale, forums for policy decision-making, such as advisory boards, can empower youths to effect change across industry and government.
Industry examples include Snapchat parent company Snap’s inclusion of youth representatives on its safety advisory board and the creation of its Council for Digital Well-Being, as well as TikTok’s efforts to create a Youth Council, to ensure that its product and policy decisions reflect and address youth perspectives and concerns.
As digital natives, youths can contribute a wealth of insights, perspectives, and innovative ideas that can guide the development of safer online spaces, while fostering a sense of ownership and accountability in cultivating a secure online environment.
Initiatives such as the Ministry of Culture, Community & Youth’s youth panels are a commendable step in giving young people a greater say in co-developing policy recommendations and research with the government.
This process ensures that the insights and experiences of young people are not just heard, but actively considered and prioritised in the shaping of laws and guidelines that govern their online experiences. Nevertheless, tech companies and digital platform providers play a pivotal role in creating safer online environments. Implementing more robust content moderation policies, developing AI that can detect and flag deepfakes or non-consensual content, and promoting transparency in AI-generated content are essential steps.

Processes for both content moderation and generation should be sensitive to the nuances of gender-based violence and the ways in which women and girls are disproportionately targeted for online abuse and harassment, to ensure that digital spaces do not inadvertently silence victims or minimise their experiences.
Collaborations with non-governmental organisations and women's rights organisations can further provide valuable insights into addressing the unique challenges faced by women and girls in their online lives.
Government intervention is equally crucial. Policy frameworks that establish robust protections for women and girls, especially in cases of online harassment as well as the creation and distribution of content tied to image-based sexual abuse, can go a long way towards safeguarding citizens' online experiences.
These should go hand in hand with educational efforts to impart digital ethics and cyber wellness, to equip younger generations with the tools they need to navigate digital spaces responsibly.
A DIGITAL FUTURE FOR EVERYONE
As technology continues to advance at a rapid pace, there is a pressing need for greater efforts to protect and empower women and girls in the digital realm.
We need both individual and collective action towards cultivating a digital environment where women and girls can freely learn, connect and express themselves without fear. Together, we can build a digital future that champions and reflects the values of equality, respect, and inclusion, ensuring that the online world becomes a space where everyone can thrive.
ABOUT THE AUTHOR:
Calissa Man is the vice-curator of the Singapore Hub of the Global Shapers Community, an initiative by the World Economic Forum that brings together youth leaders to address local, regional and global challenges.