AI-Driven Deepfake Abuse: A Growing Threat for Women in Serbia

The spread of AI technology has opened up new, dangerous avenues for digital abuse, with deepfake imagery becoming a significant issue in Serbia. Tens of thousands of users on Telegram are actively sharing manipulated images of women ‘undressed’ by AI, with current laws offering victims little recourse. The experience of Adrijana Petkovic, whose innocent bathroom selfie was transformed into a deepfake and circulated without her consent, highlights the urgent need for better legal protection and support.

A Personal Nightmare: The Story of Adrijana Petkovic

Adrijana Petkovic’s life took an unexpected turn when her husband received a manipulated image of her from a colleague, which had been shared widely in a local Telegram group. Despite her attempts to report the incident, the police could do little more than issue a warning to the group administrator, leaving Petkovic devastated. The response from authorities was dismissive, signaling that such violations have become normalized and leaving victims without real support or justice.

Deepfake Pornography: An Escalating Problem

Deepfake pornography is a form of image-based sexual abuse, primarily targeting women. It is particularly insidious as it leaves the victim unable to prove the image is false, perpetuating feelings of helplessness and shame. This form of abuse is not limited to Serbia; other Balkan nations like Kosovo and Croatia have also grappled with similar issues, though some have taken legislative steps to address them. For example, Croatia has criminalized the misuse of sexually explicit content, while Montenegro has introduced laws imposing prison sentences for such offenses. Serbia, however, has yet to follow suit.

Source

The Role of Telegram and AI Bots:

Platforms like Telegram are central to the spread of AI-generated deepfakes. Investigations have revealed that numerous Telegram channels and bots allow users to create explicit images by uploading photos and removing clothing digitally. Bots like ‘ClothOff’ and ‘InsOFF’ have gained massive followings, with hundreds of thousands of users participating in these activities. While the bot creators claim users must not process images without consent, there is no effective enforcement mechanism, enabling widespread abuse.

Legal and Social Gaps:

The lack of explicit legislation in Serbia means victims have little to no legal recourse. While some may consider filing private lawsuits, the process can be daunting and costly, often discouraging victims from pursuing justice. According to Vanja Macanovic from the Autonomous Women’s Centre in Belgrade, failure to act will only lead to more cases and the normalization of such behavior as a form of violence against women and girls.

Digital rights experts like Nevena Krivokapic Martinovic of the SHARE Foundation argue that platforms must strike a balance between moderation and protecting human rights. Without proper tools for users to report and combat such abuse, victims are left powerless. The persistence of figures like ‘Dzoni’, an administrator who continues to share explicit deepfakes despite past reports, underscores the limitations of current moderation practices.

The Impact on Victims and Society:

Victims like Petkovic and singer Goca Trzan, whose image was also manipulated and shared, represent different faces of the same issue. While public figures like Trzan may find ways to cope due to their visibility and resources, anonymous victims face more severe consequences, such as mental health issues and societal judgment. The shame and isolation they experience are compounded by the failure of friends, family, and society to take the issue seriously. Petkovic’s experience highlights the devastating effects of betrayal and the normalization of such abuse.

A Call for Change:

Serbia’s failure to amend its Criminal Code leaves women exposed to violations without adequate protection. Legal and digital rights experts stress the urgency of legislative action to punish perpetrators and regulate platforms that facilitate these abuses. Without changes, platforms like Telegram will remain a haven for those who exploit others through AI, and victims will continue to face an uphill battle for justice.

Until laws and platforms evolve to address these issues effectively, victims are urged to stay strong, as public figures like Trzan emphasize. Support from community organizations, awareness campaigns, and public outcry are essential first steps in pressuring both lawmakers and tech companies to take responsibility.

For now, victims like Petkovic continue to suffer, their only solace being the hope that their stories might inspire change and bring greater awareness to the issue.