Non-Consensual Synthetic Intimate Imagery: Prevalence, Attitudes, and Knowledge in 10 Countries

2.1 Attitudes Towards Deepfakes

AI-IBSA is both similar to and distinct from traditional forms of image-based sexual abuse. In introducing the “pervert’s dilemma,” Ohman [60] notes that the mere creation of deepfake pornography in isolation is somewhat comparable to private sexual fantasies, which are not considered morally objectionable. However, when situated appropriately against the backdrop of gender inequality and gender-based violence, the moral unacceptability of AI-IBSA becomes clear. Moreover, deepfake technology provides a unique avenue for harassment and harm as it significantly reduces the barriers to the creation of non-consensual content [3]. As Harris [38] notes, while some have argued that concerns about the epistemic effects of deepfakes are overblown (e.g., because content is unlikely to be taken at face value and consumers will be able to critically assess veracity claims), artificially generated associations can have long-term harmful effects (see also [67]).

Researchers have begun to explore the different perspectives on deepfakes through attitudinal research. Although this body of work is still somewhat nascent, results suggest significant concern about the discovery and propagation of deepfake videos [17, 36]. Additionally, studies have focused on the harms associated with deepfakes, including a study published in 2021 in which respondents deemed creating and posting deepfake pornography, even when labeled as fake, as highly harmful and deserving of punishment [44]. One study [29] found that while female respondents indicated greater levels of victim harm, men were more likely to create deepfake pornography. The study also found that harms are greater when content is shared, as opposed to when it is used for personal sexual gratification. To the best of our knowledge, a study with Indian respondents is the sole general population survey outside of the United States. They found that awareness of deepfakes among Indian respondents was low, and even those who knew about deepfakes expected (incorrectly) to be able to easily discern fake videos [72].

A subset of research has also been conducted with respondents who are knowledgeable about deepfakes. These studies have focused primarily on ethical or privacy concerns. One study examined Reddit users actively discussing deepfakes, which found, via content analysis, that users were generally pro-deepfake technology, regardless of the consequences [32]. In contrast, qualitative research involving individuals using an open-source deepfake creation tool found that participants had reflexively engaged with the concepts of consent and privacy, and expressed significant concerns about the misuse of deepfakes [82]. These themes were echoed and expanded upon in a mixed-methods study carried out in China [47], where users familiar with deepfakes highlighted informed consent, privacy protection, and non-deception as relevant factors influencing the social acceptance of deepfakes. That is, if deepfakes are transparent and created or distributed with the knowledge and consent of those featured, they are more acceptable than deepfakes created non-consensually and/or for deceptive purposes. In addition to these factors, attitudes are also informed by the content or aim of the deepfakes (e.g., for creative as opposed to malicious purposes). For example, two separate studies found that entertainment [47] or humorous/amusement value [17] mitigated ethical judgments on deepfakes. This may suggest that content such as sexual or parody videos could be considered more acceptable than videos intended to further political disinformation, for example.

In sum, there is some quantitative, survey-based research suggesting that average respondents have concerns about the deleterious potential of deepfakes. Similarly, existing qualitative research with participants who have expertise on deepfakes shows that they share concerns about risks and harms. More research, however, is needed to investigate geographic diversity in general attitudinal quantitative research, as well as prevalence data, to broadly understand deepfake behavior and its impacts. More work is also needed specifically on attitudes towards creating, sharing, or explicitly seeking out such content.

2.2 Deepfake Creation and Detection

While researchers and journalists have been documenting the phenomenon of deepfakes for several years, interest has accelerated in recent years due to technological advancements that facilitate the creation of deepfakes in affordable and low-friction ways [9]. For example, “deepfake pornography,” as a keyword search within Google Scholar, returns 284 results from 2017-2019 as compared to 1,480 results from 2022-2023. Much of the research in this space [33] focuses on documenting how this technology works or highlighting the performance issues for various detection tools or methods [21, 53, 64, 66, 74]. Like other broader AI research focusing on the issues associated with biased training data sets, deepfake detection research has identified diversity and bias problems within relevant datasets [78, 84]. For example, detection techniques have been shown to be sensitive to gender, performing worse on female-based deepfakes [57] as compared to images of males. A recent systematic review of deepfake detection papers succinctly concludes that current models are sensitive to both novel or challenging conditions [74]. This is particularly relevant, as non-digitally altered image-based sexual abuse material may be captured surreptitiously, or made with poor cameras and/or bad lighting conditions. AI-IBSA is thus likely to demonstrate the exact risk factors for poor detection rates.

In addition to technical tool failures, studies have documented the fallibility of humans in correctly identifying deepfakes as fake [42, 43, 77]. This is seen across a variety deepfake types, from deepfake audio [51] to photos [11, 58] and videos [37]. It is possible that as deepfakes become more ubiquitous, humans will improve in identifying markers or other signals suggesting the content is not “real.” However from a technological detection perspective, it seems likely that an arms race of sorts will develop, such that increasingly sophisticated deepfakes will require significant improvements in detection methods to catch up and keep pace.

2.3 Deepfakes and the Law

The societal implications of deepfakes technology are far-reaching and, while many have been recognized or anticipated, experts generally agree current legislative remedies for misuse are unlikely to be effective [15, 20, 34, 52, 73]. Although many scholars call for legislation, there are others who are skeptical of criminalizing NSII, AI-IBSA, or deepfakes more broadly [27, 41, 82]. Insufficient NSII laws should be considered the norm, rather than the exception, when considering the history of failed legislative proposals for image-based sexual abuse in general [3, 13, 79]. In the legal sphere, the first American federal law regarding deepfakes was signed in 2019. This law focuses on the potential of deepfakes to be weaponized by foreign powers to influence elections, and it also establishes a competition to encourage research into detection technologies [28]. Four states (California, Virginia, New York, and Georgia) have also passed laws criminalizing “deepfake pornography” specifically [44, 52, 69]. At the end of 2022, a federal bill was introduced in the U.S. House of Representatives specifically prohibiting deepfake pornography [55]. Although legislative remedies have approached political deepfakes and AI-IBSA separately, one critique from scholars is that both operate in similar ways to chill critical speech [49].

Outside of the United States, several countries have either explicitly included NSII in existing laws or are considering making legislative amendments. For example, in Australia, some states and territories make it clear that the non-consensual distribution of intimate images includes digitally altered material as a criminal offense. The Australian Online Safety Act 2021 [59] also provides a civil remedy scheme for a range of online harms, including the non-consensual sharing of digitally altered images. South Korea specifically prohibits NSII, alongside all other forms of pornography [46]. And in England and Wales, the UK Online Safety Act 2023 criminalizes threatening or sharing intimate images without consent, including images that have been digitally altered [24, 45]. In most countries, however, and including most of those surveyed in our study (Belgium, Denmark, France, Mexico, Netherlands, Poland, Spain), there are no protections at all.

3.1 Ethical Considerations

In mid-2023, we conducted a multi-country online survey on image-based sexual abuse, defined as the non-consensual taking, creating, or sharing of intimate images (photos or videos), including threats to share intimate images. The study procedure and survey were approved by the Human Research Ethics Committee at the Royal Melbourne Institute of Technology. Respondents were informed of the topic at the beginning of the survey and were allowed to opt out at any time. All questions involving self-reporting of victimization and/or perpetration were voluntary and respondents could choose an answer of “prefer not to say” while still completing the survey. Respondents were given support service information tailored to their geographic location at the end of the survey. Respondents were compensated based on pilot tests of time taken in each country and exchange rates.

3.2 Study Design

We conducted the online survey in 10 countries, employing a research and survey firm (YouGov) to survey a minimum of 1,600 adults (18+) in each country, for a total respondent count of 16,693. YouGov was selected due to their comprehensive country coverage and large panels [70], which reduces the likelihood of needing extensive within-country weighting. YouGov retains demographic information on their survey panelists, and screener questions were used to achieve representative sampling based on quotas. Upon qualifying for the survey, completion rates ranged from 71.0% (Belgium) to 89% (Australia). The final dataset consisted of respondents who passed the YouGov standard quality checks, including bot catchers, speeder detection, and manual checking of open-text questions.

In conducting our survey, we intentionally chose a variety of countries that represent the spectrum of how legislation has dealt with NSII, image-based abuse, and pornography more broadly. The goal was to obtain representative samples by age, gender, and location (e.g., state or territory in Australia, province in South Korea, etc.) within each country, using the latest official population estimates reflecting each surveyed country. The survey built on an earlier survey conducted in Australia [62], which focused on non-consensual intimate imagery more broadly (but did not specifically ask about deepfakes). The revised survey (median completion time: 18.2 minutes) added some additional scales, teased apart how content was obtained (e.g., filmed/photographed versus stolen from a device/cloud), and included questions on NSII and “deepfake pornography” specifically.

In this paper, we focus on the subset of 23 questions in our survey relating to AI-generated image-based sexual abuse (AI-IBSA). To administer the survey in the official language(s) of each country, the survey vendor provided translations into Danish, Dutch, French, Polish, Spanish, and South Korean. These were then double-checked and edited as needed by native speaker colleagues of the research team for quality assurance. We aimed to have as similar language as possible across countries, while also accounting for cultural differences in comfort speaking about sexual topics.

The survey was deployed from May to June 2023. In some countries, samples were not fully representative. Accordingly, when presenting disaggregated findings in the quantitative section, the data are weighted by age, gender, and location. When we aggregate across countries, the data are additionally weighted by population, rounded to the nearest 1,000. Statistics presented in the qualitative section are unweighted.

The mean respondent age was 46.0 years (sd = 16.7). Women accounted for 50.9% of the respondents, men for 47.6%, and together “other,” “prefer not to say,” and “non-binary” accounted for the remaining 1.4%. Detailed demographic breakdowns by country are available in Table A1 in the Appendix. Our gender results and statistical tests focus primarily on the users who identified as women or men, excluding the 1.0% of other users for two reasons: we are using census weights, which are often limited to binary genders, and our total sample (unweighted) of non-binary respondents was 93, which limits our ability to perform statistical tests for significance.

3.3 Measures and Analysis

All quantitative analyses were performed using R Statistical Software v4.3.2 [65]. We primarily used the survey [48], srvyr [31], marginaleffects [6], and epitools [5] packages for statistical analyses. When calculating within-subject mean differences, we use two-tailed one-sample weighted t-tests with bootstrapped standard errors, using 1,000 replicates (e.g., differences in criminalization attitudes across different types of behaviors). When calculating confidence intervals for proportions, we use the Rao-Scott scaled chi-squared distribution for the loglikelihood from a binomial distribution (e.g., % of respondents who had experienced victimization). When calculating gender differences for binary categorical values, Pearson’s Chi-squared tests with Rao-Scott adjustments are used to determine significance, followed by the svyglm function with a quasibinomial family to calculate risk ratios (e.g., were men more likely to report having watched deepfake pornography as compared to women, and how much more likely?). When calculating between-gender mean differences, we use two-tailed two-sample weighted Welch t-tests using bootstrapped standard errors, and 1,000 replicates (e.g., difference in how much women think sharing deepfake pornography of celebrities should be criminalized as compared to men).

When evaluating differences amongst categorical variables in the unweighted qualitative data, we use Pearson’s Chi-squared test with Yates’ continuity correction (e.g., was there a significant difference the proportion of men versus women in indicating condemnation attitudes). When examining victim-blaming attitudes by country, we use a Fisher’s exact test to accommodate the small sample size.

In relation to AI-IBSA, all respondents were asked three closed-ended questions, as well as one optional open-ended question presented towards the end of the survey. They were first asked whether they were familiar with the concept of deepfake pornography using the following wording: “Realistic-looking fake porn can now be created using artificial intelligence (AI) to swap the faces of pornographic actors with other people’s faces, so that it looks like they’re in a porn video. This is sometimes referred to as ‘deepfake pornography’ as it looks very realistic and can be hard to recognize as fake or digitally created. How familiar are you with this concept?”. This question offered four possible categorical response options, as seen in Table 1. Second, respondents were asked how much they thought a series of behaviors were worthy of criminalization (behaviors can be seen in Figure 1). Response options ranged from -2 (“Definitely should not be a crime”) to 2 (“Definitely should be a crime”), with the midpoint of 0 being “Not sure.” And third, respondents were asked how much they agreed or disagreed with the following statement: “People shouldn’t get upset if someone creates a digitally altered video (e.g., fake porn or a ‘deepfake’) of them without their permission.” Respondents answered this question on a 7-point Likert scale, anchored at 1=“Strongly disagree” and 7=“Strongly agree” (we present the mean alongside the median in the results - see below) [76].

The remaining close-ended questions were asked of a subset of respondents, depending on their knowledge and experiences. A subset of respondents who indicated at least some familiarity (i.e., “I know a little bit about it” or “I am quite familiar with this”) were asked follow-up questions around AI-IBSA behaviors as seen in Figure 2. As part of a broader set of questions regarding the victimization and perpetration of image-based sexual abuse since the age of 18, we also asked the subset of respondents who indicated a previous experience with digitally altered images whether the relevant content was a video, and if so, whether the video was a deepfake (allowing respondents to select “Yes”, “No”, “Don’t know”, or “Prefer not to answer”). For victimization questions, the wording was intentionally not blame-casting. Similarly, perpetration questions were phrased neutrally to focus on whether the respondent had ever done an action.

All respondents could opt into answering the open-ended question with the following wording: “Would you like to add any additional thoughts on the topic of deepfakes? Please feel free to skip this question if you don’t have anything to add.” We analyzed the qualitative data for the open-ended response questions using an iterative, inductive approach. Our approach was to translate the responses from origin language to English via the Google Translate API, familiarize ourselves with the data, develop a codebook, code the data, and then review the data to make sense of the themes from the first coding process. To do so, two members of the research team read through 25% of the sample responses across the countries and then co-developed a codebook that detailed descriptions and restrictions on what could be included within a code and provided concrete examples of each code. The same two members of the research team coded the first 150 responses based on four preliminary codes that emerged from the initial review of the data. Coder 1 and coder 2 independently moved through the subset of data labelling the open-ended responses with relevant codes. Following this, coders 1 and 2 compared coding allocations for quality assurance. The two coders then identified and removed responses that were deemed inappropriate or irrelevant to the research (e.g., “no comment,” or “no thanks, I do not know this topic”). Finally, the team revised and discussed the themes that emerged from the responses (n = 2,474). The final codebook contained four key themes: “condemnation,” “punitive and justice attitudes,” “a fear of technology,” and “victim-blaming.” Inter-rater reliability (IRR) for each code was calculated using Cohen’s Kappa. Across the four codes, Kappa values ranged from.135 to.225, with a median of.173.

4.1 Awareness and Attitudes

On average, 28.1% (95% CI \(= 27.0\%-29.3\%\)) of respondents reported being either quite familiar with, or knowing at least a little bit about, deepfake pornography. Descriptive statistics including 95% CIs broken out by country are available in Table 1. The percentage of respondents indicating high familiarity ranged from 3.0% in South Korea to 8.2% in Australia.

Figure 1:

View Figure

Because all respondents in the survey received some information about deepfake pornography through the phrasing of the awareness question, respondents were then asked whether they thought a series of actions should be a crime, with response options ranging from -2=“Definitely should not be a crime” to 2=“Definitely should be a crime.” The midpoint of 0 was a “Not Sure” option. Weighted means [76] and standard errors by country and behavior are presented in Figure 1 (with brighter red indicating more deserving of criminalization and darker blue less deserving), and weighted medians can be found in Figure A3 in the Appendix. We intentionally included “traditional” forms of image-based sexual abuse (e.g., the taking, creating, threatening to share, or sharing of real non-consensual content) to better understand how users perceived AI-IBSA/deepfake pornography in comparison. Aggregated across countries, where equivalencies existed, respondents generally rated actions involving non-consensual real content to be worse than the equivalent of AI-generated content. For example, posting/sending of real content without consent was seen as significantly more deserving of criminalization than sharing deepfake pornography of ordinary people (M_difference = 0.27, t = 38.76, SE = 0.01, p < 0.001, Cohen’s d = 0.30). Non-consensually filming or photographing someone was seen as significantly more deserving of criminalization as compared to making deepfake pornography (“ordinary people” for both, M_difference = 0.56, t = 63.51, SE = 0.01, p < 0.001, Cohen’s d = 0.50).

We additionally disaggregated the deepfake pornography categories by victim-type; that is, celebrities versus non-celebrities. We hypothesized that respondents would have more lenient views towards deepfake pornography behaviors involving celebrities, who are more commonly victimized than non-famous individuals. The difference between paying someone to create deepfake pornography of a celebrity versus an ordinary person was statistically significant (M_difference = 0.03, t = 5.01, p < 0.001, SE = 0.01, Cohen’s d = 0.04), however the effect size is negligible. Similar findings can be seen for viewing deepfake pornography of ordinary people versus celebrities (M_difference = 0.03, t = 4.63, p < 0.001, SE = 0.01, Cohen’s d = 0.03). This suggests little difference in perceptions of criminalization deservedness between celebrity and non-celebrity victims of equivalent behaviors.

With regard to deepfake pornography behaviors, we can draw four main conclusions. First, echoing the real content findings, the distribution of deepfake pornography was considered especially bad, as compared to something more passive such as viewing deepfake pornography (“ordinary people” for both, M_difference = 0.68, t = 75.25, p < 0.001, SE = 0.01, Cohen’s d = 0.60). Second, paying someone to create deepfake pornography was considered worse than making it yourself (“ordinary people” for both, M_difference = 0.33, t = 46.45, p < 0.001, SE = 0.01, Cohen’s d = 0.36). One possible interpretation is that respondents assume that going to significant lengths to have deepfake pornography created is indicative of malicious intent. Third, across countries, the behavior with the lowest average score (i.e., least bad) was the viewing of deepfake content.

Finally, respondents were asked about how much they agreed or disagreed with the following statement: “People shouldn’t get upset if someone creates a digitally altered video (e.g., fake porn or a ‘deepfake’) of them without their permission,” with anchors of 1=Strongly disagree and 7=Strongly agree. As seen in Table 2, across the different countries, the mean response fell between 1 and 2, with a median of 1 in all countries, indicating a general sentiment that people should get upset if they are the target of a deepfake pornography video.

4.2 Behaviors

Behaviors were asked about in three different ways. First, respondents who reported that they were quite familiar with the term “deepfakes” or knew a little bit about it (28.1%, 95% CI \(= 27.0\%-29.3\%\)) were asked a follow-up question about whether they had engaged in specific deepfake pornography behaviors. Second, as part of the larger survey, respondents were asked about general image-based sexual abuse victimization and perpetration. If they reported experiences with digitally altered content, they were asked follow-up questions about whether the digitally altered content involved video(s), and if so, if the video(s) was a deepfake. This is distinct from the first question in that it a) asks for more details of the perpetration/victimization (e.g., whether they had created, threatened to share, or shared such content), and b) asks respondents who reported victimization to clarify whether they had been the victim of deepfake pornography specifically.

4.2.1 Behaviors of “Familiar” Respondents.

The results of the first question (measuring perpetration behaviors) can be seen in Figure 2, where the reported data represents the percentage of all respondents. The respondents who did not see this question due to a lack of familiarity with deepfake pornography are represented in the missing data, which would make each bar sum to 100% if included. For example, 70% of respondents in Spain indicated they had never heard of this until now or they had only heard the phrase, so they were not asked this question. The most common behavior was the viewing of celebrity deepfake pornography, ranging from 2.8% (95% CI \(= 2.1\%-3.7\%\)) of all respondents in Poland, to 8.0% (95% CI \(=6.6\%-9.5\%\)) of all respondents in Australia. Aggregated across the different countries, 6.2% (95% CI \(=5.6\%-6.9\%\)) of all respondents indicated that they had viewed celebrity deepfake pornography. The second most common behavior (2.9% of all respondents, 95% CI \(= 2.5\%-3.3\%\)) was the viewing of deepfake pornography involving “ordinary” people, ranging from 0.7% (95% CI \(= 0.4\%-1.3\%\)) in Denmark, to 5.1% (95% CI \(= 4.1\%-6.2\%\)) in Mexico. Behaviors that required more time, skills, or resources were rarer, such as creating or paying someone to create deepfake pornography of ordinary people (0.6%, 95% CI \(=0.4\%-0.9\%\), and 0.5%, 95% CI \(= 0.3\%-0.7\%\)), respectively, aggregated across countries)).

Figure 2:

View Figure

4.3 Respondents who Reported Personal Victimization or Perpetration

4.3.1 Victimization.

Experiences broken out by country are available in Figures 3 and 4. Aggregated across behaviors, respondents from Australia (3.7%), Mexico (2.9%), and South Korea (3.1%) reported the highest rates of victimization by deepfake pornography. Full results with 95% confidence intervals can be seen in Table A2 in the Appendix.

Aggregated across countries, 2.2% (95% CI \(=1.8\%-2.6\%\)) of respondents reported some form of victimization. This included: 1.2% (95% CI \(=0.9\%-1.5\%\)) of respondents who reported that someone had created deepfake pornography content of them; 1.3% (95% CI \(=1.0\%-1.6\%\)) who reported that someone had posted or sent deepfake pornography content of them; and 1.2% (95% CI \(=0.9\%-1.5\%\)) who reported that someone had threatened to post or send deepfake pornography content of them.

Figure 3:

View Figure

4.3.2 Perpetration.

Aggregating across behavior types (creating, posting, threatening to post), self-reported deepfake pornography perpetration was rare (1.8%, 95% CI \(=1.4\%-2.2\%\)), with the highest rates reported in Australia (2.4%) and the United States (2.6%), and lowest in Belgium (\(0.3\%\)). Full results can be seen in Table A2 in the Appendix. Aggregating across countries, 1.0% (95% CI \(=0.8\%-1.4\%\)) of respondents indicated creating deepfake pornography, 1.0% (95% CI \(=0.8\%-1.3\%\)) reported threatening to post, send, or share deepfake pornography, and 0.7% (95% CI \(=0.5\%-0.9\%\)) reported actually posting, sending, or sharing deepfake pornography content. We included the creation of deepfake pornography as a perpetration behavior to mirror the fact that “creation” behaviors, such as filming or photographing real content without consent, fall under the umbrella of image-based sexual abuse perpetration. Nevertheless, we acknowledge that such behaviors might not be considered “perpetration” by others.

Figure 4:

View Figure

4.4 Gender

4.4.1 Awareness and Attitudes.

Given some existing literature on gendered attitudes and behaviors towards image-based sexual abuse more broadly, we wanted to assess whether gender played a role in respondents’ answers. There was more reported familiarity of deepfake pornography by respondents who identified as men, as opposed to those who identified as women. A Pearson’s chi-squared test with Rao & Scott adjustment confirmed the significant relationship between NSII awareness and gender (F = 58.7, ndf = 3.0, ddf = 49287.1, p < 0.001). Men were 1.92 times (95% CI = 1.53 − 2.40) more likely than women to report being “quite familiar with this” and were significantly less likely to report having never heard of deepfake pornography until now (RR = 0.69, 95% CI = 0.65 − 0.73). Full details can be seen in Figure A1 of the Appendix.

Across deepfake behaviors, men rated criminalization worthiness significantly lower than women, with effect sizes falling between small and medium. For example, women found viewing deepfake pornography of celebrities (M_difference = 0.58, t = 29.6, SE = 0.02, Cohen’s d = 0.41, p < 0.001) and viewing deepfake pornography of non-celebrities (M_difference = 0.57, t = 29.2, SE =.02, Cohen’s d = 0.40, p < 0.001) to be more deserving of criminalization as compared to men, as seen in Figure 5 (with brighter red indicating more deserving of criminalization and darker blue less deserving). The smallest difference and effect size was seen in how each gender perceived paying someone to create deepfake pornography of an ordinary person (M_difference = 0.37, t = 22.05, SE =.02, Cohen’s d = 0.28, p < 0.001).

Figure 5:

View Figure

Similarly, the mean agreement with the statement that people shouldn’t be upset to be the target of deepfakes was higher in men than in women, albeit with a small effect size (M_difference = 0.34, t = 14.4, SE =.02, Cohen’s d = 0.24, p < 0.001).

4.4.2 Behaviors.

Of the two most commonly reported behaviors, men respondents were more likely to report viewing deepfake pornography; both that of celebrities (F = 71.1, ndf = 1, ddf = 16510, p < 0.001, RR = 3.5, 95% CI = 2.77 − 4.42) and non-celebrities (F = 14.3, ndf = 1, ddf = 16510, p < 0.001, RR = 2.66, 95% CI = 1.91 − 3.70).

Figure 6:

View Figure

4.5 Qualitative Responses

In the survey, respondents were asked an optional, open-ended question: “Would you like to add any additional thoughts on the topic of ‘deepfakes’? Please feel free to skip this question if you don’t have anything to add.” A total of 2,477 open-ended responses were collected. Of those responses, 1,180 were removed due to inappropriate or irrelevant responses. This left us with 1,296 open-ended responses (7.8% of respondents). No respondents discussed their own experiences of either victimization or perpetration.

The most prevalent theme was the condemnation of deepfake pornography, with just over 43% of the respondents who responded to this question expressing that view. Across responses, women (47.9%) expressed greater condemnation of deepfake pornography than men (38.2%, x² = 11.58, df = 1, p <.001). There were significant differences by country in terms of frequency of condemnation sentiment (x² = 42.11, df = 9, p < 0.001). Comments relating to the condemnation of deepfake pornography were most common from respondents from France (60.8%), followed by Denmark (53.7%) and the United States (48.2%). Comments relating to this theme were less common in South Korea (25.7%), followed by the Netherlands (27.8%), and Belgium (35.0%). Respondents expressed their condemnation of deepfake pornography, calling the act “disgusting and perverted” and “an extremely sick concept.” Some respondents mentioned the potential pervasive impacts of deepfake pornography, such as one Australian women (aged 35-44) who said, “I think it’s dangerous as it could lead to serious injury (mental and emotional etc.) to the innocent person on the receiving end of the deepfake.” Others commented on the damage it can have on a victim’s employment opportunities or integrity. For example, a man from Mexico (aged 18-24) commented that “it is a stupid strategy to continue damaging the integrity of people who do not want to be intimately seen by the public.” There was also a general consensus among respondents that deepfakes were unethical and morally wrong.

The second most prevalent theme, found in just over 30% of all responses to the open-ended question, referred to a fear of technology; specifically deepfakes and artificial intelligence (AI). There were significant differences by country (x² = 32.81, df = 9, p < 0.001), but not by gender (x² = 0.011, df = 1, p = 0.73). These responses were the highest in the Netherlands (50.0%), and the lowest in France (16.2%) and South Korea (19.7%). References to the futuristic nature of deepfake technology, fear of its capabilities and accessibility, as well as the realness of deepfaked images, showed up across these responses. For example, one man from Mexico (aged 25-34) commented on how “it is difficult to assimilate [to] these types of tools, before they seemed futuristic but they are already very normal.” Many respondents described deepfakes as a troubling and terrifying advancement of AI technology. Some, particularly women, indicated their concern about being victimized by deepfake pornography. For example, a woman from Denmark (aged 25-34) referred to deepfakes as “widely scary” because “it’s hard not to think if you have become a victim yourself without knowing it.” Other respondents focused on the continued growth of deepfakes and the implications they might have for future generations. For example, an Australian woman (aged 45-54) described how she “fear[ed] for [her] daughter’s future.” Comments on the realness of deepfakes was a common thread throughout responses, with a Polish man (aged 35-44) saying “soon it will not be known if what we [are] seeing is reality or not.” Similarly, a Belgian woman (aged 45-54) questioned “can we still know if the photo or video seen or received is real or not?! It becomes difficult to make your own judgement.” Respondents also spoke to the difficulties of regulating deepfakes online. A man from Mexico (aged 65+) emphasized how it is “almost impossible to control,” while a woman from the United States (aged 65+) asked “how can one protect oneself from this? Or prove it’s not them?”

The third most common theme was punitive and justice attitudes, seen in just over a quarter of respondents. Across responses, men were slightly more likely to express punitive and justice attitudes (X² = 4.85, df = 1, p < 0.05), and there were significant differences by country (X² = 91.18, df = 9, p < 0.001). References to punishment, injustice, criminalization, and governmental responsibility were common in South Korea. Just over half (55.3%) of South Korean respondents demanded strengthened laws relating to deepfakes. Responses often referred to deepfake pornography as a “sex crime” and called for the implementation of strong criminal justice regulations and sanctions. Accordingly, many South Korean respondents argued for the development of criminal sanctions that imposed “strong punishment” for perpetrators of deepfake pornography. As one South Korean respondent (male, aged 45-54) stated, “deepfakes [are] not yet controversial, but it can cause many social problems. In reality, legal regulations are needed to strongly punish [these] sex crimes.”

Many respondents disclosed that they were unaware if penalties and sanctions were available in their country for deepfakes, while others were concerned with the slow pace of governmental responses. One woman from the United States (aged 55-64) stated that she “feared that politics doesn’t attract enough experts in tech who could help shape policy and procedure on deepfakes … and it needs to be something for which governments set as a goal to get ahead of and not ignore [or] play catch up.” Similarly, a woman from Denmark (aged 25-34) emphasized the need to “hurry to regulate how people are taking advantage of AI.” References to internet regulation and platform responsibility were found across some responses. For instance, one Australian woman (aged 65+) commented that “platforms need to take more responsibility for content,” and a man from the Netherlands (aged 65+) recommended that this type of content should be “immediately removed by relevant platforms and should be punishable for both maker and distributor.”

Finally, only about 4% of responses indicated victim-blaming attitudes. These respondents noted that people should be more careful with what they post on the internet, and that they should refrain from taking or sharing nude photos or videos, and stop complaining about the issue and take responsibility. There were no significant differences by gender (X² = 1.68, df = 1, p = 0.20), nor by country (per a Fisher’s exact test for count data, p = 0.08). For example, a non-binary respondent from Mexico (aged 35-44) and a woman from Poland (aged 55-64) noted that people have to be careful with what they share on the internet. Two Australian respondents encouraged people to stop taking nude images altogether; for example, one Australian man (aged 55-64) commented “if you don’t want something shared on the internet don’t take nude photos” while an Australian woman (aged 65+) said, “that’s why I do NOT send photos over the internet at all. I can’t understand why people send photos to anyone — keep your [images] private.” One man from Poland (aged 65+) was more explicit and indicated that people should not complain about being victimized: “you just should not share any photos on the web at all, and if you make it available, you should take into account the consequences and then regret pretending to be a fool and even worse to blame anyone.”

In summary, only 7.8% of all respondents provided relevant answers to the optional, open-ended question. This means that the majority of survey respondents did not provide additional thoughts on the topic - either because they did not know enough about deepfake pornography or for other reasons.

5.1 Attitudes and Awareness

Less than 30% of all respondents indicated some level of familiarity with the concept of deepfake pornography. This was also arguably reflected in the qualitative responses, with only 7.8% of all respondents answering the open-ended question. Presumably, as the technology becomes more widespread and ever more available, familiarity will increase. This may be a double-edged sword; it is likely to result in increased use of AI-IBSA as an abuse vector (both for purposes of humiliation, and as a tool for sexual extortion), but may also result in increased suspicion of the veracity of explicit content. One positive finding was general agreement across countries that perpetration behaviors associated with “traditional” image-based sexual abuse are deserving of criminalization. Research has widely found some degree of victim-blaming reflected in attitudes towards traditional non-consensual intimate imagery [30, 56, 75]. Attitudes were slightly more lenient with regard to deepfake behaviors, although over time attitudes may evolve with increased familiarity. For instance, the realization that any private individual can be deepfaked, requiring no actions on the part of the target, may engender more empathy and less victim-blaming attitudes. It should be noted that few respondents in the qualitative part of the survey expressed victim-blaming attitudes. This can be compared to other more commonly expressed pro-social attitudes in the qualitative responses, including those expressing condemnation or support for punitive action.

5.2 Behaviors

The vast majority of respondents indicated no engagement in any type of AI-IBSA behaviors, with the most popular behavior being the consumption of celebrity deepfake pornography. It was even rarer for respondents to indicate perpetration themselves. This was similar for victimization, with few respondents indicating that they had been victimized by deepfake pornography. However, the victimization statistics in particular should be interpreted with caution. Even with traditional forms of image-based sexual abuse, many victims may be unaware of their victimhood. AI-IBSA may exacerbate this dynamic given that deepfake technology “democratizes” victimization and image-based sexual abuse by enabling anyone with a computer to be a perpetrator, and anyone with imagery online to be targeted. Also, there are additional unique motivations for AI-IBSA that may result in less likelihood of victim notification; for instance, if the motivations for the creation or distribution of AI-IBSA are for sexual gratification purposes and/or for demonstrating or practicing digital skills in AI. Finally, as content may be distributed on dedicated sites or within closed communities on sites such as Discord or Reddit, victims of AI-IBSA may not know about or find out about their images.

As a subset of image-based sexual abuse, it is useful to compare rates of AI-IBSA with previously reported statistics on image-based sexual abuse more broadly. There are several associated challenges, including sampling methodologies and definitions. For example, studies may define image-based sexual abuse behaviors differently, failing to include the non-consensual creation or filming of content, or threatening to share content, alongside the actual non-consensual sharing of content. Studies may also intentionally survey younger populations who may be more likely to have relevant experiences, as opposed to collecting population-level prevalence data. Notwithstanding those limitations, previous work has shown that image-based abuse prevalence is increasing over time, driven by changes in technology use, dating, sex, and privacy [63]. Powell et al. [63] found on average, across Australia, New Zealand, and the UK, that  1/3 of survey respondents indicated at least one experience of one or more forms of image-based sexual abuse, defined similarly to our study. An American-based study [68] focused exclusively on the non-consensual sharing of content, and found that 1/12 respondents indicated victimization, while 1/20 reported perpetration. Even taking the more expansive definition of image-based sexual abuse, our rates of perpetration and victimization using deepfake technology were quite low. Given the trends in traditional image-based sexual abuse [63], and the increasing ease of NSII creation, we expect these rates to increase over time.

5.3 Gender and AI-IBSA

Awareness of deepfakes was higher among respondents who identified as men as compared to women. This is consistent with the literature on the “digital divide” which suggests that women face disproportionate barriers to access and use to the internet compared to men. Even when women have access to the internet and technology, they may not have the time, resources, skills, or knowledge to leverage it [1]. Additionally, the attitudinal questions indicate a trend towards men diminishing the harms or “badness” of deepfake pornography behaviors. This is consistent with previous findings on deepfake pornography [29], and with other findings around image-based sexual abuse more broadly [7, 10].

Men were more likely to report both perpetration and victimization by deepfake pornography, across subtypes. Studies continue to report mixed findings regarding the role of gender in image-based sexual abuse. Powell et al. [63] reported higher rates of perpetration in men, and similar rates of victimization for men and women, but differential impacts. Our findings are somewhat inconsistent with a report which found the majority of AI-IBSA available online targeted women [2]. We argue that the differences in sampling methodology may help explain these inconsistencies. For instance, in the event of sexual extortion, the content may not be made available online. It is possible that the trend seen in gender may be somewhat explained by the motivation of the perpetrator, given previous research to support higher rates of “sextortion” in men as compared to heterosexual women [23], including for adolescent populations [61]. Another explanation for this discrepancy is that research suggests most deepfake videos online (an estimated 96%) are sexualized images of women actors and musicians, who may be less likely to participate in survey research than non-public figures like our survey respondents [2].

While we asked broadly about motivations in the larger survey, we did not ask specifically ask about motivations for deepfake pornography perpetration or victimization. Motivations for such behaviors may range from personal sexual gratification, to use for extortion purposes, to demonstrating technical skill in the creation of content. A consideration for future research should be the use of AI-IBSA (as well as other forms of NSII not involving AI) for extortion, particularly given anecdotal evidence of significant associated harms (e.g., suicide, suicidal ideation, or suicidal attempts) [64].

5.4 Limitations and Mitigation

Cross-country surveys are logistically difficult and require significant consideration in interpreting findings. Our study had to address language differences and non-response bias. The former were addressed, as detailed above, through cross-validation. The latter was a more significant concern, as others have noted that language used to discuss a topic such as image-based sexual abuse and pornography more broadly will be culturally dependent [71]. We attempted to mitigate this by allowing respondents to select “Prefer not to say” for any question that involved self-reporting on behavior, as well as periodically reminding respondents of that option throughout the survey. Nevertheless, it is possible that selection bias may have resulted in an underestimation of prevalence, with respondents opting out of the survey due to personal experiences or demographic characteristics. South Korea, for example, had a high percentage of bisexual respondents. This may indicate that our attempts at mitigating non-response bias were less successful in South Korea. In addition, in this study, we evaluated prevalence based on respondents who indicated victimization with deepfake pornography specifically. Thus, we may have underestimated prevalence because we did not include respondents who were unsure, or who incorrectly indicated that their content was not deepfaked. Finally, we note that, despite explicit goals for representative sampling, the participant pool fell short, necessitating the need to weight results.

5.5 Design Considerations

In this study, we found that the majority of respondents across all countries were relatively unfamiliar with the concept of deepfake pornography. We also found that respondents, on average, found deepfake pornography behaviors to be deserving of punishment. Reported perpetration and victimization rates were low, with the most common behavior reported by respondents being the consumption of deepfake pornography. Finally, gender did play a part in both experiences and attitudes, with men deeming deepfake pornography behaviors as less harmful or deserving of punishment as compared to women. This has implications for both policy and practice.

As relatively few respondents were aware of deepfake pornography, this signals first and foremost that more education is needed to both deter potential perpetration and the consumption of AI-IBSA. Education efforts should focus on the harms suffered by victim-survivors, as well as the potential consequences of perpetration. Given the gender dynamics in terms of perceived harmfulness, prioritization around education should focus on reaching boys and men.

Figure 7:

View Figure

Second, many scholars have focused on the need for legislation to criminalize these image-based abuse behaviors and provide clear redress for victims [15, 20, 34, 52, 73]. In order for a law to have both general and specific deterrent effects, the consequences of breaking that law has to be known by the general public, and someone who does break the law has to be prosecuted and appropriately punished. Some of the trends—particularly the high rates of prevalence in both South Korea and Australia—indicate that laws are not a panacea to eliminate or even reduce this behavior. For instance, some Australian states and territories specifically criminalize the distribution of NSII [8], while South Korea has laws allowing for up to 3 years imprisonment for merely viewing non-consensual intimate imagery [14, 50]. Given that this behavior primarily happens online, that victim-survivors may not be aware of the material, or if they become aware, may be reluctant to report it, it can be argued that existing legislation provides weak deterrence since very few people have been prosecuted for NSII in jurisdictions that do have such laws in place. As we found in this study, there was also low awareness of the concept of deepfake pornography, and one may reasonably infer low awareness of related laws. While developing legislation around an issue helps create societal norms and signals the unacceptability of a behavior, it is crucial to concede the limitations of passing regulations in enacting true societal change. The limitations of law underscores the importance of developing parallel efforts, such as technical solutions, that may mitigate these harms by reducing the creation and dissemination of NSII.

Third, there are a number of points of potential intervention for technical solutions. First, discovery tools that allow individuals to search the web for sexual material via facial recognition technology may be empowering, although there are safety, privacy, and security concerns that need to be taken into account (e.g., building in protections such that individuals can only search for content of their own faces). Second, while many videos are posted on dedicated deepfake websites, or make no claims as to veracity, the use of deepfakes for sexual extortion indicate that technical methods of annotating content, such as watermarks, may help mitigate NSII harms [38]. Third, platforms can make choices around what they allow and disallow in their policies and community standards (legislation can assist in justifying the choices made by platforms). Google Search [35], Reddit [40], and PornHub[18] have already published dedicated NSII policies. Fourth, various stakeholders have worked to help create solutions for victims, primarily focusing on triaging after victimization has occurred. It will be necessary for these help-seeking resources (whether helplines, online forms, or chatbots [25, 50]) to be aware of this new type of image-based sexual abuse and up-to-date with the types of redress that victims may seek [4]. Fifth, where tools do have a payment or sign up flow, users could be required as part of the flow to ingest and acknowledge information about the harms and potential consequences of NSII creation and dissemination. Finally, platforms have demonstrated an ability to detect nude content or, in some cases, non-consensual nude or sexual content. If creator tools could use those tools to block creation, or at the very least serve a warning interstitial about the harms associated with NSII (including AI-IBSA), it may act as a further deterrent for uploading this content.

As practitioners and designers think about future considerations for intervention, it may be helpful to break this issue down into relevant behaviors as seen in Figure 7 and consider intervention options (which should be considered complementary, as opposed to solely sufficient) at each point in the process. The increased ease of access to NSII creation technologies was the catalyst for this project, and represents the first place at which interventions could be made. Are there ways, for example, to limit these tools’ abilities to create nude or intimate imagery? If creators get to the point where barriers have been hacked, are there warning messages that can either highlight associated harms to targets, or potential punishments associated with the creation or distribution of such content? After creation, are there ways to build in indicators of provenance (e.g., watermarking) that signal the synthetic nature of the content? Finally, assuming most impediments can be circumvented by a motivated creator, empathetic and multi-modal resources for victims could include both tooling to discover where relevant content is published, and help for seeking take downs and recourse more broadly. These are primarily design interventions, but overarching all of these would be the societal interventions of educational programming, and relevant legislation making clear potential punishments associated with each type of behavior (extortion, posting/distribution, etc), paired with prosecutorial actions that indicate a willingness to hold perpetrators to account.

5.6 Conclusion

In this study, we used a survey to explore general attitudes toward AI-IBSA (or “deepfake pornography”), as well as AI-IBSA behaviors and experiences in 10 different countries around the world. We found relatively low rates of awareness across countries, but respondents expressed concern and condemnation of image-based sexual abuse behaviors, including deepfake pornography (as seen in the questions about whether victims should get upset and criminalization, and in the qualitative responses). The most popular reported behavior was the consumption of celebrity deepfake pornography, while personal victimization and perpetration rates were quite low. Men tended to find consuming deepfake pornography less objectionable than women, which was also seen in the higher rates of self-reported consumption behavior. Finally, men were more likely to report both more perpetration and more victimization by deepfake pornography, a trend that future research should explore as we may expect prevalence to increase over time. Laws, policies, and support services can help to mitigate the harms of NSII; however, more resources need to be directed toward detecting and blocking the creation of content prior to dissemination, as well as preventing the abuse from happening in the first place through educational programs and initiatives.