Can an Awareness Campaign Boost the Effectiveness of Influencer Marketing Disclosures in YouTube Videos?

Answering the strong need for insight into how minors can effectively be informed about advertising (e.g., influencer mar‐ keting) in online content, we ran an online experiment ( N = 623 minors between 8 and 18 years old) testing the effects of two pictograms that were designed in co‐creation with minors and the potential of an awareness campaign to boost the pictogram’s effectiveness. Our findings provide three important insights that have implications for theory, practice, and regulation. First, we find that minors are able to distinguish between sponsored and non‐sponsored videos, indicating that they have developed some level of advertising literacy in this context. Second, our study shows that the two pictograms informing minors about advertising in online videos went unnoticed by most viewers and did not enhance conceptual or attitudinal advertising literacy. Third, the awareness campaign did not lead to higher recognition of the pictograms nor enhanced advertising literacy. The campaign did increase minors’ understanding of the meaning of the pictograms. However, the majority of minors also understood the pictograms without the campaign. Based upon our findings, we argue that pictograms are unnoticed by most minors and seem ineffective in enhancing minors’ advertising literacy. Although an awareness campaign can familiarize minors with pictograms and their implementation in online videos, it does not seem to boost the pictogram’s effects on advertising literacy.


Introduction
There is an ongoing debate on how to create a safe and transparent digital environment for minors.A key question in that discussion is how to protect them from hidden advertising and sponsorship (e.g., influencer marketing) on online video platforms, including YouTube and TikTok (Federal Trade Commission, 2022;Spielvogel et al., 2021).Minors have been shown to have limited information processing capacity, which may hinder the activa-tion of advertising literacy (Castonguay, 2022;Lapierre, 2019;van Reijmersdal & van Dam, 2020).Advertising literacy encompasses people's knowledge and attitudes about the source, intentions, and tactics used in advertising, such as the recognition of sponsored content as advertising, understanding that sponsored content has a persuasive intent, and critical evaluations of sponsored content (Boerman et al., 2018;Hudders et al., 2017;Rozendaal et al., 2016).When advertising is integrated into highly entertaining and emotionally appealing social media content, advertising literacy is less likely to be activated (Hudders et al., 2017), making young viewers more susceptible to such commercial messages (Castonguay, 2022;van Reijmersdal & Rozendaal, 2020).To increase transparency, legislators and child advocates have called for a uniform pictogram to signal sponsored content in online (influencer) videos that aligns with minors' comprehension capacity (Federal Trade Commission, 2022).Pictograms are easy to process due to their visual and simple nature (Tijus et al., 2007) and might, therefore, be the ideal cue for minors to activate their advertising literacy.However, insight on how such a uniform pictogram (i.e., disclosure) should be designed and implemented in online (influencer) videos to make it understandable for minors is still lacking and highly needed.
In one of our previous studies, we addressed this need by developing and testing pictograms that should help minors (8 to 18-year-olds) to signal advertising in online videos and trigger them to apply their advertising literacy (Boerman et al., 2023).In this previous study and the current study, we collaborated with NICAM, the Dutch Institute for the Classification of Audiovisual Media.NICAM is responsible for the Kijkwijzer system in the Netherlands, a classification system using pictograms to inform viewers of the potential harm in video content (see https://www.kijkwijzer.nl).Recently, there has been an expansion of the Kijkwijzer pictograms to include online videos.Therefore, we see potential in a new pictogram informing viewers of the commercial nature of video content in this established system.In our previous study, we developed such pictograms in three phases: (a) a co-creation phase in which we designed several pictograms with minors, parents, and a professional designer; (b) a survey gaining insight into minors' associations with and preferences for a selection of pictograms; and (c) an experiment testing the effectiveness of three pictograms in increasing advertising literacy.Interestingly, in this final phase, we found that none of the pictograms had the expected effects on minors' advertising literacy, even though the pictograms had been designed in close cooperation with them.
The lack of effects could be explained by the low scores in disclosure recognition: Very few minors actually remembered seeing the pictogram (Boerman et al., 2023).Other existing research also showed that disclosures, such as pictograms, are often not noticed, limiting their effectiveness (Boerman & van Reijmersdal, 2020;De Jans et al., 2018;Spielvogel et al., 2021;van Reijmersdal et al., 2017van Reijmersdal et al., , 2020)).One of the reasons for the fact that disclosures are often not seen or remembered is people's unfamiliarity with the disclosure and its meaning.Pictograms can only be effective when the receiver notices and understands them, and familiarity with a pictogram enhances comprehension (Tijus et al., 2007).This may also apply to the pictograms developed in our previous study.Given the novelty of the pictograms, minors were not yet familiar with them and thus may not have comprehended their mean-ing, or may have been unaware of their value in this context.Moreover, the implementation of the Kijkwijzer pictograms in online videos is a recent development; thus, the minors may not have been used to seeing them in online video content.Taken together, minors were probably unfamiliar with the new pictograms, making it unlikely they paid attention to them or understood their message.
A common way to increase the awareness and comprehension of pictograms or other types of warning labels, such as tobacco and nutrition labels, is through awareness or educational media campaigns.Prior research shows that campaigns that provide information on the existence and meaning of a pictogram or warning label can be effective in increasing familiarity with and comprehension of it among the target audience (e.g., Bollinger et al., 2022;Thrasher et al., 2013).Therefore, the current study aims to examine whether an awareness campaign can enhance minors' awareness and understanding of the newly developed Kijkwijzer pictogram for advertising in online video content and its effectiveness in triggering advertising literacy.We test the effectiveness of (an awareness campaign for) two pictograms that were designed and tested in our previous study (Boerman et al., 2023): One depicting #AD and one depicting an influencer showing a product with a price tag (see Figure 1).The previous study showed that the #AD icon was most strongly associated with advertising and ads by minors, and the influencer icon was most preferred by them (they mainly appreciated how the influencer pictogram depicts the content creator and the actual practice of advertising a product).
Furthermore, we aim to investigate whether an awareness campaign increases attention to and comprehension of these new sponsorship pictograms and, by doing so, boosts its effects on minors' advertising literacy.To address this aim, we first test minors' recognition of advertising in online influencer videos and the effect of the pictograms on their advertising literacy.Second, we examine the effect of an awareness campaign on minors' recognition and understanding of pictograms and advertising literacy.
Theoretically, this study provides insight into the role of awareness campaigns and the effectiveness of pictograms that aim to signal the presence of sponsored online content to minors.Practically, this study offers recommendations to policymakers to develop pictograms and awareness campaigns that can increase the transparency of sponsored content in online (influencer) videos to minors.It also contributes to ongoing societal debates on how to best protect minors from stealth advertising, such as influencer marketing, in online media.

Minors' Recognition of Advertising in Online Videos
Previous studies suggest that because sponsored content in online videos is integrated and presented by a content creator (e.g., a social media influencer) instead of an advertiser, minors are less likely to recognize the advertising, and thus are susceptible to it because they do not use their advertising literacy to cope with it (e.g., Castonguay, 2022;Hudders et al., 2017;van Dam & van Reijmersdal, 2019).Other studies, however, revealed that minors and young adults are relatively good at detecting sponsored influencer content on social media, even without any disclosure (e.g., Boerman & Müller, 2022;Boerman et al., 2023).If minors are indeed able to distinguish between sponsored and non-sponsored influencer video content, this may cause a ceiling effect, making disclosures such as pictograms ineffective and possibly even redundant.To test this, we first examine whether minors can distinguish between sponsored and non-sponsored influencer videos, regardless of disclosure.We hypothesize: H1: Minors' ad recognition will be higher for an influencer video that contains advertising than an influencer video that does not.

Effect of Pictograms on Advertising Literacy
Disclosures such as pictograms aim to increase transparency and help minors activate and enhance advertising literacy.Advertising literacy can be distinguished in a conceptual and an evaluative dimension (Boerman et al., 2018;Hudders et al., 2017;Rozendaal et al., 2011).Conceptual advertising literacy refers to minors' ability to recognize advertising and understand its persuasive and selling intent.Attitudinal advertising literacy entails general (negative) attitudes and skepticism toward advertising (Hudders et al., 2017;Rozendaal et al., 2011Rozendaal et al., , 2016)).
The advantage of pictograms as a way to transparently communicate that content is advertising lies in the efficiency of visual imagery.Pictograms can be identified faster than words and require fewer cognitive resources, making it easier to process them in suboptimal conditions (Tijus et al., 2007), such as the cluttered media environments of online videos.Combined with the abundance of studies that do find that disclosures can enhance advertising literacy, we expect the #AD and influencer pictograms to increase conceptual advertising literacy (i.e., minors' recognition of advertising in spon-sored influencer videos and their understanding of the persuasive and selling intent of this commercial content), and affective advertising literacy (i.e., skepticism and disliking of the sponsored video content): H2: Both pictograms (vs.no pictogram) will lead to higher (a) ad recognition, (b) understanding of persuasive intent, (c) understanding of selling intent, (d) skepticism, and (e) disliking of the sponsored content in influencer videos.
Our previous study showed that minors most strongly associated the #AD pictogram with advertising but most preferred the influencer pictogram (Boerman et al., 2023).Thus, both pictograms have their benefits.However, we have no theoretical grounds to predict which of the two pictograms performs best.Therefore, to compare the two pictograms, we formulated a research question: RQ1: Which of the two pictograms leads to the highest levels of (a) ad recognition, (b) understanding of persuasive intent, (c) understanding of selling intent, (d) skepticism, and (e) disliking of the sponsored content in influencer videos?

Effects of Awareness Campaign
Although pictograms can be instant reminders of an established message that can be processed relatively quickly (Tijus et al., 2007), and we cocreated our pictograms to make them comprehensible to minors, our previous study showed that the new Kijkwijzer pictograms were hardly noticed (Boerman et al., 2023).Previous research suggests that, in general, minors hardly notice disclosures (e.g., Boerman & van Reijmersdal, 2020;De Jans et al., 2018;van Reijmersdal et al., 2017van Reijmersdal et al., , 2020)), and disclosures have limited effects because they are often not understood (Rozendaal et al., 2021).Research has also shown that awareness campaigns can effectively increase people's understanding of pictograms and labels (e.g., Bollinger et al., 2022;Thrasher et al., 2013).We, therefore, developed an awareness campaign that introduced the new pictogram to minors and explained its meaning.
Drawing upon the advertising literacy theory and the persuasion knowledge model (Friestad & Wright, 1994;Livingstone & Helsper, 2006;Rozendaal et al., 2011), we postulate that such an informative awareness campaign can create association networks in minors' brains, in which the new pictogram and its meaning become linked to minors' existing advertising literacy (Du Plessis, 2005;Wright et al., 2005).These links are expected to enable activation of the meaning of the pictogram and minors' existing associative networks on advertising literacy (H5) when confronted with it in an online situation.Also, the networks are expected to facilitate information processing and retrieval (Hoek et al., 2021), resulting in higher recognition of the pictograms after exposure to an awareness campaign (H3) and to the activation of associations with Kijkwijzer, the organization behind the pictograms, and thoughts about advertising and sponsored content (H4).Based on these assumptions, we propose: H3: Exposure to an awareness campaign (vs.no exposure) will lead to (a) better pictogram recognition and (b) better understanding of the pictogram, regardless of pictogram type.
H4: Exposure to an awareness campaign (vs.no exposure) will make minors report more thoughts about (a) the Kijkwijzer and Kijkwijzer pictogram and (b) advertising and sponsored content in influencer videos.
H5: Exposure to an awareness campaign (vs.no exposure) will lead to higher (a) ad recognition, (b) understanding of persuasive intent, (c) understanding of selling intent, (d) skepticism, and (e) disliking of the sponsored content in influencer videos.
Finally, we expect an interaction effect between the awareness campaign and the pictograms, with the awareness campaign boosting the effectiveness of the pictograms.If the awareness campaign can indeed make minors attend to and understand the pictogram, combining both is expected to be most effective.In that situation, minors would be most likely to notice the pictogram and understand its message, thus resulting in the highest level of advertising literacy: H6: After exposure to the awareness campaign (vs.no exposure), the pictograms (vs.no pictogram) have a stronger effect on (a) ad recognition, (b) understanding of persuasive intent, (c) understanding of selling intent, (d) skepticism, and (e) disliking.

Design and Sample
We conducted an online experiment with two (awareness campaign: information video introducing the pictogram vs. filler video) × three (pictogram type: no pictogram vs. #AD pictogram vs. influencer pictogram) × two (video: two different sponsored YouTube influencer videos) between-subjects design.To increase the experiment's external validity and ensure our findings were not due to one specific sponsored YouTube video, we included a third factor representing two different videos (see further details in Section 2.2).Because the awareness campaign was adapted to the two pictograms (i.e., #AD or influencer pictogram), this design resulted in 14 experimental groups (see Table 3 in the Supplementary File for a detailed overview).
Participants between 8 and 18 years old were recruited through parents in a commercial panel com-pany.In total, 1,064 minors participated.We excluded those who did not finish the questionnaire (n = 270), participants who were younger than eight years old (n = 1) or older than 18 (n = 94), or those who did not consent to our conditions (i.e., 22 16-18-year-olds, 36 parents, and 20 minors).Furthermore, we excluded participants who failed both attention checks (n = 2), those who did not watch the videos (n = 3), and those who indicated that one of the videos did not work (n = 9), or those with open answers that were nonsense or said they had not seen the videos (n = 13).

Stimulus Materials
For the awareness campaign, a professional designer working for Kijkwijzer developed an information video, and the animation matched their company style.The one-minute video introduced the Kijkwijzer in general and the new pictogram for advertising in online videos.The one-minute filler animation video explained what happens when your foot falls asleep (see link to the video in the Supplementary File).
To manipulate the pictogram, we showed all participants a composite of two YouTube influencer videos: first, a non-sponsored filler video, which was the same for all participants, followed by one of the two sponsored videos.The first was an edited, two-minute filler video without sponsored content in which influencer Kalvijn and a kid play with animals and visit a goat farm.The second started immediately after the first and concerned one of the two sponsored YouTube videos.To increase the external validity of the experiment, we included two videos by Dutch YouTube influencers with two different brand placements, which were randomly assigned to the participants.The videos were edited by the researcher to limit their length and to ensure only one brand was advertised.In one video, Liefs Lotte (78k subscribers) reviews a new Taksi popsicle (ice cream; 3.20 min).In the other video, Furtjuh (981k subscribers) and his friend make a cupcake dinner using his own branded Blueband cupcake mix (4.50 min).In both videos, the product and brand were clearly visible and mentioned several times, and the YouTubers elaborately talked about the product.Depending on the pictogram condition, the video included only the AL (all ages) pictogram or the AL pictogram with the #AD pictogram or the influencer pictogram.The pictograms were shown four seconds after the start of the video, on the upper right side of the screen, for 10 seconds, and were about 20% of the screen's height (see Figure 1).In Part 1, we asked participants to watch a video (either the awareness campaign or the filler video).We told them we wanted to know what they thought of it.After watching the video, we checked whether it worked and whether they watched it ("how much of the video did you watch?" 1 = the entire video, 2 = a large part of the video, 3 = only the beginning, 4 = nothing; 92.9% watched the full video, 6.4% a large part, 0.6% only the beginning).In line with our cover story, we also asked whether they liked the video, whether they thought it was clear, whether they had any tips to improve it (these data were not analyzed).
Part 2 concerned the responses to a sponsored YouTube video and included the manipulation of the pictograms.The first video was a filler video without sponsored content, the second concerned the sponsored video (see Section 2.2).After these videos, the questionnaire started with questions about the second video (i.e., thought listing, video familiarity, video liking, ad recognition-scores 5 and 6: explain why and brand recall-understanding selling intent, and understanding persuasive intent).The questionnaire then focused on the first filler video (i.e., video familiarity, ad recognition) and then continued with questions about the pictograms (i.e., recognition of five Kijkwijzer pictograms, and open questions asking about the meaning of tested pictogram in conditions that were exposed to them).Furthermore, we asked questions about the influencer and brand in the sponsored video (i.e., influencer familiarity, frequency of watching videos of influencer, influencer liking, brand familiarity, and product use), and general questions regarding the frequency of watching videos on YouTube, frequency of posting videos on YouTube, gender, school, and class/group.Finally, participants were debriefed and could leave suggestions or feedback.
We measured conceptual advertising literacy with 6-point scales developed by Rozendaal et al. (2016; 1 = no, certainly not; 2 = no, I do not think so; 3 = no, maybe; 4 = yes, maybe; 5 = yes, I think so; 6 = yes, certainly).We measured ad recognition with two questions: "Was there advertising in the video?" and "was the video sponsored by a brand?Sponsored means that a brand has paid to make the video" (Boerman & van Reijmersdal, 2020;Hoek et al., 2020).Mean scores were calculated to create a single measure of ad recognition (Spearman-Brown = 0.79, M = 4.55, SD = 1.30).
To measure ad recognition for a non-commercial video, we asked the same questions for a filler video that did not contain advertising (Spearman-Brown = 0.88, M = 2.47, SD = 1.14).
Understanding of selling intent was measured by asking: "Was the video made to make people buy [brand]?" and "was the video made to sell [brand]?" (Rozendaal et al., 2016).Items were adapted to the brand in the video, and the mean of the two items was used as a measure of understanding of selling intent (Spearman-Brown = 0.77, M = 4.66, SD = 1.93).
To measure understanding of persuasive intent, we asked: "Was the video made to make people like [brand]?," "was the video made to make people want to have [brand]?," and "was the video made to make people think positively about [brand]?" (Boerman & van Reijmersdal, 2020;Hoek et al., 2020;Rozendaal et al., 2016).Items were adapted to the brand in each video, and the mean score of the three items was used as a measure of understanding of persuasive intent (Cronbach's alpha = 0.91, M = 4.78, SD = 1.05).
To measure pictogram recognition, we showed participants five pictograms (three existing pictograms: AL, violence, foul language; and the two selected pictograms) and asked them whether they had seen this pictogram in the video (0 = no, 1 = maybe, 2 = yes).Answers to the selected pictograms were recoded to represent correct pictogram recognition (0 = incorrect answer, 68.9%; 1 = correctly recognize the pictogram participant was exposed to, 9.6%; 2 = participants in no pictogram condition who correctly did not recognize any pictogram, 21.5%).
To measure pictogram understanding, we showed participants in the pictogram conditions (n = 431, 192 missing values = no pictogram condition), a still of the pictograms in the video that they had watched (see Figure 1), and explained that the AL pictogram meant that the video was for AL.We then asked them what they believed the other pictogram meant and provided five answer options (1 = "that advertising is made in this video or that the video is sponsored [the correct answer]," 2 = "that it is a video from the algemeen dagblad," 3 = "that it costs money to watch this video," 4 = "that there is violence in the video," 5 = "that they use swear words in the video").Answers were recoded into correct (answer 1: 13.9%) or incorrect (answers 2, 3, 4, and 5: 86.1%) understanding of the pictogram.

Control Variables
We measured age with a dropdown list ranging from 8 to 18.This continuous variable was recoded into categories (8-12 years old n = 232; 13-15 years old n = 234; and 16-18 years old n = 157).
After watching the two YouTube videos, we checked whether the video worked (three said no) and how much participants had watched (66.3% watched the full video, 29.1% a large part, and 4.7% only the start).We also asked whether they had seen the sponsored video before (video familiarity: 0 = no, 1 = maybe, 2 = yes; 5.5% was familiar, 5% maybe, and 89.6% was not familiar), and to rate on a scale from 1 to 10 how much they liked the video with 1 being very bad, and 10 very good (video liking: M = 5.30, SD = 2.27).We also asked whether they were familiar with the filler video (82.7% no, 7.7% maybe, and 9.6% yes).
Furthermore, we asked participants several questions about the YouTuber and brand in the video.These questions were adapted to match the condition participants were assigned to, and these measures were combined to generate a score for the full sample.We asked participants whether they knew the YouTuber before watching the video (influencer familiarity: 0 = no, 1 = maybe, 2 = yes; 19.3% was familiar), how often they watched videos of this YouTuber (watching influencer frequency: 1 = never, 2 = sometimes, 3 = often, 4 = very often; M = 1.23,SD = 0.54), and to rate the YouTuber on a scale from 1 to 10 (influencer liking, M = 5.20, SD = 2.31).We also asked whether they knew the brand before the research (brand familiarity: 0 = no, 1 = maybe, 2 = yes; 84.8% was familiar) and how often they used/ate the product (M = 1.22,SD = 0.50).We asked participants how often they watched YouTube videos (YouTube frequency: M = 2.83, SD = 0.84) and posted YouTube videos themselves (YouTube video posting frequency: M = 1.35,SD = 0.66).

Attention Checks
The questionnaire includes two attention checks.The first check said, "We want to check whether you read the questions, please fill out 'elephant' here" (1 = lion, 2 = tiger, 3 = elephant); the second check was similar but asked to select the "none of the above" answer (1 = YouTube, 2 = Instagram, 3 = TikTok, 4 = Snapchat, 5 = none of the above).

Manipulation Check
There was a significant difference in correct recognition between the pictograms,  2 (4) = 399.83,p < 0.001.Of the 212 participants who were exposed to the #AD pictogram, 18.9% correctly recalled seeing it.Of the 219 in the influencer pictogram condition, only 9.1% correctly recalled seeing it.Of the 192 participants who watched the videos without a pictogram, 69.8% correctly recognized that they had not seen one.These percentages align with previous research and replicate the finding that correct pictogram recognition is higher for the #AD pictogram than the influencer pictogram (Boerman et al., 2023).
All analyses included brand familiarity, influencer familiarity, video liking, product use, and watching influencer frequency as covariates.

Hypothesis Testing
To test H1, we ran a paired samples t-test with the ad recognition scores for the sponsored and filler videos.Results showed a significant difference, supporting H1: Minors' ad recognition was higher for the videos that actually contained advertising (M = 4.55, SD = 1.30) than for the non-commercial video (M = 2.47, SD = 1.42), t(622) = 28.85,p < 0.001.
Pairwise comparisons showed that compared to no pictogram, the influencer pictogram led to significantly less skepticism (p = 0.021) and less disliking (p = 0.023).All other comparisons were not significant for both skepticism (no pictogram vs. #AD pictogram p = 0.527; influencer vs. #AD pictogram p = 0.087) and disliking (no pictogram vs. #AD pictogram p = 0.694; influencer vs. #AD pictogram p = 0.055).These findings do not support H2: Both pictograms do not influence cognitive levels of advertising literacy.Moreover, the influencer pictogram does seem to make minors less skeptical and dislike the advertising less, and thus it lowers affective advertising literacy, partly answering RQ1.
To test H3a, we compared correct pictogram recognition between the awareness campaign conditions (n = 431,192, missing values = no pictogram condition).Results showed no significant differences,  2 (1) = 1.24, p = 0.265: The awareness campaign did not lead to better pictogram recognition (15.3% correct) compared to no campaign (11.5% correct).H3a was not supported.To test H3b, we compared the correct understanding of the meaning of the pictogram between the awareness campaign conditions.Results showed that the awareness campaign significantly led to a better understanding of the meaning of the pictogram (91.6% correct) compared to no campaign (84.1% correct),  2 (1) = 5.72, p = 0.017.This supports H3b, but interestingly, the high percentages also demonstrate that many minors already understood the pictograms without the campaign.
H4 concerns the thoughts minors had while watching the sponsored video.Only four minors (0.7%) reported thoughts about the Kijkwijzer (e.g., "I saw the new pictogram"), and all four were in the awareness campaign condition.Given the low number, this difference was not significant,  2 (1) = 1.71, p = 0.191.Furthermore, 10.1% of the minors reported thoughts about (some form of) advertising (e.g., "This is probably advertising," "they are baking with sponsored products," "they have a partnership with the brand").Importantly, minors significantly reported more thoughts about advertising after seeing the awareness campaign introducing the Kijkwijzer pictogram (13.3% vs 2.7% no awareness campaign),  2 (1) = 15.57,p < 0.001.These results do not H4a but do support H4b.To test H5, we ran a MANCOVA with the awareness campaign (yes or no) as a factor, and the five advertising literacy measures as dependent variables.We found no overall significant effect of the awareness campaign, Wilk's lambda = 0.99, F(5, 612) = 0.82, p = 0.535.The tests of between-subject effects also showed no significant effects of the awareness campaign on ad recognition, F(1, 616) = 1.79, p = 0.181, understanding of persuasive intent, F(1, 616) = 0.29, p = 0.589, understanding of selling intent, F(1, 616) = 1.26, p = 0.262, skepticism, F(1, 616) = 1.60, p = 0.206, and disliking, F(1, 616) = 2.01, p = 0.156 (see Table 2 for means).This means that H5a-e are not supported: The awareness campaign introducing the pictogram did not influence levels of conceptual or affective advertising literacy.
To test the interaction effect of the awareness campaign and pictograms (H6), we ran a MANCOVA with awareness campaign (yes or no) and pictogram (no pictogram, #AD pictogram, influencer pictogram) as factors and the five advertising literacy measures as dependent variables.Results reveal no significant overall interaction effect, Wilk's lambda = 0.99, F(10, 121.8) = 0.87, p = 0.565.The tests of between-subject effects showed no significant interaction effects on the five advertising literacy measures (p's > 0.398).H6 was not supported.

Robustness Check: Generalizability Between Videos
To check the robustness of our findings, we also checked whether any of the effects of the awareness campaign and pictogram differed in direction and strength between the two videos.We ran a MANCOVAs with pictogram awareness campaign and video (i.e., Lotte or Furtjuh) as factors.There were no significant two-way or three-way interactions, meaning our findings are generalizable and robust for the two videos.

Discussion
Answering the strong need for insight into how minors can effectively be informed about advertising (e.g., influencer marketing) in online content, we tested the effects of two pictograms that aim to signal sponsored content in online videos and the potential of an awareness campaign to boost the pictograms' effectiveness.
Our findings provide three important insights that have implications for theory, practice, and regulations.First, we find that minors are able to distinguish between sponsored and non-sponsored videos, indicating that they have developed some level of advertising literacy in this context.Overall, mean scores of advertising literacy were high, even without a pictogram (e.g., ad recognition M = 4.43 on a 6-point scale).As advertising literacy is believed to develop with experience, education, and age (Wright et al., 2005), these results indicate that minors have quite developed advertising literacy in the context of advertising in online videos.Although these findings may seem to imply that rules regarding disclosures are ineffective and possibly even redundant, we would be careful with drawing this conclusion: Perhaps it is the increasing presence of disclosures that the minors have learned from.Second, our study shows that the two pictograms informing minors about advertising in online videos went unnoticed by most viewers and did not enhance conceptual or attitudinal advertising literacy.Although pictograms can be processed relatively quickly (Tijus et al., 2007) and were implemented in online videos according to current legislative guidelines, very few minors remembered seeing the pictogram.These findings are in line with previous studies showing that (textual) disclosures are hardly noticed by minors (e.g., Boerman et al., 2023;Boerman & van Reijmersdal, 2020;De Jans et al., 2018;van Reijmersdal et al., 2017van Reijmersdal et al., , 2020)).As disclosures, such as pictograms, need to be noticed to have any effects, further research should focus on investigating pictogram implementations that will ensure that viewers notice and attend to them.Interestingly, exposure to the influencer pictogram did lead to lower levels of skepticism and dislike compared to no exposure to a pictogram.This finding is in line with previous qualitative research that showed that minors do not really care whether online videos are sponsored but that they do appreciate transparency (Rozendaal et al., 2021).In addition, the influencer pictogram was preferred most minors (Boerman et al., 2023), which may explain why only this pictogram had this mitigatory effect.
Third, the awareness campaign did not lead to higher recognition of the pictograms nor enhanced advertising literacy.In line with previous research into awareness campaigns (e.g., Bollinger et al., 2022;Thrasher et al., 2013), the campaign did increase minors' understanding of the meaning of pictograms.However, the majority of minors also understood the meaning of the pictograms without the campaign.Although previous research showed that disclosures have limited effects because existing disclosures are not understood (Rozendaal et al., 2021), this demonstrates that we did develop pictograms that minors comprehend.This has important implications both for other researchers in this field and for practitioners (e.g., policymakers and disclosure designers), as it shows that participatory research methods involving minors as co-researchers are effective in designing understandable and appealing disclosures for advertising in online video content.
However, more research is needed on the conditions under which awareness campaigns do succeed in improving attention to disclosures (such as pictograms) when implemented in online video content and triggering their advertising literacy.The way the current awareness campaign was designed and investigated did not have the desired effect.Perhaps the campaign video did not contain the right information or a single exposure to the video was not enough to create new associative networks in minors' brains for the pictogram and its meaning to become linked to minors' existing advertising literacy.Future research is needed to gain more insights into the potential of awareness campaigns.

Conclusions
Based upon our findings, we argue that the way pictograms are currently implemented in online videos (i.e., at the beginning of the video, on the upper right side of the screen) makes them go unnoticed by most minors and is ineffective in enhancing minors' advertising literacy.Although an awareness campaign can make minors more familiar with the pictograms and their implementation in online videos, it does not seem to boost the pictograms' effects on advertising literacy.
Thus, to create a safe and transparent digital environment for minors, our findings suggest that we should not only focus on disclosures, such as pictograms.The responsibility to protect minors from hidden advertising (e.g., influencer marketing) on online platforms is shared by various stakeholders, including educators who can help minors develop the relevant advertising literacy and the advertising brands and content creators that target minors in the digital ecosystem.

Figure 1 .
Figure 1.Stills of the sponsored videos: (a) Liefs Lotte and #AD pictogram; (b) Furtjuh and influencer pictogram.Note: Links to original videos in the Supplementary File.
The instructions said that the study was divided into two parts and that we would ask them to watch videos in Part 1 and Part 2 and fill out a questionnaire about the videos.We told them that in Part 1, we wanted to investigate what they thought of a new video, and, in Part 2, how young people react to different YouTube videos.This cover story allowed us to show the awareness campaign (or a filler video) in Part 1 and a sponsored YouTube video with one of the pictograms (or not) in Part 2.

Table 1 .
Main effects of pictograms (vs.no pictogram) on advertising literacy.Means with different superscripts in the same row differ significantly from each other at p < 0.05.

Table 2 .
Main effects of the awareness campaign on thoughts (H3) and advertising literacy (H4).
a Note: a, b = Means with different superscripts in the same row differ significantly from each other at p < 0.05.Media and Communication, 2023, Volume 11, Issue 4, Pages X-X