Pre-testing of the WHO’s Educational Video: “How to Protect Yourself against COVID-19”

The coronavirus disease 2019 (COVID-19) pandemic, caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) was declared a public health emergency in many countries, including Indonesia. The World Health Organization (WHO) released an educational video, entitled "How to Protect Yourself Against COVID-19", which explained how to prevent the spread of COVID-19. This study is a qualitative study that aims to pre-test this video, based on the Information, Education, Communication (IEC) theory, by analyzing teenagers’ perceptions of the video. The design used was a Rapid Assessment Procedure (RAP), and the data were collected by conducting in-depth interviews with participants, through virtual meetings. The data were analyzed using a qualitative data matrix and thematic analysis. The results showed that all participants found the video regarding the prevention of COVID-19 spread to be attractive and comprehensible. In addition, the participants felt that the contents of the video were not contradictory to their personal values and that the message was targeted to everyone, including themselves. The video was also perceived to be persuasive. Overall, the participants held a generally positive perception of the video.


Introduction
The coronavirus disease 2019 (COVID-19) pandemic, caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) virus has been declared a public health emergency in several countries, including Indonesia. The virus is spread through human-to-human transmission, primarily through droplets that are released when an individual coughs or sneezes. 1 Under the current COVID-19 conditions, health-related organizations, such as the World Health Organization (WHO), have continuously attempted to disseminate accurate and credible information, worldwide, via social media platforms. 2 The WHO released an educational video on the social media platform YouTube explaining how to prevent the spread of COVID-19 by regularly washing hands with soap and running water, maintaining a minimum distance of 1 meter between people, and coughing and sneezing properly. Videos represent one type of mass media that allows information to be spread quickly, reaching a wide audience. 3 The dissemination of accurate information regarding self-protection efforts during a pandemic, such as COVID-19, has been shown to encourage positive health behaviors in individuals, which could help control the spread of the pandemic. 2,4 Behaviors are influenced by each individual's knowledge and perception of the pandemic conditions. 5 Information, Education, Communication (IEC) is an approach that can be used to promote a sustainable behavioral change in individuals. The IEC approach can be applied through the use of mass media. The WHO recognizes the important role of IEC methods for the achievement of health goals and recommends the use of IEC methods in health programs, to obtain better and more sustainable results. 6 In our study, we performed a pretest analysis of the WHO's educational video, which is a type of communication or IEC material. We analyzed five components of communication material (IEC) effectiveness, which included (1) attractiveness, the elements that make people want to see and listen to the material; (2) comprehension, the clarity of the content and its presentation; (3) acceptability, whether the audience feels that they can accept, believe, and not be offended by the material; (4) self-involvement, whether the audience can identify the materials and recognize the messages that are meant for them; and (5) persuasiveness/tendency to act, whether the audience is motivated or persuaded to perform a particular action. [7][8][9][10] The present study was specifically performed among teenagers (high school students, aged 15-18 years). Despite having a lower risk of developing severe complications associated with COVID-19, teenagers remain as likely as older people to become infected and contagious. 11,12 Furthermore, teenagers represent an age group that interacts with technology, media, and the internet more frequently than any other age group. Therefore, teenagers can play a critical role in the dissemination of accurate information regarding COVID-19, through the media and internet that they access daily. 11

Method
The design of the study was a Rapid Assessment Procedure (RAP), and a qualitative approach was used to collect the data. RAP is a data collection method for obtaining and analyzing in-depth information regarding public health behaviors, during a relatively short period of time, 13,14 and can be used for IEC materials. 15 This study was performed by only five researchers, who had a short time to execute the project, as well as limited funds; therefore, we used the RAP method.
In present study, participants were selected through a purposive, snowball sampling technique, which utilized the networks of one or several informants. There were a total of 10 teenagers, who attended senior high school, five of whom lived in an urban area (East Jakarta) and five who lived in a rural area (Karawang, West Java). Data were collected by performing in-depth interviews with participants, and the results of the interview were interpreted by analyzing the contents of the participants' answers regarding their perceptions of the effectiveness of the WHO's educational video, which is a type of IEC material.
The instrument used was an in-depth, interview guideline, which was assembled based on the components of the IEC material, including effectiveness, perceived attractiveness, comprehension, acceptability, selfinvolvement, and persuasiveness/tendency to act. Prior to the use of the instrument, a trial field interview was conducted, to determine whether the interview guidelines were clear, to determine the order of the interview guidelines, and to determine how long the interviews would take. Two participants (one rural and one urban) were chosen for the trial interview. The results of the trial interview showed that, in both rural and urban areas, all of the questions included in the instrument could be understood well by the trial participants. After the trial, the researchers added two new questions to the instrument. Regarding the aspect of "acceptability", a question was asked regarding the participants' knowledge regarding the institution that published the video was added. This question was deemed to be necessary because it was associated with the informants' belief in the message being conveyed in the WHO video. Regarding the aspect of "persuasiveness/tendency to act", a question was added to ask whether the video requires translation into various languages. This question was considered to be important because it was associated with the interest of the informants to "act", by spreading the video; the language used in the video was English, and not everyone understands English. In both rural and urban areas, the interview process took approximately 30 minutes, for each participant.
Data were collected by public health graduate students, who have been trained in qualitative research techniques. Before each interview, the informants were instructed to watch the video, "How to protect yourself against COVID-19", produced by WHO, which lasted 1 minute and 30 seconds. The presented video was targeted towards a general audience and was not aimed towards a specific gender; therefore, our selection of participants did not consider gender diversity. After watching the video, semi-structured, in-depth interviews were conducted through teleconference services, including telephone lines and video conference services, including Zoom, Google Meet, and WhatsApp, between April 20 and April 28, 2020. After revising the instrument, interviews were conducted with the remaining 8 informants (four rural and four urban). During the interviews, recordings and field notes were taken by the research team members. Data were transcribed, and matrices of qualitative data were prepared. Thematic analysis was used to present the results. We performed the six steps of thematic analysis, as suggested by Braun and Clarke, 16 which are as follows: (1) familiarization with the data; (2) generating initial codes; (3) searching for themes; (4) reviewing the themes; (5) defining and naming the themes; (6) and, finally, producing the report.
To ensure both trustworthiness and data objectivity, after the completion of each in-depth interview, the research team members summarized the results of the interview and confirmed the completeness of the information. In addition, the research team members also determined the willingness of the participants to be re-interviewed, if more information was determined to be necessary.
There is no ethical clearance. Therefore, informed consent collected from each participant to comply with the Helsinki Declaration, which states that participation by individuals who are capable of providing informed consent must be voluntary and without compulsion. The participants should agree voluntarily and are allowed to continue or halt their participation in the study, without facing penalties. 17

Results
The participants in this study were eight senior high school students (grades 10 and 12), four of each from urban (code: U) and rural (code: R) regions. Their ages ranged between 15-18 years. The participants consisted of three males and five females. A summary of the participants whose data were used in our analysis can be found in Table 1.
Five aspects of communication material (IEC) effectiveness were evaluated, and representative statements provided during the interviews regarding each aspect can be observed in Table 2. a. Perceived attractiveness of the video Most of the participants believed that the animation was good, interesting, and easy to understand. However, participants U3, U4, and R3 found that the animation was not colorful enough, as it was only portrayed in black and white. According to these participants, the animation should have been more colorful, to make the video more interesting. All participants, both rural and urban, stated that the text could be read clearly. Participant U4 suggested adding a different color besides black, to highlight the most important text.
All participants stated that the sound could be heard clearly, although some participants described the speech as being too fast. The participants felt that the video was effective, not too long, and straight to the point. Participant U4 felt that the duration was ideal, but would not mind additional information that would prolong the video, such as if contents describing the use of alcohol to wash hands effectively were to be added. b. Perceived comprehension of the video For some participants (four rural participants and one urban participant), the use of English in the video made comprehension of the information and messages conveyed in the video more difficult. However, in general, the participants were able to grasp the message regarding the prevention of COVID-19 transmission. Some participants (R2 and R3) were able to mention all of the main points regarding the prevention of COVID-19 mentioned in the video, whereas others only mentioned some of the points. All of the participants stated that the function and purpose of this video were to inform people how to prevent the transmission of COVID-19. c. Perceived acceptability of the video According to all participants, nothing in the video was contradictory to their personal values. Although participant R1 stated that she did not fully agree with the implementation of social distancing, as it prevented her from meeting up with her friends, overall, she was okay with the implementation for the greater good. She also suggested adding the recommendation of using masks to the video.
Participant R1 did not identify the institution that released the video. Participants R2 and R4 identified that the WHO released the video but claimed little knowledge about the WHO. All other participants were able to identify the institution that released the video as the WHO and knew that the WHO is a global health organization.
All participants, from both rural and urban areas, stated that they believed in the contents of the video. However, the participants provided different reasons for why they believed the video. All of the urban participants and participant R3 stated that their belief was based on the fact that the video was published by the WHO, a worldwide credible organization. In contrast, most rural participants stated that they believed in the video's content because it contained a positive message. d. Perceived self-involvement of the video All participants felt that they were part of the target audience for the video and that the video was beneficial for them. They found the message to be useful because it explained the spread and prevention of the virus, encouraged them to maintain cleanliness, added information that the virus could be spread at a distance of 1 meter, and, overall, found the message to be positive. Participant U4 learned why social distancing rules are implemented after watching the video because the video states that the virus could spread if people do not maintain a distance of 1 meter. e. Perceived persuasiveness of the video All participants were interested in implementing the various messages that were conveyed in the video, such as maintaining personal hygiene, not touching facial a - reas, enacting social distancing, washing hands with soap and running water or hand sanitizers, and maintaining the cleanliness of objects by spraying them with disinfectants. All participants, both rural and urban, felt that spreading the video was important. The informants planned to spread the video to their families and friends, via social media, such as WhatsApp (through WhatsApp status posts or WhatsApp groups) and Instagram (through Instagram feed posts or Instagram story posts). Participant R4 believed that spreading the video by posting it to her Instagram feed would be more effective be-

Attractiveness
Opinions of the animation The animation is good/attractive "The video has a good animation." (U1) in the video "The animation is attractive." (R2) The animation is not attractive enough "The animation is not attractive enough, as it should've been more colorful." (U3) Opinions of the sound in Sound is clear and understandable "The sound is clear and understandable." (U1) the video Sound is spoken too fast "The sound is clear, but spoken too fast." (R4) Opinions of the text in the Text can be read clearly "The text can be read clearly." (R1) video Different colored texts could have "The text is clear, but different colored texts could have been added been added to highlight the more important parts." (U4) Opinions on the duration The duration is sufficient "The video is not too long and not too short, the video is straight to of the video the point." (R2) "I think that sufficient general information is provided. But I think it's okay to have additional time added to the video, if further information and explanations are needed." (U4) Overall opinions of the video Overall, the video is good / clear "The video provides good and informative information. The animation provides a clear depiction of the messages being conveyed, I can understand the message just by watching the animation and without having to read the text. The subtitles are good, not confusing and in sync with the sound. The sound and pronunciation are also clear, making the message easy to understand." (U1) "The video is very clear, the illustration is clear, which allows us to understand the meaning of the video, but the illustrations are not colorful enough." (U3) Comprehension Comprehension of the English Quite difficult "I don't fully understand the English, so I looked at the animation to language used in the video help me better understand the meaning." (R3) "It's quite difficult to understand." (R4) Somewhat easy to understand "The English is somewhat easy to understand, but there are some terms that are not understood." (U4) Easy to understand "The English words used were common, so it was easy to understand." (U1) "I understand the English being used." (U2) Comprehension of the infor-The virus' transmission distance "To prevent the virus, we need to implement social distancing measumation regarding COVID-19 of 1 meter res, by keeping a 1-meter distance." (R1) prevention steps obtained "The transmission can occur within an approximate distance of 1 from the video meter." (U4) Cleaning the surface of objects "Clean objects around us by spraying disinfectants or other liquids regularly that can kill the viruses." (R2) "Objects around us need to be sprayed with disinfectants so that the microbes die." (U3) Prohibition of touching eyes, nose, "Do not touch the eyes, nose, or mouth or around any part of the and mouth with dirty hands face." (R3) "Avoid touching the eye, nose, and mouth areas." (U2) Covering the nose and mouth when "If we are having a cough, we must cover it with a tissue and immecoughing, or sneezing, using elbows diately remove the tissue." (R2) or tissues "When we cough/sneeze it must be covered with our elbows." (U1) Washing hands with soap and water "Wash hands with soap, or use hand sanitizers." (R4) as the most effective method to prevent "Handwashing is done with soap and water or alcohol." (U1) spreading Comprehension of the purpose To inform/to educate the public "To inform people about how to prevent the transmission of coronaof the video virus." (R2) "To educate the public so they know more about COVID-19, espcially how to prevent it." (U2) To persuade the public "To invite all people in the world to do actions such as keeping their distance and spraying disinfectants." (R3) "To remind the public to be more aware of the prevention of COVID-19." (R4) cause more people will see it and the post can be viewed at any time. Similarly, participant U4 believed that spreading the video via Instagram stories would be more effective because she felt that more people would watch the video if it is posted on Instagram stories. All participants felt that the video should be translated into various languages. Participant R1 expressed the necessity for translations into Indonesian, the primary language used in Indonesia, but stated that the video does not need to be translated into local languages, such as Sundanese or Javanese. According to participant U1, the video should be supplemented with Indonesian subtitles and also dubbed into Indonesian because sometimes people in Indonesian are lazy to read.

Discussion
In this study, authors explored the participants' perceptions regarding the attractiveness of a video, published by the WHO, and examined the participants' comprehension, acceptability, self-involvement, and persuasiveness/tendency to act towards the video contents. Videos, as IEC media, should attract the audience's attention through animation, text, and sound. Moreover, videos should also have an ideal duration. Most of our participants expressed interest in the video's animation. Animations can help informants to better understand the information being conveyed, as described by Johari, et al. 18 Similarly, text and sounds can make the message Table 2

Acceptability
The existence of content which No content is contradictory to the "I found nothing offensive, but social distancing advice makes me a is contradictory to the partici-participants' feelings, values, and little upset because they prevent me from meeting up with my friends, pants' feelings, values, and beliefs but it's okay, as it is for the greater good." (R1) beliefs "In accordance with my values and beliefs." (U2) "There are no offensive messages." (U3) Knowledge about the institution Knows about the institution "It was published by WHO, the World Health Organization." (U3) that published the video "It was published by WHO (World Health Organization). WHO is an international organization that takes care of everything related to health." (R3) Does not know about the institution "I do not know about the institution that published the video." (R1) "The video was published by WHO, but I first heard about 'WHO' when I was invited to this interview by my friend. I don't really know much about WHO." (R2) Trust/belief towards the infor-Believes the information because the "I believe it, because the video spreads a positive message regarding mation message of the video is positive/ the prevention of the virus' spread." (R4) beneficial "Can be trusted because the video has a lot of beneficial messages." (R2) Believes the information because the "I believe it, because the information was released by a worldwide video was published by the WHO organization, so the information is accurate." (U4) Self-involvement The opinion of the participants Targeted for all groups/everyone "It is targeted for all groups, from children to the elderly." (U2) regarding whom the video is "For all people in the world. Even though the elderly are the more targeted for vulnerable group, but perhaps this video is meant for everyone: young people, young children, and the elderly." (R3) The benefits of the video for the Beneficial for preventing COVID- 19 "Very helpful in explaining the spread and prevention of the virus." participants transmission (R1) "The video is useful, and I feel more optimistic that this virus can be prevented by washing our hands. And now I know why social distancing rules are implemented, if we don't keep a distance of 1 meter, the virus can spread easily. So, the benefit is, we can help prevent the transmission of the virus." (U4) "Very useful because the video persuades us to do positive actions." (R3) Persuasiveness/ How the participants will act Implement the messages that were "I'm interested to implement the various messages." (R2) Tendency to Act regarding the messages convey-conveyed in the video "I want to implement the things that were explained in the video ed in the video because my mindset immediately said 'oh these are the ways to prevent transmission of the virus.'" (U4) Spread the video through social media, "It is important to spread the video, I want to share it to my groups, spreading the messages in the video my contacts, my relatives and through my WhatsApp story." (R3) "It is very important for people to know this, as it is official information from the WHO. I want to share this video to my family, friends, and social media, like Instagram." (U3) Feels the need for a translation of the "There needs to be a translation to Indonesian, but it doesn't need to video into the Indonesian language be translated into local languages such as Sundanese or Javanese." (R1) "Indonesian subtitles need to be added, and the video also needs to be dubbed in Indonesian because sometimes Indonesian people are lazy to read." (U1) Fadliyah et al, Pre-testing of the WHO's Educational Video: "How to Protect Yourself against  being conveyed easier to understand, by listening to the narrator's voice and reading the text contained in the video. The length of the video, 1 minute and 30 seconds, was also considered quite effective by participants because the information was conveyed clearly and completely in a relatively short period of time. This study revealed that participants' overall understanding of the video contents were considered good because they knew the function and purpose of the video, which was to provide information and education to the public regarding the transmission and prevention COVID-19 spread. An individual's understanding of information or messages can be interrupted by semantic interference, especially communication disorders caused by errors in the language used. 19 Authors identified differences in the English language comprehension levels between rural and urban participants, with four rural participants stating that they had difficulty understanding the English used in the video, whereas only one urban participant stated that they had difficulty with English comprehension. However, this obstacle can easily be overcome. Because the video is equipped with animations, the inclusion of text or subtitles in English that can enhance their comprehension.
The COVID-19 educational video published by the WHO contains five important messages: (1) the virus' transmission distance of 1 meter; (2) cleaning the surfaces of objects regularly; (3) the prohibition of touching eyes, nose, and mouth with dirty hands; (4) covering the nose and mouth when coughing or sneezing, using elbows or tissues; and (5) washing hands with soap and water, as the most effective way to prevent virus spread. 12 Out of these five messages, only a few participants were able to mention all of them completely. The contents that all the participants were able to identify were (1) to maintain a distance of 1 meter, and (2) to wash hands. Authors assumed that this occurred because these two points have previously been reported or have been frequently heard by all participants. Both rural and urban participants generally displayed a satisfactory level of comprehension for the contents of the video.
As a communication medium, messages from videos must be acceptable to the target population. Messages that can be received well are usually messages that are not offensive, do not cause distrust, do not cause disapproval, and do not cause the audience to reject the message and, ultimately, not act according to the message. 20 A similar pattern was observed among the participants' answers. Among the urban participants, who were generally aware of the WHO, trusted and believed the contents in the video because they felt that the institution that published the video was credible. Those participants, including most of the rural participants, who answered that they trusted the contents of the video but did not know what the WHO is, implied that they trusted the video because the message delivered was a positive one.
Furthermore, one of the parameters for determining the effectiveness of communication materials is "self-involvement", which determines the perceptions of the recipients based on whether the message is believed to be directed and intended for them or others. 8,9 The participants agreed that the information regarding the spread and prevention of COVID-19 was useful and targets the whole community, in general. Moreover, one urban participant was able to link the message conveyed in the video with the reasons for the implementation of social distancing regulations, despite the video not explicitly including any messages regarding social distancing regulations. 21 This result indicated that the video successfully engaged the participant.
One result of effective communication is persuasiveness or the tendency to act. 22 Authors believe that the video was able to persuade the participants to implement and spread the messages conveyed in the video. The participants expressed a preference for the use of various social media platforms to spread and share the video, indicating that one of the most important advantages of using social media is the ability to share knowledge and information online, between various groups of people. 23 For video distribution, the participants believed the video should be translated into the Indonesian language, through both subtitles and dubbing, because not everyone who watches the video can understand English. For most Indonesians, English is not their mother tongue; therefore, the English language used in the video could potentially obstruct their understanding of the messages being conveyed.

Conclusion
The information authors collected from both rural and urban participants suggested that all of them have similar perceptions of the video. Overall, the video was considered to provide clear information, and the inclusion of images made the video easier to comprehend. The partici pants also found the video to be attractive and were able to perceive that the function of the video is to inform viewers regarding how to prevent the spread of COVID-19. The participants believed that the content of the video contains trustworthy information because it was published by the WHO, an organization they considered to be credible, or because of the messages in the video itself. The participants also felt that the contents of the video did not contradict their personal values and beliefs. The participants considered everyone, including themselves, to be the target audience of the video. The informants were also interested in sharing the video with people. Additionally, the participants indicated their hope that the video could be translated into Indonesian because not everyone, especially in Indonesia, that can understand the English used in the video.

Ethics Approval and Consent to Participate
There is no ethical clearance. Therefore, informed consent was collected from each participant to comply with the Helsinki Declaration, which states that participation by individuals who are capable of providing informed consent must be voluntary and without compulsion.
The participants should agree voluntarily and are allowed to continue or halt their participation in the study, without facing penalties.