An Exploratory Mixed Method Study on H5P Videos and Video-Related Activities in a MooC Environment

In this paper, an exploratory mixed-method study is presented examining the video-related behavior of participants of a massive open online course (MOOC; N=1.238). Firstly, detailed log-file analysis of six videos has been carried out to compare clickstreams of videos with and without integrated H5P quiz integration. It shows quite different seeking and watching pattern behavior in video with H5P quizzes. In a second step, learners participated in an online questionnaire (N=707): most of them see the quizzes in videos as always or mostly helpful (67%). The survey also shows that for many, taking notes, turning on subtitles, using the transcripts, or even increasing the speed are important activities when learning with the videos in the MOOC. In a third step, interviews with 10 MOOC participants are a source for qualitative insights into how learners use the learning videos.

for these behaviors (Clow, 2013) to studies that analyzes the participant-behavior over a complete MOOC. These last approaches often lead to categories, grids or clustering-methods, such as Ferguson et al.'s (2015) or Yoon et al.'s (2021) study. Typically, videos are the core of MOOCs and are enriched by forum discussion or further readings. In many cases, researchers use data from video logs to predict completion rates (Halawa et al., 2014;Lemay & Doleck, 2020), develop visual representation methods of these data (Mubarak et al., 2021) or compare behavior between video-driven MOOCs and for-credit courses (Almeda, 2018). Other researchers focus on data gathered outside of the MOOC and try to connect these findings to aspects such as grades or dropout rates (Gamage et al., 2020). To further enhance the design process of such MOOCs and to lower down learner dropout rates, increase completion rates, enhance learner engagement, and better learning outcomes, metastudies (e.g., Yu, 2021b) have also been conducted.
However, research on interactivity, and especially the influence of certain methods, is still rather rare. Wilkie et al.'s (2018) work is an example of a study on a specific tool, in this case H5P. However, the results of the study were based upon anecdotal evidence from staff and learners at a university. To gain more insights into the influence of H5P, this study built upon the described positive findings.

LITERATURE REVIEw
Videos in MOOCs can vary from recorded traditional classes in full length to a set of short instructional to professional produced learning videos. Concerning the learning design of the videos, they can include repetition or transfer questions. However, methods such as recorded lectures are too passive and lack interactivity compared to in-person classes. Indeed, the "illusion of knowing effect" (Ertel, 2007;Salomon, 1984) is a situation where the learner falsely assumes acquisition of the learning content, which can be overcome by in-video quizzes, for example. Bjork et al. (2013) could show that learners tend to overestimate their preparedness for exams due to various reasons. To eliminate these false convictions, they suggest meaningful self-testing in the form of additional activities (e.g., quizzes), which complement the learning material. Lawson et al. (2006) also proved that learners who are provided with guiding questions along their learning material scored higher on the related test questions than learners who did not have such a form of self-assessment. Another important finding is that teacher presence in learning videos can improve academic success and intrinsic motivation, when compared to videos without lecturers in them (Yu, 2021a). Thus, it is no surprise that developers of MOOCs try to gain more interaction with their learners (Lackner et al., 2016).

Learning with H5P
A technical tool to make videos more interactive is H5P (Manacek et al., 2020). H5P is a free and open source software that allows to create interactive content for the Web. The JavaScript-based software was initially released in 2013 and it is already implemented in many MOOC platforms or learning management systems such as Moodle. H5P is labeled as an easy and interactive way to enhance online experiences with presentations, interactive videos, quizzes, timelines, a memory game, and many more (see https://h5p.org). The most important feature for this article is the addition of questions in the video that allow users to rate their knowledge while watching a lecture. A possibility to integrate H5P in MOOCs is to add such in-video questions to already existing videos. Therefore, quizzes will pop up on the screen of the learners during watching and they are asked to select an answer before continuing. This allows to give a first automated and direct feedback to the learners. There are many positive expectations concerning the usage of H5P in learning videos: According to Sinnayah et al. (2021), H5P enables such self-reflection through formative and immediate feedback. Wilkie et al. (2018) proved the positive reception of H5P elements with learners and lecturers praising the interactive nature and flexibility of the tool. Rekhari and Sinnayah (2018) observed that learners rated the positive influence of H5P questions very high, concerning their improvement of knowledge. Figure 1 shows how a H5P quiz interrupts the video (a screencast, left) with some questions (middle) and feedback (right).
Source: Lehren und Lernen mit digitalen Medien I (https://imoox.at/course/luliss22) For MOOCs, the usage of H5P in learning videos is of interest as well. As a result, higher interaction with MOOC materials through the H5P integration could lead to a smaller desertion of participants (Labrador et al., 2019). Ferschke et al. (2015) made a direct connection of MOOCs and video interaction. According to these authors, a greater number of clicks within a video increases the risk of dropping out. Li et al. (2015) developed a clustering method based on in-video interactions that observed more video interactions when weak learners are confronted with difficult topics. It has also been shown that in-video quizzes are valuable for the majority of MOOC participants, who mainly watch videos and do not use other course materials (Anderson et al., 2014;Wilder et al., 2001). Kovacs (2016) observed that participants tend to adapt their watching behavior according to those in-video questions. He discovered seek-activity surrounding in-video quizzes, such as skipping back to a certain time before a question pops up. He also observed that some participants developed quiz-oriented navigation strategies such as jumping directly to the quizzes. In this study, the authors analyzed the data of a MOOC and compared them to the above-mentioned findings. The authors' main goal was to detect such strategies and to verify the theory according to which there is an accumulation of user interaction surrounding H5P questions. Figure 1 shows screenshots of such in-video questions. Kovacs (2016) could observe a spike of seek activity surrounding in-video quizzes. He also quoted Kim et al. (2014) and noted that visual transitions such as slide transitions, lead to a decrease of socalled in-video dropouts. Literature has shown that the rate of engagement with in-video assessments is significantly higher than with other forms of assessment (Anderson et al., 2014). This means that in-video questions could lead to a lower risk of dropping out. Overall, Kovacs (2016) was able to prove that in-video quizzes are a common source of seek chains within videos. He was also able to show that participants use reverse search to check the part of the video that explains something asked in an H5P question. According to his research, users also do not tend to skip forwards or backwards over these in-video questions (Kovacs, 2016). H5P quizzes in videos are a way for the participants to test their knowledge immediately; besides, they influence the way videos are consumed.
Interactive videos are a complex topic, and they influence different aspects of learner behavior. Research has shown that they can increase engagement (Kim et al., 2015), increase the learning speed (Schwan & Riempp, 2004) or enhance the self-study context (Kolås et al., 2016). Eube and Vogt (2017) mentioned the learning-promoting effect of such interactions, with the videos in the form of quizzes or jump labels for navigation. Yu and Gao (2021) could also show that the usage of videos is dependent on various factors, such as facilitation condition, task-fit technology, perceived control over time, performance expectancy, and learning-family conflicts, at least for a flipped-classroom setting. They demonstrated that users prefer shorter or longer videos depending on these different aspects.

ISSUES, CoNTRoVERSIES, ANd PRoBLEMS
There are still "limited studies on exposing various complex aspects of clickstream interactions resulting from learner's activities within different MOOC videos" (Mubarak et al., 2021, 711). In addition, most studies that examine video consumption behavior and also use log-file analysis have not included interactive videos, yet. Despite becoming more commonly used and promoted, knowledge about H5P and its impact on learner behavior is still very limited. As a result, many insights about user behavior, gained through clickstream analysis must be viewed critically. Mir et al. (2021) conducted one of the few studies that directly investigated the impact of H5P. Their research measured different levels of satisfaction of a learning management system and could show that the value of a video in this learning context strongly depends on its level of interactivity. Learners who participated in classes that used such interactive videos were more satisfied with their learning experience. Mir et al. (2021) also stated that H5P is very suitable for boosting learners' interest, because it is easy to use and very efficient. Therefore, the goal of this study was to learn more about the influence of H5P questions on learning behavior.
The aim of the research of the authors of this paper was to verify the potential positive influence of H5P questions in videos in a MOOC. Practically, such data are available and tracked in the authors' system, but was not used so far as part of the course dashboard. In addition, they wanted to clarify what other strategies learners use around the videos. They planned and executed their research as an exploratory analysis to get a good insight into the possibilities of log-file analysis and its interpretation, and also to gain insights into other relevant aspects of learning with videos in MOOCs. Thus, the aim of their research was to explore the effects of H5P interactive videos in a MOOC learning environment. Particularly, the researchers sought to answer the following questions: 1. Are there differences of learner behavior concerning videos with H5P quizzes and videos without H5P quizzes? 2. Are H5P integrated quizzes helpful? 3. What other behaviors do learners show when interacting with the videos in the MOOC for learning?
Based on these three questions, the authors developed and applied an exploratory research design. Figure 2 shows the research design, which is a mixed-method design. The study uses three different sources, namely user behavior (log files), results of an online survey, and problem-based interviews from a single MOOC. The mixed-method design combines quantitative and qualitative data and is a good base for relevant insights and interpretation (Creswell, 1999(Creswell, , 2013. Additionally, this approach combines responsive and nonresponsive data. The following subsections describe the individual analysis steps, starting with the selected and addressed MOOC and background information.

Research Model and Procedure
To answer question (1), the researchers planned log file analysis of learners' behavior concerning the learning videos. To answer questions (2) and (3), they carried out and compared a survey amongst all MOOC participants and problem-based interviews.

Research Context and Sample
The authors conducted this study in the frame of the Learning Analytics: Effects of Data Analysis on Learning Success (January 2020 -September 2022) project of the University of Graz and Graz University of Technology (TU Graz) with the Province of Styria as a funding body (12. Zukunftsfonds Steiermark). The goal of this research project is to gain insight into the learners' learning behavior when working with videos on the MOOC platform iMooX.at. For this case, the researchers analyzed extracted data from the MOOC Learning and Teaching with Digital Media I (Lehren und Lernen mit Digitalen Medien I) from 2021. This MOOC is part of teacher education of all universities of the southeastern part of Austria. The lecture itself has a blended learning design; therefore, the MOOC and its online videos and quizzes are expanded through offline, face-to-face sessions with small groups of up to 25 participants . Although many learners are students, there are also other participants.
MOOCs at the iMooX platform (Ebner, 2021) typically embed a quiz at the end of each unit. A passed test and getting a certificate are the base for a successful participation. Therefore, each quiz has up to five attempts. Over six weeks, each week the learners watch from four to six videos and every unit is presented by different lecturers. The MOOC has 32 videos, which present topics such as didactics, informational thinking, and media law. In addition to the online videos, the learning environment offers transcripts and additional literature. Three of the examined videos contain H5P questions.
The research project itself is based on a cooperation with students from the University of Graz who did a research project work within their studies. The analysis of the data, the development of the questionnaire as well the interviews based mainly on their work. Due to the interesting insights, the researchers later decided to publish the results; this contribution is mainly in the responsibility of the Graz University of Technology staff.

Research Instruments and data Analysis
The following three subsections depict the instruments the researchers used and how they further analyzed the retrieved data. To gain insight into the behavior of the learners with and without H5P quizzes in videos, the authors analyzed the log files of their activities while using the videos. As a second step, they conducted an online questionnaire, which was obligatory for getting a participation certificate. With first insights from early data of the two previous steps, the team then developed a detailed guide for individual, so-called problem-based interviews with 10 participants.
Concerning ethics, the data, their collection, and evaluation on the iMooX platform are subject to the European data protection guidelines and TU Graz guidelines. All users have been informed about the conditions and are aware of and agree to the anonymized use of their data in scientific studies. The same applies to the participants of the online survey as well as the interviews, informed consent is verifiable.

Log File Collection and Analysis Concerning MooC Videos
For this detailed analysis, the authors selected six videos-three with, and three without H5P quiz integration from different units of the courses. The authors selected the videos in such a way that the common patterns in MOOCs-the first videos are used more often than those in later lessons-should not have a major influence. The researchers selected all week-2 videos, as they represent the first lectures fully dedicated to the content. They chose video 1 from week 4, because past insight showed that learners tend to have problems with this chapter, so finally the authors chose video 2 of week 6 because it marks one of the last inputs learners get within the MOOC.
The H5P log files are collected client-side and then transferred to the server. The log files themselves give information about playing and pausing the video, as well as seeking backwards or seeking forwards. The researchers used the data to calculate how often and at what points in the video users start their search behavior and to what destinations they jump. For the analysis, they could extract seven variables from the log files describing different activities: Closed, finished, playback seeking, pause back seeking, play forward seeking, pause forward seeking, and watched. Closed was recorded when a user closed the window while the video was still playing. Therefore, finished was registered, when a video stopped due to being at the end of its playtime. The seeking variables registered, if the seek was backwards (i.e., play back seeking and pause back seeking) or forwards (i.e, play forward seeking, pause forward seeking) and if the seek was initiated while the video was playing or paused. Then, the authors analyzed the log files in two ways: First, they compared the general, accumulated activities within their six selected videos and if there were interesting differences. Subsequently, they included the timestamps and used diagrams to visualize, and therefore analyzed according to (potential) differences of the patterns.

online Survey Amongst MooC Participants Concerning Their Learning with Videos
The students' team developed the questionnaire concerning their personal experiences and discussion with students. The researchers tested a draft as a paper version with other students, and slightly revised the questions afterward. The online questionnaire was then embedded into the MOOC, where it was possible to take part along the time frame of all Southeast Austrian lectures using the MOOC. Thus, registered MOOC participants had roughly three months to answer nine additional questions, addressing topics related to the authors' study in parallel to the MOOC. These were divided as follows: • Five questions about learning behavior and strategies had to be answered with a four-level Likert item. The participants had to give answers concerning their general opinion on the H5P questions, the usage of notes, rewatching behavior, the usage of subtitles, and changing the play-back speed. • Two single choice questions about individual transcript usage and self-assessment. • One multiple choice question about ambiguities was further added. • At the end, participants had the opportunity to address further issues or desired changes in an open question.
The authors analyzed the results of these standardized questions with descriptive statistics and provision of diagrams to show the distribution of answers.

Problem-Based Interviews with Learners
Each interview took about 20 to 30 minutes and the researchers later transcribed, tagged, and analyzed them according to Kuckartz' (2018) qualitative content analysis. They selected the sample of 10 participants from learners, the main target group of the MOOC, who attended the face-to-face lecture that enriched the online experience during the semester. The researchers aimed for an equal distribution of male and female participants. Within these interviews, the focus was to get a better understanding of the information, available through an early look at the log files and the questionnaires. However, additional topics such as time management or suggestions for improvement were also allowed. Table 1 gives an overview of the collected data within the presented research. The MOOC counted 1,238 registered participants, who did not all watch all videos (see more details in Table 2). In total, 707 participants answered the online questionnaire, and the researchers conducted the problem-based interviews with three female and seven male learners. Table 2 shows a total listing of all interactions across the selected six videos. It presents the length, the number of users, the sum of their activities, and the means of videos. It shows that the mean of the sum of activities of videos with H5P is higher than of the videos without and that they had more users.  Figure 3 shows a total listing of all interactions, so seeking, watching, and other activities per user across the selected six videos. Since the videos vary in length and the number of users who have watched them varies, the authors showed the activity in relation to the number of users and the length of the videos. In relation to the length and users, video 2 and the H5P video 2 shows the most activities.

Accumulated Activities on Learning Videos with and without H5P Quizzes
The researchers also checked the influence of the order of the videos in the MOOC, and observed the typical decrease (attrition effect, Sinha et al., 2014). As a result, while the overall relation of seeking and watching behavior did not differ drastically throughout the different videos, the authors observed some (small) differences, depending on if H5P questions were included or not. Firstly, the number of users of H5P videos with quizzes is slightly higher, so it could be that these videos are more attractive because of this implementation, which can be seen before video starts. However, when comparing the activities per user and per second of lengths, this does not show clear differences in relation to H5P.

Video Activities over Time
The comparison of the visual representation of the log files over time shows clearer differences between videos with and without H5P quiz questions. Indeed, the authors' analysis shows that an accumulation of seeking activity can be noticed around in-video questions. An example for this behavior is in Figure  4, where data from the week-2 video shows clearly that the participants increase the number of their interaction around H5P questions. This phenomenon can especially be seen when compared to the interactions visualized for a video without H5P questions, as in Figure 5, where seeking behavior is far more evenly distributed over the playtime of the whole video. Additionally, the activities in those videos with H5P questions were centered around the timestamps where questions occur. In the case of the video H5P video1, time spans around the H5P questions are much more often the destination and origin of seeking activities than any other time stamp, as Figure 4 depicts. The participants tend to use these H5P questions as a sort of guidance for their interactions. On the one hand, some participants seek backwards after the quiz-potentially, when the question could not be answered. The log files show as well that a huge number of interactions is connected to fast forwarding or jumping directly to an in-video question. The type of seeking behavior across all three H5P videos is quite balanced.

RESULTS FRoM THE LEARNER SURVEy CoNCERNINg H5P VIdEoS
In the questionnaires and interviews, the authors used these findings as a foundation for further investigation. Overall, the H5P questions were very well received by the learners (Figure 6). In the questionnaire, 185 of 707 (26%) participants answered that they were "always helpful," while 287 (41%) answered that they were "mostly helpful." Only about 8% of all the participants had the impression that the in-video questions were only helping them rarely or not at all.
The open questions also showed that 35 participants wanted more integrated questions. This marks the second most common category after shorter videos, with 77 participants. The next subsection provides other findings from the open questions.

RESULTS FRoM THE LEARNER INTERVIEwS CoNCERNINg H5P VIdEoS
In the interviews with 10 learners, most of them see H5P questions in the videos as activating and boosting their concentration. According to interview 2, the mere existence of an H5P question within a video already enhanced the concentration, because they wanted to be able to answer it correctly when it occurred. In interview 1, the learner said that the questions are good for self-assessment, because of the instant feedback. The fact that the video stops at the moment an in-video question occurs was also highlighted by the same participant, as this triggers a moment of attention. Interviewee 2 compared this phenomenon to an in-person class, where the lecturer asks the audience and the attention of everybody is increased abruptly. They also suggested that this could bring many people to jump back in the video and watch a certain section again, because they think they might have missed something. Two of the 10 participants said that they used these in-video questions as a moment to rewind, if they were not able to answer it correctly. Interviewee 10 answered that they started to search for the exact moment in the video, where the topic of an H5P question is talked about in such a case.
Beside this positive feedback, there was also some criticism. Interviewee 8 found the questions too easy, but fitting for the setting. Interviewee 3 also criticized the used H5P questions. On the one hand they said that some people might get distracted by them or their flow of watching the lecture might get interrupted. On the other hand, they said that many of the questions were written in a way that they would forget the topic again rather quickly. However, this participant was the only one with a negative opinion on the H5P questions.

AddITIoNAL INSIgHTS INTo ACTIVITIES ARoUNd VIdEoS THRoUgH THE SURVEy
Beside the relevancy of H5P questions and behavior, the MOOC participants were asked if they took notes while video watching, if they speeded up the video, if they activated subtitles, and if they watched the videos more than once. Figure 7 shows the distribution of answers concerning a set of statements: From these opportunities, note-taking is the favorite behavior, followed by rewatching. Nevertheless, both speed-up while watching and the usage of subtitles are also a regular behavior for many learners-more than a third had already used one of these possibilities. Another interesting finding was that 43% of all the learners used transcripts while watching the videos, while 7% only read them and did not watch the videos. In contrast to this, 50% answered they did not use the transcripts at all (Figure 8).
Besides, participants had the opportunity to add additional opinions, wishes, and suggestions in the form of three open questions. As the authors already mentioned, 35 people requested more integrated questions within the videos. Apart from 289 participants, who did not have any wishes for changes, this was the second most common suggestion after shorter videos with 77 answers. Other common suggestions were less monotonous speakers, more visualizations, a faster talking speed or the change of disturbing background music. When asked about aspects, the learners liked and disliked about the MOOC self-assessment and appreciated H5P once again. In particular, learners praised the possibility to reflect on one's knowledge and they had a feeling of being well prepared after completing them. Transcripts, the grade of independence, and the general structure of the lectures were additional aspects many learners liked in the open question. On the other hand, the secondary literature was by far the most disliked aspect. Participants criticized the complexity and length of the texts and mostly relied on the videos and transcripts. A few learners also disliked the quiz and the integrated questions. In detail, they answered that the questions were sometimes too short and not sufficient for real self-assessment.

AddITIoNAL INSIgHTS INTo BEHAVIoRS ACCoRdINg To THE INTERVIEwS
The strategy of most interviewees was to simply watch the video and only interact with it, when the H5P question occurs. Taking notes is important for some interviewees: Interviewee 5 said that they used bullet points noted down during the video, for better recognition and as a guidance, when coming back to the topic later. They also mentioned that having a notebook and using handwriting is essential for their learning success. Interviewee 7 stated that they started to take notes during the early lectures, but changed to only using the transcripts later. Interviewee 4 used screenshots to document the most important details during the videos. There were also multiple interviewees (six of 10) who did not take any notes.
The interviewees watched certain parts of a video again when they did not fully understand it, as in the case of interviewee 6: "Then, if I did not understand something, I watched it again." Interviewee 6 also said: "I watched the videos, then I read the literature and, if it was necessary, I watched the video again." Interviewee 8 said that they were doing online research after a certain video and discussed the topic within a group. Interviewee 1 answered that they might watch certain videos again later, when learning for the final test. Interviewee 10 said that many videos were good enough to motivate them to watch them a second time.
There are multiple reasons for using the available transcripts. Some learners used them for remembering if they had already watched a certain video or if they were already familiar with the topic: "Afterwards, when I was not sure, I looked into the transcript and read over it quickly. Then, you will instantly see if you have already watched the video" (Interview 5). Overall, some learners (Interview 5) received the "combination of video and transcript" very well. Some participants also mentioned that they used the transcript when they could not understand the lecturer in terms of audio quality or when they wanted to search for a specific piece of information quickly.
Some participants also addressed the relevancy of subtitles. Most participants answered that they did not use them at all. Interviewee 3 said: "Yes, I saw them, but I am not a friend of subtitles. I like to watch it without them, but that is just my personal preference." However, those who activated the subtitles found them "pleasant and useful for reading along" (Interview 4). The same participant compared it to watching TV shows, where they also activate subtitles. In this way, they have the option to read along together with the visual input sometimes.
Within the interviews, the opinions on changing the playback speed were quite differentiated. Some of the participants said that they never or rarely sped up the videos, while a majority of them reported the opposite. Interviewee 4 said: "I think everyone I spoke to did watch them with increased playback speed. I think it is quite motivating when you see that a video is ten minutes long, but I can even do it in seven or eight minutes." Interviewee 1 appreciated the function and saw it as an advantage when compared to in-person classes. Some learners also mentioned that using the function was dependent on the topic and the complexity of the video.

SUMMARy: HyPoTHESES CoNCERNINg THE RESEARCH QUESTIoN
There are several similarities between the results of the three forms of data collection and analysis, which are now summarized and compared lead by the authors' research questions. As the authors adopted an exploratory research approach, they present theirr results as hypotheses that might be verified in future and potentially experimental study designs .
Hypothesis One: There are differences of learners' interaction patterns and effects concerning videos with H5P quizzes and videos without H5P quizzes, but not regarding the amount of activities.
The H5P questions influence the watching behavior of learners. As the analysis of the log files suggested, these in-video questions work as a sort of navigation help for learners. Through the questionnaire and interviews, the researchers could find multiple reasons for this behavior. Some learners said that they jump directly to the question and review it first, before deciding whether they watch the sequence before it or not. Others may rewatch specific portions of the video, connected to the H5P question. The pure existence of the questions alone also seems to increase the concentration and attention, as the interviews showed.
Hypothesis Two: H5P integrated quizzes in videos in MOOCs are helpful.
The majority of learners also rated the helpfulness of the H5P feature for learning quite high. Therefore, the general positive opinion on the in-video questions was confirmed through the questionnaire and the interviews. In general, there was little criticism concerning this tool. Overall, the study could prove that a high level of interaction was evoked by the H5P questions and that the learners found them helpful.
Hypothesis Three: There are additional relevant behaviors around videos in MOOCs that need more attention in research, such as speed regulation and transcripts.
Another interesting insight was that around half of the learners used the transcripts for their learning. However, the scope of this usage and the reasons for using them differ from learner to learner. Some participants used printed versions to read before, along or after watching the videos. Another portion of watchers used them as a navigation tool because the content of each lecture can be reviewed faster in written form, according to some interviewees. A relevant fraction also seems to learn with transcripts only. Additionally, the possibility to fasten the playback speed is an interesting insight. Although this might potentially be tracked with the log-files (other activities), in this study the questionnaire and the interviews revealed the (relevant) frequency of the behavior. This opportunity helps users to learn in their individual pacing and it keeps motivating them.

dISCUSSIoN
The authors' exploratory strategy and field setting is not precise enough to prove their results, so they coined it as "hypotheses," as they did not apply any approved questionnaire or experimental setting. For example, the 10 interviewees were students from Graz University of Technology, so their digital behavior might differ from others.
Concerning the usage, relevance, and helpfulness of H5P videos, this study gives potentially relevant inputs for future research, as it shows that H5P videos not directly increase the activity per se, but influence the corresponding patterns and also have a stimulating effect. Indeed, videos without in-videos questions contained roughly the same percentage of interaction as those with integrated questions. However, there are some caveats concerning this finding. On the one hand, the learners and their behavior could have been influenced through the videos with H5P questions and adapted their behavior to the other videos. The research data in this study were also rather small, as the authors analyzed only six videos, and these came from the same year. However, with 26% of all questionnaire participants answering that the H5P questions were always helpful and 41% answering that they were mostly helpful, this feature was received very well. Only about 8% thought that this function did not help them at all.
Overall, H5P can be an effective enhancement for more learner engagement in a self-directed environment and it fits in line with the results from the log files. However, scope and difficulty of these questions should be chosen adequately and adapted constantly. The authors also experienced that the visualization of the video data is not trivial. Surprising was the insight that they did not recognize from the log files that and how often the speed of the videos is adjusted. Potentially, this activity is lost under "other activities." This exploratory study with different sources and research steps revealed other limitations and potential "blind spots" of current research in the usage of videos: The most important additional aspects were taking notes, transcript usage, and changing the playback speed. The authors' experienced research team, was surprised about the amount of learners using transcripts, as this activity is not visualized in their dashboards for MOOC creators. Thus, practically, the researchers got relevant information to adapt their metrics here as well.