Developing a Framework for Interactions in CBT-Based Serious Games on Smartphones

While there are numerous serious games that explore cognitive behavioral therapy (CBT) techniques through gamification on smartphones, the framework for developing interactions is not often thoroughly discussed. The objective of this study is to outline the process of combining CBT techniques, narrative setup, and game mechanics to create two types of interactions (verbal-and physical-based) using the player interaction framework (PIF). The PIF consists of three key sections: setup, aim, and execution. In the setup section, it utilizes the ABCDE (activating events, beliefs, consequences, disputation of beliefs, and effective new approaches) model to combine narrative and CBT techniques such as gratitude and self-monitoring. The aim section is used to break down the intended experience of both the player and their main character, at the same time clarifying the CBT goal of the interaction. Finally, the execution section includes the representation of how players will interact in the game through input and feedback. The efficacy of the framework in visual narrative serious games remains to be investigated through a randomized controlled trial after the completion of our serious game, BlueLine.


DeVeLoPING A FRAMewoRK FoR INTeRACTIoNS IN CoGNITIVe BeHAVIoRAL THeRAPy BASeD SeRIoUS GAMeS oN SMARTPHoNeS
Millennials' lives are inextricably linked to their phones, which they use for work, travel, and entertainment.Social media apps have become a hangout during traffic and a temporary respite from work (S.Kim, 2018).Smartphones may no longer be things we use regularly but rather a hub of where we live.In Thailand, social media users spend approximately three hours daily, mainly on YouTube and Facebook (Kemp, 2021).However, long-term use of social media may result in adverse effects such as social comparison (Wang et al., 2020).In most cases, negative effects come in the form of upward social comparison, self-evaluation of someone superior (Verduyn et al., 2020).According to the Department of Mental Health's survey in 2023, 48.54% of Bangkok's millennials are at risk of depression (Department of Mental Health, 2024).At the same time, this population has to confront the substantial financial cost to access effective mental healthcare (Prukkanone et al., 2012).
In recent years, numerous projects and papers have focused on the gamification of cognitive behavioral therapy (CBT) and therapeutic techniques in serious games and mHealth applications (Cechetti et al., 2019;Seaborn & Fels, 2015).CBT involves the patient identifying the triggering event and correcting dysfunctional beliefs through the therapist's guidance (Beck, 2020).Components of CBT can be gamified, for example, in Quest -Te Whitianga, a mHealth app for Māori (indigenous New Zealand people) youth with emotional difficulties (Christie et al., 2019).The player plays as "Tata," who goes on an adventure to meet with the guardians to gain ancient wisdom about well-being.The efficiency of these applications has not been fully established, but the field does show potential for further exploration (Erten Uyumaz et al., 2021;Torous et al., 2017).
From a game developer's perspective, existing serious games have explored many gamification elements such as points, badges, and leaderboard (Richter et al., 2015).However, only few studies have examined the nuance of these reward systems (Nicholson, 2015) and the input system (Smith & Chaparro, 2015), especially in mHealth-related apps.The nuance of the player's experience and their input can amplify the effectiveness of CBT and its therapeutic components further.The input range on smartphones is not limited to the mouse and keyboard; the user's finger can make the interaction of elements in applications more tangible (Djajadiningrat et al., 2007).While the framework of each serious game is distinct, the potential of a structured universal framework for interactions in visual narrative serious games could increase the consistency of immersive experiences for players.Immersion leads to more robust player engagement and improves the gaming experience (Jennett et al., 2008).
This paper proposes and outlines the player interaction framework (PIF) used in designing mobile interaction for our CBT-based serious game, BlueLine.We aim to provide a detailed description of how we reach and finalize the two types of interactions: verbal-and physical-based.

LITeRATURe ReVIew
Among many definitions of the term serious game, it is widely accepted that they are games with purpose on value creation and not primarily focused on entertainment (Laamarti et al., 2014).Over the years, serious games have been integrated into multiple fields such as education and psychology (Ávila-Pesántez et al., 2017;Alvarez & Djaouti, 2011).In order to accommodate the development process of this magnitude, there are many proposed frameworks to create structure for serious games (Mitgutsch & Alvarado, 2012;Yusoff et al., 2009).This research allows serious game developers to understand structure at the macro level, the big picture of their game projects.
In addition, there are records of design process in existing serious games such as Triangle of Life (Assigana et al., 2014), a side-scroller platformer game that conveys the cognitive triangle concept of feelings, thoughts, and behaviors, and Merlynne (Chan et al., 2021), a role-playing game that motivates participation in peer-to-peer CBT.These serious games showed positive results in creating values to the user through CBT techniques through small-scaled prototypes.Despite that, the use of CBT in serious games is still in its early stages as a mental health intervention tool (Coyle et al., 2011).It is plausible that an extra step is required to scale up short gameplay experiences to a full-length serious game.This presents an opportunity for the PIF framework to provide structure in the expansion of CBT-based interactions.

Related works
Our past game project in 2018 was Yumi's Home (Siswojo et al., 2018), a visual narrative game that reflects depressive symptoms and obligations in family.The game utilizes repetition of physical-based interactions to reinforce positive behavior, which aims to improve the player's reaction to triggering events.The narrative setup is based on a loop of daily activities on a weekly basis.While there are verbal responses in the form of player feedback, there are no verbal-based interactions because there is no framework set up in place.The PIF is used to fill this void so that in our current project, BlueLine, it is possible to implement both physical-and verbal-based interactions in BlueLine.

MeTHoD
PIF is the structure of gameplay interaction to develop BlueLine, a visual narrative serious game about a Bangkok millennial's struggles to live through societal norms.The game portrays the routines of Blue's daily life and the influences on his virtual life and relationships.It is a mobile-platform-based game whose primary target is Bangkok's millennials.The expected gameplay length of BlueLine is approximately 80 minutes.
PIF is based on the player's journey in BlueLine (Figure 1).Players play as Blue (the main character), and both have objectives that align.For example, Blue wants to go out for a walk, while the player wants to see Blue stop spending time hiding in his bedroom.The scenarios in the game are structured using the activating events, beliefs, consequences, disputation of beliefs and effective new approaches (ABCDE) model developed by Albert Ellis.It is a mnemonic device developed in rational-emotive behavior therapy that targets beliefs as a fundamental course of treatment.It guides Source: Sriwatanathamma, 2023 the individual to break down the event, carefully identify the trigger, and examine the causes and effects, so they may respond better to the situation if it relapses (Field et al., 2015).This model is used in BlueLine to incorporate CBT techniques and therapeutic components: behavioral activation, self-monitoring, interpersonal skills, positive psychology, relaxation and mindful activities, and problem-solving (Sriwatanathamma et al., 2023).
Each BlueLine scenario gameplay is created through the player's interaction with the main character, who exists inside the game's world (an artificial environment in the game).The intended gameplay experience in BlueLine is for the player to live through scenarios in the game's world as Blue.By observing the consequences of Blue's actions, both the player and Blue have the awareness to monitor themselves.It is a process where individuals identify the relationship between their thoughts and emotions (Cohen et al., 2013).

PIF
PIF supports organizing CBT and related therapeutic techniques in game development to maintain a consistent gameplay experience for the player.It aims to assist developers in coping with the reiterative nature of CBT by showing the big picture of how one scenario links to another (Berg et al., 2022).Table 1 indicates that three key sections work concurrently to create the framework: setup, aim, and execution.
In the setup section, the framework can support developers in seeing the relationship between the narrative moments and their character's current mental health stage.The correlation between player experience and their main character's experience during the scenario is in the aim section.Finally, there is an overview of the in-game representation of the scenario in the execution section.

Setup
The setup section overviews the scenario where the player's interaction will occur.The goal is to identify the moment of disputation of beliefs, as that tends to be the focal point of gameplay.It is to ensure that CBT techniques are weaved into the narrative structure during the development process and make the interaction itself intrinsically rewarding.It is to lessen extrinsic rewards (points and high scores), which may limit the sustainability of player engagement (Richter et al., 2015).Exposure of game elements to the player while playing.

Player agency.
Actions to be made by the player.
The assigned method of player's interaction.
In-game outcomes according to the player's input and interaction.
Three aspects of the setup section are as follows: • Scene: a breakdown of narrative moments from a chapter in the story.
• The Diagnostic and Statistical Manual of Mental Disorders, fifth edition (DSM-5): This diagnostic tool displays the main character's mental state, which is player-controlled.It shows moments where depressive symptoms exist in the span of the narrative.• ABCDE model: Each plot point further breaks down smaller beats where Blue goes through each step of activating events, beliefs, consequences, disputation of beliefs, and a compelling new approach.Ellis developed this behavioral therapy model and aimed to convert irrational beliefs and promote changes in mindset (Field et al., 2015).
Once completed, the setup section shows the game's overall pacing through the combination of narrative and CBT components.From this point, developers can determine CBT techniques and the amount of repetition throughout their game.This information is crucial for the next step, aim.

Aim
The aim section establishes the connection between the player and their main character concerning CBT goals in a scenario.This process is iterative; details of these three aspects can be switched when the interaction does not meet the CBT goals.
Three aspects of the aim section: • Player experience: The qualitative outcomes of interactions are between the player and the game, such as flow and immersion (Wiemeyer et al., 2016).This segment covers the intended player experience.• Primary character experience: The details of the intended experience for the main character are based on their actions in the story.• CBT goals: The expected outcome from CBT and related therapeutic techniques is based on the player's interactions.
Separating the intended experience for the player and the main character is crucial because their motives for taking action are sometimes different.For example, the main character wants to surprise his date with a handcrafted gift, while the player's motive can be to progress through the story.At times, the game guides the player to take actions the main character might be unwilling to take, such as confronting one's dysfunctional beliefs.

execution
Once the game has established setup and aim sections, developers can determine the appropriate player interactions in a scenario.From a game development point of view, this is the production stage where the scene implements art assets and gameplay.
Four aspects of the execution section: • Representation: the details the player perceives while playing, such as visual, animation, and sound.It can also be the nuance of a character's emotion, shown through facial expressions and body language.Selecting a representation that best reflects the setup and aim sections is crucial in a scenario.• Interaction: This is an action that needs to be taken by the player in order to progress in the game.
This moment plays a crucial role in raising player agency: the ability to have a meaningful impact within the game's world (Day & Zhu, 2017).
• Input: the assigned method of player interaction.BlueLine focuses on finger gestures since the game is made primarily for smartphones.Depending on the scenario, the input can be performed in a sequence of actions as long as the interaction justifies its context.For example, the intuitive input for tossing a ball up in the air would be a swipe up over a swipe down.• Feedback: The player's input results in a gameplay response, the consequences of their actions in the game's world.The player's experience determines successful feedback in relation to the aim of that scenario.A player's experience is considered meaningful when the action has immediate significance and also in the larger context of the game (Tekinbas & Zimmerman, 2003).
Among the three sections, execution is the most iterative process due to extensive playtesting to ensure that the target player interacts, reacts, and experiences within a respectable scope of the aim section.The representation of each scenario should depict the narrative in the setup section to increase the immersion of the player experience (Naul & Liu, 2020).In this last step, the success of the interaction depends on the nuance of the input and feedback, which will be discussed in the next section.

ReSULTS AND DISCUSSIoN
This section discusses the relation between PIF and the two types of interaction in BlueLine: verbal and physical.The discussion focuses on the breakdown of design decisions in the execution section and how they align within the scope of the aim and setup sections.

Verbal Based Interactions
In BlueLine, players converse with other nonplayer characters (NPCs).Their conversation dialogue is a medium for related CBT components and gamification strategies (Sriwatanathamma et al., 2023).It can be in the form of reinforcement of successful behavior and positive feedback on players' performances (Cugelman, 2013).One of the core interactions in BlueLine is the dialogue crafting system, which represents player verbal agency within the game's world.This mechanic is designed where the player completes a sentence that the main character would speak through an user interface (UI) as shown in Figure 2.
In BlueLine, the variation of fixed and switchable options allows us to control the balance of player agency.This leads to a smooth flow of the player navigating through the intended experience.The sentence spinner mechanic from the game We Should Talk has inspired us to create the dialogue crafting system for BlueLine ("Insatiable Cycle," n.d.).In moments, the game utilizes choice bubbles where symbols can communicate better than words.It also helps offset the overabundance of text in the game.

Verbal Based Scenario One: Positive Psychology
The CBT technique in this example is positive psychology, a study of improving one's well-being through positive outlooks (Lee Duckworth et al., 2005).In this scene, the player plays as Blue, who has just returned home from years of working abroad.His dear parents greet him as he enters the house.The player is then given two choices via the dialog crafting system (single-choice sentence spinner), as illustrated in Figure 3 and shown in Table 2.
The feedback stays positive among the two choices given to the player.Instead of having one right and one wrong choice, each with its gameplay responses, we chose neither since the CBT goal is to show the importance of family support.The scene portrays that no matter how adverse Blue's action is, his parents will always share their kindness, a positive trait that can improve one's wellbeing (Cregg & Cheavens, 2022).The CBT technique in this example is gratitude reinforcement, the product of behavioral expressions and grateful contemplation (Rash et al., 2011).In this scene, the player plays as Blue on a video call with his parents.They would like him to return home during the pandemic lockdown instead of staying at his apartment.However, he recently moved in with his girlfriend, Line, and she is right next to him during the call.He feels conflicted because the other party will be sad no matter what he says.Blue is at the consequence stage in the ABCDE model.
Ultimately, the resolution of this gameplay is to showcase that being open to external help can improve his well-being.This is shown through Line's action that is based on therapeutic techniques, gratitude and compassion (Rivier University, 2023).She resolves Blue's dysfunctional beliefs (D in the ABCDE model) by encouraging him to be with his family and that their relationship will still be strong even if they are apart as illustrated in Figure 4 and shown in Table 3.
Feedback can come through the main character's conversation, body language, and visual elements.For example, after Blue has disputed his dysfunctional belief, the elevated atmosphere is represented through the increase in color saturation throughout the scene (Kurt & Osueke, 2014).Sound effects and background music are another viable medium to evoke and convey sensations to the player (Zhexuan & Zhong, 2023).For example, when Blue is in a phone conversation with someone and he hesitates to respond, static noise sfx and tense background music will be played.If he happily responds back, there will be no distortion or noise to offset the mood, and the music will be lighter.This allows the player to have a better understanding of what the atmosphere of the scene is and how the character currently feels.

Physical Based Interactions
In BlueLine, physical based activities intend to incentivize players to take actions infused with CBT components.The following setup is from one of the chapters in BlueLine, where the couple decides to take a break from job burnout, as shown in Table 4.The next segment will break down the execution of three scenarios according to the setup above: • The first scenario (Table 5) adapts traditional Buddhist practices in the form of loving-kindness meditation (Zeng et al., 2015).• The second (Table 6) focuses on exercise in games to alleviate anxiety (Abd-alrazaq et al., 2022).
These examples have the same CBT goal: to encourage both Blue and the player to engage in relaxation and mindful activities (H.Kim & Kim, 2018) and reward them with positive experiences applicable in real life.

Physical Based Scenario One: Respect and Gratitude
The first example refers to Theravada Buddhism, which is the dominant religion in Thailand, and the characters' positivity toward the religion (Keyes, 2016).The act of wai with flowers in temples shows respect and gratitude; it boosts life satisfaction and well-being based on multiple types of research (Laurent et al., 2021;Wallace & Shapiro, 2006).The player swipes up to mimic the motion of wai.For BlueLine's target group, Bangkok millennials, this is an activity where they could participate outside the game as illustrated in Figure 5 and broken down in Table 5.They feel that their bosses could be happier with their performance.
They feel trapped inside their cubicles, wanting to escape but obligated to keep working.
Even though they are motivated to work, the burden catches them off guard.
Together they decided to vacation outside the city to recharge their energy.Physical Based Scenario Two: Outdoor Activities The second example focuses on displaying the joy and benefits of exercising as it can improve mental health disorders, such as anxiety and stress (Mikkelsen et al., 2017).The gameplay utilizes a simple rhythm-tapping mechanic in major music game franchises such as Rock Band (Harmonix, 2010).Difficulty-wise, it is designed to be less challenging than other titles within the same genre because the gameplay should be accessible to a wide audience.Instead, we focus on juice, a game design term for enhancing the state of visual and audio effect feedback to heighten gaming experience (Johansen & Cook, 2021).
Even though player interactions are limited within the context of BlueLine, it is our aim to encourage players to take part in outdoor activities in real life.There are scenarios where outdoor environments, such as a forest, can promote remission of depressive symptoms (W.Kim et al., 2009), as positive incentives can lead to a positive therapeutic experience (Holtforth & Castonguay, 2005).This leads to our gameplay being a calm morning walking exercise in a forest setting as illustrated in Figure 6 and broken down in Table 6.

Physical Based Scenario Three: Art Therapy
The third example is based on art therapy, expressing oneself through a piece of art as a form of psychotherapy (Case & Dalley, 2014).While not all individuals appreciate art therapy, art creations can improve one's mental health in the form of well-being and reduction of post-traumatic symptoms (Chiang et al., 2019).Players decorate a wooden charm to express their wishful thoughts at the temple Blue raises the flower to perform a 'Wai' action.
The player swipes up a gesture to raise the flower.
Blue performs a 'Wai', a slight bow with both hands put together in a prayer-like fashion.It is a sign of showing respect and gratitude in Thailand.Source: Sriwatanathamma, 2023 by using their finger as a stylus to write or draw instead of pressing UI buttons.The more involved the player is in the activity, the greater the effect of immersion (Jennett et al., 2008).Player tension is distributed throughout BlueLine to not exhaust the player from constant complex interactions as illustrated in Figure 7 and broken down in Table 7.

General
In this paper, we propose and outline the application of the PIF in designing interactions for our CBT-based serious game on smartphones, BlueLine.This results in two types of interactions: verbalbased and physical-based.The framework functions as a supporting tool for visual narrative serious  Blue writes or draws a message on the charm.
Using a finger as a stylus to write or draw on the charm.
The message created by the player will be on the charm, which is placed on the tree.game developers.As shown in Table 1, the framework consists of three key sections: setup, aim, and execution.
The main factor of creating meaningful interactions starts with the setup section.CBT techniques and narrative must maintain balance that is achievable through the ABCDE model.The purpose of the aim section is to affirm that the narrative in the ABCDE model makes logical sense from the point of view of the player and the main character, as shown in Figure 1.The interaction should resonate with both the player and the main character in order to maximize engagement and immersion.Finally, the execution section shows the gameplay details presented to the player, such as UI, animation, and sound.
From a developer's point of view, player interaction and gameplay feedback should align with the CBT goal for the design of the interaction to be successful.In addition, this process is iterative and requires multiple playtest sessions within and outside the development team.With this framework, developers should be able to quickly identify aspects that do not fit within the interaction concerning the CBT goal.Implementing this framework in BlueLine allows us to understand the big picture of how CBT techniques are implemented in player interactions over 80 minutes of gameplay.

Strengths
The PIF offers developers a systematic structure for integrating and organizing CBT techniques in large scale visual narrative serious games.Through the ABCDE model, it is possible to bridge CBT techniques and narrative together to create lengthened gameplay interactions (Sriwatanathamma et al., 2023).The player can obtain benefit from CBT techniques such as mindfulness, which reduces stress and improves well-being (Davis & Hayes, 2012).
The framework guides developers to understand the intended experience of the player and main character before designing the interaction.This way, it steers the design process toward the intended experience, an optimal design to sustain player engagement with intrinsic rewards (Richter et al., 2015).
A side benefit of PIF is that it can be implemented in ongoing projects and create the structure for CBT and narrative components.However, based on our experience, it is understandable that each serious game has its uniqueness, whether in-game mechanics or narrative structure.For this reason, the framework is tunable to fit each project.For example, Life Is Strange ("Square Enix," 2015) has two main characters, Max and Chloe; hence a second column for the second main character can be added to the aim section in the PIF.

Limitations
The effectiveness of serious games still needs to be investigated further based on multiple studies (Bellotti et al., 2010;Six et al., 2021).For example, physical-based interactions positively affect mood and well-being (Bartholomew et al., 2005).Nevertheless, most trials have limited sample sizes and short follow-up periods (Penedo & Dahn, 2005).Developing a suitable set of dialogue choices for the player is also challenging, as each person can come from different cultural backgrounds.Our solution is to focus our design on our primary target, Bangkok millennials.At the same time, we consult with CBT experts in Thailand who treat millennial cases.
While PIF is being effectively used to develop a visual narrative serious game, such as BlueLine, the framework's efficacy in other game genres has yet to be proven.The framework is designed for narrative driven games as it focuses on breaking down narrative structure into scenarios.On the one hand, multiple studies suggest the importance of narrative in serious games (Kampa et al., 2016;Naul & Liu, 2020).On the other hand, there are limited studies regarding frameworks for visual narrative serious games.This study attempts to fill this genre's need for an established framework through a practical, serious game, BlueLine.However, more data are needed to establish the effectiveness of the PIF.
When creating gameplay interactions, it is essential to consider addiction to prolonged gaming.It is up to the developer to utilize strategies to minimize the addictiveness, as it could lead to increased stress (Griffiths et al., 2012) and occupational impairment (King & Delfabbro, 2020).It is vital to consider that, while designing the narrative, the length of each act in the story is within a manageable timeframe.Another method is to design non-narrative features such as a reminder feature; it is used in internet-based CBT applications to notify players (Radomski et al., 2019).
PIF framework is implemented on two platforms during the development of BlueLine: smartphone and personal computer (PC).The input for the project is through touch on mobile and a combination of keyboard and mouse on PC.Even though the framework is based on relevant theories and references, it should be thoroughly tested before fully integrating into other platforms, such as consoles.

Future Directions
At the time of this article, BlueLine has passed the pre-production stage with assistance from the Department of Mental Health.In the short term, we plan to explore integrating different narrative structures with the current version of PIF in BlueLine.The game is scheduled for completion in mid 2025 where a randomized controlled trial will be conducted in Thailand with the support of the Department of Mental Health.Through the guidance of Thai CBT experts, the trial will assess the efficacy of the PIF.This would open the possibility of identifying the current limitations of PIF and opportunities to develop the framework further.

CoNCLUSIoN
In conclusion, PIF combines CBT techniques, narrative setup, and game mechanics to integrate two types of interactions: verbal-and physical-based in visual narrative serious games.It utilizes the ABCDE model with the narrative breakdown to create the foundation for interaction.It focuses on developing achievable CBT goals that are executable in game development.The first serious game that utilizes PIF is BlueLine.Hence, the efficacy of the framework in visual narrative serious games remains to be investigated through a randomized controlled trial after the completion of BlueLine's production stage.

Figure 4 .
Figure 4. Illustration of Blue's phone call with his parents during the pandemic: (A) Before gameplay, (B) During gameplay, and (C) After gameplaySource:Sriwatanathamma, 2023

Figure 5 .
Figure 5. Illustration of Blue and Line praying at the temple: (A) Before gameplay and (B) After gameplaySource:Sriwatanathamma, 2023

Figure 6 .
Figure 6.Illustration of Blue and Line going up the temple stairs: (A) Before gameplay and (B) After gameplay