Ecological Momentary Assessment Using Smartphone-Based Mobile Application for Affect and Stress Assessment

Yong Sook Yang; Gi Wook Ryu; Insu Han; Seojin Oh; Mona Choi

doi:10.4258/hir.2018.24.4.381

Abstract

Objectives

This study aimed to describe the process of utilizing a mobile application for ecological momentary assessment (EMA) to collect data on stress and mood in daily life setting.

Methods

A mobile application for the Android operating system was developed and installed with a set of questions regarding momentary mood and stress into a smartphone of a participant. The application sets alarms at semi-random intervals in 60-minute blocks, four times a day for 7 days. After obtaining all momentary affect and stress, the questions to assess the usability of the mobile EMA application were also administered.

Results

The data were collected from 97 police officers working in Gyeonggi Province of South Korea. The mean completion rate was 60.0% ranging from 3.5% to 100%. The means of positive and negative affect were 18.34 of 28 and 19.09 of 63. The mean stress was 17.92 of 40. Participants responded that the mobile application correctly measured their affect (4.34 ± 0.83) and stress (4.48 ± 0.62) of 5-point Likert scale.

Conclusions

Our study investigated the process of utilizing a mobile application to assess momentary affect and stress at repeated times. We found challenges regarding adherence to the research protocol, such as completion and delay of answering after alarm notification. Despite this inherent issue of adherence to the research protocol, the EMA still has advantages of reducing recall bias and assessing the actual moment of interest at multiple time points that improves ecological validity.

I. Introduction

Ecological momentary assessments (EMA) or experience sampling methods (ESM) are a design for longitudinal repeated measurements and started with paper diaries combined with electronic wristwatches or pagers [1]. These methods have been receiving high interests from researchers from various scientific background since they are considered to reduce the recall bias and improve accuracy by capturing momentary changes in real-life [2 3]. As technology has advanced, personal digital assistants and smartphone applications have been utilized to collect the data using EMA or ESM [4]. An early study used a palm-top computer to assess cigarette smoking cessation and relapse [5]. In this case, EMA repeatedly measured a behavior, mood, and activities as factors to track down the quitting process over time by electronic diaries five times per day for several weeks. In 2008, the first application developed as a software downloaded on a mobile device and utilized [6]. Since then, EMA using smartphones has increased rapidly and expected to grow continuously.

EMA has a number of advantages. It reduces recall bias and prevents aggregation by obtaining information at the actual moment of interest at multiple time points, and improves ecological validity by measuring the variables in natural and real-life environments [2]. Thus, EMA achieves increased accuracy and sensitivity in assessing changes in psychological properties [4]. EMA is considered a suitable method for observing momentary changes in affect and stress and understanding the psychological changes by yielding abundant and accumulated data from the repeated assessments in various situations [7 8 9].

Smartphone applications are broadly utilized to collect data used for EMA; this is attributable to the developed logistics and convenience of carrying smartphones. Researchers have developed their own applications or also utilized some commercial applications to conduct EMA through mobile or smartphones [10 11 12 13 14]. This case report describes the process of utilizing an EMA application to collect data on affect and stress, including the momentary activity, place, and persons whom the subject is with at the moment. Additionally, we also addressed issues during implementation of the EMA application.

II. Case Descriptions

We will describe the specifics of the EMA application and a case of preliminary implementation of utilizing the application.

1. Structure of Mobile EMA Service

To conduct an EMA study, researchers prepare survey questions and plan the scheduling of data collection points in advance. Then an administrator registers survey questions in advance in an EMA portal web application and establishes the schedule for survey questions. Furthermore, a participation schedule is set up separately for each participant (subscription). According to the schedules determined previously, the scheduler on the service uses Firebase Cloud Messaging (FCM) and Access Point Names (APNs) to automatically send alarms regarding responses from a participant. In some cases, an alarm can be sent out several times for just one scheduled question. To encourage participants to respond actively during the subscription period, gift-icons can be sent as an incentive. As for the gift-icons, various goods can be provided through a separate company, and depending on the case, interoperating with other companies is also possible.

Various response data collected in this process can be analyzed by using a data analysis tool. The mobile EMA service can extract the data in comma-separated values (CSV) format from such various data, thereby providing users with opportunities to analyze such data by using a tool familiar to them or any specialized tool. Furthermore, a stable service can be provided through cloud networking, and if necessary, it can be easily scaled up/scaled out. The structure of the mobile EMA service is presented in Figure 1.

2. Flow of Mobile EMA Service Process

The process of the mobile EMA service is summarized as follows. After an administrator, usually a researcher prepares the survey questions and gets ready to distribute them to participants. The administrator sends the application URL to a participant. The participant downloads the application to her/his smartphone and sends a request to be registered to the administration. Then the administrator allows the participant to register, and the study participant registers with a username and password. The administrator sets up subscription dates and time schedules for each participant. According to the date and time determined, a push notification is sent to the corresponding participant, and the participant receiving the push notification can answer the survey questions immediately, which is an advantage in comparison to the conventional paper survey method. When the participant completes the survey, she/he submits the responses to the administrator. If the survey questions are not answered on time, the researcher can send a push alarm separately to the corresponding participant as a reminder to participate in the survey. The data sent to the administrator are stored on a cloud server and exported in CSV or Excel format for further analysis (Figure 2).

In the case of using the mobile EMA service, the subscription dates can be set differently for each participant, and the push notifications can be defined according to the corresponding subscription dates. Therefore, the researcher can obtain the participants' momentary data, for example, momentary affect and stress that the participant experiences at a given moment.

3. Features and Advantages of Mobile EMA Service

The conventional paper survey method required as much efforts as the number of survey participants in terms of organizing the question items and collecting the data. Questions on paper entail a reliability issue for the responses of participants (e.g., responses are all composed at the same time, response is not composed on time, and participants may omit a question).

Meanwhile, in mobile EMA design, researchers can easily create the corresponding survey items, compose the desired questions using various formats and setup an explanation for each question. This will help greatly in saving resources and time, thereby helping the researchers perform activities more efficiently. When comparing digitalized survey method to the conventional paper survey method, a participant can manage and respond to the survey questions by using mobile devices easily. The administrator can receive the survey responses from each participant without laboring to transfer the data collected on papers to Excel one by one. Furthermore, the response results can be obtained as raw data in an Excel file. When multiple participants are targeted, the registration and management of participants are easy in comparison to the paper survey method. When a response request signal is received through a smartphone, a participant responds immediately by recording data, such as current activity, environment, affect, stress, and so forth. The administrator can distribute survey questions with a minimal effort since functions such as sending alarm notifications for response requests to each participants are automated (Figure 3).

4. Utilizing EMA Application for Data Collection

A convenient sample was drawn from police officers who were recruited from the police stations in Gyeonggi Province of the Republic of Korea in 2018. The study was approved by the Institutional Review Board of Yonsei University (No. Y-2018-0035). Our research team installed a mobile application into the participants' smartphones, and participants were requested to answer a set of questions of a total of 25 items assessing their momentary affect (13 items; 9 for positive affect and 4 for negative affect) and stress (8 items), including their shift of work, what they do, whom they are with, and where they are at the moment of responding to the questions. Most participants spent 2 to 3 minutes to answer those questions. The momentary affect and stress questions [15 16] were translated into Korean language using translations and back-translations and obtained content validity from a group of experts in the related fields. We also measured the participants' perception of the data quality and technical challenges in using this mobile application in terms of usability and these items were derived from the previous study utilizing EMA and modified for this study [17].

We asked multiple repeated responses, four times a day (08:00 to 9:00, 13:00 to 14:00, 17:00 to 16:00, and 21:00 to 22:00) for seven consecutive days, which yielded 28 data points per person. The application sets alarms at semirandom interval with 60-minute blocks. For example, if the notification alarm for a morning survey is sent at 8:30 to the participant, which is randomly set by the EMA application, she or he is supposed to respond by 9:30, within 60 minutes.

The participants were 97 police officers, and the mean age was 37.7 ± 10.8 and ranged from 22 to 60 years old. Ninety-two persons (94.8%) were males. About two-thirds (66.0%) were college graduates or higher. The mean completion rate was 60.0%, ranging from 3.5% to 100% per person. For affect, the means of positive and negative affect were 18.34 ± 4.57 of total score of 28 and 19.09 ± 9.75 of total score of 63, respectively. The mean of momentary stress was 7.92 ± 5.76 of total score of 40 (Table 1).

We also assessed the usability of the mobile EMA application from the participants. Eighty participants responded that the mobile application correctly measured their affect (4.34 ± 0.83) and stress (4.48 ± 0.62) on a 5-point Likert scale. They also felt that the mobile application was easy to use (4.05 ± 1.04), and they were comfortable to answer the questions in the application (4.11 ± 0.89) installed on their smartphones (Table 2).

III. Discussion

This case report described the process of utilizing a mobile application and methodological strategies to assess affect and stress of from police officers by using EMA methods. While we distributed the application and collected the data from the participants, we identified some issues. It was difficult for participants to adhere to the protocol because EMA methods usually require the collection of multiple data points for a relatively long period of time from the participants. Therefore, the burden of participating can be an issue. We provided approximately $18 (20,000 KRW) as an incentive—a small gift when obtaining informed consents from the participant to encourage participation and beverage gifticons on completion of the EMA survey. Although the mean completion rate was 60.0%, which is comparable to the rates of other studies of EMA measuring affect or stress (50% to 82%) [9 12 13], it would be better to provide the incentive first, in the middle, and the end of the EMA data collection to maintain a high completion rate. The medium completion rate may be related to the job characteristics of police officers, who sometimes work at the site.

Even though we found some limitations in utilizing the mobile application, our participants responded positively regarding their perception that the mobile application correctly measured their momentary status, and easiness to operate this mobile application, and to answer the questions in the application on the acceptability questions. We believe that this case report can provide a helpful reference for researchers who plan to utilize EMA design to measuring psychological and physical aspects in the daily lives of study participants. Based on the EMA findings, researchers should consider the implementation of intervention to the participants for further research.