ⅠIntroduction

As understanding travel behavior is crucial to achieving sustainable transportation, researchers and planners in urban and transportation planning fields have sought to collect high-quality travel behavior data that includes detailed non-motorized trip activities. Smartphones, which feature programmable applications able to capitalize on a range of sensors that include global positioning systems and accelerometers, provide an opportunity to collect behavioral data in an unobtrusive and cost-effective manner. The smartphone has inherent advantages as an activity survey tool. For example, smartphone users almost always carry their phones with them, making it possible to capture trips and activities throughout the day. The sensors installed in the smartphones provide location, movement, and temporal data, which can be used to infer user activities, travel modes, and routes.

The well-known issues of traditional self-reported, paper-based travel surveys include small sample sizes, incomplete responses, under-reported trips, and inaccurate trip departure and arrival times [1]. These problems pose a non-trivial challenge to implementing advanced agent- and activity-based behavioral models. Advanced sensing capabilities, enabled by smaller and more powerful computers and sensors, hold great potential to upgrade activity surveys, by reducing the number of missed trips, and improving the accuracy of trip times, locations and paths. Location-enabled smartphones epitomize these advances, offering an unprecedented opportunity to collect the more detailed and precise data needed for emerging - albeit data-hungry - transportation and behavioral models [2].

Smartphone-based travel and activity surveys have been developed worldwide. These include MoALS [3], TRACT-IT [4], CycleTracks [5], Quantifiable Traveler [6], ATLAS [7], and UbiActive [8] – with several moving towards large-scale implementation. The Future Mobility Sensing (FMS) is a smartphone-based prompted-recall travel survey that was originally developed and piloted in Singapore [9]. In a 2012 field test, more than 1,000 respondents who participated in Singapore’s Household Interview Travel Survey (HITS), a traditional travel survey instrument, also participated in the FMS. While the HITS-FMS pilot data from the two instruments were not collected for the same days, a comparison of the FMS and HITS reveals that the FMS captures more detailed activity patterns than the HITS [10].

This paper describes a pilot experiment carried out in Korea with the objectives of testing the worldwide applicability of the FMS, and assessing its relative accuracy and reliability. Specifically, we compare the FMS to the Korea Passenger Travel Survey (PTS), a traditional, paper-based, self-reported travel survey that is widely used in Korea. To collect comparable data, participants in the experiment logged their activities and trips through both the PTS and FMS for the same time period. Through comparison of the behavioral data collected through different methods for the same days, we aim to provide insights into the strengths and weaknesses of a smartphone-based approach.

Following this introduction, Section II reviews relevant studies. Section III introduces the PTS and FMS, and describes the experimental design. Section IV compares the data collected across the two instruments, while Section V concludes the paper.

ⅡBackground

Self-reported travel diaries, typically paper-based, have long been the primary source of data on travel behavior. Many countries carry out their own versions of national travel surveys (e.g., the National Household Travel Survey (NHTS) in the U.S., the German National Travel Survey (GNTS), and the Dutch Travel Survey (DTS) [11, 12]. However, the validity and reliability of traditional travel survey instruments have been questioned. Since the 2000s, new global positioning system (GPS)-based technologies have emerged with the potential to eventually replace traditional self-reported travel survey. The most notable technological innovation is the development of small, portable GPS devices that individuals carry with them. These new technologies hold the promise of improving the validity and reliability of travel survey data.

Many studies have compared traditional travel survey and GPS-based methods. Comparisons have mainly focused on the journey (or trip) frequency, journey duration, total travel time, and travel distance. Stopher et al. compared the Sydney Household Travel Survey (HTS) with GPS travel data from the households recruited among the HTS participants. The participants completed prompted recall surveys via phone, Internet, post, or face-to-face methods. They found that traditional survey respondents under-reported their travel by 7.4%, and the majority of the missed trips were short trips [2].

A study comparing GPS data with Dutch Travel Survey data found a similar result. Based on the research done in the Netherlands with more than 1,000 respondents carrying portable GPS devices for a week, Bohte and Maat found that the numbers of tours per day were identical, but the numbers of trips per tour were quite different [12]. Minor trips within a tour tended to be easily forgotten, and thus under-reported by the traditional survey. Kelly et al. systematically reviewed eight studies comparing the self-reported and GPS-measured average trip durations. Their finding suggested that subjects consistently overestimated the self-reported trip duration [13].

Since the late 2000s, technological innovation in wearable GPSs has evolved to the smartphone, which beyond the GPS, also has accelerometer, WiFi, and cell phone tower locational capabilities. Can these devices, which users own and carry with them almost all of the time, improve the accuracy and reliability of travel behavior and activity data collection? Can they replace or complement existing methods? Only a handful of studies of the reliability of data from smartphone-based methods exist to date, and most remain focused on the accuracy of travel mode detection.

In Vienna, an Austrian research team utilized travel data extracted from a smartphone application, and automatically classified travel modes [14]. The most accurate mode detection occurred for biking (98%), followed by walking (92%), and riding trains (80%). In a similar study conducted in Shanghai, Xiao et al. compared the travel modes retrieved from a smartphone-based GPS application with the mode data collected from a prompted-recall survey conducted by telephone interviews [15]. They used a Bayesian network to improve the predictability of travel modes. The precision was over 80% for all modes, and exceeded 97% for walking.

Earlier research done in Japan collected travel data from both GPS-equipped cell phones and Internet travel diaries [3]. The participants carried their GPS-equipped cell phones, and edited their travel diaries on the web after returning home. This method was administered for five weekdays with 31 respondents. One-day paper-based travel survey was also administered with the same group, but four days prior to the GPS-based survey. Their comparison revealed that paper-based survey respondents under-reported in terms of “trips per person per day”.

Table 1 compares the systems to the FMS. While several systems passively detect travel modes from GPS and accelerometer records, completely passive activity or trip purpose detection are not yet fully developed. The FMS infers trip purpose based on history and contextual data. The experiment below aims to bring additional evidence to bear on the accuracy of traditional self-reported travel surveys vis-à-vis the data available from smartphone-based data collection methods.

ⅢThe Passenger Trip Survey (PTS) and Future Mobility Sensing (FMS)

This section introduces the traditional paper-based Korean PTS and the smartphone-based FMS, and describes the data collected through the experiment in Seoul.

1The Future Mobility Sensing (FMS)

The FMS is an activity and travel data collection method, comprised of a smartphone application, an interactive web interface (front-end), and data storage and analytics back-end <Fig. 1>. The smartphone application acquires user movement data through sensors commonly embedded in contemporary smartphones: the Global Positioning System (GPS), Accelerometer, WiFi, and Mobile Communication System (GSM, CDMA, and UMTS) [9]. The back-end runs stop and mode detection algorithms that analyze the raw data collected through the sensors, and generates estimated stops and modes that a web-based activity diary presents to users, as <Fig. 2> shows. The diary presents back-end inferences to users via a web interface to validate or correct the system-generated stops and modes [16].

The long-term goal of FMS is to unobtrusively collect travel behavior data without user intervention. Therefore, the stop-mode detection algorithms play a key role. Figure 3 illustrates the stop-mode recognition process. The back-end infers stops, modes, and activities from logged raw data. The ‘process raw data’ step includes cleaning and composing the raw data for subsequent inferences. ‘Stop inference’ consists of several steps: (1) generating candidate stops by matching the GPS location sequence to spatial and temporal windows, (2) checking against frequently visited places, such as home or work place, which are recorded during registration or user validation, and (3) merging stops by using WiFi, GSM data, and accelerometer data to detect ‘still’ periods when users stay in one place [17]. The ‘mode inference’ step selects travel modes for trips from among car, bus, subway, walk, and bicycle, based on accelerometer/GPS data and personal travel history through a machine-learning algorithm [18]. The ‘activity inference’ integrates personal history from validation and contextual information, such as Points of Interest (POIs), and transportation routes and stops, in order to select the most probable activities at a location [19].

One of the main constraints to smartphone-based travel surveys is battery life. If an application uses too much battery charge of the device, it reduces the period over which data can be collected, and/or discourages users from continuing to run the application. For this reason, minimizing battery drain is a main consideration in the FMS use of GPS and other on-phone sensors. The FMS uses a ‘phased sampling’ approach, collecting GPS data for a continuous period, then deliberately turning GPS off. While GPS is sleeping, the app still gathers GSM, Wi-Fi, and accelerometer data. This approach attempts to strike a balance between conserving battery life and maximizing the probability of capturing reasonably accurate behavioral data. Inevitably, data collection suffers under this approach, and refinements to the FMS method are ongoing [9, 20].

2The Passenger Trip Survey (PTS) Instrument

The PTS, as part of a National Transport Survey, is a paper-based survey conducted in Korea every five years [21]. The PTS instrument is designed to collect travel behavior data for a typical weekday, including trip destination, trip purpose, mode, departure time, arrival time, and parking fee. The PTS defines a trip as a one-way journey to a destination. The original PTS does not record short trips, such as walking trips to nearby supermarkets or bus stops. However, to make PTS data comparable to FMS, which aims to detect all trips, including short trips and walking access trips to other modes, we modified the PTS for the experiment to include such trips. The survey also collects socio-demographic characteristics of the households and individual travelers.

We further modified the PTS to match the FMS categories of activities and transportation modes to facilitate accurate comparisons between the two approaches. As a result, the modified PTS classifies user activities into seventeen categories: Home, Other’s Home, Work, Work-Related Business, Shopping, Personal Errand/Task, Medical/Dental, Sports/Exercise, Change Mode/Transfer, Recreation, Education, Meal/Eating Break, Social, To Accompany Someone, Entertainment, Pick Up/Drop Off, and Other. We included ten transportation modes: Foot, Car/Van (Self), Car/Van (Passenger), Bus, LRT/MRT (Subway), Bicycle, Taxi, Motorcycle/Scooter, Air, and Other.Other.

3Data Collection

The experiment compares PTS and FMS data for the same days traveled by the same participants, using Seoul, Korea as the setting. Smartphone penetration is high in Korea, being 82.4% as of 2014. The penetration rate is almost 100% among those in their twenties. Seoul is one of the densest cities in the world, with a population density of approximately 17,200 inhabitants per square kilometer. The city is served by well-developed public transportation systems, including nine subway lines with 306 stations, and a bus system that connects destinations not easily accessible by subway. The combination of high smartphone penetration rate, high development density, and advanced public transportation system makes the city a desirable experimental site to test the smartphone-based travel survey.

The subjects of the experiment were 46 undergraduate students. This convenience sample of people in their early twenties represents a group relatively familiar with smartphones and new applications. The majority of participants were male, and while most came from households with at least one motor vehicle, only 50% had a driver’s license <Table 2>. According to the survey data, few students drove cars for their trips.

The FMS introductory session for the students was held in May, 2015. After a brief introduction to the FMS and PTS, the students installed the FMS application, and completed the registration survey that collects user socio-demographic information. The modified PTS instruments were also distributed. The students were asked to log their trips using the PTS instrument, while also carrying their FMS-equipped smartphone. To ensure the FMS app was properly functioning, and to provide the FMS back-end machine-learning algorithm with initial validation history, we conducted a test run before the experiment: students completed one day of PTS while carrying their smartphone with the FMS application; we then held a validation session to resolve problems that students might have experienced, and to instruct students on validating their FMS data via the web interface. Subsequently, the students were asked to complete the PTS for four typical weekdays (from midnight to midnight) while carrying their smartphone with the FMS for two weeks (May 15 to May 28, 2015). We asked that during the four days, although the students were carrying their smartphones, they not validate their FMS data, to prevent them from completing the PTS by relying on the FMS information. This aimed to minimize the possibility of contaminating the PTS data with that gathered and processed through the FMS app and back-end. After the four days of data collection, the students then validated the FMS data for those four days during a second validation session.

4Discrepancies between the PTS and FMS

The participants provided records on a total of 114 days, collected through the PTS and FMS. Among these days, we matched the PTS and FMS data of 86 days. For the other days, it was impossible to match the two survey data, because either the PTS or FMS records were completely missing, or only one or a few trips were recorded. We compared the activity and trip records of the matched days and identified discrepancies between the two survey data, treating the PTS as a benchmark to assess the FMS accuracy and reliability. In reality, we cannot know which survey data are accurate; the FMS is prone to failure with the sensors, app, phone (battery shutdown), and user validation, while the PTS depends on user memory, and accurate perception and reporting of travel times (people tend to perceive travel times as longer than reality, and also tend to round their travel times (e.g., [2, 3, 10]). Three types of errors lead to differences in the data collected through the two instruments.

First, although we treat it as the benchmark, the PTS includes incomplete or erroneous records, particularly due to participants forgetting to complete the instrument, or completing them inaccurately. For example, some PTS records indicated trips made only during the morning, while the FMS data indicated that the user had also traveled in the afternoon. Also, many students forgot to record trips to transfer to other modes.

A second type of error occurred when the battery level of the student smartphone was low. Figure 4 presents an illustrative example. When the battery level was low between 10:30 am and noon, and between 3 pm and 5 pm, the GPS and accelerometer stayed inactive; the app thus failed to collect activity data, producing data gaps. This may be because users turned off their smartphones, or logged off the applications [10]. During these gaps, the FMS failed to capture activities at university, which were captured by the PTS. These errors lead to unreasonably long trip records (e.g., a 2-hour walking trip), because the FMS erroneously recorded the time spent at the university as a travel time.

The third type of error appears to arise from user mistakes when validating their FMS data. Although the FMS aims at unobtrusive data collection, participants must still review the collected data, and correct it if necessary through the web interface. For example, they might add missed stops, or delete false stops. However, this validation process can be difficult, especially for those unfamiliar with the FMS interface. Some students mistakenly deleted correct stops, leading to trip records with unrealistically long travel times. The delay between FMS data collection and validation – instituted to minimize FMS influence on PTS records – may also have affected the FMS record accuracy, as participants were validating data from up to 4 days prior.

Among the three types of discrepancies, the second type, data gap due to battery drainage, was most frequently observed. Among the 86 days, 41 days (47.7%) were excluded due to PTS error (15 days, 17.5%) and FMS error (26 days, 30.2%). After excluding the erroneous records, the dataset comprised 45 days of 24-hour data. This result confirms that minimizing battery consumption remains the biggest challenge to the FMS. This challenge may be compounded in the particular subjects in this experiment, who are likely highly intensive smartphone users. Comparing the two surveys reveals only a few days with perfectly matched PTS and FMS trip records. We excluded daily data with serious discrepancies (e.g., missing major stops or trips), but maintained data with relatively minor discrepancies (e.g., missing short walking trips from home to bus stop).

ⅣComparative Analysis

Table 3 presents the number of trips and total travel time during the successfully matched days without discrepancies. As the participants are all students, walking and public transportation are their major travel modes, while few drive cars or ride bikes.

Overall, the FMS tends to capture more trips than the PTS, suggesting that the FMS captures trips that users failed to manually record. In particular, the FMS captured short walking trips (e.g., walking to close destinations, or to transfer), detecting almost twice as many walking trips than the PTS. The FMS also collected more trips by other modes than the PTS, primarily because the FMS detects short stops for transfers (e.g., transfer from a subway line to another subway line), while users often overlook these stops. This is consistent with previous studies showing that short trips tend to be under-reported in self-reported travel surveys [2, 3]. Among the short stops, false stops generated by the FMS also exist: for example, short stops waiting at a traffic signal.

On the other hand, the PTS tends to result in higher travel times. The total travel time recorded by users for all days in the PTS is approximately 1.2 times longer than that by the FMS (Table 3). Table 4 compares daily travel times collected by the PTS and FMS. Similarly, daily travel times recorded in the PTS are also significantly longer than in the FMS (Table 4), consistent with previous evidence that people tend to overestimate their travel time [2].

To test the travel time reliability, we compare total travel times and travel times by modes across the two data sources. In Table 5, the All Days column shows the correlation between overall travel times and travel times by modes. The Days with Travel Records Available column excludes days with no trips when estimating travel time by mode. For example, we calculated car travel time reliability with only four days’ data.

Overall, the FMS travel times correlate highly with the PTS travel times, suggesting that after data cleaning, the FMS travel times are reliable. Some major differences emerge, especially for walking travel time, most likely due to walking trips under-reported by the PTS. Therefore, the FMS can possibly overcome the difficulty in detecting walking trips, which are often under-reported in self-reported travel surveys. Correlation coefficients for other modes are moderate to strong, although excluding no-trip days (in the Days with Travel Records column) weakens the strength of the correlation. However, this result of significant difference in average travel times between the PTS and FMS needs to be carefully interpreted. The PTS travel times are not ground truth, due to imperfect human memory and perception.

ⅤConclusion and Future Direction

1Conclusion

Smartphones offer a promising method for collecting travel and activity behavior. The FMS is a smartphone-based activity and travel survey instrument originally developed and piloted in Singapore. This paper presents an initial study of the FMS applied in Seoul. Although only a pilot study with a small group of students, the experiment provides insights into the strengths of the FMS in stop and mode detection relative to traditional paper-based survey, as well as weaknesses concerning battery drainage and user interface.

Forty-six undergraduate students completed two survey instruments over the same days: a traditional paper-based travel survey instrument widely used in Korea, the PTS, and the FMS. Comparing the trip records of total 86 matched days revealed three types of discrepancies: incompleteness of errors in the PTS; inactive GPSs and accelerometers, due to low battery levels in user smartphones; and mistakes in user validation via the FMS web-based interface. Battery capacity remains one of the biggest challenges for the FMS, which needs to simultaneously satisfy multiple and conflicting needs: minimizing battery usage, collecting high resolution data, and not disrupting normal user smartphone usage [20]. In terms of the web-interface for validation, although younger subjects may well be familiar with web-based user interfaces, the results suggest they still found the process confusing. Improving the app’s energy efficiency, and making the user interface simpler and clearer remain FMS development priorities.

Comparing the number of trips and travel times of the same participants from the PTS and FMS shows that the FMS captures more trips than the PTS, collecting short trips often overlooked in the self-reported survey, and access and egress trips that are regarded as a part of long trips. Therefore, FMS data tend to include more detailed activity patterns within the day. Another noteworthy difference, that of the longer travel times recorded by the PTS, is likely because people tend to perceive their trips as longer than actual, overestimating their travel time. In contrast, the FMS provides travel times as measured from smartphone devices. Thus, the experimental results imply that the FMS can supplement the PTS, overcoming the weakness of paper-based survey.

2Limitation and Future Direction

Although suggesting the potential of the FMS as an alternative data collection method in Seoul, this pilot experiment is limited by the small and biased sample. Also, we have not yet focused on travel time and frequency, and the reliability of activity and mode detection. Additional work should investigate the usability and reliability of the FMS in capturing activities by different subjects and trips by various modes.

The results revealed barriers to reliable large-scale smartphone-based data collection in Korea. The collected smartphone-based data contained considerable errors due to the low-battery issue, and trip and mode detection failure. Also, the system requires the validation process of survey participants, which is a considerable burden to implementing a large-scale survey.

Hence, large-scale FMS implementation in Korea will likely require improvements to the apps’ use of the phone battery, and in the validation interface. Also, reliable activity and mode inference are essential components to improving the system. Therefore, the long-term goal is to develop the FMS towards a completely passive system that requires no intervention from participants.

Despite the limitations, large-scale smartphone data are expected to feed data-hungry transportation and behavioral models, such as activity-based models and integrated land use-transportation models, making possible more detailed analysis and prediction of travel behavior. The smartphone-based system also provides opportunity to gain location-based survey data, such as traveler satisfaction levels at certain locations, which enable more location-specific analysis of the built environment attributes and human behavior interaction.