Exercise Lock Screen
CPSC 581 Fall 2021 - Project 2
CPSC 581 Fall 2021 - Project 2
Created By: Marcello Di Benedetto, Abraham Beauferris, Christopher Rodriguez
This project revolves around creating a novel replacement to the “slide to unlock” style lock screens common to modern smartphones that still resists accidental activation. This specific replacement aims to utilize sensor based user input supported by modern smartphones. The utilized sensors could include any combination of the phone’s accelerometer, gyroscope, cameras and microphone. Though, as a requirement of the design process, the replacement lock screen should have a side benefit to the user in terms of self-betterment or the encouragement of healthy habits. The 10+10 design method was put to use for this project to narrow in on a final design with useful and intuitive features.
The final design chosen by our group aims to motivate the user to complete workout routines as a prerequisite for their phone to unlock. This design prompts the user with a few different workouts and will use the phone’s camera along with Google’s Teachable Machine, to recognize their completion of the workout. Once the workout is complete, their phone will unlock. The intent behind this design is to leverage the common compulsion to check one’s phone to motivate quick and periodical exercises.
As this sensor-based project was developed in parallel with a touch-based project, the following concept sketches include only the touch-based sketches from the concepts.
This concept utilizes the microphone to facilitate a back and forth conversation with the phone in the form of a knock knock joke. The user would firstly set up a “punchline” to be used as a voice based passcode in the phone’s settings. The user would prompt the phone with “knock knock”, the phone would then reply with “who’s there?”. The user would then repeat their passphrase/punchline to the phone in order to authenticate and unlock the phone. This concept was not pursued further as it does not have a clear secondary benefit to the user.
This concept aims to provide a way to authenticate with face id while wearing a facemask. The user would be required to wear a QR code printed or stuck on their facemask, which the phone will scan and use to authenticate the user. This QR code would have to be unique to each user. While this concept is genuinely useful, we wanted to go with a more novel, behaviourally beneficial approach to our final design.
This concept utilizes the accelerometer and gyroscope to track motions taken by the user. These motions would be classified as repetitions of short workouts required to unlock the phone. The phone would prompt the user with specific workouts and the amount of repetitions required to unlock the phone. This concept was chosen for further iteration, as it is directly beneficial to the user, and would allow us to explore interesting technologies during implementation.
This concept is an extra requirement on face id, where a specific facial expression is also required to unlock. In this case, a smile is required to unlock the phone. While this concept can be beneficial to the user, it does not stray far enough from traditional phone unlock processes. Due to this, this concept did not undergo further iteration.
This concept utilizes the front facing camera to register a series of blinks from the user, acting as a passcode. These blinks will be timed and parsed into morse code, then compared to the user’s configured passcode. This concept was not pursued as it does not have a clear behavioural benefit to the user.
This concept utilizes the phone’s gyroscope and accelerometer to simulate the motions required by a dial lock to enter a numeric passcode consisting of three numbers. The user would rotate their phone accordingly to align the indicator (which always points up) with numbers on a dial in the traditional right-left-right sequence. This concept was not pursued further as it did not have a clear behavioural benefit, and could be quite cumbersome to use.
This concept leverages touch and microphone capabilities to provide an alternate way to enter a sequence based password. This concept was inspired by The Legend of Zelda: Spirit Tracks spirit flute minigame. The user must click and drag to position a flute in the center of the screen underneath an indicator then blow into the microphone to produce a tone. The user would repeat this four times blowing flutes in the correct order to unlock the phone. The order of the flutes would be configured ahead of time by the user as a passcode. This concept was not pursued further as we wanted to stray from traditional passcode entering systems.
This concept utilized the accelerometer and gyroscope to simulate rolling a billiard ball into the four corners of the screen. As the ball enters each corner, the corner will turn green to indicate to the user which corners are complete. Once all four corners have turned green, the phone will unlock. This concept was not pursued as it does not have a clear behavioural benefit to the user.
This concept utilizes the camera to recognize the user’s hand gestures. The user would be able to set up a specific gesture as a form of passcode. When this gesture is correctly presented to the phone, the phone will unlock. This concept was not pursued as it does not have a clear behavioural benefit to the user.
This concept utilizes the gyroscope to recognize a series of “flicking” motions. The phone would prompt the user with a cardinal direction, then the user must flick their phone in the specified direction. They will be prompted multiple times to avoid accidental activation. After all prompts have been successfully completed, the phone will unlock. This concept was not pursued as it does not have a clear behavioural benefit to the user
From the initial concept sketches, we decided to go with a design that had great side benefits for the health of the user. Our main design goals were utility, ease of use, and novelty, and using those to inform our ideation process, one of our team members came up with a fitness-based lock screen. During our ideation process, we came up with many different concepts for sensor-based lock screens. We internally decided which ideas we liked the most, then presented that handful of views to our class to gauge which images resonated the most. The exercise concept garnered the most engagement and interest, so we felt it was the best idea to pursue. Initially, the idea included gyrometer-based tracking, but through experimentation, we decided that placing the phone on a surface was more elegant and accurate.
The following are some details and variations of our chosen concept, the “Exercise to unlock” lock screen.
This iteration's focus enabled more than one exercise to encourage a more diverse array of movements. This idea made it into our final implementation through the selection between squats and pushups.
This iteration focused on the sensors used to detect the exercises, with an accelerometer proposed for running and gyro for specific repetitive exercises such as pushups. We ultimately decided against these as the camera-based approach was more accurate and allowed users to perform the activities without holding their phones.
This iteration describes a pedometer-style lock instead of a set of exercises. While this would have been easier to implement via accelerometer data, we also felt it lacked novelty and "wow-factor."
This iteration's goal was to prevent cheating by detecting elevated heart rates using the camera and flashlight. Ultimately we decided this massive increase in complexity wasn't justified, especially since the accuracy of such heart rate detection methods is spotty.
This detail thought up a stretching mode instead of an intense exercise. We decided against it as most stretches are recommended to be upwards of 60 seconds, slightly too long for a lock screen use case.
Like the time of day-based iteration, this concept allowed for the pre-programming of exercises to enable custom exercise programs depending on the day of the week. While helpful, we decided the payoff given for the complexity required to implement this was not high enough. The more elegant solution was a simple button tap on the lock screen to indicate which exercise you prefer.
This detail describes locking out the user until they finish a workout routine instead of letting the exercise routine be optional. We opted for this, as we felt most users would get lazy and choose not to exercise.
This variation allowed for the pre-programming of exercises to enable custom exercise programs depending on your schedule. While helpful, we decided the payoff given for the complexity required to implement this was not high enough. The more elegant solution was a simple button tap on the lock screen to indicate which exercise you prefer.
This iteration's primary focus was increasing the feature's usability, with tutorial-like videos showcasing how to perform the exercise. This idea made it into our final implementation through the addition of gifs depicting the proper form of each exercise.
This concept describes a parental control style lock screen to incentivize the exercise of a particular party. It requires two users to function, with one user setting exercise requirements for the other user's phone. While novel, we ultimately deemed the concept a little cruel and abusable by the user with controls.
For the final design, the user can do ten pushups or ten squats and use the camera and a machine learning model. The phone will detect exercises performed and unlock the phone upon completion. Our main design goals were utility, ease of use, and novelty. To this end, our final design used the camera, trained model approach due to ease of use and accuracy. Also, it implemented other aspects of our iterations, such as short tutorials through looped gifs and providing an easy option to choose which exercise you want to perform at the lock screen.
We used Google Teachable Machine, a web-based tool to train our models that detect the exercises. We used the pose recognition feature within the tool and fed the model image samples of people doing the exercises above, differentiating each activity state as a different class. This tool was handy due to its accuracy and ease of use. We built the apps themselves in HTML, CSS, and JavaScript.
Each group member had an active role in the development of these concepts. We coded collaboratively on glitch.com, with myself primarily handling the gesture lock screen, Abraham managing the sensor-based lock screen, and Chris helping both and contributing to training the models, rotoscoping for the assets, and more.
Site Link: https://marcellod1.github.io/sensor-based-lockscreen
Source Code: https://github.com/Marcellod1/sensor-based-lockscreen