Projects

  "SenSay", the Context-Aware Mobile Phone
 


Instead of thinking about our phone, how about a phone that thinks for us?
It would Sense your environment and enable you to Say what you need to. We call it SenSay.


Presented at the 2003 International Symposium on Wearable Computers



Press & Coverage

Project Description

What if a phone could...
  1. Change ringer volume when appropriate?
  2. Let emergency calls through?
  3. Help you remember to call people back?
 SenSay does. It eliminates...
  1. Unwanted interruptions
  2. Missed important calls
  3. Forgotten call-backs
 ...thereby creating:
  1. Alleviation of cognitive load
  2. Socially-acceptable phone use


User States

Uninterruptible: user is in scheduled meeting, conversation, or giving a talk. [Ringer off. If phone in pocket, vibrate on. If not, vibrate off.]

High Activity: user is jogging or at a concert, bar, or disco. [Ringer extra loud, vibrate on.]

Idle: user is waiting for the bus, eating alone. [Suggest people to call based on calling history and patterns.]

Normal: user's normal settings.


Threshold Analysis

We used this in-house GUI to visulize sensor values in order to conduct threshold analysis. With this information we came up with formal definitions of conversation, jogging, a night at the bar, etc. For example, we had to test the "high activity" user state by simulating a user jogging. Watch a (humorous) short video of us testing this.

  

State Transitions

If I'm having an important conversation with somebody for 10 minutes and we stop talking, I don't want my phone to immediately exit 'busy' mode. We have to take into account the duration of the conversation before we attempt to transition to a new state. The duration of an event can be interpreted as its "importance".

At each second for 10 minutes, we stored the sensor values and what state the phone was in. Then we looked at different lengths of time, calculated the averages, and applied the state priority algorithm to place the phone in the correct state.



Interface

The normal SenSay user interface. Notice the ability for the user to place the phone in "Auto" or "Normal" mode because we knew the phone would reguarly make mistakes (remember, this was a rapid prototype). Right now, phones cannot discriminate between urgent or unimportant phone calls. Using today's technology, we came up with a solution. If the user was detected as being busy when an important call was being received, an SMS message would be sent back to the caller. The SMS message if the above text and then starts a timer on the user's phone (SenSay). If the same caller calls back within two minutes, the user is interrupted.

Remembering who to call back and who you tried to call seemed like an easy thing for phones to do. So, we figured out when the user was idle and then showed suggestions to the user. Here, a certain person has, throughout the day, attempted to call the user but failed to connect. Now SenSay is offering to return the call. The settings screen. We knew the SenSay would never be perfect when detecting the user's state and so we wanted the system to be configurable.

My Role & Responsibilities

  1. Designate the feature set by conducting informal contextual interviews with college students
  2. Create the decision module (how the phone should react to the user's state) and flow diagrams
  3. Design poster and write much of academic paper; help direct and write demonstrational video
  4. Create formal rules for state transitions
  5. Present this project several times to different audiences including Microsoft Research Technology Evangelist, Project AURA, and the Pittsburgh Community.
  6. Field testing and threshold analysis
  7. Conduct formal research in mobile phone usage patterns college-aged students