This is individual work as part of my Master program in Human-Centered Design and Engineering. I conducted secondary research, defined target users, scenarios and mapped the experience. I used Figma for concept design, prototyped and tested with Unity, C# and the Oculus Interaction SDK.
HCDE 536: Interaction Design and Prototyping | Fall 2022
Professor: Mike Altaman | Google
Masters of Science in Human-Centered Design and Engineering
University of Washington
Secondary Research
Experience Design
Systems Design
Interaction Design
Spatial Design
3D Modeling
Prototyping
Elicit AI & UW Libraries
Figma
Experience Mapping
Wireframing
Sketchup
C4D
Unity3D & Oculus Interaction SDK
This project, although primarily focused on systems design and prototyping, incorporated secondary research to establish a foundational understanding of user needs associated with IoT devices, and the nature of individuals' engagement with their digital environments.
● People's digital identity continues detached from their day-to-day reality. Research shows that people are increasingly interested in achieving consistency across all digital communication channels.
● People's digital life is expanding, spending considerable amounts of time in front of narrow-sided monitors or mobile phones.
● Current projections point that by 2030, IoT devices at home will increase from 14 billion to 30 billion.
● Recent research on smart home devices and their comms. with servers shows that people’s data can be easily tracked and hacked, and that devices can be a very effective channel towards obtaining sensitive information about users.
In an increasingly digital and connected world, managing IoT devices at home can be complex and feel disjointed. There is a need for an integrated system that simplifies interaction with these devices and blends physical spaces with persistent digital overlays for improved productivity and lifestyle.
The target user for this solution is a tech-savvy, young professional willing to try new technology to increase their productivity and well-being.
"I need more efficient ways to manage my digital life at home"
36 Years Old
Single Dweller
San Francisco, CA
Software Engineer
- Remote
● Smartphone
● Smart Lights
● Thermostat
● Air Quality Sensor
● PC
● Smart Watch
Sarah is a software engineer who has been working remotely for a major tech company for the past two years. She lives in a smart home, equipped with various IoT devices, such as smart thermostats, lights, security systems, and kitchen appliances.
Being a tech enthusiast, she has a deep interest in emergent tech and is always on the lookout for innovative solutions to make her life easier. She likes to stay organized and is constantly juggling multiple tasks at work and at home. Despite being tech-savvy, she finds managing her IoT devices and digital tasks scattered across various platforms inconvenient and time-consuming.
1. Seamlessly integrate her digital life with her physical environment.
2. Efficiently manage her IoT devices.
3. Maintain productivity and organization in her work and personal life.
4. Keep her data secure and protect her privacy.
1. Finding an intuitive way to manage all her IoT devices.
2. Achieving consistency across all her digital products.
3. Keeping track of her digital tasks across various platforms.
4. Ensuring the security of her personal data while using IoT devices.
Before diving into the specific actions, I wanted to showcase what a "Day in the Life" would be for our user. Let's follow Tess on a typical day working from home:
Tess is starting her workday, and decides to put on her augmented glasses on.
Her "focus" setting populates her house with customized productivity AR widgets.
She looks into a responsive calendar, she sees an important meeting coming up.
To dress for the meetup, her closet offers suggestions, and tracks her choice to match her digital avatar.
She steps into a designated “Virtual Arena” , where she can freely interact with her colleagues as if present in their office.
Later, she changes her Hombox environment to “fun” mode, widgets are replaced with a calming environment.
To set up her digital home and start bringing her widgets to life, Tess will require following a few steps to map her home and start setting up her preference:
First, Tess puts on her augmented glasses to activate the Hombox home tracker.
To create a digital copy of her home, the system guides her through all spaces.
Afterwards, Hombox creates a digital twin, with naming suggestions for each room.
Tess can visualize, rename and control any space of her home in any of her devices.
She can connect with any smart appliance at home, tracking its precise location as well.
Hombox also provides its own home widget, which she can manipulate directly!
This diagram integrates thoughts and actions with sensor and system logic, allowing us to understand the entire user experience and potential scalability, including all the touchpoints that need consideration. It is separated in three distinct phases.
Solving all of these is out of the scope in this research, but are important to consider as an integral system.
Persistent augmented reality widgets will be part of the user's environment for long periods of time. User experience in AR is no longer only focused on productivity and efficient cognitive loads, but in enabling well-being, harmony and a seamless integration of technology with users' physical context.
The goal is to make these widgets more than just pieces of interactive information, giving them warmth and texture, creating digital companion's that complement the people's surroundings, such as adding a touch of homeliness akin to a nice board of Scrabble, or a high-end mirror. Nevertheless, personalization is at the core of these elements, for users to adapt their AR environment to their individual style.
Each of the widget studies below represent a unique approach to create persistent environmental representation of a key aspect of the user's digital life, including a time assistant calendar, a responsive wardrove that represent the user's current avatar for online meetings, a home control panel giving access to every single IoT in the home (and representing their location), a live stock exchange visualization, and a desk-based augmented reality work environment.
These are mere examples of the potential for these systems to become permanent AR fixtures at home. The challenge is guaranteeing their flexibility, accessibility and ease of setup.
Most prototyping focused in the user's work-from-home and home automation dynamics through augmented reality. I put focus in thinking of novel design or interaction design problems.
These studies were designed using Figma and Adobe Illustrator for user interface design, modeled in Cinema4D and Rhinoceros 3D, then prototyped and tested in Unity 3D, where I added interactors and integration into the Oculus SDK ecosystem, enabling passthrough and spatial matching.
A persistent time scheduling widget that shifts between a circular "clock" style, which is more in line with interior design aesthetics, and a more traditional calendar overlay. Users can see and adapt their upcoming meetings at a glance. They can also locate this artifact anywhere around their house for visibility.
By providing two interchangeable options for time and events visibility, user's can switch from environment to productivity-oriented settings.
As seen in the video prototype below, the settings interface, which includes the action to switch between modes, is hidden until the user reaches their hand to interact with the widget.
Final Iteration
First prototype
1. This work station provides users with an augmented reality workstation that can be used in any desktop for accuracy and speed. Users can have a general translucent canvas in which interfaces can move around.
2. Most of their interactions provide automatic feedback thanks to the touch of the table. Windows layouts can be enabled in a traditional, semi-immersed, and completely immersed way to provide flexibility and adaptability.
3. The tabletop helps add touch and precision to keyboard and mouse inputs, likewise, hand tracking is enabled at all times, allowing users to easily transition between interaction modes.
This home control panel gives users a rapid understanding of the state of their home environment. It allows them to change and create routines based on their favorite settings and visualize them in real-time on a digital twin of their home. Through different inputs, users can visualize smart lights, temperature, humidity, and integrate any other IoT device or sensor that might be in their home.
This product addresses user's concerns about the ever increasing number of gadgets and IoT devices in their lifes. By creating a one-stop solution, without complicated names for rooms or individual lights, users have more control of their environment while wearing augmented reality headsets.
