Nicole Yeo

The Mushroom Fan

dexie

Interface design for humanoid that supports dementia and long-term care patients

About Dexie

Dexie is an artificially intelligent humanoid that is programmed to run social activities for dementia and long-term care patients. Such activities offer cognitive stimulation and psychosocial support for them while alleviating the burden on their caregivers. Dexie can be controlled from any device with the Dextron software installed.
View product

role

  • Contextual Inquiries
  • User Interviews
  • Design System
  • Usability Tests
  • Product Features
    and backlog
  • Task Flows
  • Wireframes
  • Prototypes
  • Mockups
  • User Manual
  • Wireframe Annotations

team

  • 1 BD Manager
  • 1 Software Engineer
  • 1 3D Robotics Designer
  • 3 Robotics Engineers
  • 1 UI/UX Designer

time

The development of A.I. robots and how they can shape our future has been a long-standing talking point. There is so much potential in what A.I. robots can do in the healthcare sector, yet its possibilities raises fear and concern for those who work with it.
How can we revolutionize the way we provide health care for dementia and long-term care patients while providing support for their caregivers?
Dexie is an innovative solution designed to enhance these elderly patients' quality of life while relieving healthcare providers of some mental load to reduce burnout.

My Role

At Dex-Lab, I had the opportunity to understand the back end system of how humanoid works, in order to simplify the front end process so that healthcare providers can operate Dexie independently and confidently. I was responsible for building the design system for ‘V.1’ of the Dextron software and the overall user experience (research, prototyping, interface).

Defining our users

We have 2 groups of users – the healthcare providers and the elderly and long-term care patients with dementia. Both to which play an equally crucial role to each other in their user experience with Dexie.

Primary Users

Our primary users are healthcare providers and therapy assistants who provide care for residents in nursing homes.

They are the ones who will directly interact with Dexie’s interface.

01

Secondary Users

02

Our secondary users are dementia and long-term care patients residing in nursing homes.

They mainly participate in the programmes run by Dexie but do not manage the interface themselves.

Empathising with our users

For Dexie to be calibrated to their needs, I needed to understand what my users were going through and the frustrations encountered in their daily routines. From casual conversations to field studies, the information I've gathered gave me a better understanding towards their behaviours and sentiments.


This is what I've learnt:
Click on the pointers to read more

Healthcare providers

1

Hectic schedules

Healthcare providers are responsible for several wards during their shift. Each of these ward has approximately 15 to 25 patients that would require constant care and attention.

Given their tight schedules and short breaks, our solution should not ask them for more time than they are already limited to.

2

Mental load & capacity

Huge responsibilities in a high-stress environment will inevitably take a mental and emotional toll on healthcare providers. It is common for people in this profession to feel burntout.

Our solution should allieviate healthcare providers' mental load so that they have more bandwidth for their patients.

3

Language barrier

Many healthcare providers are not local and may face communication issues with their patients since many of these elderly are dialect speakers. Sometimes information can be lost in translation.

Dementia & long-term care patients

1

Cognitive impairment

Patients suffer from different stages of cognitive impairment and will need healthcare providers' assistance to carry out day-to-day tasks. One way to slow the progress of dementia is to stimulate the brain with cognitive therapy.

Our solution should help healthcare providers juggle these tasks by focusing on one area.

2

Physical disabilities

Some patients have lost part or all motor skills in their upper/lower limbs. We should consider design accessibility in our solution.

3

Need for social support

Some of these elderly patients have gone through significant losses, be it the death of a spouse, separation from family, or purpose in life. This can trigger feelings of loneliness and isolation and may become overwhelming for an individual. A healthy way to cope with this is to talk to someone about these feelings.

Our solution should provide a sense of comfort and validation to these elderly patients at times when healthcare providers are unavailable to them.

* Disclaimer: The information synthesized is based on the nursing homes I have visited.  It may not be representative of all nursing homes in Singapore.

This information guided me in deciding which UX principles to adopt and goals to achieve.

Usability Problems

When I first got acquainted with operating Dexie, I found several usability problems.
Here are some to name a few:
Hover over the cards to read more

01

Overwhelming and complicated setup process
Users had to learn how to connect to the server, IP address, engine motors of the humanoid, etc.

A total of 22 steps for Dexie to be online.

02

Disorganized file naming and structure system.
Back and front end naming conventions were combined, making it hard to decipher what the action was.

The system was not categorised yet, so users had to read through every card before a selection.

03

UI buttons are too small
It was difficult to tap the buttons with accuracy despite the interface being on a tablet.

04

Counterintuitive steps required to run certain programmes
Due to a lack of system categorisation, users had to go back and forth between pages to select specific actions for the programme to run.

05

Lack of visual feedback for chat-mode
Users won't know when the chat mode is activated.
I believe these problems will frustrate healthcare providers at some point, and that's when they will abandon Dexie.

Validating with research & testings

To validate these concerns, I returned to the nursing homes once again.

I wanted to observe how healthcare providers would interact with Dexie (V.0) under accurate environmental conditions. Their feedback on Dexie's operational usability will help me substantiate the right features and functionalities to focus on to support them better. The contextual inquiries and usability tests would also help me discover hidden opportunities and recognize roadblocks that healthcare providers might face.

Here are some of the feedback from healthcare providers:

Doubts & Opportunities

Roadblocks

Dropoffs

”Im still very stressed because they [patients] are still aggressive. If you lose them in the game, then it’s game over for them. They won’t pay attention anymore.”

“I know it’s new but a lot of things need to be done to prepare Dexie. I cannot continue conducting a session if there’s any issues with her”

”Youtube is quite user friendly. If Dexie hangs, we cannot do anything. We just end up using Youtube.”

“They cannot hold tablet properly so we have to choose residents who can do chit chat sessions. Not everyone can engage in this. Sometimes smaller will be easier for them to hold. Button very small”

”We normally don’t do chat sessions. Don’t know how to do troubleshoot if got problem.”

“Sometimes the engine server never change. I call Dexie team how to do. Still cannot, so stop. Never use Dexie in the end.”

Defining UX goals

Dexie was built to offer therapeutic value to dementia and long-term care patients while alleviating some work load off healthcare providers.

The last thing we want is to make our healthcare providers feel like using Dexie is additional work for them.

Here are some UX goals set out to help push the design iterations forward.
01
Exclude unnecessary information to optimise user flows
02
Adopt progressive onboarding to relieve users of cognitive load
03
Include large touch areas for users to execute tasks with speed and accuracy
04
Use familiar and simple UI designs to give users the confidence to carry out tasks

Design solutions

Design iterations for V.1 Dextron software

Switching from tablet to mobile interface

The chat function is one of the programmes designed for dementia and long-term care patients to have personal conversations with Dexie, especially when there's an onset of sundowning behavior.

As anticipated, some elderly patients with physical disabilities may not have the strength or ability to use both hands to hold the tablet while pressing the button to converse with Dexie.

By switching the default interface to mobile, we hope to make this more accessible and inclusive to all our secondary users.

At the same time, this is less cumbersome than the tablet for our healthcare providers to hold while facilitating the programmes on the go.

V.0 Tablet interface for chat mode
V.1 Mobile interface for chat mode

Future considerations

Making elderly patients press a button before speaking still may not be the most intuitive action since many are unfamiliar with such functions on a mobile device. We could perhaps consider conducting an A/B test with microphones and corded phone handsets instead, to see if such devices would make conversing with Dexie more natural.

Shortening task flow steps

Right now, we've got a very daunting setup process and lengthy task flows. This is not ideal at all because it affects Dexie's usability, adoption, and eventually retention rate. This is evident from the feedback given by our healthcare providers when they indicated a dropoff from Dexie whenever they encountered a roadblock.

Information like server names, list of engine motors, and IP addresses are irrelevant to the healthcare providers and would contribute to their confusion.

What we needed to do was to hide what the users don't need to know.
V.0 Screenshots from the 22-step setup process
This is some text inside of a div block.

We took careful steps to filter down the information and tried to keep it at its bare minimum. Restructuring the information architecture (which will be elaborated on in the next point) also helped a fair bit in shortening the task flows for the setup and each programme. This allows healthcare workers to execute their tasks more efficiently and quickly.

V.1 Proposed setup process

Task flows

setup
BEFORE

A minimum of 17 steps to complete the software setup of Dexie.
V.0 Existing task flow for software setup of Dexie

setup
after

The number of steps are reduced by more than half.
It will now take 7 steps to complete the software setup of Dexie.
V.1 Proposed task flow for software setup of Dexie

exercise
before

It takes 6 steps to get Dexie to run an exercise routine.
V.0 Task flow for running exercise programme

exercise
after

Steps are reduced by half. It will only require 3 steps to run the exercise programme, and another 3 to end it.
V.1 Proposed task flow for running exercise programme

chat
before

It takes a minimum of 12 steps to execute chat programme with Dexie.
V.0 Task flow for starting chat programme

chat
after

Steps are reduced by half. It will only require 3 steps to run the exercise programme, and another 3 to end it.
V.1 Proposed task flow for starting chat programme

Restructuring information architecture

In V.0 of Dextron software, we have information but with weak architecture. The main page itself has 12 cards consisting of actual programmes, different parts of specific programmes, technical software functions (that users will never need), and other random functions that are completely unrelated to this project.

V.0 Tablet Home Screen

With information all over the place like this, it was difficult for our healthcare providers to find and run the intended programmes systematically.

The irrelevant content is constantly competing with relevant ones for visibility, which takes up time and effort to sieve through.

If healthcare workers find this process too tiring or complicated, there's a risk they might abandon it altogether.

Revisions to the information architecture of the home page

To keep the content simple and easy to navigate, I restructured the information architecture according to the 4 main programmes offered and left out unnecessary distractions.

V.1 Revised Home Screen

This follows Miller's law of usability, that keeping content organized in smaller chunks (ideally 7 or fewer) helps users to process, understand and memorize the content easily. Conjointly, this brings us to the next point of reducing the complexity of choices.

Reducing the complexity of choices

The aforementioned weak information architecture has led to many complicated and counterintuitive steps that healthcare providers needed to take to run a programme. A simple game of Bingo, for example, required 18 steps. Part of the complication involves users needing to decide and search for the game's audio language from the main menu and eyeballing the animation's progress indicator so that another animation can be selected before the current one ends.

V.0 game selection

V.0 exercise selection

V.0 song selection

Such decisions and complex processes take up more time and attention from healthcare providers than intended. To resolve this, I referenced the principles from Hick's law for V.1's iteration.

By adopting a progressive onboarding approach while minimizing complicated processes, I hope to improve my user's cognitive load and decision time

V.1 Game selection

V.1 exercise routine selection

V.1 song selection

Healthcare workers now only need to decide which programme they would like to run. The subsequent decisions and input will be prompted to them one step at a time. They also no longer need to manually select certain animations as it is now automated.

In addition, I've taken the liberty to make animations that are utilized often to be pinned at the top of the card deck with a different colour to give it visual priority. Relieving healthcare providers of some decisions will give them more confidence in running the programme on their own while having more time to tend to their elderly patients.

Readjusting space and size of touch targets

The touch targets on the tablet are visible but difficult to press because they are too small and close to each other. These are everything that Fitt's law has warned us not to do when it comes to the functionality of a button. Expectedly, this resulted in healthcare providers taking more time to tap on the buttons accurately.

Now that the default interface has changed to a mobile screen, all the more so I had to be sensitive about the sizing and spacing of the touch targets.

V.0 Tablet Play Bar
V.0.2 Mobile Play Bar
V.1 Revised Mobile Play Bar

Following the guiding principles of Fitt's law closely, I increased the size and spacing of the touch targets while ensuring that they were placed in a reachable position for users if they were to use the device with one hand. I've also decided to make the primary call to action (play/pause) button larger to give it greater visual prominence so users know to press that first.

V.0 Tablet play bar attributes
V.1 Mobile play bar attributes
These changes will help our users execute programmes with better speed and accuracy.

Creating familiar user interface

According to Jacob's law, users prefer familiarity and like it when things work the same way as what they already know. Leveraging on this, I've decided to draw inspiration from existing popular music platforms such as Spotify and Youtube Music to create similar interface designs for V.1’s music playlists.

Spotify play bar screen
Youtube Music play bar screen
V.1 Music play bar screen
This reduces their mental load to learn how to operate Dexie.

Making the user experience intuitive will also heighten their confidence in running the programmes independently.

Providing visual feedback

One of the usability heuristics by Jakob Nielsen is providing visible feedback on a system's status. People want to know if their actions will garner a reaction.

In V.0's interface design, the chat programme did not have any visual feedback when users pressed the button to speak. This cast some doubts amongst the elderly patients and healthcare providers when speaking to Dexie because they simply do not know if their speech was successfully captured.

To fix this problem, V.1 of the Dextron software will show the following visual changes when the microphone button is pressed.

V.1 Default state with instructions to activate chat function
V.1 Chat function is active when the microphone icon turns from blue to green and the filled ellipse changes to a blueish purple stroke.
V.1 Speech to text A.I will provide feedback to the user on the voice that was captured
This feedback will give a sense of security and comfort to the user that Dexie is listening.

Challenges & limitations

1
Information - both qualitative and quantitative - was limited by restricted opportunities to engage with healthcare providers due to privacy agreements and the sensitive nature surrounding nursing homes.
2
Usability testing conditions for V.0 of the Dextron software are inaccurate as our users underwent a training program on operating Dexie before they tested the app's usability.
3
Dex-lab ran on a tight timeline toward the product's launch and could only afford to make minor changes to the system. Issues with the animation nesting groups had to be compromised and resolved next time.
V.1 Dextron software

Wireframe annotations

Wireframe annotations lay out the context and provide clear information on the app's interactive features. It helps to communicate design decisions and concepts more comprehensively and succinctly to developers and external stakeholders.

Response from users

Our dementia and long-term care patients seem to get along well with Dexie.
Here are some of the responses noted during a trial session with Dexie.
The numbers
The observations
The compelling
  • 5/16 elderly patients immediately followed Dexie’s actions without prompts when they first saw her.

  • 6/8 elderly patients participated in the exercise routine conducted by Dexie

  • 80% of the elderly patients were paying attention to Dexie even if they did not physically follow through with the actions she was doing.

  • Elderly patients were replying Dexie's questions and one even continued singing after Dexie has finished a song.

  • One of the elderly patients was actively participating in the exercise routines despite an arm impairment.

  • One elderly patient was crying, but when he saw the rest exercising with Dexie, he slowly stopped crying and joined in with the rest.

  • Healthcare providers from 2 different nursing homes were amazed and shared that a particular elderly patient who has never participated in their exercise sessions was now joining the exercises conducted by Dexie.

  • A few elderly patients who are still sharp and alert asked about the possibilities and limitations of Dexie, and one even asked to learn how to operate her because he used to be an engineer.

Wireframe sketches

Wireframe sketches for V.1 design iteration:

Next steps...

  • Conduct a usability test for V.1 of the Dextron software
  • Consider design accessibility e.g. higher contrast colour palettes, button sizes, suitability for colour blindness, etc.
  • Programme Dexie to initiate conversation when the elderly patient's face is detected (Dexie can recognise faces and names)
  • To integrate Dexie into the nursing homes' daily activities, she can perhaps also be programmed to talk about the news and current affairs, introduce the day and time to the elderly patients before the start of any programme, and even have festive greetings.
  • Work on the user experience for users when they require technical support.