KARA: Facilitating trust in EMS through a conversational agent

A Research and Design Case Study

Transporting sick patients is hard and stressful work. Emergency medical technicians have clear protocols for managing emergency situations and stabilizing patients, but stress happens during transport.

Kara is a future-oriented concept for an intelligent conversational situated in the driver's compartment of an ambulance. Designed for Medics, it is not intended to replace them, but to integrate into the workflow of medics and become a helpful member of the team.

How it works

Kara uses a pervasive multi-modal interface situated in the front of the ambulance to facilitate the safe transport of patients. Aside from voice, the system also includes a heads up display and haptic feedback.

Kara augments the foresight capacity of emergency transport personnel using computational power, leveraging external data feeds for traffic and healthcare infrastructure, as well internal data logs for patient care and equipment management. Through conversational patterns attuned to the communication style of medics, Kara provides support by sharing navigational logic, patient case analytics, and vehicle maintenance monitoring alerts to facilitate successful transportation in stressful situations.

Experience Prototypes

Interaction Model

Kara is designed to integrate into a team of emergency personnel using a model of symbiosis through co-action. By taking on shared responsibility, shared logic, and shared memory, Kara is able to optimize specific areas of concern such as navigation, patient case patterns, equipment management, etc. This means that medics are able to better adapt to unforeseen diruptions that arise during pateint transport, and focus attention to reponding quickly and appropriately in complex situations.

Kara takes on a friendly, conversational tone in team communications, adopting a shared vernacular, and following social graces. It is seamlessly able to transition between passive and active states of listening as speech through triggers and invocations, and interjects at appropriate moments, presenting information verbally, visually, or haptically based on context. Kara builds trust initially through transparency in communication, and through reliable action over time.

Process

Background

Conversational user interfaces (CUIs) are positioned to change the way we interact with technology on a day-to-day basis. As part of the Microsoft Design Expo, our project brief was design a future solution that demonstrates the value of CUIs in building a symbiotic relationship between humans and machines, keeping in mind J.C.R. Licklider’s definition of human-computer symbiosis.

Identifying a design opportunity

With a relatively open brief, we began by identifying a broad area of investigation in automotive technology. Inspired by rapid advancements in autonomous vehicles, we saw an opportunity to design conversational user interfaces that could make people feel more comfortable in these environments. We began scoping the design challenge by choosing a user group to focus on.

After speaking to a variety of stakeholders, we landed on emergency medical transport as a rich area for potential design intervention.

How might conversational user interfaces be used in the future of emergency medical transport?

Getting to Know the Stakeholders

Through a dozen light-touch interviews with a diverse range of emergency responders, from experienced EMTs to volunteer medics, we were able to create a macro-level view of emergency medical support infrastructures, and understand the relationships between multiple stakeholders.

Points of Stress

While being a medic is a high-stress job, medics often rely on protocols they've been trained to follow while actually dealing with patients. However, through our interviews, it surfaced that the most high-stress points of a dispatch call occur when the ambulance is in transit. There is a large number of unknown variables in these situations, and it is this uncertainty that requires foresight and quick decision-making.

Getting to know the environment

A touchstone tour of the inside of an ambulance to understand current interface and interaction challenges. We learned that EMTs currently use several complicated and outdated interfaces, and even incremental multimodal interaction designs could seriously alleviate much of the issues around ease of use and intuitiveness.

16 hours doing ridealongs with EMTs gave us a more empathetic view of their routines, challenges, coping mechanisms, and stresses.

Generative Research

We conducted a series of co-design workshops with Pittsburgh paramedics and EMTs to help us gain a deeper understanding of the challenges and pain points, and integrate qualitative inputs from end-users to inform the development of the design.

Journey Mapping

A simple exercise to visualise the end-to-end experience and identify moments of stress. Specific scenarios emerged as key moments for design intervention in day-to-day shifts.

Collage

By asking research participants to represent their ideal teammate, we learned how EMTs collaborate and build trust in each others’ intuition, judgment, and problem solving skills.

Role-Playing

A set of future-looking provocations to test the boundaries of what participants were comfortable with using and interacting with.

Research Insights

EMTs are active problem-solvers.

"So much of what I need to know has to be committed to memory." What paramedics know is how to quickly assess and respond in the right way in complex and stressful human situations.

Communication is important, but trust is deeper than that.

"The nightmare scenario is a partner who doesn't know what to do." "Sometimes a gesture or look is enough." Each person on the team must play an active role. During moments of intense activity, each person is their own unit and EMS professionals must and do trust each other's intuition, judgment, and response.

Some noise is good noise.

"It's good to hear what's happening around the city." EMS professionals need to multi-task but rely on a variety of signals to gather information.

Design Principles

Let people do what they do best, let computers do the rest

We don't want to remove or replace the job that EMTs really do. Let's design for where computers have strengths and augment what is working.

Build for trust through reliable action

Conversation is the tip of the experiential iceberg but it underlies an agent that should demonstrate trustworthy behavior by showing consistent actions and approaches over time.

Not reductive but additive

Rather than reducing information, we should provide information to contextualize situations at the right time, in the right way.

Resulting Directions

Based on insights gathered from generative research, we identified some possible roles the CUI could play, as well as the forms it could take.

We realised some of these initial roles were focused on logistical tasks, which led us to focus on the “front of ambulance” experience with the goal of alleviating stress from unexpected events.

Focused Design Opportunity

How might we alleviate stress for emergency medical transport teams by augmenting foresight through a conversational agent?

Concept Generation

With a direction coming into clearer focus, and with key insights and design principles to guide us, we brainstormed a total of 100 user interface concepts to fully externalize our findings as a team. Through affinity mapping and in-depth ideation through sketches, we generated 12 narrative storyboards intended to test the boundaries of our design outcomes. We took these back to the EMT teams to illicit reactions and gather feedback. We also conducted a card-sorting exercise to determine which qualities and features could anchor the experience.

Through reactions and feedback on our initial concepts, we established a set of heuristics for interactions within the system to support further ideation.

We also established that our users were not looking for direct stress relief, because they had individual coping mechanisms that were very effective. Instead, we focused our design on 3 main scenarios: navigation, patient case patterns, and vehicle maintenance.

Prototyping and Testing

We began designing multimodal interactions — verbal, visual, haptic — in the focused scenarios through dozens of iterative prototypes using scripts, audio, mock-ups, projections, and role-plays. We experimented with a variety of tangible forms, personalities, voice and tone for the CUI, as well as visual information displays. Prototypes were frequently tested and evaluated with various stakeholders to finally arrive at a complete design concept.

Refinement

In the last leg of the design process, we codified design patterns and interaction models, and assembled refined prototypes to communicate the concept.

Reflection

Our presentation elicited interesting responses – from how this could be retrofitted onto existing systems (the ‘smartness’ of the smart ambulance was something we hadn’t really discussed concretely), to the use and sharing of data across different platforms (our stakeholder interviews and map showed that this would increase efficiency), to what happens when the human and computer disagree (the computer knows when and how to mediate relationships between people but doesn’t have control over what they do).

They appreciated the CUI with the GUI and that we had created solutions for their particular pain points. Probing a little deeper, they told us that they have to deal with a lot of paperwork on antiquated systems that are not integrated. We also asked them about the ‘anticipatory’ stage before receiving a dispatch call. They said that though it was an interesting time when a lot of data could potentially be helpful, as EMS personnel on the road, they wouldn’t be interested in data patterns. They also reiterated how being in the EMS is a job, discrete from their personal lives. All in all, it was great to talk with them about the blue-sky possibilities and understand that incremental changes that need to be made to get to Kara.

Detailed process documentation available here. (link: https://thebluebarracudas.wordpress.com/)

< Previous

Next >

Saumya Kharbanda