Ethnographic Interviews

To ground the project in lived experience, I conducted two ethnographic interviews with people who interact daily with non-verbal users: a pedagogy student who accompanies a nine-year-old child with cerebral palsy at school, and the sister of a non-verbal autistic adult. Both interviews focused on daily communication patterns, moments of uncertainty, and the emotional cost of interpretive ambiguity.

Two interviews is a small sample — enough to surface real patterns and shape design decisions at this scope, but not enough for statistical validation. The insights below should be read as directional, not conclusive.

What they told me

Both described moments where they knew something was wrong but could not identify whether it was pain, hunger, sensory overload, or sadness. This uncertainty generates anxiety and frustration, because the response depends on reading signals that are often invisible to anyone else. They also noted that only people with long-term proximity know how to interpret certain cues, which isolates the responsibility of care and makes it impossible to share that knowledge with others.

Key Insights

Empathy is not about recognising emotions. It is about creating conditions for the human to exist fully within the technology. The AI’s role is not to replace human care, but to reduce the caregiver’s emotional load and make their knowledge shareable.

The biggest challenge is not identification, but intervention: deciding how, when, and how far the technology should act without overstepping. FLUID.ai knows its limits and escalates to human intervention when needed.

Multimodality is essential. Combining visual, haptic, and sonic stimuli creates a more gradual and less invasive reading of emotional states, respecting different sensory sensitivities.

Colour and rhythm are language, not decoration. Chromatic codes linked to emotional states are not labels but a translation system between two different emotional worlds.

Benchmarking

FLUID.ai was designed in dialogue with existing assistive tools, each one revealing a gap the project set out to address.

• Avaz AAC Gives the patient a way to express intent through symbols, but assumes enough motor and cognitive control to operate a grid interface — and says nothing about the caregiver’s side of the loop.
• EmbracePlus (Empatica) Reads physiological signals with clinical precision for seizure and stress detection, but surfaces them as data on a screen for a clinician to review — not as an ambient signal a caregiver reads at a glance, in real time.

The gap was clear: nothing translated involuntary physiological signal directly into ambient, multimodal output for someone who cannot operate an interface at all.

Beyond direct assistive-tech comparisons, two bodies of work shaped the project’s register: Refik Anadol’s Machine Hallucination and teamLab’s Resonating Lamps informed how the lamp’s light could move and feel alive; Kristina Höök’s Soma Design and The School of Life’s trauma-to-healing work informed designing with the body, not just for it.

Narrowing the scope

The brief asked a deliberately broad question: can an AI agent make the invisible constructs of the mind, emotions, sensations, ways of thinking, visible or perceptible? Read literally, that opened the door to any non-verbal patient, not only autistic individuals, but also people left non-verbal by conditions such as a cerebral ischemia, aphasia, or other acquired communication loss.

I chose to narrow that scope to non-verbal autistic individuals specifically. Trying to design for every cause of non-verbal communication at once would have flattened very different lived experiences into one generic profile, and made every downstream decision, from the ethnographic questions I asked to the sensory calibration of the system, vaguer than it needed to be. Focusing on one population let the research actually sharpen the design, instead of just broadening it.

Exploring the object

Before settling on an ambient lamp, I considered two other form factors, each rejected for a reason specific to this population rather than a generic design preference.

The lamp won for the opposite reasons: it asks nothing of the patient’s body, it’s always present in the room without demanding attention, and its light can be seen, felt as warmth, and mirrored remotely, all at once.

Two connected objects

The choice of a lava lamp as the primary interface was not aesthetic — it was conceptual. The slow, organic, repetitive movement of the lamp’s internal fluid creates a visual metaphor for emotional states: expansion and retraction, calm and agitation, flow and stillness. This behaviour communicates without requiring complex interpretation, making it legible even to someone with limited verbal comprehension.

The lamp is grounded in Anna Vallgårda’s (2013) understanding of computational artefacts: objects that take form through the relationship between physical form, temporal form, and interactive gestures. FLUID.ai is not just a functional tool. It is an emotional mediator that creates conditions of safety, calm, and mutual understanding.

The system infers emotional state from two sensing sources: a camera reads posture, movement, and facial cues, while a wearable biometric sensor tracks heart rate, skin temperature, and stress markers. Together they form the layer beneath every colour, vibration, and sound decision described below.

Multimodal interaction

The system communicates across four channels simultaneously, calibrated to different sensory tolerances and contexts of use.

Escalation logic

One of the core principles of FLUID.ai is knowing its own limits. The system follows a clear escalation model: it first attempts autonomous self-regulation through sound and colour; if the patient does not improve, it alerts the caregiver; if the caregiver cannot resolve the situation, it contacts emergency services. At every step, human agency is preserved.

Refining the form

With the lava lamp already chosen, the form still needed refining: rather than a traditional triangular shape, the object needed softer edges and space for a speaker, through which the AI communicates with the patient via voice and sound. Several sketches explored this balance between the lamp’s organic form and the functional requirements of speaker placement.

Wireflow and user flow

The interaction model was mapped through two parallel documents: the conceptual wireflow (the system’s decision logic, from emotional detection to escalation) and the user flow (the lived experience of both the patient and the caregiver across an emotional episode). These became the backbone of the interface design. Every screen, every vibration pattern, every colour transition was derived from them.

←

→

1 / 8

Interface screens

The onboarding flow connects the caregiver’s watch to the FLUID.ai lamp, establishes the patient’s profile, and calibrates the emotional sensitivity thresholds. The watch interface uses the same fluid graphic language as the lamp, with colour as the primary information layer and vibration as the secondary alert.