Agentic Workflows for Conversational Human-AI Interaction Design
In short
Talking to an AI assistant is open-ended: people often do not know what to ask, and designers rarely see what they were trying to do. We built a chat app in which helper agents use a person’s context, such as a photo, the place, the time and the weather, to suggest goals and sharpen prompts. We redesigned it four times with ten participants.
Why this work
Two things make conversational AI hard to design for. One is ambiguity: goals are open-ended, and it is unclear what the AI can do. The other is transience: exchanges are brief and rarely revisited, so designers get little sustained contact with the people using them.
We asked how agentic workflows, in which several AI agents each take a part of the job, could help users and designers with both.
What we built
The probe is a chat web app running a small multimodal model on our own server. Over four versions it moved from suggesting ready-made prompts based on a photo, to adding a step for choosing a goal first, to letting people combine one kind of context at a time with goals they typed themselves.
In the last version, designers could talk with an AI stand-in for a participant, built from that person’s logged interactions.
How we studied it
We used research through design: four cycles in about a month, with ten participants who each used a version for several days in different situations and then answered a survey. I was one of the two authors who sorted participants’ prompts into categories.
What we learned
Suggestions offered upfront often missed what a person actually wanted. Working up to a prompt step by step, mixing context with the person’s own input, worked better.
More context is not always better. An accurate detail about a person could send the agent in the wrong direction, and sharing context has a privacy cost. Participants’ prompts fell into six kinds: action, classification, context analysis, search, recommendation and instruction.
Cite this paper
Arthur Caetano, Kavya Verma, Atieh Taheri, Radha Kumaran, Zichen Chen, Jiaao Chen, Tobias Höllerer, and Misha Sra. 2025. Agentic Workflows for Conversational Human-AI Interaction Design. arXiv:2501.18002. https://arxiv.org/abs/2501.18002
BibTeX
@misc{caetano2025agenticworkflowsconversationalhumanai,
title = {Agentic Workflows for Conversational Human-AI Interaction Design},
author = {Arthur Caetano and Kavya Verma and Atieh Taheri and Radha Kumaran
and Zichen Chen and Jiaao Chen and Tobias Höllerer and Misha Sra},
year = {2025},
eprint = {2501.18002},
archivePrefix = {arXiv},
primaryClass = {cs.HC},
url = {https://arxiv.org/abs/2501.18002}
}