Beyond Chat: Blending UI For An AI World
One thing that's been on my mind a lot lately: Is a purely chat-based interface the optimal UI for humans to work with AI?
As much as I love natural language interfaces provided by ChatUX, there are many use cases where there are better, more efficient ways to interact.
I think the future is going to be about hybrid, fluid UI. A combination of chat and other more "classic" UI that we're used to.
Here's what I'm experimenting with as a very rudimentary approach to incorporating some "classic" UI into a chat interface:
Expose the ability for the LLM to request user inputs (of varying kinds) as "tools" or (functions). This ability for LLMs to use tools is a Very Big Deal -- but in most cases, we're using these tools to do back-end integrations, call APIs and access data.
Recommended by LinkedIn
But we can take tool use much further. We can give LLMs tools that allow human input.
For example, if the LLM needs to get a date from the user, it uses the tool to get that input, and we show an inline UI control for the user to enter a date. That control could allow natural language input (next Monday) or a popup calendar. Similarly, we could support all the UI primitives that are common (text input, dropdown selection, etc.)
Not the fanciest of UIs, but even with the basic primitives, there's a lot we can do. And, I like how it keeps orchestration power in the hands of the LLM and lets it worry about when human input might be required and a way to get it.
Over time, we could expand that model with more powerful UI controls (grids, visualizers, image editors, etc.)
I can also conceive of a way to get human input asynchronously (for longer-running tasks). For example we could have a "send_user_approval_link_by_email". This would send the user an email with a link to approve/continue the agentic workflow.
There are likely much better ways to blend the UI, but this one has the value that it could actually be implemented by mere mortals like me, and it would be a step in the right direction.
Human-Centered Data/AI Leadership
2wI want this. Have a chat decide that it needs to know exactly 6 things from the user right now. Yes, you could ask the user to answer everything and then parse the natural language response, but that feels really stupid when we invented UI primitives decades ago... I really could use a UI that had tool use like this. Any progress?
Building AI Agents | Full-Stack Engineer | Open to opportunities.
1moThis is an amazing take on extending the capabilities of Language models beyond chat interfaces. But, as AI use-cases are still evolving, majority is focused on implementing the chat based services. I think in future we'll use these generative capabilities in much more expressive ways. Just like various technologies of the past.
Founder at Avinashi.ai
1moI think most of top line AI chat platforms are utilising Artifacts as outcomes. But mostly they are like Markdown previews, with versioning (advanced outcome). However I think, Artifacts with specific UI (e.g. Spreadsheet view if my outcome is Table) would be the next experience as connecting dot further. Just my 2 cents :)
🎖️ | Vice President | Top 1% Business Incorporation Expert – ADGM | Dubai | BVI | Global Markets | Brand Management | ex-American Express | Multilingual: 🇬🇧🇮🇳🇪🇸🇫🇷
1moGreat insights, Dharmesh! Blending conversational UI with classic UI seems like the perfect approach to balance innovation and user familiarity. Your thoughts remind me of the importance of continuously adapting our strategies based on robust market analysis and user feedback. Looking forward to more of your ideas!
AI Product Manager at The Marketing Cloud
2moDharmesh it’s not just about making it easier and engaging for the user, it’s also a way for companies to pass through their brand identity via the agent. Imagine when an agent takes an action in Hubspot via MCP, instead of responding with just a success message, you also send a hubspot branded confirmation window back to the agent to display it to the user.