Building a Local AI Agent (Part 2): Six UX and UI Design Challenges

Dev.to / 4/30/2026

💬 OpinionIdeas & Deep AnalysisTools & Practical Usage

Key Points

  • The article explains how the author designed Reiseki, an open-source local AI agent, focusing on UX/UI choices rather than the technical ReAct-loop problems covered in Part 1.
  • Reiseki balances two sometimes-conflicting goals: being usable by non-technical users without setup/config work, while making the agent’s knowledge (memories, conversation history, and file access) fully transparent and removable.
  • To improve transparency, the UI includes a live tool trace that displays each tool call in real time with its arguments and returned results.
  • The system also introduces a dedicated memory management panel so users can directly view and manage what the agent remembers.
  • The author notes that key layout improvements—such as workspace setup during installation and model selection via in-app dropdown—were introduced in version 0.1.3 and 0.1.4 during testing.

In Part 1 I covered the six technical problems behind Reiseki's ReAct loop — iteration caps, context management, persistent memory, and security. If you haven't read it, start there.

This part is about the design decisions. Even though I had an initial concept in mind, the current layout actually developed over time while testing. I asked the following questions: Who is this for? What should be adjustable by the user and what be automated? And for an agent that has access to your files and remembers your conversations, how can I make this as transparent as possible?

Reiseki is open source: github.com/Flo1632/reiseki

Two Goals That Pulled in Different Directions

Reiseki was designed with two principles in mind that don't always sit comfortably together.

The first: it should be usable without any technical knowledge. No terminal, no Python, no config files. If you can install an app, you can run Reiseki.

The second: the user should always know what the agent knows about them. Memories, conversation history, file access — all of it as visible as possible and deletable at any time.

The tension is that transparency often means complexity. Surfacing the right information without overwhelming the user.

Here's how I approached it.

The Six UX/UI Challenges and How I Solved Them

No setup required — just download and start

The challenge: If you can install an app, you should be able to run Reiseki. Everything else — model configuration, database setup, ... — should happen invisibly in the background.

The solution: Reiseki ships as a Windows installer. You pick a workspace folder during installation, and that's it. No Python environment to configure, no config files to edit, no command line. Given you have Ollama installed and downloaded a model, the agent opens with a setup screen asking for a name and a short description of your goal — and then it's ready.

The idea for the workspace setup during installation and to change the model with a drop-down menu directly in the Reiseki-App came in the last versions 0.1.3 and 0.1.4, while I was testing it.

Live tool trace

The challenge: How to show what the model is doing and how to make it visible to the user to create transparency?

The solution: Every time the agent calls a tool, the UI shows it in real time — which tool was called, with which arguments, and what came back.

Memory management panel

The challenge: I want to make the memories visible in the UI directly for full transparency.

The solution: In first versions I wanted to automate memory saving, but smaller models tend to forget the automation, even if it is included in the system prompt. So I gave the user the call with a designated "Save-Memory"-button, which actually triggers the save_memory tool call. The advantage: The user decides when a conversation is worth preserving, not the agent.

There is also a memory panel on top, which lets you see every stored memory and delete individual entries with one click. No asking the agent to forget something and hoping it complies — you delete it directly. The agent's knowledge about you is a list you can edit.

Conversation history modal

The challenge: Chat log transparency and editability

The solution: The chat log persists across sessions in SQLite, which means the agent has a record of past conversations. The history modal makes that visible and gives you a delete button. The entire log can be cleared at any time.

Smartphone access via QR toggle

The challenge: Making smartphone use possible without any app and as easy as possible.

The solution: The agent runs as a local web server, which means it's technically reachable from other devices on the same network — but only if you explicitly enable it. The QR modal has a toggle for this. When it's on, a QR code appears that you can scan from your phone. When it's off, the server blocks all non-localhost requests at the middleware level.

The key design decisions here were giving the user control over this functionality + using a QR code for easy access.

Model switcher

Ollama supports multiple models and switching between them is a common workflow — sometimes you want a faster model for quick questions, a more capable one for complex tasks. Requiring a server restart to change the model creates unnecessary friction.

The challenge: Making the model switch as easy as possible.

The solution: The model switcher lets you change it directly in the UI. The change takes effect on the next message. No switching between Ollama and Reiseki necessary, if you already downloaded the models before in Ollama.

Additional Thoughts

The tension between "simple for everyday users" and "using as little RAM as possible" while "keeping it as functional as possible" never fully resolves — it just gets managed. A few things I'd approach differently next time:

The database in the background is a great and simple basis, but I like the Markdown-file approach of Claude very much and I think this gives even more flexibility. (I used Claude.md and other md-files during the Claude Code sessions and it worked great!)

The first setup and user-guidance: I still think a tutorial like an automated session or a video at the beginning could help users discover the different functionalities in better ways.

The small model trade-off: During development I shifted from Qwen 2.5-coder:7B to Gemma 4:e2b, which made the agent and its tool calls significantly better. I hope that we see even more advanced small models in the future. On the other hand, more context and larger models would also make some of the coding challenges described in part 1 obsolete and would in general provide an even better experience. We currently cannot have it all.

This project was built entirely with Claude Code. The technical decisions and design goals are mine; Claude handled the implementation.

What UX/UI problems have you run into when building tools for non-technical users? Very curious about your experience