Just a chat? Four levels of LLM integration

“We’d like to add an LLM to our web app.”

That can mean very different things to different businesses. Let’s look at what this request might involve, from a simple chat to a dynamically generated user interface.

1. Just a chat

The simplest option is a chat window, like the ones you know from ChatGPT or dozens of support bots.

The user asks a question and the LLM responds. The interface looks like WhatsApp or any other chat application.

You can have a conversation, summarise content and answer questions. With a little Markdown, the response can also look good when it contains code, a formula or a table.

But there is no interaction with the existing application. A chat is easy to implement, but it remains an add-on to the actual application.

2. Tools

The next step is to give the LLM access to specific functions in the application.

Now the model can do things instead of just answering questions. For example, a user of a GIS system might ask:

“Show me this point on the map.”

The LLM can then decide to call a goToLocation function. The application responds by zooming in on the relevant point.

The key difference is that the LLM can now control parts of the application.

3. RAG and a little more

Retrieval-Augmented Generation (RAG) is a technique that retrieves relevant information at runtime and makes it available to the LLM as context.

But instead of simply providing a few documents and asking for references to them, the sources themselves can become part of the user experience.

Users can see the document and the relevant passage. Metadata and other useful information can be shown directly, too.

The result is more than an answer with a citation: it is an answer with the relevant evidence.

This is especially useful when someone needs to find specific information and trust in the result matters.

4. Generative UI

At the fourth level, the LLM does not just receive context—it decides how that context should be presented.

If someone asks for an overview of the current project status, the application could show a dashboard with metrics, charts and a list of current activities.

After another question, the dashboard might look completely different.

Instead of one fixed interface for every possible question, a new one is generated on demand to best answer the current question. Unlike level two, this is not just about using predefined tools; the LLM also decides what the interface should look like.

The application no longer feels like two separate products, but like one integrated application that adapts dynamically to the question.

The way forward

These four levels do not replace one another, and a higher level is not automatically better.

Each level represents a different kind of interaction.

The question to ask is not “Where can I add an LLM?”, but rather:

“Which problems can the right kind of interaction actually help solve?”

This distinction helps create a product that is more than a product with an LLM attached—it is a product that uses an LLM to solve problems better.