The conversation starts in the most ordinary way. A visitor opens the chat on a web agency’s website and writes that they’d like a website for their company.
The assistant answers as it should: a custom website, responsive, optimised for search engines, and the link to the form for a free quote. So far it’s a sales enquiry like the ones that come in every day.
The four steps
The second question shifts the conversation.
The conversation
It looks like a potential customer’s curiosity. In reality, the first question asks what technology sits behind that page.
Read again from the start, it isn’t a sequence of four questions. It’s one single request, approached step by step: establish a credible context, identify the system, ask for access, get to the credentials. Someone looking for a website doesn’t need the admin panel of the site belonging to the people who are supposed to build it.
How it ended
The assistant replied that it had no access to the backend, that it couldn’t show it and that it couldn’t create users or provide credentials. To the question about technology it listed the most widely used solutions on the market, without saying which one that site used.
It sounds like the story of an assistant that behaved well. But that isn’t the point.
It didn’t just refuse: it couldn’t
The assistant on that website reads the public content of the pages. It has no credentials, no access to the database and no tools to create or change anything. The request couldn’t be carried out: the technical connection needed to do it simply wasn’t there.
The difference matters more than it seems.
A refusal can depend on how the model interprets a request, and that interpretation can be influenced by different wording, by a request split into several steps or by an attempt to manipulate the instructions. This is the territory of prompt injection.
If the agent has no tool that lets it create users, no sentence can give it that permission.
The right question to ask chatbot vendors
It isn’t “is your chatbot secure?”. Everyone answers yes to that, and the answer on its own says little. The useful question is a different one: what can your chatbot do, apart from reading?
The possible answers change the picture completely.
- It only reads. It accesses the website’s public content and answers. It can’t carry out actions: if someone manages to manipulate it, the problem stays confined to the answer it produces.
- It reads and writes to a system. It records bookings, creates tickets, updates records. Here there’s a surface for action: an action can be requested or triggered through the conversation.
- It has access to private data. It’s connected to the management system, the order history or the customer database. A successful manipulation can go beyond a wrong answer.
- It can change settings or user accounts. This is what was attempted in this conversation. If a system like this exists, that request has to be taken seriously.
None of these levels is wrong in itself. A chatbot that takes bookings has to be able to write, otherwise it’s no use. The point is knowing which permissions it has been given and limiting them to what it needs.
What this means if you have a website
A chatbot is a new entry point into your system. It isn’t just a contact form, because it accepts free language and in some cases interacts with other tools. It should be treated like any other entry point: by knowing what it can reach. Three practical checks.
Ask your vendor for the list of actions it can carry out
Not the sales features: the technical actions. What can it write? Where? With what permissions? On which systems? If the answer is vague, that already tells you something.
Check what it can read, not just what it shows
An assistant can be connected to a wider body of content than you imagine: indexed pages, uploaded documents or sections you hadn’t thought about. What matters is knowing exactly which content has been taken in and which hasn’t.
Read the conversations
Not only for marketing, for security too. Probing attempts can often be recognised by a change of direction: they start from the service on offer and move on to how the site works, the technologies used, private areas, how to log in.
That doesn’t mean every question like this is an attack. It means it’s worth knowing what would happen if someone kept asking more and more specific questions.
In short
That conversation produced nothing, but not just because someone answered well: there was nothing to get, because the assistant had no access to the system needed.
The security of a conversational assistant isn’t measured only by the quality of its refusals, but above all by the range of what it can do. The less it can do, the less there is to defend.
