An LLM Will Answer Your Question
That Is the Problem
Ask a chatbot whether a second palletizer gets Line 3 to next year’s volume and it will tell you. Fluently, with a number, in seconds. It has no model of your line. It has read a great deal about palletizers and is producing the most plausible continuation of your sentence — and you cannot tell that answer from a correct one by looking at it.
What it is actually doing
A language model predicts text. Given your question, it produces what a well-informed answer to that question tends to look like, assembled from everything it has read. That is a genuinely remarkable capability, and for a great many tasks it is the right tool.
But your line is not in its training data. Not the buffer between your filler and your capper, not the changeover you lose forty minutes to on Tuesdays, not the fact that your palletizer is starved rather than slow. It has never seen your historian. When it returns “roughly a 12–15% throughput improvement,” that figure is not a calculation. It is the kind of number that appears in sentences like the one you asked for.
Why that is worse than being wrong
A tool that fails visibly is manageable. You see the error, you stop, you find another way. The failure mode here is different: the answer is indistinguishable from a correct one. It has the right shape, the right units, the confident register, and often a plausible-sounding justification underneath it.
So the usual defence — “we’d check it” — does not work, because there is nothing to check against. Checking means comparing to something that knows your line, and if you had that, you would not have asked the chatbot.
In most contexts a confidently wrong answer costs you an afternoon. In a capital decision it costs you the decision, and you find out two years later when the line still cannot make the volume.
The question is not whether the model is clever. It is whether anything it said was derived from your plant.
What we do instead, and where we do use AI
This is worth being precise about, because we are not making a general argument against AI. We are building an AI / MCP add-on — connect an assistant to your validated model, ask it questions in plain language, run scenarios conversationally, against a local model if you want one. It is not on sale yet; pricing is announced at launch.
So this is a commitment about a product we have not sold you rather than a defence of one we have: when it ships, it will sit on top of a validated model, and it will not be the thing that produces the number.
The distinction is what produces the number:
Two different jobs
| What produces the answer | Can you check it? | |
|---|---|---|
| A chatbot answering directly | text prediction over things it has read | No — there is nothing underneath it |
| ReliaSim | a discrete-rate and reliability model of your line, built from your own stop history, run thousands of times | Yes — against the output you already measured |
We do not use a language model to build the model. The model is built from your line event data — the stops and starts your historian is already recording — with distributions fitted per failure mode. An assistant helps you ask questions of it and read the answers back in plain language. It is the interface, never the engine. Those are different jobs, and conflating them is the whole error this page is about.
The test that separates them
There is one question worth asking of any tool that offers to answer a what-if about your plant: what did it reproduce before it predicted?
A peer-reviewed WSC 2020 study modelled a multi-line food plant — twenty-plus unit operations, up to twenty failure modes each — and agreed with the plant’s measured OEE to within one percentage point across a full year. That model was rebuilt in ReliaSim and independently re-run within 1% of both the measured figure and the original.
That is not a claim about cleverness. It is a claim that can be falsified, and was not. A chatbot cannot make it, because there is nothing to compare: it did not reproduce your year, it produced a sentence about it.
Where a language model genuinely helps
It would be dishonest to leave it there, because the useful uses are real — and they are what the add-on is for:
- Asking the model questions without learning its interface. “Which failure mode costs the most on Line 2?” is a better way in than a menu.
- Reading results back in plain language, including the ones you did not think to ask for.
- Getting oriented in an unfamiliar field — what a buffer does, why micro-stops matter, what OEE leaves out.
- The writing around the work: the summary, the business case, the email to your plant manager.
Every one of those sits on top of something that knows your line. None of them is the thing that produces the number. That is the whole design, and it is why the assistant is an add-on to a validated model rather than a product of its own.
What to ask a vendor
- “What is this answer derived from?” If the honest answer is “a language model,” it is a guess in a confident voice.
- “What did it reproduce before it predicted?” A tool that cannot match a period you already measured has not earned a question about one you have not.
- “Would it give me the same answer twice?” A simulation run again on the same inputs behaves the same way. Text generation does not.
- “Can I see the assumptions?” A model has inputs you can argue with. A sentence does not.
Related: why predictive maintenance can’t answer what-if makes the same argument about a different tool — one that knows your plant intimately, and still cannot tell you what a change returns.