How to Know Your Supply Chain Model Is Right
Reproduce History Before You Trust a Scenario

A supply chain model earns trust the same way a new planner does, by getting last year right. Before you ask it what would happen if you closed a DC, make it reproduce a period you already lived through, and check it where it’s most likely to be wrong.

Reproduce history before you run a scenario

Every scenario is a change to a base case. If the base case doesn’t behave like your network, every scenario inherits the error, and the difference between two wrong answers can look perfectly reasonable. So the first run of any network model should be a period you already know: last year, or the last two quarters, with the demand, the sites, the lanes and the policies that were really in place.

Then compare what the model did with what the network did. If they agree, you have a base case. If they don’t, you’ve learned something about either the model or the data, and that’s worth finding before anyone builds a decision on it.

Freeze the inputs for the base period

A fair test gives the model the same inputs the network had. Pull them for one period and don’t change them while you compare.

• Demand as it was ordered, by customer or region, by product and by day or week. Use actual orders, not the forecast.
• Sites and lanes as they operated in that period, including any DC that opened or closed partway through.
• Policies as they were run, meaning order points, order quantities and review periods, not the ones in the planning manual.
• Transit and production times as distributions from your shipment and production records, not single averages.
• Starting stock at every site on the first day, so the model doesn’t spend its first weeks filling up.

What to compare

Compare the things the network records and the model reports, at the level where mistakes hide.

Flows by lane

Units shipped on each lane, by week or month. A model can match total volume while sending it down the wrong lanes. That usually means a sourcing rule or an assignment is wrong, and it shows up lane by lane long before it shows up in a total.

Stock by site over time

Inventory at each site through the period, not just the average. The shape matters. Look for the same reorder rhythm, the same peaks before a season and the same lows. A site that holds the right average stock with the wrong rhythm is being replenished by the wrong logic.

Lead times

Order-to-delivery time by lane or region, as a distribution. Compare the middle of the range and the long tail. The tail is where late suppliers and congested DCs show up, and it’s the part an average hides.

Production by plant

Output by plant and product, week by week. If plants make the right totals at the wrong times, the network downstream sees a different pattern of supply than it really had.

Compare the parts, not only the total

A model can match the total for the wrong reasons, with one lane overstated and another understated. Checking each lane, site and product separately rules that out. It’s the same rule the ReliaSim Method applies inside a plant, where each failure mode is checked against the historian on its own rather than trusting the line total. The published OEE validation shows what that looks like on a real plant.

Agree in advance on how close is close enough for each measure, and write it down before the first comparison. A tolerance chosen after you’ve seen the results tends to be exactly as wide as the gap.

Hold out a period

If you adjust a model until it matches last year, it will match last year. That shows you can tune it, not that it’s right. So split the history. Build and adjust the model on one period, then run it on a later period you didn’t touch and compare again.

The held-out period should include something the first one didn’t, such as a peak season, a promotion or a new customer. When a model tracks a period it has never seen, you can start asking it about the future.

Start the run where the network was

A model that starts empty spends its first weeks filling up, and during that time it moves product faster than the real network ever did. For a long study you can discard that warm-up. For a short horizon you can’t, because the warm-up is most of the run. The fix is to start the model from where stock and shipments really were on day one. The VinLogic case study describes a vehicle network model started from the tracked position of every vehicle, and what that did to the first weeks of its forecasts.

When the model doesn’t match

Most gaps come from the data rather than the model logic. Check these before changing anything else.

• Units. Cases in one table and eaches in another is the most common error there is.
• Missing lanes. Emergency shipments and transfers between DCs often live outside the main shipment table.
• Policies on paper. Planners override order points. The model has to run what they did, not what the system says.
• Averaged times. A single transit time makes every shipment arrive on schedule, so the model never sees the late ones.

Resist tuning a parameter just to close a gap. If you can’t explain why a number changed, you’ve hidden the problem rather than fixed it.

After the base case holds

Once the model reproduces history and the holdout, save that version as the base case. Every scenario is then a change to it, one change at a time where you can, so the difference between runs comes from the change and nothing else. What is supply chain simulation? covers the scenarios that come next.