From a Million-Row Lifecycle Table to AI-Driven Digital Twins

How do you know where several million items, across a few hundred item types, sit in their lifecycle, when you cannot go and inspect them, and when that lifecycle shifts with both time and external events? We started where most teams start: one very large table, cron jobs and triggers, then an LLM to keep it up to date. The lesson came early: we first needed a formal description of the item type and of the item itself, and then a lifecycle we could observe and control.

Our first attempt modelled both as state-machine workflows. The lifecycle rules turned out to be a poor fit for AI, and workflows brought their own problems at this data volume. The alternative, running models straight against the big table, was no better.

Digital twins gave us the model we were missing. We moved through several implementations, from fixed lifecycle rules to an AI-driven lifecycle, and now run on Fabric Digital Twins with AI agents driving item lifecycle – a ship-of-Theseus migration where the system keeps running while every part is replaced.

This session is the data and AI view of that journey, with a live demo of what we run today – and a chocolate-sweet finish, as always.

Key takeaways:

• Where a “big table plus cron plus LLM” approach breaks, and the signals that tell you before it does
• Why rule-based state machines and AI-driven lifecycle prediction pull in opposite directions
• How to split responsibilities between the twin model and the agents acting on it