Your AI keeps giving you a different architecture. Here's how we fixed it.

Imagine you work at a company and every Monday someone walks in, looks around, and renames every team.

Monday: “Auth,” “Users,” “Billing.”

Tuesday: “Authentication Layer,” “User Module,” “Payment Flow.”

Same people. Same work. Different names.

That’s what asking an AI to name the subsystems in your code feels like.

The problem

We ship a thing called the domain graph. You hand us your code. We hand you back a short list of the big pieces, what each one does, and which files live inside. Like an architecture diagram, but made from the real code instead of drawn in a meeting.

The first version worked fine on one run. The second run was the problem. You’d run it Monday and get back AuthService, UserMgmt, PaymentFlow. You’d run it Tuesday with the same code and get back AuthenticationLayer, UserModule, Billing.

That is useless. A map that changes names every time you look at it is not a map. You can’t put it in your wiki. You can’t use it for PR labels. You can’t onboard new engineers with it. The AI was doing something reasonable each time. It just never did the same reasonable thing twice.

The reason is simple. An AI that has no memory of last week will come up with new names this week. Not because the code changed. Because the AI is guessing from scratch every time.

The fix

We showed it last week’s names.

That’s it. We save the domains we got last time — just the names and how many pieces each one has — and we pass them back in when we run it again. The prompt says: “Here are the names you picked last time. If the code still looks the same, use the same names.”

Small change. Big result.

The numbers

We ran it on two big projects.

• Directus: about 18,000 code nodes.
• Next.js: about 12,000 code nodes.

Before, 0–60% of the domain names survived from run to run. Some runs kept nothing. Some kept about half.

After seeding, 87–100% of the names survived. Nearly every one.

The count of sub-pieces also stopped drifting. Before, it wandered up and down every run. After, the drift was between 0 and 1.2.

Same code. Same AI. Same model. The only thing we changed was handing it last week’s map.

One extra trick: ghost domains

There was a small catch. Sometimes the AI was too helpful. We’d hand it last week’s names, and it would keep them even if the code that used to live in that domain was gone. We’d get a domain called LegacyBilling with zero files in it. A ghost.

So we added one more step. After the AI writes the map, we walk through each domain and check: does it have any files? Any functions? Any sub-pieces? If the answer is no to all three, we throw it out.

A domain with no code is not a domain. It is a memory of a domain.

Why it matters

A map that is the same every time you look at it is a map you can actually use.

You can paste it in your wiki and it will still be right tomorrow.

You can label your pull requests with the domain they touch and the labels will mean something.

You can show it to a new hire and they won’t have to relearn a different set of names when they come back from lunch.

None of that works when the map changes names on its own.

The quiet lesson

The fix was not a smarter model. The fix was giving the model a little bit of memory.

Most problems that look like “the AI keeps making stuff up” are really “the AI has nothing to hold on to.” If you give it something to hold on to — the last answer, the last map, the last names — the new answer stops drifting.

A map that stays put is cheap. It just takes one argument.

Try it

If you use the CLI, you already get this. The tool saves the names on every run and sends them back in the next one. You don’t do anything.

Install it:

npm install -g @supermodeltools/cli

Go to your project and run:

supermodel setup
supermodel watch

The first run names your domains. Every run after that keeps the names steady. Paste them in your wiki, use them for PR labels, hand them to a new hire. They’ll still be right tomorrow.

If you’re calling the API yourself, pass a previousDomains query parameter on your next call to /v1/graphs/supermodel or /v1/graphs/domain. It’s a small JSON array of {name, subdomainCount} from your last run.

Run it once. The names stick.