One morning a bug landed on my desk: a user reported that a link in one of our web app’s chat responses was 404’ing. Not a remarkable problem on its face — there are a lot of links out there, and some of them are bound to break.
What was supposed to happen
A lot of our web app’s job is to expose insights to our customers about their pipelines and deals. To keep things organized, we relate things together in a useful hierarchy. Users have accounts and accounts have all sorts of things, like for example, contacts.
If you’re using the Endgame web app to learn more about “John Smith” at one of your accounts, our agent will respond with some helpful links in a custom markdown format to make it easy to learn more about that contact. That link will also reflect this hierarchy too: the account’s UUID first, then the UUID of the related resource you’re drilling into.
Under the hood, the raw markdown generated by the Endgame agent for our web app looks something like:
<SourceLink ref="/abd84396-9e13-4954-ad6e-e5e4c3b51ca1/sources/e3423847-e9d5-4a87-8f4e-29c7e62fa95d">
John Smith
</SourceLink>What happened instead
Well, that link did not take you anywhere useful. If you were looking to get more information about John Smith, sorry, you’re out of luck.
The next question was why?
This problem is not widespread, any other account contact links generated work great, and there’s no difference in the URL format.
I know the account exists. I’m looking at the account’s details in the web app as I’m trying to debug this issue.
This contact must exist too. The agent found it during its research phase, that’s why it knew about “John Smith” at all. I can use the UUID from the broken link in the bug report and query the database and get John’s record no problem.
A memory game
We’ll get back to John Smith, but first, let’s try something really quick. Take a moment to memorize the following UUID:
0cb5bcef-b5f5-48fe-9dff-5e4a9870c66fNow close your eyes, and try to repeat it back to yourself.
How’d that go?
Okay let’s try again. How about this one:
11111111-2222-3333-4444-555555555555Any difference?
If you found that second one easier, you wouldn’t be alone and it wouldn’t just be humans who agree with you.
Cognitive load
Under it all, LLMs are token prediction machines. They’re quite good at exploiting statistical regularities to anticipate what should be said next.
UUIDs are designed to be maximally random. They are stringified entropy. They don’t mean anything. Knowing the first half of a UUID won’t help you predict what the second half will be, and this is a difficult problem for LLMs given their probabilistic nature. Without any tricks at their disposal, they instead just have to memorize it.
If we look at how GPT-5.x tokenizes strings, that first UUID in our memory game occupies 26 tokens. The second, simpler, UUID occupies 17 tokens. That first, real, UUID must be broken down into smaller chunks. Just like you, an LLM will have to spend more effort when recalling that UUID. The pattern of the second one gave you a heuristic to hold onto, the real one requires brute force.
It turns out this brute force has a cost, and ended up being the cause of my broken links.
Not like the other
Okay, one more game. What is the difference between this relative URL:
/abd84396-9e13-4954-ad6e-e5e4c3b51ca1/sources/e3423847-e9d5-4a87-8f4e-29c7e62fa95dand this one?
/abd84396-9e13-4954-ad6e-e5e4c3b351ca1/sources/e3423847-e9d5-4a87-8f4e-29c7e62fa95dIf you caught the stray 3 wedged into the account UUID of the second link then you’re more astute than I was.
The first URL is the correct format, and the second is the URL included in the agent’s generated response. This malformed UUID was the source of my error, and it was hiding in plain sight. That UUID is part of the input to the agent’s own research process. If it were wrong, I reasoned, then nothing should work.
The problem
So why did it almost work? The UUID the agent was reading was simply not the same UUID it wrote back. This was a new, LLM-era problem I wasn’t used to solving: there was no issue fetching the data at all — the problem was in how the response got transcribed.
Years of classical software trained me to assume that finding John Smith must be the failure point. I’d taken for granted that API responses couldn’t have typos — anything present in the input is available in the output. I’d never considered the added burden of transcribing 122 bits of entropy from its own context, and unfortunately John Smith had paid the price.
The fix
In this case, the fix is multi-stepped but straightforward. The data structure maps the relationships of accounts to contacts, there’s no need for the URL (and the agents who have to write them) to include this information. The URL can be shortened to:
/sources/e3423847-e9d5-4a87-8f4e-29c7e62fa95dWhere then any account details can just simply be looked up by the relationship this contact has.
By doing so, we’ve cut our chances for error in half. The next step would be to remove our last UUID and have the agent emit a placeholder or name, and then deterministically stamp the UUID in its place.
If each UUID an LLM has to write is a series of dice rolls, the easiest way to improve our odds is to roll fewer dice. The best way to stop an LLM from forgetting a UUID is to never ask it to write one.