I’ve noticed a growing number of people trying to tackle complex puzzles like the 3301’s Liber Primus or Kryptos using AI. We often say these puzzles can’t realistically be solved this way—at least not today—but I wanted to explain why and what real evidence looks like. This is aimed at those using large language models (LLMs) for these challenges. AI is a broad term, so let’s break it down.
Today, there are two kinds of AI: Narrow AI and General AI.
- General AI is the stuff of science fiction—an intelligence that can reason across disciplines, form independent thoughts, and solve problems creatively. Think Data from Star Trek or Skynet from Terminator. Fascinating in theory, but it doesn’t exist on Earth today.
- Narrow AI, on the other hand, excels in specific domains. It powers cancer detection, quality control in factories, and yes, LLMs that can generate code or text. Impressive, but still limited in scope.
So where do LLMs fit? They’re conversational tools trained to predict the next likely word based on patterns in their data. If you ask for a PowerShell script that prints “Hello World,” it will deliver because it’s seen that pattern before. But if you feed it the runes of Liber Primus and ask for a solution, it will produce something—likely wrong—because there’s no real data on solving that puzzle. That’s what we mean by “hallucination.”
LLMs shine in practical tasks:
- Automating repetitive work like generating units tests or boilerplate code
- Improving the tone of documents
These are things you could do yourself, but LLMs make you more productive. However, if you use an LLM to validate a solution, remember: it’s subjectively true—true for you and the model at that moment, not objectively verifiable.
If you want your solution to be considered objectively true, here’s what evidence requires:
- No LLM needed to reproduce the output
- Steps are clearly documented
- Output is reproducible
- Steps are validated
That’s it—four steps, one of which requires no extra work. Meet these criteria, and your solution stands on solid ground.