Why AI Game Masters Get the Rules Wrong

An AI game master gets the rules wrong in three ways players report again and again: it settles rolls with a bare d20 and no modifiers, it uses the wrong edition's wording, and its dice keep falling the player's way. Any player can cut most of this down today by naming the system and edition, giving the GM the rules in play, and rolling and adding their own modifiers. A game run by the book lets the written rules decide each result and lets the roll stand.

This post is part of our guide, Can an AI Be a Good Game Master?.

What Rules Mistakes Do AI Game Masters Make?

AI game masters make three common rules mistakes. They settle everything with a bare d20 roll and none of the character's modifiers, they use wording from the wrong edition of a game, and their dice keep falling in the player's favour. Each mistake is small on its own, and over a campaign they add up to a game where the numbers on your character sheet stop meaning anything.

A forum thread titled "ChatGPT is not a Good DM" describes a chatbot settling everything "by rolling a D20". On EN World, a player caught an AI using "a 2014 wording rather than 2024" for a rule. When Matt Evans tested chatbots as GMs for TechRadar, he found that with Gemini "all my rolls were successful", though he credited both chatbots with a good grasp of the rules.

Dice that always roll in your favour are part of a bigger problem, a GM that says yes to everything, which Why Your AI Game Master Lets Every Plan Work covers.

Why Do AI Game Masters Muddle the Rules?

In my experience, an AI game master muddles the rules because applying the right rule at the right moment is a judgement it has to make again every turn, and that judgement slips as play goes on. I spent a long time trying to make chatbots run my games. Told to follow a rule, a chatbot would obey for a while and then start breaking it again.

The chatbots I tried could hold the rules, since a language model can read a great deal of text at once, enough for a campaign's facts and the rules in play. They slipped on choosing which rule mattered right now.

Two plainer gaps made it worse. A chatbot that hadn't been given the rulebook answered from whatever it already knew, which is how a rule from one edition turns up in a game of another. And in a plain chat, the chatbot did its own sums, so a wrong total looked exactly like a right one.

How Do You Get an AI GM to Follow the Rules?

An AI GM follows the rules more closely when it has the rules text to hand, the player handles their own dice and modifiers, and someone looks at the totals. These five habits work with any tool, including a plain chatbot.

• Name the system and the edition. Say which game and which version you're playing at the start, and say it again when a ruling looks off.
• Give the GM the rules you're using. Paste in the section that matters, such as the grappling rules or a monster's stat block, before the scene that needs it.
• Roll and add your own modifiers. Give the GM your total and show the sum, so a missing modifier is easy to spot.
• Ask for the target number before you roll. A number named first is harder to bend afterwards.
• Check the numbers when a fight turns. Damage, hit points and conditions are worth a quick look at the big moments. Correct a mistake as soon as you see it, briefly and once.

Can an AI GM Run a Game by the Book?

An AI GM can run a game by the book when the written rules decide every result, the player rolls their own dice and the result stands. By the book is an aim, and it still allows the odd mistake, so a good AI GM owns a slip in the game and puts it right.

We aim to run Fable Smith by the book, because an AI GM needs those guardrails.

Does Fable Smith Run Games by the Book?

It aims to: it waits for your rolls, your system's rules judge each one, and the result stands. It runs games of fifth edition (5e), Draw Steel, MUNE or a freeform story with no rules system, in Discord.

Fable Smith is vivid where it matters: a world that moves, and a GM who knows what to show you. When a bandit swings at you on a rope bridge, the GM waits for your roll, and if it fails, you're hanging from the ropes with the bandit above you.

• You roll your own dice. When a moment calls for a test, the GM asks for the roll and waits. You can press the Roll button under its post, use the Discord dice bot you already use, or use the /test command, which fills in your modifiers from your character sheet in 5e and Draw Steel.
• The rules judge the roll, and the roll stands. Your result is judged by the rules of your system and is never softened.
• You should rarely need to correct the GM. When it does slip on a rule, it owns the mistake in the game and puts it right.

Fable Smith is in a free closed beta, and today you play on your own, with the GM in your own Discord channel. It's slow, since a turn can take minutes, and it still makes mistakes sometimes.

To test any AI GM's handling of dice in a few minutes, try the risky-move test in Can an AI Run Your Solo Campaign?.

Want a GM that aims to run your game by the book and waits for your rolls? Join the waitlist for the free closed beta on the Fable Smith page.

Sources

• D&D Beyond forums, ChatGPT is not a Good DM, for a chatbot GM settling everything with a bare d20.
• EN World forums, AI GMs, for an AI using the wrong edition's wording.
• TechRadar, Matt Evans, As a human D&D Dungeon Master, AI will never replace me, 2 August 2026, for every roll succeeding with Gemini.
• Anthropic, Context windows, for how much text a language model can read at once.
• The Forge & The Fable, Fable Smith, for every claim about Fable Smith.