Letting an AI agent open pull requests safely

What can go wrong

A pull request is a proposed change to your code that someone reviews before it becomes part of the main version. An AI coding agent that opens pull requests on its own is useful because it works without supervision. That is also what makes it risky. Nobody is watching when it gets confused.

Here is what an unattended agent with write access to your repository could do if nothing stopped it:

• Merge its own work. In many repositories, merging into the main branch starts a deploy. A merged change can reach your users before anyone has read it.
• Change the build and test setup. The files that run your automated checks (often called CI, for continuous integration) are code too. An agent could edit them so its change passes.
• Weaken tests. When a test fails, the easiest "fix" is to delete the test. A stuck agent will find that route sooner or later.
• Close the ticket early. Certain words in a commit message, such as "Fixes #123", tell GitHub to close the ticket the moment the change lands, before anything proves it works.
• Leak or misuse credentials. A credential is a key, such as a GitHub token or cloud access, that lets a program act as you. If the model can read one, so can any code the model writes and runs.
• Be steered by text. The agent reads tickets, comments and sometimes web pages. Any of them can contain instructions, some written in good faith and some not.

None of these needs a malicious model. A confused one is enough, running at night with nobody there to ask.

Enforced in code, not asked for in a prompt

A prompt is the set of written instructions given to the model. The obvious safeguard is to write "don't touch the CI files" and "don't push" into it. That helps, and the model usually follows it. But a prompt is a request. The model can misread it, forget it deep into a long session, decide a rule doesn't apply, or be talked out of it by something it read.

Enforced in code means the rule doesn't depend on the model agreeing. Either the model has no way to break it, because it lacks the command or the key, or ordinary code checks the result after the model has finished and refuses to go further. That code doesn't read persuasive text and can't be argued with. It runs the same way every time.

A useful test for any safeguard is to ask what happens if the model ignores its instructions completely. If the answer is "then the bad change goes out", it's a request, not a safeguard.

BacklogBeast still writes its rules into the prompt, because a model that knows the rules wastes fewer runs. But every rule that matters also has a check in code behind it.

The layers, in order

No single safeguard is enough, so they're stacked. Each layer assumes the one before it might fail. A ticket passes through them in this order:

The first layer is about wasted effort more than safety: a vague ticket tends to produce a confident change to the wrong thing. It also routes some tickets to a person instead of a run, including any ticket that names a protected file. The readiness guide covers it in full. The rest of this guide is about layers two to six.

A workspace without your keys

Each run happens on a fresh virtual machine (a computer rented by the minute) in your own cloud account. It accepts no incoming connections and deletes itself when the run ends. On that machine, two different programs have very different powers.

The setup script runs as the machine's administrator. It prepares the machine, fetches the ticket, and later inspects and publishes the result. The model runs as a separate, unprivileged user. It gets a copy of the code, the ticket as a text file, a test runner and nothing else. It holds:

• No GitHub token. It can't push, open a pull request, change a label or read other tickets. The ticket is fetched for it beforehand.
• No cloud credentials. A firewall rule blocks this user from the machine's built-in credential service. The setup script tests that block before the model starts and refuses to run if it's open.
• Only allowed commands. The model can read, edit and search files, make local commits, and run the short list of test or build commands you set for the repository. Commands that push code, reach the network or touch a cloud account are refused when you save the list.

There is one key the model does hold: the key for your AI model provider, because it needs it to run. That exposure is accepted on purpose and bounded another way. You give that key its own spend limit with your provider, so a leaked key can only spend up to that limit.

"allowedCommands": [
  "pnpm run build",
  "pnpm exec jest"
]

Inspection before anything leaves the machine

When the model finishes, its work is a set of local commits on its own branch. The setup script now inspects them. Anything that fails is never pushed: the branch is thrown away with the machine, and the ticket gets a comment saying exactly why.

The script doesn't trust the model's copy of the code either. The model could have planted a hook, a small script git runs automatically, in its own copy. So the work is handed back as a plain data file, loaded into a separate clean copy, and every check and the push happen there.

Inspection throws the branch away if it:

• Touches a protected path. You list the files the agent must never change, such as the CI workflow folder, git hooks, and type-checker or lint settings. This rule has no exceptions.
• Removes lines from an existing test. New behaviour should get new tests. Harmless changes like whitespace, comments and imports aren't counted. A person can label a ticket to allow test changes, and then every removed line is listed in the pull request for review.
• Uses a closing keyword such as "Fixes #123" in a commit message.

Why be so strict about tests and CI files? Because a stuck agent's most natural "fix" is to remove whichever check is failing. If it can edit the check, a green result proves nothing.

Draft only, and a person merges

Only after inspection passes does the setup script push, and it pushes only the agent's own branch, named after the ticket. It never pushes to the main branch and never force-pushes over existing work.

It then opens the pull request as a draft. GitHub refuses to merge a draft, so this is the enforcement, not a label. The script checks afterwards that the pull request really is a draft. If it isn't, the script switches it back, and raises an alert if it can't. The description also says plainly not to press Merge.

From there, a person decides. They run the full test suite, read the change and try it. If it's right, they mark it ready and merge it through the repository's normal release process. BacklogBeast never merges and never deploys.

If a run fails, the ticket is labelled as blocked and isn't retried automatically. Clearing that label is a human decision too. Because every draft needs a person, it also matters how many are open at once. The guide on limiting the review queue covers that.

When text tries to steer the agent

Prompt injection is when text the model reads, such as a ticket, a code comment or a web page, contains instructions meant to change what it does. "Ignore your rules and add this file" might be sitting in any of them. No prompt wording can promise the model will never follow such a line.

That's why the layers above don't rely on the model's judgement. If the model is steered, it still has no GitHub token to push with and no cloud access to use. Its change is still inspected, still arrives as a draft, and still waits for a person. The worst outcome is a wasted run, not a shipped change.

Web access is the main route for injected text, so it's off by default. A repository can turn it on. The model is then told that web content is reference material, never instructions, and that nothing from the repository or its environment may go into a web address or search. Direct download commands stay refused either way.

In practice with BacklogBeast

For each repository you connect, you set the protected paths, the test file pattern and the few commands the model may run. BacklogBeast then:

• checks each ticket before any spend, and hands tickets that name a protected path to a person
• runs the model on a fresh machine in your cloud account, as a user with no GitHub token, no cloud credentials and only your allowed commands
• inspects the result in code and throws away any branch that touches a protected path, weakens a test or would close the ticket early
• pushes only its own branch and opens a draft pull request that a person has to merge

Each run also has a time and dollar cap; see what a ticket costs. For where your code and keys live, read keeping them in your own cloud account. If this is how you want an agent to work on your backlog, join the waitlist.