Build a GitHub PR reviewer

What you will build

You will create a GitHub pull request (PR) reviewer fleet from a Definition file. It will receive PR events and post review comments.

Before you begin

If you need hosted access, join the waitlist. Public website access links open the waitlist, including Sign in links. If you already have access, sign in to the dashboard to manage your workspace. Choose Manage account for your profile and sign-in security. You can also open account settings directly. You need Bun 1.4 or later, an agentsfleet account, and a GitHub repository that you can manage. Your workspace owner must enable a model with credentials, an API format, an endpoint, and token limits. Copy its eligible ref from GET /v1/workspaces/{workspace_id}/models?purpose=authoring; step 6 calls this <MODEL_REF>. Create a GitHub token that can read the repository and post comments. Keep the token ready for step 4. These steps use a repository webhook and a saved token. For a connected GitHub App, follow Configure a webhook instead.

Steps

Install the agentsfleet command-line interface.

bun install --global @agentsfleet/cli

installed @agentsfleet/cli with binaries:
 - agentsfleet

Check the installed version.
agentsfleet --version
<varies>
Sign in through a browser on the same computer as your terminal.
agentsfleet login
Connect your terminal Open this link on this computer: login_url: <varies> Check code: <varies> Waiting for authorization… (Esc to cancel) browser: opened <varies> Finalizing sign-in… ✓ signed in as <varies> — <varies>
Compare the check code in your browser with your terminal. Choose Authorize only if you started the sign-in and both codes match. For a remote terminal or another device, use agentsfleet login --no-open and follow manual sign-in. Login keeps your previous workspace if it is still available; otherwise it selects the first available workspace. Check agentsfleet workspace show before saving secrets. Use agentsfleet workspace use <WORKSPACE_ID> to switch, with an identifier from agentsfleet workspace list.
Save the GitHub secret in your active workspace. Replace <GITHUB_TOKEN> with your repository token. Generate <GITHUB_WEBHOOK_SECRET> as a random value of at least 32 bytes; GitHub will use the same value in step 9.
This inline example can leave credentials in shell history. For regular use, pipe JSON from your secret manager to agentsfleet secret create github --data=@-.
agentsfleet secret create github --data='{"host":"api.github.com","token":"<GITHUB_TOKEN>","webhook_secret":"<GITHUB_WEBHOOK_SECRET>"}'
✓ Secret 'github' stored in vault.
host binds the outgoing token to api.github.com. The client stores the values without printing them again.

Write the fleet’s Definition to pr-reviewer.json.

{
  "schema_version": 1,
  "description": "Reviews GitHub pull requests and posts review comments.",
  "instructions_md": "Read `repo` and `number` from the event.\nFetch the pull request diff from https://api.github.com/repos/{repo}/pulls/{number} with http_request and ${secrets.github.token}.\nPost one review with event COMMENT that lists correctness, security, and test gaps.\nNever approve, merge, or close a pull request.",
  "config": {
    "name": "pr-reviewer",
    "x-agentsfleet": {
      "triggers": [{"type": "webhook", "source": "github", "events": ["pull_request"]}],
      "tools": ["http_request"],
      "credentials": ["github"],
      "network": {"allow": ["api.github.com"]},
      "budget": {"daily_nanos": "2000000000"}
    }
  }
}

daily_nanos caps spending at 2 dollars a day. Fleet budgets explains the unit.

Open a private draft from the file and review it. Use the <MODEL_REF> from Before you begin. This file starts a draft without making an authoring-model request.
agentsfleet chat --definition pr-reviewer.json --model <MODEL_REF> --no-input --review --name pr-reviewer
✓ Draft <DRAFT_ID> created Instance: pr-reviewer Runtime model: <varies> <varies> Review digest: <DIGEST>
The review prints the setup, runtime model, and required access. Check them before you save. The authoring model and runtime model can differ.
Save the fleet with the digest from the review.
agentsfleet chat <DRAFT_ID> --no-input --save --name pr-reviewer --review-digest <DIGEST>
<varies> ✓ Fleet pr-reviewer: active
A fleet that reads installing is waiting on a setup check, such as an integration grant to approve. The lines after it name each check and its next step.
Read the fleet’s webhook URL. <FLEET_ID> comes from agentsfleet list.
agentsfleet --json fleet show <FLEET_ID>
<varies>
The webhook_urls entry with source set to github holds the URL.
Add the webhook in GitHub. Open Repository settings → Webhooks → Add webhook. Use the webhook URL from step 8 as the payload URL. Choose application/json as the content type. Use the same webhook secret from step 4. Select Pull requests as the event. GitHub now sends PR events to your fleet.
Open a PR in the repository. GitHub sends the event after you open the PR. The fleet then attempts the review and posts its comment.

Verify it works

<FLEET_ID> comes from agentsfleet list.

agentsfleet logs <FLEET_ID>

Event Stream
  <varies>  webhook:github  <varies>

The activity stream should show a GitHub event. Check the PR for the fleet’s review comment. Review text and tool calls vary by model. If the run fails, inspect the event and follow Troubleshoot a fleet.

What you learned

You installed the client, signed in, and saved a secret. You reviewed a fleet’s setup, saved the fleet, and connected GitHub to it.

Command line

Tutorials

Build a GitHub PR reviewer

What you will build

Before you begin

Steps

Verify it works

What you learned

​What you will build

​Before you begin

​Steps

​Verify it works

​What you learned

​Related pages

What you will build

Before you begin

Steps

Verify it works

What you learned

Related pages