Self-hosted AI code review GitHub Action

The MergeJudge GitHub Action runs the whole review inside your own GitHub Actions runner. It doesn't call the hosted MergeJudge service and doesn't need a MergeJudge account. It talks to the GitHub API with the workflow token. If you configure an AI provider, it sends the review prompt to that provider using your own key. With no key, it runs the deterministic rules only.

Every pull request gets a lane:

• auto: low risk (docs, styles and similar paths, under the size limit).
• review: needs a human.
• block: don't merge until a human has looked.

The lane comes from deterministic rules (protected paths, size, labels). The AI step adds line comments and rubric results, and it can only raise a lane, never lower it. The result is posted as a sticky comment and a mergejudge/lane check.

The Action and the review engine are open source (MIT): gauravsharmaGG/mergejudge-runner.

1. Add your AI key as a secret

Store the key as a repository (or organisation) secret. Use the GitHub UI under Settings → Secrets and variables → Actions → New repository secret, or the GitHub CLI:

gh secret set MJ_API_KEY

The secret name is up to you. The workflow passes it in as an environment variable, and api_key_env tells MergeJudge which variable to read. MergeJudge doesn't log or post your key.

Supported provider values: anthropic, openai, zai, openai-compatible (any Chat Completions endpoint, set base_url), or none for rules-only. See LLM providers for example configs.

2. Allow the Action to open the learning PR

The weekly learning run opens a pull request. By default, GitHub's GITHUB_TOKEN can't create PRs. Turn on Settings → Actions → General → Workflow permissions → Allow GitHub Actions to create and approve pull requests. You can skip this if you don't want learning proposals. Triage and feedback still work without it.

3. Add the workflow

# .github/workflows/mergejudge.yml
name: MergeJudge
on:
  pull_request:
    types: [opened, synchronize, reopened, ready_for_review, labeled, unlabeled]
  issue_comment:
    types: [created, edited]      # feedback checkboxes arrive as "edited"
  schedule:
    - cron: "0 9 * * 1"           # weekly learning proposal, Mondays 09:00 UTC
  workflow_dispatch: {}           # run the proposal by hand
permissions:
  contents: write                 # learning branch + proposal PR (contents: read is enough for triage only)
  pull-requests: write
  checks: write
jobs:
  lane:
    # Skip issue comments that aren't MergeJudge feedback (saves minutes).
    if: >
      github.event_name != 'issue_comment' ||
      contains(github.event.comment.body, 'MergeJudge') ||
      startsWith(github.event.comment.body, '/mergejudge')
    runs-on: ubuntu-latest
    steps:
      # Only needed for deep mode (step 4). Safe to remove otherwise.
      - uses: actions/checkout@v4
        if: github.event_name == 'pull_request'
      - uses: gauravsharmaGG/mergejudge-runner@v1
        env:
          MJ_API_KEY: ${{ secrets.MJ_API_KEY }}   # omit for rules-only
        with:
          provider: anthropic                     # or openai, zai, openai-compatible, none
          model: your-model-name                  # check your provider's docs for current names
          api_key_env: MJ_API_KEY

The Action picks what to do from the event (mode: auto):

Event	Mode	What happens
pull_request	triage	Review the PR, post the comment, the check and line comments
issue_comment	feedback	Record a lesson from ticked boxes or /mergejudge learn …
schedule, workflow_dispatch	propose	Open or update the learning proposal PR

Inputs: github_token, app_id / app_private_key (optional GitHub App credentials), mode, config_path (default .mergejudge.yml), provider, base_url, model, api_key_env. Outputs: lane, proposal_pr.

Without a .mergejudge.yml, built-in defaults apply. The config is always read from the PR's base branch, so a PR can't change the rules that review it.

Fork pull requests

For PRs from forks, GitHub gives the workflow a read-only token and no secrets. There's no key, so the review is rules-only, and posting the comment fails. To avoid failed runs, limit the review to same-repository branches by adding this condition to the job:

if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository

(Combine it with the issue_comment filter above using && if you keep both.)

Don't switch to pull_request_target with a checkout of the PR head. That runs untrusted code with your secrets.

4. Deep mode: give the model the rest of the repo

In the default diff mode, the model sees the changed hunks, your policy files and rubrics. Deep mode adds the code the change touches. Set it on the base branch:

# .mergejudge.yml
review:
  mode: deep

and keep the actions/checkout step. The Action then:

• indexes the checked-out files in memory, for this run only;
• finds importers, imports, callers, files that mention changed names, strings and env var names (up to two hops out);
• sends those snippets to your AI provider next to the diff, in a single pass.

Limits: tracked files only. .env* files, .pem and .key files, common lockfiles, node_modules, build output folders, binaries and files over 100 KB are not indexed, and symlinks are never followed. At most 5,000 files, 50 MB or 2 minutes are indexed, and a partial index is reported. Without a usable checkout, the review falls back to context mode (files read through the GitHub API) and says so.

5. Learning: feedback → weekly proposal PR

1. Feedback. The MergeJudge comment ends with checkboxes (Correct / Too lenient / Too strict / Wrong reason / Missed something, plus a box for each failed rubric item). When someone with write access ticks one, or comments /mergejudge learn <note>, the issue_comment run records a lesson in lessons/lessons.jsonl on the mergejudge-learning branch. Nothing is written to the reviewed PR. In the Action, merges and reverts alone don't create lessons. Only explicit feedback does.
2. Weekly proposal. The scheduled run reads the lessons. Once there are at least learning.propose_after (default 10), it opens or updates a PR from mergejudge/proposal-learning. It may suggest more lanes.protected paths or new rubric notes in .mergejudge.yml, with the evidence listed.
3. You decide. The proposal PR is labelled do-not-merge and needs-human and is never approved or merged by MergeJudge. A guard rejects any edit that would loosen the rules.

More: Learning and AGENTS.md and policy files.

Where your data goes

With the Action, there are only two parties: GitHub and the AI provider you configure.

Data	GitHub	Your AI provider	MergeJudge servers
PR diff (minus diff_exclude, e.g. lockfiles, build output, images)	yes (it's your repo)	yes, when a provider is configured	no
PR title and body	yes	yes	no
Policy files (AGENTS.md, REVIEW_POLICY.md, …)	yes	yes	no
Related code snippets (context and deep modes)	yes	yes	no
Lessons (lessons.jsonl)	yes, on the mergejudge-learning branch	no	no
Your API key	stored as a GitHub secret	sent to the provider to authenticate	no

So your code does leave your runner. It goes to the AI provider you chose, under your account and that provider's data terms. Pick a provider and plan whose data handling you're comfortable with, or point openai-compatible at an endpoint you run yourself. With provider: none (or no key), there is no AI step and the diff goes nowhere except GitHub.

The MergeJudge hosted service is never called by the Action.

Versions

Pin @v1 to get compatible updates, or pin a full commit SHA if your policy requires immutable references. The Action runs a committed bundle (action/index.cjs) on Node 24.

Other ways to run MergeJudge

• Hosted GitHub App: no workflow file. Reviews run on the MergeJudge service with your provider key.
• Run on your own machine: the hosted service plus the open-source runner doing the AI step with your own login.
• Self-hosting the MergeJudge Worker itself isn't public yet.