Step 1
Private task set
Saved tasks, each tied to a pull request your team merged
Step 2
Managed benchmark runs
Short-lived Fullbeam sandboxes that use your own model provider keys
Step 3
Recommendation
Both setups on the same tasks, with a recommendation for each kind of work
Step 4
GitHub check
A check on the exact commit, one summary comment and a rollout plan your own tools carry out
Isolated Fullbeam sandboxes · Short-lived workspaces · Your own model provider keys
Benchmark runs happen in isolated, short-lived sandboxes that Fullbeam manages. They use the repository access you authorize and your own model provider keys. GitHub stays the source of truth for commits and pull requests. Your existing GitOps, MDM, agent vendor or internal platform still handles deployment and rollback.
Can do
Cannot do
Signing in tells us who you are. It does not give you access to any repository. The active GitHub App installation decides which repositories are included, and Fullbeam separately checks that your GitHub account is currently a member of each one. A workspace role can't get around that check. Installation owners choose the repositories. Each reviewer owns only their own checkpoints.
| Permission | Why we need it | What it cannot do |
|---|---|---|
pull_requests: write | Post review comments and check runs labeled as coming from the App ("via Fullbeam on behalf of @you"). Needed for chapter comments and the reviewer checkpoint check. | Cannot merge, close or reopen pull requests. Cannot act as a user. Every write is labeled as coming from the Fullbeam bot. |
checks: write | Create the Fullbeam Qualification check on the exact commit, linked to the Decision Report (sign-in required). The check only blocks merges if a GitHub admin makes it a required check. | Cannot modify existing checks created by other apps or CI systems. |
contents: write | Read pull request diffs and limited repository context. During a benchmark run you authorized, Fullbeam makes a short-lived checkout of the exact commit inside an isolated managed sandbox. If someone in your workspace chooses to publish an attempt for review, Fullbeam creates new branches under fullbeam/reviews/ and opens a draft pull request between them. | Cannot push to, force-push or merge your existing branches. It only creates new fullbeam/reviews/ branches, and only when a person asks it to. Cannot access repositories you didn't select during installation. Cannot keep the benchmark checkout after the attempt ends. |
issues: read | Read the issue linked to a merged pull request so it can become a task description. Only used for repositories where an owner turned on issue capture, or when you add an issue and pull request pair yourself. | Read-only. Cannot create, edit, comment on or close issues. |
metadata: read | GitHub requires it for every App installation. It provides pull request metadata (title, description, number, state) used to build the Story. | Read-only. No write access to repository settings. |
Webhooks received
Every webhook delivery is signature-verified, size-limited and processed only once per delivery GUID.
Runtime data from Claude Code, Codex and OpenCode is connected separately, and only after a user authorizes it. When Fullbeam records what actually ran, it keeps the link to the version of your agent setup it observed, plus a limited set of session details. Missing or partial data is shown as a gap in the record. We never fill it in from code or Git history. GitHub stays the source of truth for commits, pull requests and review outcomes.
Runtime collection records a limited set of facts: session lifecycle, tools, capabilities, permissions, verification, duration and clean Git state, along with which sources each fact came from. Prompts, responses, source code, command text and raw tool output are not part of the data we require. Missing data stays marked as unavailable or partial. We don't guess it or show it as zero.
Benchmark runs are configured for private, short-lived Daytona sandboxes, with the EU as the target region. Your model provider keys stay encrypted behind the Fullbeam model gateway, and the sandbox only receives a short-lived token for that one attempt. Before production use, we still need recorded proof from the deployed system for region placement, network egress limits, cleanup and key rotation. You can't choose a hosting region today.
Fullbeam stores versioned agent setups built from Git, records of which setup was assigned (as reported by outside systems), limited facts about what actually ran, exact Git and GitHub identities, pull request revisions, checks, reviews, the Story versions that were cited and each reviewer's own checkpoints. Dependency analysis may list the files at the exact revision and read up to 40 supported text files, 1 MiB in total. It does not clone the full repository.
Every setup record, runtime view, Story, source and write action is scoped to an account and a repository. Server-side checks require a current link to your GitHub identity plus membership of the repository. Row-level security protects ordinary database reads, and changes to the GitHub App installation revoke repository access that no longer applies. Admin database clients skip row-level security only inside narrow services that run the same account and repository checks themselves.
Today an internal outbox records each account's GitHub actions, with their payloads and delivery state, so they can be sent and retried safely. A customer-visible audit log that also records who acted and which installation credential was used is planned. It does not ship today.
Each item below is labeled Current when we can show the control in our code, Verification requiredwhen it still has to be proven in the deployed system before release, or Planned when it isn't built yet.
Fullbeam sends pull request diffs and metadata to AI models to write Story chapters, risk tags and citations. When an owner imports merged pull requests as benchmark tasks, it also uses them to draft each task's spec. We use these providers:
We update this list when providers are added or removed. Agents in benchmark runs reach your own model provider through the Fullbeam model gateway, using your keys, so that traffic runs on your own provider account.
Sentry handles error monitoring. Funnel and operational telemetry stay in Fullbeam's own Supabase and ClickHouse systems.
When Fullbeam posts a comment or check to GitHub for you, it is labeled "via Fullbeam on behalf of @your-handle". The label is honest attribution, not cryptographic proof of who wrote it. You make the final review decision (approve, request changes, merge) yourself in GitHub, with your own credentials. Fullbeam never submits a GitHub review approval under your name.
Ask for the data-flow diagram, a permissions review, retention details or a security questionnaire through the contact form. In our reply, we'll say which controls we can show in our code, which have proof from the deployed system and which are still planned.
Fullbeam is invitation-only in private preview. Invited teams connect the repositories they choose and their own model provider keys, then set up a benchmark with a defined scope. We share any Fullbeam plan limits, what the results cover, retention and support terms before any paid commitment.
Model usage is billed to your own provider account. Creating a workspace does not give Fullbeam permission to deploy or enforce your coding-agent setup.