Threats to the runs and the workflows

← Security design · Threats and countermeasures

What can go wrong around the AI, in the runs of the workflows, their packages and their logs, with the countermeasure and the evidence of each.

Threat	Countermeasure	Evidence
A package of the Gemini CLI or the Copilot CLI, or one that they depend on, is replaced on npm's registry, or a new release of one is broken or malicious	analyze installs only the lock files of engines/, with npm ci, which takes each package at its locked version, refuses one whose SHA-512 hash isn't the locked one and runs no script of a package; an update of a lock file merges only once the required check engine contracts of its pull request passed, which verifies the signatures of npm's registry and the attestations of every package that npm installs from it on GitHub's Ubuntu x64 runner, and that npm refuses another hash, as the packages of the engines shows	test_the_lock_file_pins_every_package_with_its_hash, test_analyze_runs_the_engine_with_its_secret_alone, test_a_failed_install_says_what_npm_said in tests/test_engines.py; test_the_lock_file_passes_when_its_signatures_verify_and_npm_refuses_a_hash in tests/test_engine_contract.py
An issue keeps the assistant busy, so that no analysis of the repository finishes, because each one reads many issues; or a repository's tens of thousands of issues and pull requests do	An analysis reads every open issue but only the max_issues closed issues and the max_discussions discussions that changed last, in pages of a hundred without pull requests, and a reply only its issue. A first analysis learns from the max_analyzed issues analyzed last, from at most 5 pages of the comments written last, and asks for the permission of at most 20 people behind a 👎, never of a bot. Each text is read up to GitHub's 65,536 characters, by patterns that look at each character a bounded number of times; the AI reads at most 20,000 characters of each text of its issue, and HTML comments are removed as GitHub hides them, also one that starts a line and never ends	test_the_context_reads_every_open_issue_and_the_closed_ones_that_changed_last, test_the_discussions_end_at_their_limit, test_no_text_makes_reading_an_issue_slow, test_no_text_makes_the_cleaning_slow, test_forms_read_as_they_always_did in tests/test_assistant.py; property tests against the earlier patterns in tests/test_properties.py; fuzzing with Atheris in fuzz/fuzz_issue_text.py, which fails on a reading that takes longer than half a second
Someone steers the assistant with a command: mutes it, closes an issue as a duplicate of their own, asks the AI to post something, or makes it analyze again and again	A command counts only from a maintainer or the reporter of the issue; from anyone else, it is an ordinary comment. The reporter may only say not-duplicate and ask once per issue for a new analysis, while it waits for triage, within the limits of the day, which record the run in the marker of the assistant's own analysis. The question of /assistant ask reaches the AI in comment.md, never in the prompt, and its answer passes every check of apply, once for each question; the event job runs no AI	test_a_command_is_the_first_line_of_a_maintainer_s_or_the_reporter_s_comment, test_a_command_of_anyone_else_is_an_ordinary_comment, test_the_reporter_may_ask_for_a_new_analysis_once, test_the_reporter_s_again_counts_for_the_limits_of_the_day, test_the_answer_to_a_maintainer_s_question_is_checked_and_posted_once, test_an_answer_goes_only_to_a_maintainer_s_question in tests/test_assistant.py; test_a_whole_comment_mentions_nobody_and_links_only_allowed_places in tests/test_properties.py
Someone starts the assistant by hand to have it answer an issue again and again, spending the tokens of the maintainer: a collaborator with write access, an app with the right to run workflows, or someone who runs a maintainer's run again	The job event asks GitHub for the role of the person who started the run, or who ran it again, before it writes anything and before the AI runs: only write, maintain and admin start a run, and only maintain and admin override the one-comment rule; an app's own token has no role. Without the override, a triage or follow-up that has its comment stops without the AI. The job apply checks the role again before it posts a second comment, and the summaries name each override. The instruction of a run follows the task as the words of that person, and the issue stays material	test_only_someone_with_write_access_starts_a_run_by_hand, test_only_a_maintainer_may_override, test_a_run_by_hand_without_override_leaves_an_answered_issue_alone, test_the_person_who_runs_it_again_needs_the_role_as_well in tests/test_by_hand.py
A label name or the model of config.toml adds arguments to the command line of the engine, or breaks it, so that no issue gets an analysis	The schema writes an apostrophe of a label name as \u0027, which ends no quotes and reads back as the same JSON. check and the engine command refuse a model that is not a name of letters, digits and ._:@/[]-, and one that starts with a dash, which the engine's action would read as another flag	test_the_schemas_fit_into_the_engine_arguments, test_a_model_that_would_add_arguments_is_refused in tests/test_assistant.py; test_a_model_that_would_add_arguments_fails_the_check_and_the_engine in tests/test_setup.py
Anyone opens many issues or discussions, also from many accounts and all at once, or comments again and again after a release, so that the paid engine runs without end or uses up a subscription that other repositories share (denial of wallet)	The event job starts an analysis, a first answer to a discussion, a follow-up or a reply after a release only while the 24 hours before hold fewer than max_per_day new issues and discussions of the categories that get answers of people who don't maintain the repository, follow-ups and replies after a release; a new issue counts every issue opened before it, also one that a limit kept from the AI, so neither a flood at once nor issues that keep the AI from answering get past the count. A new issue also needs its reporter to have opened fewer than max_per_author_per_day, and the AI reads only release_replies comments of the reporter after each release. The event job records each follow-up and reply after a release as it starts, in the marker of the assistant's own comment that the run answers, which only the assistant can change, so neither runs that still go on nor deleted comments get past the count. Over a limit, the issue keeps its labels and needs-triage for the maintainer, and the run says so in a notice. A limit that is no whole number fails check and stops the run instead of lifting the limit	test_a_new_issue_beyond_the_daily_limit_waits_for_the_maintainer, test_only_the_new_issues_of_reporters_before_this_one_count, test_the_answers_of_the_day_count_toward_the_daily_limit, test_a_follow_up_counts_while_it_still_runs, test_one_reporter_gets_three_analyses_a_day, test_the_ai_reads_two_comments_of_the_reporter_after_the_release, test_deleted_comments_after_the_release_get_no_more_replies, test_maintainers_are_never_limited, test_a_limit_that_is_no_whole_number_stops_the_run in tests/test_assistant.py; test_config_problems_are_found in tests/test_setup.py
Someone comments while a reply of the AI waits in line, so that it judges their comment instead of the one that started it: a "still broken for me too" that reopens an issue whose reporter just confirmed the fix, or a question of someone else that marks the issue as waiting for its reporter	The context step and the apply job read the comment of the run's event. The AI judges that comment alone, in comment.md, and apply reads it again and acts only when its author is the reporter (release-reply) or a maintainer (maintainer-reply); a run by hand judges the newest such comment	test_a_reply_reads_the_issue_and_the_comment_that_started_it, test_a_release_reply_acts_only_on_the_reporter_s_comment_that_it_judged, test_a_maintainer_reply_acts_only_on_the_maintainer_s_comment_that_it_judged, test_context_and_apply_judge_the_comment_of_the_event in tests/test_assistant.py
A job that writes to discussions breaks the analyses of a repository whose template grants discussions only reading, or a compromised step of another job writes there	A called workflow that asks for more than its caller grants doesn't start, so the jobs of the issues only read discussions, and reusable-discussion-assistant.yml, which only discussion-assistant.yml calls, has the one job that writes there: apply, which runs no AI, behind Harden-Runner. Its job analyze is the one of the issues, which a test keeps equal	test_only_the_apply_job_of_the_discussions_writes_to_discussions, test_the_discussions_analyze_as_the_issues_do in tests/test_templates.py
Text of an issue runs as a shell command in a workflow	Every input goes through the environment; no expression of untrusted text stands in a run: script	test_untrusted_text_never_reaches_a_shell_script, test_the_action_passes_every_input_through_the_environment
A pull request runs its code with the permissions of the lifecycle	The lifecycle checks out the default branch and runs no code of the pull request	test_merges_and_releases_start_the_lifecycle_without_running_pull_request_code
A compromised action or tool in a job with a write token sends the token elsewhere	Harden-Runner blocks every host but api.github.com and github.com in every job without the AI, before any other step, and lets the findings reach GitHub's results service of their cache besides, and turns off sudo and containers so that no step lifts the block; the Action test runs the action behind the same block	test_jobs_without_the_ai_reach_only_github, test_the_action_runs_in_ci_behind_the_block_of_the_templates in tests/test_templates.py
Harden-Runner goes on without its agent, as in an outage of GitHub, when StepSecurity is unavailable or when the agent doesn't start, so a job with a write token runs without the block	Every job that blocks stops at Make sure that Harden-Runner blocks, the blueprint's script, before any step that could reach a host or get a secret, and no step that runs after a failure runs without the block (decision 0029)	test_every_job_that_blocks_stops_when_harden_runner_doesnt, test_the_step_that_makes_sure_that_harden_runner_blocks_is_the_blueprint_s, test_the_step_stops_the_job_when_harden_runner_doesnt_block in tests/test_templates.py
A workflow of the default branch that runs untrusted code, such as one on pull_request_target, writes an entry into the cache of the findings: an archive whose paths replace the action that the job with security-events: write runs next, or a record that keeps the Findings workflow passing on an alert that opens	actions/cache unpacks every path of an archive, so only the job record, without any scope of the token, checkout or issue assistant, unpacks the cache, and the findings only look up the key of its newest entry, without downloading it; it hands on a record of alert numbers and times in at most 64 KB alone, through the environment. The record can only keep a run from failing: it names numbers and times, the run still names every open alert in its summary, with a warning for each that an earlier run reported, and dismisses only what .github/findings.toml accepts. A record that isn't one counts as none, so every open alert fails the run again; a newer entry than the record read keeps only that run from failing, writes no record, and the next run fails for the alerts that its record lacks; the alert stays in the Security tab (decision 0026)	test_a_broken_record_counts_every_open_alert_as_new, test_of_two_overlapping_runs_only_the_first_fails_for_a_new_alert, test_a_run_without_a_newer_record_fails_for_a_new_alert in tests/test_findings.py, test_the_findings_keep_the_record_of_the_reported_alerts_in_the_cache, test_no_job_that_may_write_unpacks_a_cache, test_only_a_checked_record_leaves_the_job_record in tests/test_templates.py
A pull request brings a real finding of a rule that .github/findings.toml accepts in the same file, so that the Findings workflow dismisses its alert and resolves its review thread before anyone reviews it, or adds an entry that accepts its own finding	After the scan of a pull request, the Findings workflow dismisses only alerts that findings.toml of the default branch accepts, from the checkout of workflow_run, never the file of the pull request, and only in a pull request of the repository itself, whose author can push to it and dismiss alerts anyway; a fork's pull request keeps its threads for a person. It dismisses no more than the runs of the default branch would after the merge, and an entry with a path keeps that narrow (decision 0028)	test_the_run_knows_the_pull_requests_whose_scan_started_it, test_the_scan_of_a_fork_s_pull_request_asks_github_nothing in tests/test_findings.py; test_the_findings_workflow_may_only_dismiss_alerts_and_runs_no_ai in tests/test_templates.py
A public log or timeline reveals a possible vulnerability	The Findings workflow names an open alert by number and link only, and can only dismiss alerts. The comment on a possible security report only points to private reporting, the timeline shows a fixed reason for its labels, it gets no priority, and the prompt asks for one neutral sentence without details in its summary, analysis and reason	test_other_findings_stay_open_and_name_only_number_and_link, test_the_findings_workflow_may_only_dismiss_alerts_and_runs_no_ai, test_a_possible_vulnerability_is_not_discussed_in_public, test_a_security_report_gets_no_public_priority
GitHub renames or ends its egress-firewall runner, a technical preview, so every analysis waits in the queue for a day, and a repository moves the AI to a runner without any block	The variable ISSUE_ASSISTANT_AI_RUNNER names a runner in its place without a release: the job event accepts only GitHub's runners of Ubuntu with a fixed version and names the hosts of .github/egress-firewall.yaml for Harden-Runner, which blocks every other host as the first step of analyze-in-reserve, the job that runs the steps of analyze there; analyze itself, on the firewall runner, has no step of Harden-Runner, whose agent would start there too. Before the engine runs, a step fails the job when requests to example.com still get through after a minute. The Canary fails when the job of the AI, analyze or, with the variable set, analyze-in-reserve, waits for longer than 30 minutes	test_the_ai_runs_on_another_runner_only_behind_harden_runner, test_no_job_that_may_run_on_the_firewall_runner_has_a_step_of_harden_runner, test_the_ai_runs_only_where_the_block_holds in tests/test_templates.py; test_harden_runner_gets_the_hosts_of_the_firewall_policy, test_a_runner_where_nothing_would_block_the_hosts_is_refused in tests/test_setup.py; tests/test_canary_queue.py
GitHub's egress firewall stops enforcing the policy without a warning: a change of its preview, a policy that it didn't load because the job may not read the repository, or mode: log, so the AI and the engine's secret run with open network access in every repository	Before every step that gets the engine's secret, in analyze of both reusable workflows and in replay of the Evaluate workflow, on every runner, the step Make sure that the network block holds asks for example.com, a host of no policy, and api.github.com: two failed requests to the one in a row and an answer of the other, within a minute, let the AI run, and the job stops otherwise. The Action test runs the step on ubuntu-24.04-firewall in every pull request and every week, records how the firewall refused the host, and runs it on plain ubuntu-24.04, where it has to stop the AI; the Canary runs it in its analyze (decision 0025)	test_the_ai_runs_only_once_the_network_block_holds, test_the_ai_runs_only_where_the_block_holds, test_every_job_that_makes_sure_of_the_block_runs_one_script, test_the_network_block_of_the_ai_s_runner_runs_in_ci in tests/test_templates.py; the jobs the network block of the AI's runner and the probe of the network block on a runner without one of the Action test
A repository's copy of a workflow drifts into something weaker	check compares every workflow with its template, and the tests pin the security rules of the templates	tests/test_templates.py