Skip to content

Threats and countermeasuresΒΆ

← Security design Β· Threats to the runs and the workflows

What an issue can try to make the AI do or write, with the countermeasure and the evidence of each.

Threat Countermeasure Evidence
An issue tells the AI to run a command, read a secret or reach a server Each engine runs with only tools that read, as the boundary of each engine shows, with a read-only token or none, and the egress firewall allows only the hosts of the engine of config.toml test_each_engine_can_only_read_the_checkout, test_only_the_analyze_job_runs_an_engine_and_only_with_read_permissions, test_the_firewall_template_allows_only_named_hosts in tests/test_templates.py
A text of an issue or a comment passes for the maintainer, for another comment or for the AI's earlier answer, with a heading like those of issue.md Each text stands in a code fence one backtick longer than its longest run of backticks, so that nothing in it can end the fence; only the assistant writes the headings with author, role and date, and the prompt says that a fence holds only that person's text test_no_text_forms_a_heading_or_a_comment_in_issue_md in tests/test_assistant.py; test_no_text_forms_the_structure_of_issue_md_or_hides_in_it in tests/test_properties.py
An issue hides instructions for the AI in characters that GitHub shows as nothing, such as Unicode tags, zero-width spaces or the controls of the direction of writing, also for the analyses of later issues The format characters of Unicode and the code points it calls default ignorable are removed from issue.md, the excerpts of the other issues and discussions, and the AI's answer before its checks; a notice in the log counts them test_the_invisible_characters_are_those_that_unicode_names, test_no_invisible_character_reaches_the_engine, test_the_answer_loses_its_invisible_characters_before_the_checks, test_the_engine_reads_no_text_that_github_hides, and test_no_file_of_the_project_hides_characters for the project's own files, in tests/test_assistant.py; test_no_text_forms_the_structure_of_issue_md_or_hides_in_it and test_the_engine_reads_no_text_that_github_hides in tests/test_properties.py
An issue makes the AI post a mention, a link, an image or HTML apply removes images, HTML and headings, turns mentions, e-mail addresses, references and addresses that GitHub would link elsewhere into code, and keeps only links to allowed sites and to issues that exist; it places the text in the comment so that its code stays code test_mentions_images_html_and_headings_are_removed, test_only_links_to_known_places_stay_links, test_tricky_text_is_cleaned_safely, test_what_github_would_link_elsewhere_becomes_code, test_the_whole_comment_mentions_nobody_and_links_nowhere_else, test_no_table_cuts_code_in_two in tests/test_assistant.py; property tests of the text, of the places it takes in a comment and of whole comments in tests/test_properties.py, judged by the oracle of tests/oracle.py; fuzzing with Atheris in fuzz/fuzz_sanitize.py and fuzz/fuzz_render.py
The AI posts a secret that it found or was given An answer with anything that looks like a token or a key is never posted, as What counts as a secret shows test_an_answer_with_something_like_a_secret_is_never_posted, test_ordinary_long_words_links_and_paths_are_no_secrets, test_a_file_that_git_tracks_is_no_secret in tests/test_assistant.py; test_a_token_anywhere_in_the_answer_is_refused, test_a_path_that_git_tracks_is_never_a_secret in tests/test_properties.py
An answer runs a workflow command in the log of the run, such as ::add-mask::, ::stop-commands:: or a forged ::error::, or its comment ends the code block of the preview in the summary of a dry run Every error and notice of the assistant is escaped as GitHub's Actions toolkit escapes it, and a key of the answer that the schema doesn't know shows as a short JSON string. The Codex action prints what Codex prints into the log, so Codex prints JSON events alone, one per line, and no log of its own (--json and RUST_LOG=off); the Gemini CLI and the Copilot CLI print nothing there, because analyze keeps their output. The summary turns workflow commands off with a random token while it prints the engine's words, and the fence of a preview is longer than any run of backticks in the comment test_no_text_of_the_engine_runs_a_workflow_command, test_a_workflow_command_keeps_its_message_on_its_line, test_each_summary_stops_the_commands_with_a_token_of_its_own, test_only_a_terminal_outside_a_runner_gets_the_summary_alone, test_a_preview_can_t_close_its_fence in tests/test_assistant.py; test_each_engine_can_only_read_the_checkout in tests/test_templates.py
An answer mentions someone or links elsewhere in a reason outside its comment: a line of the log and the summary, or the reason for a label or a duplicate on the timeline Such reasons are cleaned like a comment, joined into one line before the cleaning, so that no code block keeps a mention as it is, and shortened to 280 characters before the cleaning, so that no cut loses the backtick that keeps a mention in its code test_the_reasons_on_the_timeline_are_cleaned_like_comments, test_the_engine_s_reasons_in_the_log_are_one_cleaned_line, test_a_shortened_line_keeps_its_cleaning in tests/test_assistant.py; property tests in tests/test_properties.py; fuzzing with Atheris in fuzz/fuzz_sanitize.py
An issue steers the analyses of later issues into calling them its sure duplicates, so that they close after 3 days Only an open original that a maintainer opened or triaged gets the notice that closes a duplicate; any other becomes a suggestion for the maintainer. A comment of anyone, and a thumbs down of the reporter or of anyone who may push to the repository, keep the issue open, and so does an original that is closed or no longer vouched for when the 3 days are over test_a_sure_duplicate_of_an_untriaged_issue_is_only_suggested, test_a_thumbs_down_of_the_reporter_or_a_maintainer_keeps_the_issue_open, test_other_reactions_do_not_object, test_a_notice_against_an_untriaged_original_closes_nothing in tests/test_assistant.py
Someone writes a correction for the AI to learn from: a title that reads as a maintainer's correction or an instruction in corrections.md, or a πŸ‘Ž that makes a good analysis look wrong The script writes every line of corrections.md from GitHub's events, the markers of the assistant's comments and its reactions: the number, the labels of labels.md and fixed words, with the title as a JSON string without invisible characters, all in a code fence that no title can close. Only people with triage access can change labels, a πŸ‘Ž counts only from someone who may push, and the AI reads the file as data for issues like the listed ones, never as instructions test_a_first_analysis_learns_from_the_maintainers_corrections, test_a_duplicate_notice_is_objected_to_by_people and test_an_issue_learns_from_its_own_corrections_and_not_beyond_the_window in tests/test_assistant.py
A discussion makes the AI speak for the project in public: answer where the maintainers didn't ask for answers, answer again and again, mark its own answer as the accepted one, or post after a maintainer answered Only the categories of [discussions] answer_categories get an answer, none by default, and never a discussion of a maintainer or a bot. apply reads the discussion again and posts one comment, with every check of an analysis, only while it is open, unlocked, without an answer and without a maintainer's comment, and never marks it as the answer; the footer names the AI and that it can be wrong. A bug or a feature request only gets a link to the form of an issue, never one test_other_categories_maintainers_and_bots_get_no_answer, test_nothing_changes_while_no_category_is_named, test_the_first_answer_is_checked_and_posted_once, test_the_answer_waits_for_nothing_that_happened_meanwhile, test_the_text_of_the_ai_is_cleaned_as_every_analysis in tests/test_discussions.py; test_the_answer_to_a_discussion_is_one_comment_on_it in tests/test_graphql_contract.py; whole comments in tests/test_properties.py
An answer that breaks the schema changes the wrong labels or issues The answer is checked against the schema of its task, the labels and the issues that exist test_an_answer_that_breaks_the_schema_is_refused, test_any_answer_is_checked_or_refused