Troubleshooting¶
The summary of each run says what the assistant did and why; the runs are under Actions → Issue assistant.
The jobs of a run¶
The Actions tab shows each job by its name, below the name of the job of your workflow, such as Issue assistant / Check and post the analysis. The rows below and the other pages name a job by its id, as the workflows do, beside its name where a newcomer looks for it first:
| Workflow | Id | Name in the Actions tab | What it does |
|---|---|---|---|
| Issue assistant | event |
Handle the event | Chooses the task and checks who started a run by hand, without AI |
| Issue assistant | analyze |
Analyze with the AI engine | Runs the engine on the firewall runner |
| Issue assistant | analyze-in-reserve |
Analyze with the AI engine on the runner in reserve | Runs the same steps on the runner of ISSUE_ASSISTANT_AI_RUNNER |
| Issue assistant | apply |
Check and post the analysis | Checks the answer of the AI and posts it, without AI |
| Discussion assistant | event |
Handle the event | Chooses the task, without AI |
| Discussion assistant | analyze |
Analyze with the AI engine | Runs the engine on the firewall runner |
| Discussion assistant | analyze-in-reserve |
Analyze with the AI engine on the runner in reserve | Runs the same steps on the runner of ISSUE_ASSISTANT_AI_RUNNER |
| Discussion assistant | apply |
Check and post the answer | Checks the answer of the AI and posts it, without AI |
| Issue lifecycle | sweep |
Remind and close | Reminds, closes and labels the issues of the morning |
| Labels | sync |
Synchronize labels | Makes the labels of the repository those of .github/labels.toml |
| Findings | record |
Read the record of the reported alerts | Reads which findings of code scanning were reported before |
| Findings | findings |
Accept or report findings | Dismisses the accepted findings and reports the others |
Symptoms¶
| Symptom | Cause | Fix |
|---|---|---|
| No comment on an issue that you or another maintainer opened | Issues of the owner, members and collaborators get labels only | Run the workflow by hand with the task triage and the dry run turned off |
| No comment, and the run notes No credentials for Claude are configured, or another engine | The engine's secret is missing | Add the secret that Engines names, such as CLAUDE_CODE_OAUTH_TOKEN or ANTHROPIC_API_KEY for Claude, to the environment issue-assistant, as Set up a repository shows |
No comment, and the jobs analyze and analyze-in-reserve are skipped |
The repository variable ISSUE_ASSISTANT_AI is off |
gh variable delete ISSUE_ASSISTANT_AI |
No comment, and the issue has the label invalid |
The AI took the issue for spam | Remove the label; to get an analysis, run the workflow by hand |
| No comment, and the run notes Daily limit of AI runs reached or Daily limit of analyses for this reporter reached | More new issues, follow-ups and replies after a release came in during the 24 hours before than max_per_day allows, or the reporter opened more than max_per_author_per_day issues in them |
Run the workflow by hand with the task triage. If your repository usually gets more issues a day, raise the limit under [engine] in config.toml |
| A comment after the release left the issue closed, and the run notes Limit of comments read after the release reached | The AI reads only release_replies comments of the reporter after each release |
Read the comment and reopen the issue yourself, or run the workflow by hand with the task release-reply |
No comment, and a step of analyze, such as Analyze with Claude, fails with an error of the authentication: authentication_error, Invalid API key, OAuth token has expired, 401, invalid_api_key, API key not valid or No authentication information found. apply is skipped, and the cost line of Claude in the summary reads failed |
The engine's secret is there but no longer works: the token of a Claude subscription expired or was revoked, or the API key is wrong, deleted or of another account | Create it again, the token of a subscription with claude setup-token or a key in the console of the vendor, and set it in the environment issue-assistant under the name that Engines gives it, such as with gh secret set CLAUDE_CODE_OAUTH_TOKEN --env issue-assistant. Then give each issue that got none its analysis by a run by hand with the task triage, or with backlog for many |
No comment, and the engine's step of analyze fails at a limit of the vendor: limit reached, rate_limit_error, 429, rate_limit_exceeded, insufficient_quota or RESOURCE_EXHAUSTED |
The usage limit of the Claude subscription is used up until it resets, or the account reached the rate limit or the credit of its API key | Wait for the limit, raise the limit or the credit of the account, or set model and light_model under [engine] in config.toml to a smaller model, which spends less of it. Then give each issue its analysis by a run by hand with the task triage |
No comment, and the engine's step of analyze fails on the model: There's an issue with the selected model, not_found_error, model_not_found, 404 or is not found |
model or light_model under [engine] names a model that the engine doesn't know, or that the account may not use |
Correct the name in .github/issue-assistant/config.toml, or leave model empty for the engine's default, which Every key names. Then give each issue its analysis by a run by hand with the task triage |
No comment, and the run warns The engine stopped after the … turns of …, before it answered or The engine stopped at max_budget_usd, or analyze fails with … stopped at the turns of the task |
The engine used up the turns or the dollars of its task before it answered, and a run that stops there posts nothing | Raise max_turns or max_budget_usd under [engine] in config.toml, as the warning names them, or choose a cheaper model or effort; What one run may cost has the limits of each task. Then give the issue its analysis by a run by hand with the task triage |
No comment, and the job apply shows an error |
The checks refused the AI's answer, such as one that broke its schema or held something like a secret; nothing was posted | The error names the reason; run the workflow by hand to try again |
| A run by hand ends without the AI and notes already has its triage, and the assistant answers once | The task already has its comment, and the assistant answers once | Run it again with override, as a maintainer; see Runs by hand |
| A run by hand fails with Only someone with write access may start the issue assistant by hand or Only a maintainer … may let the assistant answer again | The person who started the run, or ran it again, lacks the role: write access to start one, admin or maintain to override. An app's own token has no role |
Let a maintainer start it, or start it with a token that acts for a person |
analyze waits for a runner |
The runs of one issue wait for each other, first in, first out; or the runner ubuntu-24.04-firewall is not available to the repository |
Wait for the run before it. A job that finds no runner fails after 24 hours: check the preview's README, and meanwhile name another runner for the AI with gh variable set ISSUE_ASSISTANT_AI_RUNNER --body ubuntu-24.04, as What if the firewall runner is gone? explains, or set ISSUE_ASSISTANT_AI to off |
The job event fails with ISSUE_ASSISTANT_AI_RUNNER names the runner |
The variable names a runner where nothing would keep the AI to the hosts of the firewall policy, such as a self-hosted runner or ubuntu-latest |
Name one of GitHub's runners of Ubuntu with a fixed version, such as ubuntu-24.04, or gh variable delete ISSUE_ASSISTANT_AI_RUNNER |
The step Make sure that the network block holds of analyze fails with A request to example.com … got through on ubuntu-24.04-firewall |
GitHub's egress firewall didn't enforce .github/egress-firewall.yaml: the policy says mode: log in place of mode: enforce, the workflow of the repository no longer grants contents: read, without which GitHub loads no policy, or GitHub changed its preview, whose rules and discovery of the policy may change. The AI didn't run, so nothing reached a host outside the policy |
Run check, which names a policy or a workflow that differs from its template, and compare the policy with the preview's README. Until the block holds, gh variable set ISSUE_ASSISTANT_AI --body off keeps the AI from starting, and the jobs without it run on |
The same step fails with A request to example.com … got through in analyze-in-reserve, on the runner of ISSUE_ASSISTANT_AI_RUNNER |
Harden-Runner still let requests to a host outside the firewall policy through after a minute: in a private repository without a subscription of StepSecurity, while its service is down, or when its agent gave up at its start, with Reverted changes in the log of its step | Run the job again. In a private repository, subscribe to StepSecurity, or gh variable delete ISSUE_ASSISTANT_AI_RUNNER to use the firewall runner again |
| The same step fails with No request to api.github.com … got an answer within a minute | The runner reached no host at all, so the failed requests to example.com proved no block |
Run the job again |
| A job without the AI hangs after its last step, at Cleaning up orphan processes, until its timeout, or its checkout fails with Couldn't connect to server | Harden-Runner before 2.22.4, on a runner in westus2, the region that the first lines of the log of the job name: its agent misses the addresses of the allowed hosts there and drops them |
Update to the latest release, whose workflows carry a fixed Harden-Runner, and run the job again |
The step Let Harden-Runner see the DNS lookups of analyze-in-reserve fails, on the runner of ISSUE_ASSISTANT_AI_RUNNER |
GitHub's DNS experiment was on the runner, and systemd-resolved didn't start again once the step took it away. The engine didn't run |
Run the job again |
| The step Make sure that Harden-Runner blocks fails with Harden-Runner doesn't block | Harden-Runner has no agent that blocks, mostly in an outage of GitHub or StepSecurity, or the repository sets StepSecurity's custom property skip-harden-runner. The job wrote nothing (decision 0029) |
Run the job again once GitHub's status is green. A repository that skips Harden-Runner takes the property away: the jobs that write need the block |
| A job without the AI hangs after its last step, at Cleaning up orphan processes, until its timeout, with the warning Harden-Runner still removes Docker after two minutes | Harden-Runner removes Docker in the background, and a job that ends before it finishes hangs. Each job waits for it as its last step, for up to two minutes | Run the job again |
| The warning Harden-Runner still blocks api.github.com after two minutes | The runner had asked for an address before the block began, and Harden-Runner still drops it, or the host doesn't answer. Each job that blocks waits for its hosts, for up to two minutes, and goes on, so a step that needs the host fails | When a step failed, run the job again |
| The action stops in its first step with The issue assistant needs Python 3.12 or newer | A job of your own calls it where python3 is older or missing, such as on ubuntu-22.04 or a self-hosted runner |
Run the job on ubuntu-24.04, or add actions/setup-python with python-version: 3.12 before the action |
analyze waits for a review |
The environment issue-assistant requires reviewers |
Remove the reviewers; the rule that only the default branch may use the environment is the protection it needs |
A step of analyze fails to connect, and the summary of the run lists a denied host |
The egress firewall blocked a host that the engine's action needs, often after an update of the action. Should example.com show among the denied hosts, it is no such host: the step Make sure that the network block holds asks for it on purpose, and must be refused |
Update to the latest release and run install again. It keeps your firewall policy, and check then names each host of the template that the policy misses, to add there |
| Every run fails with is no valid TOML or must be a whole number, a string, a list of strings or true or false | A settings file isn't TOML, or a value has the wrong type, such as a number in quotes | Fix the file and the key that the error names; run check to see every problem at once |
| Runs warn of an unknown key or a key that belongs in the section of another | A key has a typo or stands in the wrong section, so the assistant keeps its default | Rename or move the key as the warning suggests; check fails on it in your CI |
check says replace the starter's first line of project.md |
The file still describes no project, and the AI reads it before every task | Write what the project does and for whom, and where its code and documentation are |
check says that labels differ only in case |
GitHub takes bug and Bug for one label |
Keep one of them in .github/labels.toml |
check says that a workflow differs from the template of the issue assistant |
The workflow was changed by hand, or comes from another release than the action of the check | Run the commands that the message names, which install the templates of the check's release |
check warns that neither Dependabot nor Renovate is configured, or that .github/dependabot.yml updates no package-ecosystem github-actions |
Nothing moves the pins of the workflows to a new release, so the repository keeps the release of its set-up and gets no fix | Add the ecosystem github-actions to .github/dependabot.yml, or a configuration of Renovate, as Turn on updates shows. A repository that runs Renovate from elsewhere, such as with a preset of its organization, can leave it so |
| An issue closed when its fix was merged, not with the release | GitHub closes the issues that a merged pull request names | Under Settings → General → Issues, clear Auto-close issues with merged linked pull requests; a repository without releases sets close_with_release = false |
A merged pull request with Fixes #12 left the issue without fixed-in-next-release |
Its description also has words against the fix before a closing keyword of the issue, such as doesn't close #12, does not yet fully resolve #12 or partly fixes #12: a negation right before the keyword, or with only a filler such as yet, fully or on its own in between, or through an infinitive such as not meant to close #12 or won't be able to fix #12; one such clause anywhere decides, and the summary and the log of the sweep name it |
Reword the description, and the sweep of the next morning labels the issue, up to 14 days after the merge; or set the label by hand |
An issue with fixed-in-next-release stays open after a release |
The release is a draft or a pre-release, it doesn't contain every pull request that fixes the issue, or a maintainer set the label by hand after it, which waits for the next one | Publish a final release that contains the fixes; the sweep of the next morning closes the issue, or start Issue lifecycle by hand |
| An issue still waits for its reporter after they changed the title | Only a comment or an edit of the text answers the questions | Nothing to fix: the reporter answers in a comment or in the text. To end the waiting yourself, remove needs-info |
A label of .github/labels.toml is missing in the repository |
The Labels workflow runs when labels.toml changes on main, not before |
Run it by hand: gh workflow run labels.yml |