Skip to content

Replace github-script with file-based approach to avoid large JSON output max limit#177

Open
lindseywild wants to merge 2 commits intomainfrom
replace-github-script-file-based-argmax
Open

Replace github-script with file-based approach to avoid large JSON output max limit#177
lindseywild wants to merge 2 commits intomainfrom
replace-github-script-file-based-argmax

Conversation

@lindseywild
Copy link
Copy Markdown
Contributor

@lindseywild lindseywild commented Mar 26, 2026

Summary

Replaces the actions/github-script@v8 "Set results output" step with a file-based bash/node approach to avoid Linux's ARG_MAX limit when processing large JSON payloads (~138KB+).

Problem

The previous implementation interpolated large filings JSON directly as a CLI argument to node via github-script. This exceeded the ARG_MAX limit on Linux runners, causing:

An error occurred trying to start process 'node' ... Argument list too long

Changes

  1. Replaced github-script@v8 step with two bash steps:

    • A step that writes filings/fixings to temp files via heredocs (avoids CLI arg limits)
    • A step that processes the data via a node heredoc script and writes scanner-results.json
  2. Added results_file output — a new composite action output pointing to the JSON file, for consumers that need to handle large datasets without output size limits.

  3. Switched cache saving from the value-based gh-cache/cache action to the file-based gh-cache/save action, copying scanner-results.json to the cache key path first.

Backward compatibility

  • The results output is still set via GITHUB_OUTPUT heredoc delimiter for existing consumers.
  • The new results_file output is additive and opt-in.

[Staff only] Resolves github/accessibility#10354

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the root composite GitHub Action to avoid Linux ARG_MAX failures when handling large JSON payloads by moving results processing to a file-based flow, while preserving the existing results output and adding a new file-path output for large datasets.

Changes:

  • Replaces the actions/github-script@v8 “Set results output” step with bash + inline Node that reads/writes JSON via files.
  • Adds a new results_file output pointing to the generated scanner-results.json.
  • Switches caching of results from value-based gh-cache/cache to file-based gh-cache/save by copying scanner-results.json to the cache path first.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@lindseywild lindseywild marked this pull request as draft March 26, 2026 18:15
@lindseywild lindseywild force-pushed the replace-github-script-file-based-argmax branch 2 times, most recently from bb1f458 to 1b3db1c Compare March 27, 2026 15:11
@lindseywild lindseywild changed the title Replace github-script with file-based approach to avoid ARG_MAX limit Replace github-script with file-based approach to avoid large JSON output max limit Mar 27, 2026
@lindseywild lindseywild marked this pull request as ready for review March 27, 2026 15:19
@lindseywild lindseywild marked this pull request as draft March 27, 2026 15:19
The 'Set results output' step used actions/github-script@v8, which
interpolated large filings JSON (~138KB+) as a CLI argument to node.
This exceeded Linux's ARG_MAX limit, causing 'Argument list too long'
errors.

Changes:
- Replace github-script step with bash heredoc + node heredoc approach
  that writes data to temp files, avoiding CLI arg limits
- Add results_file output for consumers needing large dataset support
- Switch cache saving from value-based to file-based using gh-cache/save
- Keep results output for backward compatibility

Resolves github/accessibility#10354

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@lindseywild lindseywild force-pushed the replace-github-script-file-based-argmax branch from 1b3db1c to b55df88 Compare March 27, 2026 15:31
@lindseywild lindseywild marked this pull request as ready for review March 27, 2026 15:48
@lindseywild lindseywild marked this pull request as draft March 27, 2026 15:48
@lindseywild lindseywild marked this pull request as ready for review March 27, 2026 16:19
The 'Output cached value' step cats a JSON file into GITHUB_OUTPUT
followed by an EOF delimiter, but if the file has no trailing newline
the delimiter lands on the same line as the content, causing:
'Invalid value. Matching delimiter not found EOF'

Add a bare echo to ensure a newline before the closing delimiter.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

@abdulahmad307 abdulahmad307 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the code looks good. I think we need to go through the run logs and make sure issues were opened (or re-opened) correctly before merging

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants