Skip to content

feat(pipeline-run): surface API HTTP failures as clean errors#38

Open
arseniy-pplx wants to merge 1 commit into
TangleML:masterfrom
arseniy-pplx:transfer/clean-sdk-http-errors
Open

feat(pipeline-run): surface API HTTP failures as clean errors#38
arseniy-pplx wants to merge 1 commit into
TangleML:masterfrom
arseniy-pplx:transfer/clean-sdk-http-errors

Conversation

@arseniy-pplx

@arseniy-pplx arseniy-pplx commented Jul 20, 2026

Copy link
Copy Markdown

Summary

  • add a shared HTTP-error formatter and surface_http_errors context manager for SDK commands
  • report API status, reason, method, URL and a bounded, whitespace-collapsed response body as one-line command errors, with URL userinfo/credential query parameters stripped and response-body secrets redacted before truncation
  • redact credential-bearing query keys transport-neutrally, including presigned/SAS signature keys (signature, X-Amz-Signature, sig, awsaccesskeyid, googleaccessid) and bare api_key, kept in sync with the sibling transport helper so the guarantees do not drift
  • redact recognizable key=value / key: value credential assignments in non-JSON bodies (form-encoded, plain text, HTML); the ambiguous colon form is gated to opaque-looking values so non-sensitive keys (page=2) and natural-language detail (credential: invalid) are preserved
  • wrap pipeline-run manager API calls (status, details, cancel, graph-state, logs, list/search, export, submit) and annotation commands while preserving client-level recovery, post-submit run recovery, and fallback behavior
  • report per-run graph-state failures with the same formatted message in each result's error field

Context

Pipeline-run SDK commands currently leak raw requests tracebacks when the API returns a non-2xx response. The command layer already converts PipelineRunError to a clean non-zero exit; this change makes HTTP failures on sdk pipeline-runs commands cross that boundary instead. Client-internal recovery (the 404 run-id -> execution-id fallback, submission-id run recovery after a failed submit) keeps handling the statuses it can; only unrecovered errors are formatted.

Behavior note for downstream hook subclasses: on_poll_error and on_submit_error now receive PipelineRunError (with the original requests.HTTPError chained as __cause__) instead of the raw HTTPError when an API call fails with an HTTP status error. Non-HTTP submit errors reach on_submit_error unchanged.

Testing

  • uv run pytest tests/test_sdk_http_errors.py tests/test_pipeline_runs_cli.py
  • uvx ruff check on the touched source files
  • uv lock --check
  • git diff --check

@arseniy-pplx
arseniy-pplx marked this pull request as ready for review July 20, 2026 18:11
return f"Tangle API request failed: {exc}"
request = response.request
target = (
f"{request.method} {request.url}"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(AI-assisted)

Please redact the request URL and backend detail before rendering this error.

This currently interpolates both request.url and response.text verbatim. For example, formatting a request to https://alice:hunter2@api.test/x?access_token=QUERYSECRET with an error body containing credential=BODYSECRET exposes all three values in the resulting stderr line.

Could we sanitize URL userinfo/credential query parameters and pass structured response bodies through the existing sensitive-key redaction before truncation? Ideally the requests/httpx error paths in #38, #45, and #46 should share transport-neutral redaction helpers so their guarantees do not drift.

@Volv-G Volv-G left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(AI-assisted)

Approved per reviewer direction. The inline credential-redaction feedback remains available for follow-up.

@arseniy-pplx
arseniy-pplx force-pushed the transfer/clean-sdk-http-errors branch from 3d0c42a to 0f8300e Compare July 24, 2026 09:59
Pipeline-run SDK commands leaked raw requests tracebacks whenever the API answered non-2xx. Manager and annotation methods that call the client now run under a shared error-surfacing context manager that re-raises requests.HTTPError as PipelineRunError with the status, reason, method, URL and a single-line, trimmed response body, so sdk pipeline-runs commands (including submit and annotations) exit non-zero with a one-line message. Client-internal recovery such as the 404 run-id to execution-id fallback and post-submit run recovery keeps handling the statuses it can; per-run graph-state failures reuse the formatted message in each result's error field. on_poll_error and on_submit_error hooks now receive PipelineRunError with the original HTTPError chained as __cause__.

Harden the shared redaction so echoed URLs and bodies cannot leak credentials: URL query redaction now also covers presigned/SAS signature keys (signature, X-Amz-Signature, sig, awsaccesskeyid, googleaccessid) and bare api_key, matching the sibling transport helper so the guarantees do not drift; and non-JSON response bodies (form-encoded, plain text, HTML) now have recognizable credential key=value / key: value assignments redacted, with the ambiguous colon form gated to opaque-looking values so natural-language detail is preserved.
@arseniy-pplx
arseniy-pplx force-pushed the transfer/clean-sdk-http-errors branch from 0f8300e to a90101c Compare July 24, 2026 11:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants