Skip to content

Add unwrapped dict task inputs#48

Merged
Volv-G merged 5 commits into
masterfrom
piforge/tangle-compile-oss-migration/dict-unwrapped-task-inputs-aefbc0b
Jul 23, 2026
Merged

Add unwrapped dict task inputs#48
Volv-G merged 5 commits into
masterfrom
piforge/tangle-compile-oss-migration/dict-unwrapped-task-inputs-aefbc0b

Conversation

@Volv-G

@Volv-G Volv-G commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds a first-class @task(unwrap=...) API for Python-authored tasks that need to receive a caller-provided dict as explicit component inputs instead of one JsonObject input.

@task(image="python:3.12", unwrap="run_data")
def combine(run_data: dict[str, str]) -> str:
    ...

combine(run_data={"shop": shop_task.Output, "catalog": catalog_task.Output})

The compiler now flattens the traced call-site dict into synthetic inputs such as run_data__shop and run_data__catalog, so normal argument/edge emission can wire each dict value independently.

Mechanism

  • Extends @task(...) with unwrap="param" and unwrap=["param", ...].
  • At trace time, flattens marked dict arguments into explicit task arguments (param__key) and records the exact key order per task call site.
  • Persists a hydrate-safe local_from_python.unwrapped_inputs schema in the generated components sidecar:
    • original parameter name;
    • synthetic input prefix/name;
    • caller-provided key list/order;
    • inferred Tangle type from dict[..., V];
    • optional flag.
  • Uses the same persisted schema in component generation during both compile and hydrate, so regenerated component inputs are deterministic without access to the original Python call site.
  • Re-wraps flattened CLI args back into the original dict inside the generated runtime Python wrapper before calling the user function.
  • Adds schema-hashed sidecar fragments for unwrapped tasks (task-name--<hash>) so the same function can be called with different dict key sets without component schema collisions.
  • Preserves function-like introspection attrs across CallableRef.named()/bind()/with_annotations() so schema generation works for named task call sites.

Tests

Adds coverage for:

  • @task(unwrap=...) API normalization/validation.
  • Type inference from dict[str, V] into flattened input types.
  • Generated runtime source re-wrapping flattened inputs into the original dict.
  • End-to-end component generation of flattened inputs.
  • Pipeline compile sidecar emission with local_from_python.unwrapped_inputs.
  • Compile-time flattening of task arguments and task-output edges.
  • Schema-hashed fragments for different key sets of the same function.
  • Hydrate forwarding of unwrapped_inputs for compile↔hydrate round-trip determinism.
  • Edge cases: empty dicts, non-dict annotations, invalid names/keys, and argument collisions.

Version

Bumps tangle-cli from 0.1.6 to 0.1.7 so this can be tagged/released after merge.

Design reference

Implements the design tracked in the PiForge project wiki page design-dict-unwrapping-task-inputs.

Local validation

  • uv lock --check
  • uv run ruff check packages/tangle-cli/src/tangle_cli/component_from_func.py packages/tangle-cli/src/tangle_cli/component_generator.py packages/tangle-cli/src/tangle_cli/pipeline_compiler.py packages/tangle-cli/src/tangle_cli/pipeline_hydrator.py packages/tangle-cli/src/tangle_cli/python_pipeline/ref.py packages/tangle-cli/src/tangle_cli/python_pipeline/task.py packages/tangle-cli/src/tangle_cli/python_pipeline/graph.py tests/test_component_from_func.py tests/test_pipeline_compiler.py tests/test_pipelines_cli.py tests/test_packaging.py
  • uv run pytest805 passed, 4 warnings

Assisted-By: devx/b49d7847-4e8a-45eb-8304-dacd6e78458e
@Volv-G
Volv-G requested a review from Ark-kun as a code owner July 23, 2026 15:38
Volv-G added 4 commits July 23, 2026 12:08
Assisted-By: devx/b49d7847-4e8a-45eb-8304-dacd6e78458e
Assisted-By: devx/b49d7847-4e8a-45eb-8304-dacd6e78458e
Assisted-By: devx/b49d7847-4e8a-45eb-8304-dacd6e78458e
Assisted-By: devx/b49d7847-4e8a-45eb-8304-dacd6e78458e
@Volv-G
Volv-G merged commit f1d2dee into master Jul 23, 2026
2 checks passed
@Volv-G
Volv-G deleted the piforge/tangle-compile-oss-migration/dict-unwrapped-task-inputs-aefbc0b branch July 23, 2026 20:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant