poc: BigQuery - Add column updated_at to table execution_node

[yuechao-qin]

Summary

Adds a self-maintaining updated_at column to execution_node, needed as a change-tracking watermark for an upcoming BigQuery replication pipeline (hack-days prototype). The column stamps itself on insert and on every update via SQLAlchemy's insert_default/onupdate, so no application code needs to set it explicitly.

Changes

• backend_types_sql.py: Add _utcnow() helper and un-comment/replace the updated_at column on ExecutionNode with a mapped_column using insert_default=_utcnow and onupdate=_utcnow. onupdate fires on any UPDATE to the row, not just specific field changes — this matters because PipelineRunsApiService_Sql.terminate() only ever sets extra_data["desired_state"], never container_execution_status directly.

• database_migrations.py:

  • migrate_add_execution_node_updated_at_column: idempotent ADD COLUMN, using Alembic's batch_alter_table so the same code path covers SQLite/MySQL/PostgreSQL. Returns True only when it actually adds the column (no logging on the no-op path, since this runs on every startup).
  • backfill_execution_node_updated_at: idempotent bulk UPDATE ... WHERE updated_at IS NULL, stamping every pre-existing row with a single flat "now" timestamp.

• database_ops.py: Wires the migration into migrate_db(), running the backfill only in the same startup that actually added the column (guarded by the migration function's return value).

• Tests:

  • test_database_migrations.py: coverage for the migration (adds when missing, no-ops silently on second call, column is queryable afterward) and the backfill (backfills nulls, leaves existing values untouched, idempotent on second call).
  • test_sql_event_listeners.py: confirms updated_at is set on creation and bumped on both a status change and an extra_data-only change (no listener needed — plain onupdate covers it).
  • test_api_server_sql.py: regression test confirming terminate() bumps updated_at on a running descendant ExecutionNode, since it mutates extra_data rather than container_execution_status.

Testing

Ran the full test suite locally — all passing (449 tests).

Notes

This is a hack-days prototype (poc: prefix). The reverse migration (dropping the column) intentionally does not live in this repo — it's admin-only, testing/rollback tooling that lives entirely in oasis-backend since that's the only place with real admin auth.

[yuechao-qin]

• poc: BigQuery - Add column updated_at to table execution_node #302 👈 (View in Graphite)
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.