gh-152548: Convert test_audit to the test.support.isolated() decorator#152565
gh-152548: Convert test_audit to the test.support.isolated() decorator#152565serhiy-storchaka wants to merge 5 commits into
Conversation
Run a test in a fresh interpreter subprocess, so that it does not share global or interpreter state with the rest of the test run. It can decorate a test method (only that method runs in a subprocess) or a TestCase subclass (the whole class runs in one subprocess, with its setUpClass()/setUp()/tearDown()/ tearDownClass() running once there). Failures, errors and skips, including those of individual subtests, are reported for the test and show the original subprocess traceback. The subprocess inherits the parent's resource, memory and verbosity configuration, so that requires_resource(), bigmemtest() and similar behave the same in both processes. The test.support.running_isolated flag is true in the subprocess, so that fixtures can choose what to run there. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fixes the ruff F401 "imported but unused" lint failure for the re-exports. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Documentation build overview
8 files changed ·
|
|
@zooba, please look at this. I think it reflects the intention of the test, but is more clear. |
@isolated() always runs the test in a subprocess, so skip it in the parent process on platforms that lack subprocess support, the same way the tests it replaces were guarded by requires_subprocess(). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
zooba
left a comment
zooba
left a comment
There was a problem hiding this comment.
Overall, I love it. I trust your implementation (and CI to detect when it doesn't work yet), and it's just the extra imports during test cases that worry me. Maybe those are dealt with in a way I didn't spot.
|
|
||
| @support.isolated() | ||
| def test_marshal(self): | ||
| import_helper.import_module("marshal") |
There was a problem hiding this comment.
These are intentionally trying to skip the test if the module can't be imported, so it'd be nice to preserve that somehow (marshal is unlikely to be missing, of course, but I assume there are others here that are less unlikely)
| from test import support | ||
| from test.support import import_helper | ||
| from test.support import os_helper | ||
| from test.support import script_helper | ||
| from unittest.mock import ANY |
There was a problem hiding this comment.
This is the bit that worries me most about the change. Previously, we weren't importing all these for each test case, and now we are? I don't know that it'll change anything, but most critically it may lead to modules being imported early when we intended to import them during the test.
There was a problem hiding this comment.
os and unittest.mock were already imported in Lib/test/audit-tests.py, at the top level.
I moved the rest at the methods level. But the one big left -- test.support itself, which imports many other modules. Even if we handle it somehow, unittest will left. This requires thought.
Move the helper into a public test.support.isolation submodule (used as "from test.support import isolation"), drop the test.support re-export, and document running_isolated and isolated() under that module. Replay expected failures and forward subprocess durations to the parent, so an @expectedfailure isolated test is no longer misreported as an unexpected success and reported timings reflect the subprocess run. Add test.test_support.TestIsolated covering the outcomes, subtest reporting, traceback-as-cause, duration forwarding and the no-subprocess skip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
sys.addaudithook() cannot be undone, so each test needs a fresh interpreter. With @isolation.isolated() each test becomes an ordinary TestCase method using self.assert* directly in the subprocess, replacing the separate audit-tests.py driver script and the parsing of events out of its stdout. Per-test imports stay inside the methods, and a local requires_module() decorator checks optional modules in the parent process, so a missing module skips the test without first spawning the subprocess. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
serhiy-storchaka
force-pushed
the
test-audit-isolated
branch
from
e3dff23 to
8f064af
Compare
2 participants
Each audit test needs a fresh interpreter because
sys.addaudithook()cannot be undone. With@support.isolated()these become ordinaryTestCasemethods that useself.assert*directly in the subprocess, removing the separateaudit-tests.pydriver script and therun_test_in_subprocess()/do_test()/run_python()code that parsed events out of the subprocess's stdout.test_excepthookstays a small subprocess test (now inline viascript_helper): thesys.excepthookaudit event fires only for an exception reaching the interpreter's top level, which an@isolated()body (run under unittest) never does.This is built on top of #152551 (the
test.support.isolated()decorator). The first two commits belong to that PR and will drop out once it is merged and this branch is rebased.