Skip to content

CMP-4475: Install OpenShift Virtualization operator for OCP-Virt e2e#78

Open
Vincent056 wants to merge 6 commits into
ComplianceAsCode:mainfrom
Vincent056:cmp-4475-ocpvirt-e2e
Open

CMP-4475: Install OpenShift Virtualization operator for OCP-Virt e2e#78
Vincent056 wants to merge 6 commits into
ComplianceAsCode:mainfrom
Vincent056:cmp-4475-ocpvirt-e2e

Conversation

@Vincent056

@Vincent056 Vincent056 commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator

What

Adds the setup and plumbing to run the CIS OpenShift Virtualization (cis-vm-extension) CEL profiles e2e on a cluster that doesn't ship CNV.

  • CNV installer (helpers/virtualization.go, gated on -install-virt, default false): creates the openshift-cnv namespace, an OperatorGroup, a Subscription for kubevirt-hyperconverged (channel stable, redhat-operators), waits for the HyperConverged API, creates the HyperConverged CR (retrying to tolerate the operator webhook), and waits until it is Available. Idempotent; leaves existing CNV in place. Called from helpers/Setup.
  • CEL ProfileBundle support: bumps the vendored ComplianceAsCode/compliance-operator API to v1.9.0 (adds ProfileBundle.spec.celContentFile) and sets pb.Spec.CELContentFile for the ocp4 bundle when -cel-content-file is provided, so CEL profiles are parsed. The k8s stack is aligned to match v1.9.0 (k8s.io/* v0.30 -> v0.35.3, controller-runtime v0.20.4 -> v0.23.3) — no harness code changes were needed for the upgrade.
  • Flags + Makefile wiring: -install-virt, -cel-content-file, plus CONTENT_IMAGE / LOG_DIR make variables passed to make e2e-profile and make e2e-profile-remediations. LOG_DIR is optional in CI (the openshift/release prow lane relies on the default /logs/artifacts, which is writable there); only local runs need to override it.
  • Log-dir robustness: the assertion/mismatch/report writers now create the log directory before writing, so runs don't fail when it doesn't already exist.
  • Opt-in live tests TestInstallVirtLive (RUN_VIRT_LIVE=1) and TestCELProfileBundleLive (RUN_CEL_PB_LIVE=1).

Usage, the proposed CI lane, and further details are in a PR comment.

Validation (OCP 4.21 + AWS + CNV)

  • make e2e-profile passed end-to-end on a fresh cluster (247s): the target self-installs CNV (HyperConverged Available), creates the ocp4 ProfileBundle with the typed celContentFile (VALID), scans the cis-vm-extension CEL profile to completion, records the 5 rule results, and cleans up the scan binding.
    make e2e-profile PROFILE=cis-vm-extension PRODUCT=ocp4 \
      INSTALL_OPERATOR=false INSTALL_VIRT=true \
      CEL_CONTENT_FILE=ocp4-cel-content.yaml CONTENT_IMAGE=<image> LOG_DIR=<dir>
    
  • TestInstallVirtLive and TestCELProfileBundleLive both passed against the cluster (CNV install; typed CELContentFile -> ProfileBundle VALID -> CEL profile with 5 rules).
  • go build / go vet / gofmt clean.

Follow-ups (not in this PR)

  • Assertion generation for the CEL + mixed rule set (CEL platform + manual who-can + OpenSCAP node).
  • CI lane in openshift/release (e2e-aws-ocp4-cis-vm-extension) — see the PR comment.
  • Apply the scanner RBAC from CMP-4425 in the flow.

Jira: https://issues.redhat.com/browse/CMP-4475

🤖 Generated with Claude Code

Add an optional setup step that installs the OpenShift Virtualization (CNV)
operator and creates the HyperConverged CR before the tests run, so the CIS
OpenShift Virtualization profiles can be scanned on a cluster that does not
already have CNV. Gated by the new -install-virt flag (default false) and
idempotent if CNV is already present.

Also adds an -install-virt and -cel-content-file config plumbing; wiring the
CEL ProfileBundle (spec.celContentFile) is marked as a follow-up TODO pending
a bump of the vendored compliance-operator API to >= v1.9.0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@openshift-ci

openshift-ci Bot commented Jul 2, 2026

Copy link
Copy Markdown

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

Vincent056 and others added 3 commits July 7, 2026 02:20
Retry HyperConverged CR creation to tolerate the operator webhook not being
ready right after the CRD appears. Add an opt-in TestInstallVirtLive
(RUN_VIRT_LIVE=1) that exercises the installer against a real cluster.

Validated on an OCP 4.21 cluster: CNV installs and HyperConverged reaches
Available in ~4 minutes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…docs

Expose INSTALL_VIRT and CEL_CONTENT_FILE make variables and pass the
corresponding -install-virt / -cel-content-file flags from the e2e-profile
and e2e-profile-remediations targets, so a CI lane can run the OCP-Virt CEL
profiles (which need CNV installed and a CEL content file).

Add docs/ocp-virt-e2e.md documenting the flags, local run, the CNV install
step, a proposed CI lane, and the on-cluster validation results.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Keep the Makefile wiring; the OCP-Virt e2e usage notes live in the PR
discussion instead of a checked-in doc.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Vincent056

Vincent056 commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator Author

OpenShift Virtualization (CIS VM Extension) e2e — usage & validation

The OCP-Virt profiles (e.g. cis-vm-extension) are CEL profiles that check OpenShift Virtualization resources. Running them e2e differs from the SCAP profiles in two ways, both handled by this PR:

  1. The cluster needs OpenShift Virtualization (CNV) installed. CI clusters (e.g. ipi-aws) do not ship CNV, so the suite installs it during setup.
  2. The profile is delivered via a CEL content file, not the XCCDF datastream, so the ProfileBundle needs spec.celContentFile set.

Flags

Flag Make var Default Purpose
-install-virt INSTALL_VIRT false Install the CNV operator + HyperConverged CR during setup. Idempotent; no-ops if CNV already present.
-cel-content-file CEL_CONTENT_FILE "" Path (inside the content image) to the CEL content YAML, e.g. ocp4-cel-content.yaml. When set, a CEL ProfileBundle is created so CEL profiles are parsed.

Running locally

# 1. Build the content image with the CEL content bundle (in the content checkout):
./build_product ocp4 --datastream --cel-content=ocp4
#    -> build/ssg-ocp4-ds.xml and build/ocp4-cel-content.yaml
#    Package both into an image and push to a registry the cluster can pull.

# 2. Run the OCP-Virt profile e2e (installs CNV itself):
export PROFILE=cis-vm-extension PRODUCT=ocp4
export INSTALL_VIRT=true CEL_CONTENT_FILE=ocp4-cel-content.yaml
make e2e-profile   # go test also needs -content-image=<your image>

What the CNV install step does

installVirtualizationOperator (helpers/virtualization.go), gated on -install-virt:

  1. Creates the openshift-cnv namespace, an OperatorGroup, and a Subscription for kubevirt-hyperconverged (channel stable, source redhat-operators).
  2. Waits for the HyperConverged API to be served.
  3. Creates the HyperConverged CR (retrying to tolerate the operator webhook not being ready immediately) and waits until it reports Available.

Opt-in live test for just this path:

RUN_VIRT_LIVE=1 go test ./helpers -run TestInstallVirtLive -timeout 40m -v

Proposed CI lane (openshift/release, tracked separately)

- as: e2e-aws-ocp4-cis-vm-extension
  steps:
    cluster_profile: quay-aws
    test:
    - as: test
      commands: |
        export PROFILE=cis-vm-extension PRODUCT=ocp4
        export INSTALL_VIRT=true CEL_CONTENT_FILE=ocp4-cel-content.yaml
        make e2e-profile
    workflow: ipi-aws

LOG_DIR is not set in the openshift/release prow lane — CI's default /logs/artifacts is writable there, so it's omitted. Only local runs need to override it (e.g. LOG_DIR=/tmp/artifacts).

Validation (OCP 4.21 + AWS)

  • TestInstallVirtLive installs CNV and HyperConverged reaches Available in ~4 min (no metal nodes required for the operator/HCO config the CEL rules read).
  • With a CEL content image, a ProfileBundle using celContentFile: ocp4-cel-content.yaml parses VALID; the operator creates the cis-vm-extension CEL profile and its scannerType: CEL rules; a ScanSettingBinding runs the scan to completion and produces per-rule ComplianceCheckResults.

@Vincent056 Vincent056 marked this pull request as ready for review July 7, 2026 09:29
…tFile

Bump the vendored ComplianceAsCode/compliance-operator API from v1.7.0 to
v1.9.0 (the release that adds ProfileBundle.spec.celContentFile), aligning the
k8s stack to match: k8s.io/* v0.30 -> v0.35.3, controller-runtime v0.20.4 ->
v0.23.3, plus the openshift/api and openshift/client-go replaces from v1.9.0.

With the typed field available, set pb.Spec.CELContentFile directly in
ensureTestProfileBundles for the ocp4 bundle (removing the earlier TODO /
unstructured workaround), so the harness creates a CEL ProfileBundle for the
OCP-Virt profiles.

go build/vet/test-compile pass; the k8s/controller-runtime upgrade required no
code changes in the harness.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…g dir

- Add CONTENT_IMAGE and LOG_DIR make variables (passed as -content-image /
  -log-dir) so e2e-profile can test a custom content image (e.g. one built with
  CEL content) and write artifacts to a writable directory off-CI.
- Create the log directory before writing assertion/mismatch/report artifacts
  (GenerateAssertionFileFromResults, SaveMismatchesAsYAML, GenerateMismatchReport)
  so runs don't fail when the dir does not already exist.
- Add opt-in TestCELProfileBundleLive validating the typed CELContentFile path.

Verified end to end on a fresh OCP 4.21 cluster: make e2e-profile installs CNV,
creates the CEL ProfileBundle (VALID), scans the cis-vm-extension profile, and
records results.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Vincent056

Copy link
Copy Markdown
Collaborator Author

/retest

@openshift-ci

openshift-ci Bot commented Jul 10, 2026

Copy link
Copy Markdown

@Vincent056: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-rhcos4-moderate db231a2 link true /test e2e-aws-rhcos4-moderate
ci/prow/e2e-aws-ocp4-cis db231a2 link true /test e2e-aws-ocp4-cis
ci/prow/e2e-aws-ocp4-stig db231a2 link true /test e2e-aws-ocp4-stig
ci/prow/e2e-aws-openshift-platform-compliance db231a2 link true /test e2e-aws-openshift-platform-compliance
ci/prow/e2e-aws-openshift-node-compliance db231a2 link true /test e2e-aws-openshift-node-compliance

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@vickeybrown

Copy link
Copy Markdown

Looks good! Only problem is the go version is mismatched with the CI so the tests are failing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants