Invalid#22766
Closed
jianyizh wants to merge 11 commits into
Closed
Conversation
…nt division - Identifies arith.divui/arith.remui by scalar kernel args in gpu.func - Hoists magic/shift precomputation to host (before gpu.launch_func) - Adds magic/shift as new kernel arguments, replaces division with fast mul - 28% speedup on Intel Data Center GPU Max 1550 (1048576 work-items) - Benchmark and integration test files included - Detailed build & run guide in README_SCALAR_HOIST.md
…ONSTANT) - Forward dataflow analysis classifies every SSA value in gpu.func body - arith.divui/remui by SCALAR_ONLY divisor triggers magic/shift hoist - Supports multi-level scalar computation chains (not just block arguments) - getScalarOperand() extracts scalar operand from MIXED binary operations - Verified: 28% speedup on Intel Data Center GPU Max 1550
- 1024 threads, 32 subgroups, 100 iterations of 4 divisions each - Baseline: 2414 ns avg → Optimized: 1995 ns avg (21% speedup) - Pass adds 2 magic/shift args for divisor S (8→10 total args)
…r division BiasAdd: 1M elements (f32), urem+udiv by chw/hw → 8409→6718ns (+25%) GroupNorm: f16→f32 Welford scan + f32 compute + division by S → 17836→17286ns (+3%)
- N=1024 equivalent via 64 groups × 20 iterations - Baseline: 16,179ns avg → Optimized: 15,729ns (+2.8%) - Division is minor part of Welford-heavy kernel (but passes are correct) - BiasAdd: 8,499ns → 6,812ns (+25%) confirmed
- sum+sum_sq+affine norm with gamma/beta lookup per element - 4 x arith.divui by runtime scalar S per vec-4 per thread - Baseline: 30,538ns → Optimized: 21,811ns (+40%) - BiasAdd: +26% confirmed
- Shape: tot=67108864, C=16, N=16, chw=4194304, hw=262144 - 262144 blocks × 256 threads, each thread: remui + divui + fadd - Baseline: 588,682ns avg | Optimized: 498,727ns avg - Larger shape is bandwidth-bound, still 18% from division hoisting - Drop GroupNorm: math.rsqrt SPIRV lowering unsupported, workgroup attributions incompatible with ScalarHoist pass
- General claim: cross-boundary scalar hoisting via dependency classification + algebraic decomposition (5 rules) + cost model - Hoisting categories: int div (magic multiply), FP div (reciprocal), transcendentals (sqrt/exp/log), compound scalar expressions - Primary embodiment: integer division magic-multiply with measured 18% speedup on 64M-element BiasAdd (Intel Max 1550) - FP worked example: physics kernel, 89% per-work-item cycle reduction - Replaces custom dialect design with direct gpu/arith/math ops
More precise terminology aligned with GPU compiler community (uniform/divergent analysis). Affects title, problem definition, summary, and detectability sections.
Clarify that the method is realized through MLIR's GPU dialect as the device-aware IR, modeling host-device boundary, cross-device value transfer (kernel arg ABI), and precomputation scheduling.
Introduce minimal scalar_hoist dialect (allowUnknownOperations, no
TableGen) with two ops visible in IR dumps:
- scalar_hoist.precompute: wraps host-side magic/shift computation
in a region, taking scalar divisor as input, yielding magic+shift
- scalar_hoist.yield: terminator for precompute region
New pipeline: gpu-scalar-hoist -> lower-scalar-hoist -> spirv pipeline
- gpu-scalar-hoist: emits scalar_hoist.precompute ops on host side
- lower-scalar-hoist: inlines regions, erases dialect ops
BiasAdd 64M verified: ~500us optimized (consistent with prior results).
Files:
ScalarHoistDialect.h - dialect class (header-only, no TableGen)
LowerScalarHoist.cpp - lowering pass (region inlining)
ScalarHoist.cpp - modified to emit dialect ops
Passes.td - lower-scalar-hoist pass registration
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.