[AMDGPU] Fold OR of {SIGN|ANY}_EXTENDed SGPR booleans to SELECT
`or x, (sext cc)` with an SGPR boolean materializes the 0/-1 mask and
then ORs it in. `select cc, -1, x` does the same in one instruction.
After legalization, fold i32 `or x, (sext/anyext cc)` to
`select cc, -1, x`. The high bits of ANY_EXTEND are unspecified, so
they can be chosen to match sign extension. ZERO_EXTEND stays
excluded: `or x, (zext cc)` is `select cc, (or x, 1), x`, which keeps
the OR.
Two checks avoid code-size regressions. The extension must have one
use; otherwise it is still materialized and the select may need an
e64 cndmask. A divergent OR also skips the fold when its uniform
condition has other uses: the select needs the condition as a lane
mask, and the other use would then convert it back to SCC.
Split out of #230701, which extends the existing AND fold to
ANY_EXTEND. Both patches add the same isBoolSGPRExt helper.
[2 lines not shown]
CodeGen: Simplify def segment extension in shrinkToUses (#242869)
During extendSegmentsToUses, Segments holds only one def segment per
value, and none extend past their block. Live-in segments are kept
separately in LiveIns. To extend a def to a use, find the last segment
starting before the use, check that it starts in the use's block, and
move its end to the use. This replaces extendInBlock, whose general
search and merging of overlapped segments never applies here.
Also combine the per-block live-out and live-in tables into one struct.
llc instructions:u:
CTMark x86_64: -0.07%
CTMark arm64: -0.10%
AMDGPU sample: -0.27%
Largest AMDGPU improvements:
rocprim warp_sort (gfx1100): -1.89%
rocsolver stedc (gfx942): -1.70%
[4 lines not shown]
[AMDGPU] Move the OR boolean select fold to a separate patch
Keep this patch to accepting ANY_EXTEND in the existing AND fold, as
requested in review. The OR fold and its tests move to a follow-up PR.
Co-Authored-By: Claude Opus 5.5 <noreply at anthropic.com>
[lldb-dap] Hide internal options from --help (#243124)
Hide internal options for lldb-dap launcher and debugging options. These
are not stable options and may change in future versions.
[ExecutionDomainFix] Ignore regmasks in early-exit check (NFC) (#242997)
Calls clobber registers in RC with regmasks, preventing the early exit
in functions that do not use them.
CTMark instructions:u geomean (x86_64): O3 -0.05%, ThinLTO -0.03%.
[mlir][OpenMP] Update declare SIMD lowering for upstream APIs
Add the property dictionary required by strict assembly formats and update declare SIMD tests to use canonical property syntax. Use the OpenMP translation helper instead of the removed ModuleTranslation builder accessor.
Co-authored-by: Copilot <223556219+Copilot at users.noreply.github.com>
AMDGPU: Update physreg LiveIntervals accurately in SIOptimizeExecMaskingPreRA (#242860)
Instead of dropping the exec/vcc/scc register unit ranges after any
change, remove all physreg defs of the instruction. This fixes the
redundant exec copy fold leaving a stale def in the exec range for
COPY $exec, implicit-def $exec, which was hidden by always
recomputing.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[mlir] Remove forward and friend declarations of non-existent types (NFC) (#242851)
This patch removes forward and friend declarations of classes and
structs that are not defined anywhere in the codebase or do not exist
in their respective namespaces.
Assisted-by: Antigravity
[lldb] Treat calls that do not return as branches when stepping over (#243115)
A Wasm return_call is a call and a barrier, so the breakpoint that
step-over put after it was never reached.
[clang][deps] Flush the in-process module cache on a background thread
The in-process module cache keeps every PCM built during the lifetime of the
dependency scanning service in memory, and flushes them to disk at the end.
Long-running builds that scan many variants of modules (e.g. with different
search paths or macro definitions) accumulate tens of gigabytes worth of
scanning PCMs this way. Moreover, the final flush is single-threaded and
happens after all scanning work is done.
This PR makes the service start a background thread that flushes each PCM to
disk as soon as it's written, and then replaces its in-memory buffer with the
memory-mapped file. Readers of the in-memory buffer keep it alive on their own
(shared ownership), and importers of the freshly built PCM never wait for the
IO, since the file gets written without holding the entry lock. Reads and
timestamps are still served from memory. The final flush is kept, but only
writes PCMs the background thread failed to write. With
`-no-flush-module-cache`, nothing changes.
I tried several approaches, all measured on a scan of the macOS SDK where
[52 lines not shown]
[clang] Extract built PCMs from their container without copying
When a module is built implicitly, the importing instance extracts the AST
file from the module cache's view of the written PCM and stores a copy of it
in its `InMemoryModuleCache`. That's an extra heap copy of every built PCM,
kept alive for the lifetime of the translation unit.
This PR resolves the FIXME by storing a buffer that keeps the written PCM
alive and exposes just the AST file: the `__clangast` section for PCMs
wrapped in an object file (`-gmodules`), or the whole PCM otherwise. The
in-process module cache now shares the memory of its entry with the
importing instance, and the cross-process module cache gives it the
memory-mapped file.
For a regular implicit modules build importing Foundation and friends, this
reduces peak memory footprint from ~150 MB to ~100 MB. For a TU importing
Foundation with `-gmodules` (31 PCMs), the copy increased the peak memory
footprint by 12-17%.
[clang] Use the module cache's view of built PCMs (#242383)
When building a module implicitly, the compiler hands the PCM contents
to
`ModuleCache::write()` and then extracts the AST file from the PCH
container
of its own buffer, storing a copy of it in the `InMemoryModuleCache` of
the
importing instance.
This PR makes `ModuleCache::write()` return a buffer with the written
contents, which the importing instance now extracts the AST file from.
The
in-process module cache returns a buffer sharing ownership of its entry,
the
cross-process module cache returns the written file, read back from the
same
file descriptor it was written through (memory-mapped for all but small
files). This is a prerequisite for storing the module cache's view in
[15 lines not shown]
[ConstraintElim] Address compile-time regressions from getKnownSign. NFC
As a follow-up to PR235159, re-order some of the checks to reduce compile time
impact, and add early bail-outs for ConstantInt's where we can to avoid
KnownBits queries.
(cherry picked from commit feb4e0f5b8d9efe1a206ee932af8035db9f8d616)
Reapply "CodeGen: Create register unit dead defs in slot order (#242577)" (#243143) (#243171)
This reverts commit cf1cd3d96bfbd1f3542c6dd1aef80227bae1d5e3.
Fix conflict with other recent LiveInterval compile time patches.
Fixes for declare simd lowering
- Handle allocatables as ref-like in linear step rescaling and default
modifier selection (IsAllocatableOrPointer instead of IsPointer alone).
- Add TODO for non-int/float element types in linear step rescaling
(complex, character, derived types are rare but possible with
linear(ref:)).
- Assert SubprogramDetails exists instead of silently skipping (declare
simd can only appear in subprograms, not main programs).
- Use dyn_cast_if_present for LinearModifierAttr extraction.
Fix declare simd linear stride rescaling and arg_types verifier
1. Rescale constant linear steps from source-level element counts to byte
strides in Flang's processLinear(). For reference-like parameters
(pointers or non-VALUE dummy arguments) with Linear or LinearRef ABI
kind, the step must be multiplied by the element size in bytes. This
matches Clang's rescaling in CGOpenMPRuntime.cpp. Val and UVal kinds
are not rescaled as they describe value changes, not pointer strides.
Var-strides are also not rescaled as the value is an argument index.
2. Add a verifier check in DeclareSimdOp to ensure 'arg_types' length
matches the number of function arguments, preventing out-of-bounds
access during MLIR-to-LLVM IR translation.
Also restructure processLinear() to compute stepOperand per-variable
instead of appending the same operand for all objects in the clause,
enabling per-variable rescaling.
Assisted with copilot.
[mlir][OpenMP] Translate omp.declare_simd to LLVM IR
Translate omp.declare_simd operations to LLVM IR by computing Vector
Function ABI (VFABI) mangled names and attaching them as function
attributes. This reuses function parameter mangling and codegen logic
in OpenMPIRBuilder which was extracted form Clang [2][3].
For each omp.declare_simd, lowering computes:
- ParamAttrs: one entry per function argument, classifying it as
Vector / Uniform / Linear (+ step or var-stride) / Aligned.
- Branch kind: Undefined / Inbranch / Notinbranch.
- VLEN: either from simdlen() or derived from ISA-specific rules.
x86 (SSE/AVX/AVX2/AVX-512):
Emits mangled names following the x86 Vector ABI [1]:
_ZGV<ISA><Mask><VLEN><ParamAttrs>_<name>
[33 lines not shown]
[MLIR][VectorToGPU][VectorToNVGPU] Only convert add contractions to MMA ops (#236857)
`convert-vector-to-gpu` lowers `vector.contract` to
`gpu.subgroup_mma_compute`, and `convert-vector-to-nvgpu` lowers it to
`nvgpu.mma.sync`. Both are matrix multiply-accumulate (MMA) ops: they
compute `C += A * B`. Each pass checks the iterator types and the
indexing maps of the contract in its own
`contractSupportsMMAMatrixType`, but not the combining kind. So a
contract with another kind, like this one, was lowered to an MMA op too,
and its maximum became a sum:
```mlir
%d = vector.contract {indexing_maps = [...], iterator_types = ["parallel", "parallel", "reduction"],
kind = #vector.kind<maxnumf>} %a, %b, %c : vector<16x16xf16>, vector<16x16xf16> into vector<16x16xf16>
```
This patch makes both passes accept only the `add` kind, with
`ContractionOp::isAddKind()`. Contracts with other kinds stay as
`vector.contract`.
[4 lines not shown]
[CIR] Re-apply Implement 'ctor/dtor returns this' (#242922)
This is a blocking feature for a few of the ABIs, plus is a command line
option for existing itanium ABIs. This patch fills in the handful of
"NYI" spots.
The real 'interesting' part of this is the tests. This was well-tested
in the CodeGenCXX classic codegen test, so this patch enables those of
which can be enabled trivially (or with minor SSA name removals).
SOME of those run-lines are not yet converted because they have
unsupported platforms in their RUN lines, but we intend to convert those
eventually.
FIXED since revert: The test wasn't updated for a different patch that I
wrote that changed how vtable lookups happened. Woops!
Attributor: Deduce noundef returns if poison-generating attributes hold (#231797)
Instead of giving up on noundef for returns with poison-generating
attributes, check that the attributes hold for all returned values, like
FunctionAttrs already does.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>