CodeGen: Remove -verify-machineinstrs from single -run-pass MIR tests
Since #228248 / 66366908, llc verifies MIR output by default. MIR
inputs were always verified so for an llc invocation running exactly
one pass with -run-pass, the -verify-machineinstrs flag does nothing
so these flags are pointless. We should more aggressively trim
-verify-machineinstrs uses in the future, but this covers the cases
that don't change behavior. This removal was performed by generated
script.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
AMDGPU: Fix folding 64-bit move imm into subreg use in SIShrinkInstructions (#237040)
foldImmediates folded the full 64-bit immediate of a move into a
subregister use, rather than the value of the used half. Extract the
immediate value of the used subregister. Global addresses were similarly
broken.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[BoundsSafety][Docs] Add an index page and upstream status for -fbounds-safety
Add BoundsSafety/index.md as the landing page for the -fbounds-safety
docs. It covers how to contribute, standardization efforts in WG14,
open-source adoption efforts, RFCs, and talks.
Add BoundsSafetyUpstreamStatus.md, which lists what is available
upstream, what is in progress, and what is planned, with links to the
relevant PRs and issues.
Leave a stub at the old BoundsSafety.html location so existing links keep
working.
Assisted-by: Opus 5.5
[ARM][MIPS] Show reloc comments in compressed instruction byte-order (#233599)
Little endian instruction encoding for Thumb/microMIPS consists of
higher order half-word followed by lower order half-word, each in
little-endian order. Relocation bits in this encoding need special
mapping logic.
Example encoding (starting MSB): AA BB CC DD
Layout in memory (starting low byte) [BB AA] [DD CC]
Also fix test cases for Thumb/microMIPS that captured the previously
broken relocation comments.
Co-authored-by: Faraz Shahbazker <faraz.shahbazker at mediatek.com>
[AMDGPU] Preserve non-canonical values in packed f64 fneg (#236405)
Select divergent v2f64 fneg with sign-bit XORs. Restrict the
V_PK_ADD_F64 pattern to canonicalize-of-fneg so bare negation preserves
signaling NaNs and all operand bits except the sign.
Add explicit fneg and canonicalize-of-fneg coverage, and update the
affected packed vector negation checks.
Fixes #236341
---------
Co-authored-by: Jay Foad <jay.foad at gmail.com>
[AMDGPU] Fix scale_offset with wrapping offsets (#238016)
On gfx1250, vector memory scales the sign-extended offset in 64 bits,
but SMEM scales the offset in 32 bits and then zero-extends it. Both
selectors set scale_offset either way, so global and flat accesses
miscompiled a 32-bit scaled index that wraps, and SMEM loads an index
whose 64-bit scaled value reaches 4 GiB.
Keep scale_offset only when nsw or known bits prove that the hardware
computes the same address. Fix the SMEM tests that expected the wrong
code.
Fixes #237993
[X86] Enable SlotIndexes compaction by default
Compacting SlotIndexes before greedy allocation drops the entries of
erased instructions, so live range sizes and instruction distances
reflect the code being allocated rather than how much was erased
before it. AMDGPU has done this since #235806; do the same for X86.
Codegen changes only through allocation: no new crashes, verifier
errors or allocation failures in the lit suites, including
EXPENSIVE_CHECKS, and a stage-2 build of LLVM, clang and lld with
-verify-machineinstrs passes its own tests. On llvm-test-suite at -O3,
spills drop 0.3% and .text 0.06%, with compile time unchanged.
Most test updates are regenerated checks. misched-matmul now inserts
20 spills instead of 24. live-debug-vars-stale-slotindexes.mir gets
one more use of %27 so that accumulate_fit still reaches the stale
index path it was written for.
Revert "[CoroSplit] Use resume-switch weights on the final-suspend de… (#238125)
…stroy branch (#237526)"
This reverts commit c797aca0865eae6328ddd4f327d905d4115f719f.
If we have some code like this that calls f (from the introduced test
case):
```cpp
for (int i = 0; i < 100; ++i) {
Task t = f();
while (!t.crt_handle.done())
t.crt_handle.resume();
}
```
We call f.destroy 100 times, and hdl is null 100% of the time, which
contradicts the branch weights synthesized by the patch (~19/81).
[AMDGPU] Return early in scale_offset wrap checks
Each wrap check now returns false itself, so the matchers no longer
need to guard the final Offset update with ScaleOffset. The update has
to stay after the checks, because the callers use Offset even when
the matcher fails. No functional change.
[BoundsSafety][Docs][NFC] Move -fbounds-safety docs into BoundsSafety/
Move BoundsSafety.md, BoundsSafetyAdoptionGuide.md and
BoundsSafetyImplPlans.md into a BoundsSafety/ directory without changing
their content, and update the references to them. A follow-up adds an
index page for the directory.
Assisted-by: Opus 5.5
[SelectionDAG] Expose the concat fold for DAG combining
Make FoldConcatVectors a SelectionDAG method shared by node construction
and DAG combining. Invoke it directly when revisiting concat operands
instead of using getNode to rerun construction-time folds.
AMDGPU: Fix folding 64-bit move imm into subreg use in SIShrinkInstructions
foldImmediates folded the full 64-bit immediate of a move into a
subregister use, rather than the value of the used half. Extract the
immediate value of the used subregister. Global addresses were similarly
broken.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
AMDGPU: Fold global address into subregister use with lo/hi relocation
Split 64-bit relocations into the 32-bit halves when applying a
subregister extract, similar to how immediates are modified when
folding. The same split is applied when 64-bit mov pseudos are
expanded.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
SystemZ: Remove kill flags on virtual registers from MIR tests (#236919)
The verifier will soon disallow these, so prepare test inputs. Delete
tests that were identical except for the kill flags.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
libunwind: Use libgcc's function context layout for sjlj
Make call_site an i32 and the __data words pointer
sized. compiler-rt's SjLj_Function_Context was not defined in the same
way as libgcc.
This was broken for all 64-bit targets. Every field was i32, so the
exception pointer in __data[0] was truncated. It just happens that no
64-bit target defaults to SJLJ exceptions except VE. Instead of fixing
that, a VE special case was introduced in the libunwind sources and
SjLjEHPrepare, which had a special TargetMachine hook for this data
size. In reality it should just be a uintptr_t which we can already
infer from the DataLayout.
This is an ABI break for 64-bit targets other than VE when using
libunwind built for SjLj: the __data, personality, lsda and jbuf
offsets move, so compiled code and libunwind must be updated
together. That configuration was already broken by the pointer
truncation and is not the default anywhere. 32-bit targets are
[4 lines not shown]
[AMDGPU] Reword zext scale_offset test comment
SelectionDAG keeps scale_offset in this test only because it narrows
the mask to the demanded bits first. GlobalISel keeps the wider mask,
whose shift can wrap. The old comment suggested that GlobalISel misses
a proof.
[AMDGPU] Only demand the exponent bits of V_CVT_SCALEF32 scale operands (#229889)
The `V_CVT_SCALEF32_*` conversions from F32/F16/BF16 to
FP8/BF8/FP6/BF6/FP4 only read the exponent field (bits [30:23]) of their
f32 scale operand. This is documented in the CDNA4/CDNA5 ISA.
Frontends that build the scale from an E8M0 byte or an fp32 value often
mask or shift it first, e.g. `and %x, 0x7f800000` or `shl (zext i8
%e8m0), 23`, and those instructions currently survive to the final ISA.
Frontends like Triton lower straight to the ROCDL intrinsics, so the
logic in ArithToAMDGPU that works around this, doesn't reach them. Those
instructions are wasted work in the scale computation of kernels. This
patch tags the down-conversion intrinsics with a new
`AMDGPUExponentOnlyScale<ScaleArg>` class and adds a searchable table
that maps each intrinsic to its scale argument
(`AMDGPU::getExponentOnlyScaleArgIdx`). In InstCombine, the scale
operand drops any `fneg`, `fabs` or `copysign`. When the scale is a
bitcast of an i32 and every user of the bitcast is one of these scale
operands, the i32 is simplified with only the exponent bits demanded. In
[3 lines not shown]
[BoundsSafety][Docs] Add an index page and upstream status for -fbounds-safety
Add BoundsSafety/index.md as the landing page for the -fbounds-safety
docs. It covers how to contribute, standardization efforts in WG14,
open-source adoption efforts, RFCs, and talks.
Add BoundsSafetyUpstreamStatus.md, which lists what is available
upstream, what is in progress, and what is planned, with links to the
relevant PRs and issues.
Leave a stub at the old BoundsSafety.html location so existing links keep
working.
Assisted-by: Opus 5.5
libunwind: Use libgcc's function context layout for sjlj
Make call_site an i32 and the __data words pointer
sized. compiler-rt's SjLj_Function_Context was not defined in the same
way as libgcc.
This was broken for all 64-bit targets. Every field was i32, so the
exception pointer in __data[0] was truncated. It just happens that no
64-bit target defaults to SJLJ exceptions except VE. Instead of fixing
that, a VE special case was introduced in the libunwind sources and
SjLjEHPrepare, which had a special TargetMachine hook for this data
size. In reality it should just be a uintptr_t which we can already
infer from the DataLayout.
This is an ABI break for 64-bit targets other than VE when using
libunwind built for SjLj: the __data, personality, lsda and jbuf
offsets move, so compiled code and libunwind must be updated
together. That configuration was already broken by the pointer
truncation and is not the default anywhere. 32-bit targets are
[4 lines not shown]
[AArch64] Split composites between x7 and the stack in Windows varargs (#234836)
On Windows on Arm, the arguments of a variadic function, named ones
included, are laid out as if on a stack whose first 64 bytes are loaded
into x0-x7 (Microsoft's ARM64 ABI, "Addendum: Variadic functions").
A composite lowered to `[2 x i64]` or `[2 x ptr]` whose first half lands
in x7 is therefore passed in x7 and the first stack slot.
That is what MSVC emits and what clang's own `va_arg` lowering expects.
`CC_AArch64_Win64_VarArg` delegated such a block to the AAPCS64
consecutive-register block allocation, which puts it entirely on the
stack. For a variadic argument, the callee then read it and every later
argument 8 bytes off.
Allocate the elements of a consecutive-register block one at a time in
`CC_AArch64_Win64_VarArg`.
SelectionDAG also flags a byval pointer as a block when its pointee is
an array; it keeps the existing stack copy, since byval is only lowered
as a stack argument.
[6 lines not shown]
[flang] Carry a requested alignment on hlfir.copy_in (#235854)
A copy-in buffer stands in for the variable it copies at a call site, so
a variable with an alignment requirement needs its buffer aligned the
same way. This patch gives `hlfir.copy_in` an optional `alignment`
attribute (a positive power of two, verified), to be set by lowering
when the actual argument carries such a requirement, and honors it in
both places that allocate the buffer: the loop-inlined form in
`InlineHLFIRCopy` and the `hlfir.copy_in` codegen in `ConvertToFIR`.
Copies made by the runtime (assumed-rank, polymorphic) do not honor it
yet.
The `FIRBuilder` temporary helpers (`createTemporaryAlloc`,
`createTemporary`, `createHeapTemporary`, `createAndDeclareTemp`,
`createArrayTemp`) gain a trailing optional alignment applied to the
`fir.alloca` or `fir.allocmem`, and `shouldUseStackForCopyin` takes the
alignment so the placement policy counts its padding against the stack
budget.
[5 lines not shown]
[InstCombine] Don't drop provenance when merging stores into successor
mergeStoreIntoSuccessor built the phi in the type of the store being
visited, and cast the other stored value to it. Casting a byte to a
non-byte type, or a pointer to an integer, is lossy. Merge in the byte
or pointer type if either store has one.
Fixes #238063.
[mlir][acc] Return after NYI for non-NVIDIA per-row barrier (#238116)
createPerRowBarrier reports that a per-row barrier is not yet
implemented on non-NVIDIA devices, but it does not return afterwards.
Execution falls through to the NVVM named-barrier code, which asserts
that the device type is NVIDIA, so acc-cg-to-gpu aborts instead of
failing cleanly. This is reachable with the default device type for any
thread-partitioned loop wider than a subgroup. For example,
acc-cg-to-gpu-barrier-after-thread-loop.mlir crashes if its
device-type=nvidia option is removed.
Return after emitting the diagnostic, matching the other NYI sites in
this file, and add a test that checks the diagnostic for this case.
CodeGen: Replace TargetOptions::DebuggerTuning with module flag
Read the debugger tuning from the module's "debugger-tuning" flag instead of
TargetOptions. Delete the now-dead -debugger-tune= cl::opt and the
TargetOptions::DebuggerTuning field.
Update the tests relying on it to directly use the module flag. The cases
that exercise multiple values do a text substitution to swap the value.
Alternatively this could be changed to synthesize the module flag in the
llc flag like some of the other widely used module flags do.
Co-authored-by: Claude Sonnet 5 <noreply at anthropic.com>
