[AMDGPU][NFC] Compute prologue frame size/alignment in 64-bit (#224734)
In `SIFrameLowering::emitPrologue` the stack-realignment sequence computes its
`S_AND` mask (`-Alignment * scale`), `S_ADD` offset (`(Alignment-1) * scale`)
and the final SP adjustment (`RoundedSize * scale`) from 32-bit locals; the
multiply is done in 32 bits, which CodeGen static analysis flags as a possible
truncation (`cpp/integer-multiplication-cast-to-long`). Widen `NumBytes`,
`RoundedSize` and Alignment to 64-bit. The `AsmPrinter` still truncates via
`printImmediate32`, so emitted asm is unchanged; only the raw MIR immediate of
the realign `S_AND_B32` changes (e.g. `4294443008 -> -524288`). The seven
autogenerated .mir frame tests are regenerated accordingly.
Assisted-by: Cline
[Comgr][transpiler] Read PC-relative literals from the source image
When a kernel computes an address from its own program counter and loads
through it, it reads a literal the compiler laid down next to the code, not
memory the kernel was handed. Emitting that load would point a source
code-object address at target memory, where nothing of the source image is
mapped.
s_get_pc_i64 now writes the address the next instruction sits at, which the
decode already knows as a constant, and marks the SGPR pair as holding a
source-image address. A 64-bit scalar add or subtract of a constant carries the
mark to its destination, s_mov_b64 carries it across a copy, and a scalar load
off a marked pair reads the captured .text and .rodata bytes straight into
register constants.
Anything that does not resolve to a literal while raising is refused with its
reason: a register offset, an address no captured section covers, arithmetic
that leaves the address space, or a source address put somewhere the raise
cannot track it.
[2 lines not shown]
[TableGen] Handle the non-`CoveredBySubRegs` case in `computeRegUnitLaneMasks` (#223653)
Currently `computeRegUnitLaneMasks` ignores non-leaf subregisters as
"their lane masks are fully covered by the leaf subregisters". However
this only holds when `CoveredBySubRegs` is set to true. Otherwise it
causes subregisters with `CoveredBySubRegs` set to false to have missing
lanes.
For example removing `hi16` from SGPR in AMDGPU causes `sub1` to have a
lane mask of `0x8`, which is equal to `sub1_lo16`, whereas the correct
mask should be `0xC` as although `hi16` is not addressable `sub1` covers
both halves.
This patch adds a logic that identifies unclaimed lanes and backfills
them into appropriate subregister units.
Assisted-by: Claude Code
[mlir] Enable strict assembly properties for the builtin dialect
Print ModuleOp visibility in the property dictionary and retain its
separate discardable attribute dictionary. Update affected assembly tests.
Assisted-by: Codex
[mlir][tosa] Update reverse tests for direct axis syntax (#225751)
Use the axis(...) assembly syntax in block-scaled reverse tests. This
matches the existing TOSA operation format and restores test parsing.
Fix post merge #225058
Assisted-by: Codex
[flang] Lower loops whose branching is confined to their body structurally
Such a loop was classified separately by a previous change but still
lowered as a raw CFG, so its structured form was lost.
Lower it structurally instead, with its body folded into a region that can
hold the branching. The loop keeps its bounds on the op, so it remains
available to whatever transforms or parallelizes it. Only the body is
folded: the loop control statements are emitted as they are for any
structured loop, since a branch from outside may target either of them.
Loops an OpenACC or OpenMP directive owns are lowered the same way, so
they keep their form too.
[flang] Detect loops whose branching is confined to their body
A DO loop is classified as either structured or unstructured, and a single
raw branch anywhere in its body forces the loop -- and every construct
enclosing it -- onto the unstructured path.
That is stronger than necessary. A loop keeps its structured control flow
as long as its branching neither leaves its body nor enters it from
outside. Classify such a loop separately from a fully unstructured one.
This only classifies: lowering is unchanged. PFT dumps mark the new
classification with '~', which is what the tests key on.
[flang] Record the evaluations that branch to each evaluation
The PFT records where each branch goes, but not where it comes from, so
asking whether anything branches into a construct means walking the whole
procedure.
Record the reverse edges beside the forward ones, and print them in PFT
dumps so both directions of the branch graph are visible.
[compiler-rt] Add 'csan' library for the concurrency sanitizer
Summary:
Adds the runtime for the concurrency sanitizer, both CPU and GPU.
Fundamentally, this works using the following pseudocode:
```c
static u64 watchpoints[N]; // Hash-indexed, zero is empty.
// Emitted before the access, so we never trip on our own write.
void check_access(volatile void *addr, u32 size, u32 type) {
// Every access probes. A read conflicts only with a watched write, a
// write conflicts with either.
if (u64 *wp = find_watchpoint(addr, size, type))
consume(wp, this_pc()); // Hand our location to the owner.
if (!should_sample()) // Wave-uniform, 1-in-N chance.
return;
[17 lines not shown]
AMDGPU: Mark dead carry-out when rewriting scalar carry op to VALU
If an operation is expanded with a dead scc def, the resultant vcc def will
also be dead, so preserve the dead flag. Reduces implicit reliance on
LiveVariables recomputing dead flags later.
Co-authored-by: Claude (Opus 4.8) <noreply at anthropic.com>
[libsycl][NFC] Suppress shadow warnings from LLVM headers (#225089)
The warning for a local variable shadowing a class member is enabled for
libsycl, but not LLVM as a whole. Suppress those for included LLVM
headers.
AMDGPU: Mark dead carry-out when rewriting scalar carry op to VALU
If an operation is expanded with a dead scc def, the resultant vcc def will
also be dead, so preserve the dead flag. Reduces implicit reliance on
LiveVariables recomputing dead flags later.
Co-authored-by: Claude (Opus 4.8) <noreply at anthropic.com>
AMDGPU: Mark dead carry-out when rewriting scalar carry op to VALU
If an operation is expanded with a dead scc def, the resultant vcc def will
also be dead, so preserve the dead flag. Reduces implicit reliance on
LiveVariables recomputing dead flags later.
Co-authored-by: Claude (Opus 4.8) <noreply at anthropic.com>
[orc-rt] Move test utils into orc_rt::test, drop qualification (#225733)
Move unit test helpers from the global namespace into orc_rt::test. This
makes them consistent with the newly added ErrorMatchers.h utils, and
allows us to drop the 'orc_rt::' qualification on a lot if identifiers.
AMDGPU: Preserve carry-out dead flag in V_ADD_CO_U32 custom inserter
EmitInstrWithCustomInserter rebuilds V_ADD/SUB_CO_U32_e32 but did
not carry over the dead flag. Preserve the flag to reduce implicitly
depending on dead flags recomputed by LiveVariables.
Co-authored-by: Claude (Opus 4.8) <noreply at anthropic.com>
[mlir][OpenMP] Split generated operation definitions into three TUs
Compile independent and helper-backed OpenMP op definitions separately from
the handwritten dialect implementation. Move shared assembly helpers into
the dialect namespace and keep registration in the independent TU.
Outline the repeated block-argument interface body once for all 15 models.
The generated operation partitions cover all 72 OpenMP operations.
Assisted-by: Codex
CodeGen: Mark dead PHI destination copies dead in the LiveIntervals path
When PHIElimination lowers a PHI whose destination is dead, it records a
dead def in LiveIntervals but did not set the dead flag on the lowered copy's
def operand. The LiveVariables path set this via addVirtualRegisterDead.
Avoids "Instruction ending live segment on dead slot has no dead flag" verifier
errors once LiveVariables is removed.
Co-authored-by: Claude (Opus 4.8) <noreply at anthropic.com>
[flang] Detect loops whose branching is confined to their body
A DO loop is classified as either structured or unstructured, and a single
raw branch anywhere in its body forces the loop -- and every construct
enclosing it -- onto the unstructured path.
That is stronger than necessary. A loop keeps its structured control flow
as long as its branching neither leaves its body nor enters it from
outside. Classify such a loop separately from a fully unstructured one.
This only classifies: lowering is unchanged. PFT dumps mark the new
classification with '~', which is what the tests key on.
[flang] Lower loops whose branching is confined to their body structurally
Such a loop was classified separately by a previous change but still
lowered as a raw CFG, so its structured form was lost.
Lower it structurally instead, with its body folded into a region that can
hold the branching. The loop keeps its bounds on the op, so it remains
available to whatever transforms or parallelizes it. Only the body is
folded: the loop control statements are emitted as they are for any
structured loop, since a branch from outside may target either of them.
Loops an OpenACC or OpenMP directive owns are lowered the same way, so
they keep their form too.
