[flang] Refine fir.assert verification and target lowering
Restrict diagnostic values to i8, i16, i32, i64, and index, requiring
producers to choose how boolean values are extended. Use target global
address spaces for diagnostic strings and cast their addresses for calls.
Use runtime entry-name macros and a named fixed-value limit, remove a
redundant verifier check, and cover incompatible declarations, symbol
conflicts, explicit boolean extension, and global address spaces in tests.
[flang] Add fir.assert and its LLVM lowering
Introduce a non-speculatable assertion with typed guarded results and a
write effect on a non-addressable Fortran runtime resource. Verify its
message and diagnostic operands, and erase constant-true assertions.
Keep attribute and optionality queries able to inspect guarded values
without making rewriting operations transparent to the guard.
Lower assertions to a weighted branch, a noreturn runtime call on failure,
and guarded operands on success. Pack larger diagnostic payloads into an
array on the failure path and preserve source locations and device symbol
scope. Add dialect, code generation, effect, and variable-query tests.
This introduces the operation without emitting it from source lowering.
[Reassociate]Reject fma pairs with a shared addend (#228751)
A kept pair whose addend has other uses is frozen against the rank based
re-sort of the enclosing expression. That hides the shared value from
the
other terms and turns b + c*c + b*b + c into two independent pairs
instead
of the chain produced before #215873. Fixes the TSVC s1244 regression
left
over after #220640.
Assisted-by: Cursor
[CIR][NFC] Share getSuccessorRegions across region-branch ops (#220512)
Four of the ten CIR ops that declare `RegionBranchOpInterface` directly
had identical `getSuccessorRegions` bodies. The parent may enter any of
their regions, and every region exits back to the parent. This moves
that body into a new TableGen base class,
`CIR_EnterAnyRegionBranchOpBase`, and puts `ScopeOp`, `CaseOp`,
`SwitchOp` and `TernaryOp` on it. The stale doc comments above the
deleted `ScopeOp` and `TernaryOp` definitions go too.
`IfOp`, `GlobalOp`, `AwaitOp` and `CoroutineOp` keep their own
definitions because their successors differ. `IfOp` exits straight to
the parent when the else region is empty, `GlobalOp` skips empty ctor
and dtor regions, and `AwaitOp` and `CoroutineOp` send each region to
specific successors.
`CleanupScopeOp` and `TryOp` have the same successors today, but both
are wrong. The cleanup region runs after the body, and a catch handler
runs after the try body unwinds, so neither is an entry from the parent.
[3 lines not shown]
[CIR] Fix cir.try successor regions
Stacked on #221222, which adds the `requireAllReachable` flag the tests use.
`TryOp::getSuccessorRegions` reported the try region and every handler region as
alternative entries from the parent, and every region exit as going back to the
parent. That lets a handler run without the try body. A handler runs after the
try body unwinds, not instead of it.
This reports what the op executes. The parent enters the try region. A
`cir.resume` ending the try region unwinds into the handlers, and the exception
type decides which one runs. Every other exit, including a handler's own
`cir.resume`, leaves the op.
A try body ends in `cir.resume` once FlattenCFG has flattened an inner cleanup
scope, or an inner `cir.try` with an unwind or filter handler, and
`CIRTryOpFlattening` then sends each such resume to the handler dispatch block,
so this edge matches the lowering. A try with no handlers has nowhere to unwind
to, so its `cir.resume` leaves the op.
[21 lines not shown]
llc: Disable test from #228248 for EXPENSIVE_CHECKS builds (#231455)
Do what the llc-pipeline tests do and just avoid dealing with the extra
verifier runs inserted by default in EXPENSIVE_CHECKS.
[CIR] Fix cir.cleanup.scope successor regions
Follow-up to #220507, split out of the review of #220512.
`CleanupScopeOp::getSuccessorRegions` reported the body and the cleanup region
as two alternative entries from the parent, and every region exit as going back
to the parent. That lets the cleanup run without the body and leaves no edge
from the body to the cleanup. The comment right above the code said the
opposite.
This reports what the op actually executes. The parent enters the body. A body
exit that triggers the cleanup enters the cleanup region. The cleanup region
exits to the parent.
Whether a body exit triggers the cleanup depends on the terminator and the
cleanup kind. `cir.resume` leaves while unwinding, so it runs the cleanup when
the kind includes EH. `cir.yield` and `cir.co_return` are normal exits, so they
run it when the kind includes normal. `cir.resume` appears in an unflattened
body once an inner cleanup scope has been flattened, and `FlattenCFG` handles
[29 lines not shown]
[lldb][minidump][test] Enable more minidump tests on AArch64 (#231448)
I want to get to a point where we can have an annotation like
requiresMinidumpSave. Instead of spelling out the supported
architectures each time.
This first step is enabling them on AArch64 and seeing what happens.
They seem fine on my Graviton instance, but there might be some problems
on systems with different register state.
Some of the register mismatches might be bugs. I'm not going to fix
those, I just want to document the current state.
[RISCV] Model Zvzip costs for fixed-length shuffles (#227677)
Teach the RISC-V TTI to recognize interleave and deinterleave shuffles
that can be implemented with vzip.vv or vunzipe.v/vunzipo.v. Compute
their costs using the operand containing the interleaved data: the
destination of vzip.vv and the source of vunzipe.v/vunzipo.v.
Assisted-By: Trae CLI (GPT-5)
[Machine Outliner][RISCV] Input section aware machine outlining (#226126)
[Machine Outliner][RISCV] Input section aware machine outlining
Currently machine outliner refuses to outline from named section
functions because the callers of the outlined function could potentially be in
different input sections and since outlined function can only be in one of
the input section, this will lead to cross-section calls that breaks the
assumption around the code being all in the same section.
This PR fixes the issue by partitioning the candidates of a repeated
sequence by their parent function's input sections and emitting the outlined
function within the same section. An outlined function inherits its parents'
input section.
This input section based outlining behavior is opt-in via
`TargetInstrInfo::supportsSectionAwareOutlining()`, which
defaults to false and has been enabled only for RISCV.
[4 lines not shown]
[mlir][toy] Add missing spacing to tensor literal closing bracket (#223275)
There's a missing space before the right closing bracket of tensor
literals for the Toy language. This PR fixes the AST dumper to include
it.
[CodeGen] Let canInsertSelect() describe the select expansion (NFC) (#230027)
`canInsertSelect()` only reports latencies, but `insertSelect()` may emit
more than one instruction and may clobber physregs. For example, on
AArch64 a `cbz`/`tbz` condition needs a flag-setting `subs`/`ands` before the
`csel`, which clobbers NZCV.
Replace the out-parameters with a new SelectExpansion struct that holds
the latencies plus NumInsts and ClobberedPhysRegs.
These new fields are not used yet in this commit, but will be wired in a
follow-up commit to Early If Conversion.
Note that these fields are only implemented for AArch64 for now. Targets
which do not support the new struct will return `0` in NumInsts and
default to the existing behavior.
This PR on it's own is an NFC, but the new API will be used by #218415
IR: Add wasm_emscripten_invokecc calling convention keyword
WASM_EmscriptenInvoke had no textual IR spelling and was printed as
cc99. Tests using cc99 as an arbitrary custom convention now use cc1000.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[CIR] Fix cir.try successor regions
Stacked on #221222, which adds the `requireAllReachable` flag the tests use.
`TryOp::getSuccessorRegions` reported the try region and every handler region as
alternative entries from the parent, and every region exit as going back to the
parent. That lets a handler run without the try body. A handler runs after the
try body unwinds, not instead of it.
This reports what the op executes. The parent enters the try region. A
`cir.resume` ending the try region unwinds into the handlers, and the exception
type decides which one runs. Every other exit, including a handler's own
`cir.resume`, leaves the op.
A try body ends in `cir.resume` once FlattenCFG has flattened an inner cleanup
scope, and `CIRTryOpFlattening` then sends each such resume to the handler
dispatch block, so this edge matches the lowering. A try with no handlers has
nowhere to unwind to, so its `cir.resume` leaves the op.
[18 lines not shown]
[SPIRV] Don't cache SyncScope IDs across LLVMContexts in getMemScope (#229444)
`getMemScope()` kept the SyncScope IDs of `subgroup`, `workgroup` and
`device` in function-local statics, initialized from the LLVMContext of
the first call. SyncScope IDs are per context and assigned in
registration order, so a later context that registered these scopes in a
different order gets the wrong Scope operands. For example, after a
first module that uses `syncscope("device")`, `syncscope("workgroup")`
and `syncscope("subgroup")` (in that order), a module in a new context
that uses them in the reverse order is emitted with `Device` scope for
its subgroup atomics and `Subgroup` scope for its device atomics.
This doesn't show up in `llc`, which compiles a single module, but it
does when the back-end is used as a library that compiles each module in
a fresh context. We hit it in Julia's GPU compiler, where device-scope
atomics in kernels compiled later in the session silently became
workgroup or subgroup atomics.
The fix looks the IDs up in the given context on every call.
[6 lines not shown]
TableGen: Move value reference locations into Record (#231411)
RecordVal kept reference locations that only the TableGen language
server uses, but every copy of every field paid for them. Keep them in
the owning Record instead. This shrinks RecordVal from 48 to 32 bytes.
Parsing AMDGPU.td with llvm-tblgen --null-backend, peak memory drops
from 834 MB to 638 MB with 22% fewer page faults.
Add a language server test for references to fields and template
arguments, which were not covered before.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[flang-rt] Preserve fatal diagnostics when allocation fails
Validate the array-entry format and value count before printing error message.
Print the validated message in bounded pieces through the existing
termination helpers, avoiding allocation of a formatted message and the
custom decimal conversion. Concurrent failures may interleave their output.
Document the fixed-entry caller contract and mark all three fatal-error
entries for device compilation. Test large mismatched counts,
including long text and integer limits.
Check the real runtime diagnostic in the array-entry tests and remove the
allocation-failure regression and its static-runtime lit feature.
[VPlan] Switch fn in replaceUsesWithIf to User ptr (NFC) (#229144)
Changing the function to accept a User ptr instead of a User ref helps
improve some code.
[lldb][minidump][test] Re-enable and xfail TLS test (#231425)
This has not been running / never was, because it did not have the test_
prefix. On both AArch64 and x86_64 (the two architectures we can save
minidumps on), we find the variable but its value is 0. I assume it has
always been this way, so xfail it.
[CIR][CodeGen] Share the EH personality selection logic (#223421)
Deduplicates the EH personality selection (`getEHPersonality`,
`getCXXEHPersonality`) between CIR and classic CodeGen, taking the
classic
implementation. CIR's copy lacked the z/OS, Wasm and GNUstep-on-CygMing
cases;
none are reachable in CIR today, so no test changes.
Assisted-by: Claude Code (Claude Fable 5.1).
[CodeGen] Expand unsupported vector (de)interleave operations into shuffles (#210494)
Added generic expansion support for vector_(de)interleave operations to prevent crashes on backends that do not provide custom lowering.
Consolidated the expansion logic, previously scattered across multiple locations, into vector legalization. This also removes the expansion logic from the SelectionDAG builder, centralizing the handling of these operations.
[orc-rt] Split connection support out of bedrock. NFC. (#231431)
Move the code for reaching a controller in another process out of
orc-rt-bedrock and into two new libraries:
orc-rt-connect: ConnectionSpec, ConnectorRegistry, SimpleRemoteCA
and VettedPeer, which every transport shares.
orc-rt-connect-socket: SocketHandle, SimpleRemoteCAOverSocket and the
socket:adopt connector.
Bedrock no longer assumes any particular transport. Headers move from
orc-rt/bedrock/ to orc-rt/connect/ and orc-rt/connect-socket/.
Assisted-by: Claude
[flang] Mark loops LICM moves into gang-redundant code of an acc parallel region
In a compute region from an acc.parallel construct, code outside a loop
with gang-level worksharing is executed redundantly by all gangs. A loop
that LICM moved there from inside such a loop was not marked as running in
that mode, so later passes could distribute it across gangs. When its
result is used in the same kernel, each gang then only computed part of it,
giving wrong results.
Mark loops moved into gang-redundant code with acc.gpu_block_redundant so
they keep being executed by every gang.
Co-Authored-By: Claude Opus 5.5 <noreply at anthropic.com>
[NVPTX] Set default value of nvptx-allow-ftz-atomics to false
This restores the strict FTZ default originally introduced in #200732
and temporarily relaxed in #206154.
Frontends can now explicitly opt in to native FTZ-mismatched atomics on a
per-instruction basis using !atomic.ignore.denormal.mode metadata (#217585,
#217586), which is supported in Clang, MLIR, and XLA. Users who need the
previous behavior globally can still pass -nvptx-allow-ftz-atomics=true.
Update test cases to verify the new default disallow behavior.
