[SLP]Release scheduling deps of copyable reassoc operands only once
A flattened reassoc node may have a scalar that is both a modeled
copyable tree operand (child entry) and a direct IR operand of a node
member. Scheduling such a member released its ScheduleCopyableData
dependency twice - in the per-lane operand scan and again in the
reassoc operand cleanup - driving UnscheduledDeps negative. Dedup the
cleanup release through the same Checked set the operand scan uses.
Reviewers:
Pull Request: https://github.com/llvm/llvm-project/pull/213079
[CIR] Allow cir.copy to work across address spaces- (#211873)
LLVM Memcpy supports copying across address spaces, so we should too. IN
C, this is legal, and there are no associated AST nodes to reasonably
put in a cast, so this patch just legalizes it to match the LLVM
behavior.
Note: I've added a 'CopyTypes' printer/parser as well, because it seems
unfortunate to make EVERY copy have to show all of its types.
[Modules] Handle decl attributes on deserialization the same as during parsing. (#208348)
Replace manual handling of 2 attributes with
`Sema::mergeDeclAttributes`, which is called during parsing too. Also
propagate attributes not from a previous redeclaration but from a
previous redeclaration outside of the current module. This is done to
avoid double propagation because attributes from a previous decl in the
same module are already handled when a module is built.
Call `ASTDeclReader::attachLatestDecl` after each decl is added to a
redeclaration chain, not once per `ASTReader::loadPendingDeclChain`
call. This is done to maintain correct redeclaration chain for each
`ASTDeclReader::attachPreviousDecl` call because [newly added]
`mergeDeclAttributes` requires a correct redeclaration chain.
Separated `Parser::Initialize` and `Parser::ConsumeToken` so can move
the initialization earlier, into `Parser` constructor. This way
`Parser::Initialize`, `Sema::Initialize`, `ASTReader::InitializeSema` are
executed before deserialization. This specific case is verified by
[6 lines not shown]
[CIR]Implement lowering for simple sin/cos/tan/exp10 builtins. (#211614)
Discovered these just looking around, they are pretty simple/trivial
translations to LLVM-IR intrins, and the infrastructure to do so is
already in place, so this is a bit of simple wiring up!
Implements f/h/hl versions of sin/cos/tan. Also implements base-10
exponent, since it is also trivial.
[CIR] Fixup type of no-prototype functions when emitting an alias. (#212308)
If a function is first used through a no-prototype declaration, then
defined later as an alias, we erased the old declaration and created the
new one with the new type. However the earlier references still used the
old type, which caused a verification error.
This patch just makes sure we replace the uses of it similar to how we
do with a similar problem with normal functions.
Note: there is a bit of inconsistency in how we're setting the type of
the function between OGCG and CIR that I'm not sure of the impact of,
but that is prexisting.
[libc++] Fix difference_type expectations in range adaptor iterator member_types tests (#201853)
This fixes incorrect expectations in:
```
libcxx/test/std/ranges/range.adaptors/range.adjacent/iterator/member_types.compile.pass.cpp
libcxx/test/std/ranges/range.adaptors/range.zip.transform/iterator/member_types.compile.pass.cpp
libcxx/test/std/ranges/range.adaptors/range.adjacent.transform/iterator/member_types.compile.pass.cpp
```
These iterators define difference_type as `range_difference_t<Base>`.
For `ForwardSizedView` and `InputCommonView`, the `difference_type` of
the underlying iterators is `std::intptr_t` in `test_iterators.h`.
The tests instead assert `std::ptrdiff_t`. That happens to hold on
platforms where `std::intptr_t` and `std::ptrdiff_t` are the same type,
but fails elsewhere.
---------
Co-authored-by: Louis Dionne <ldionne.2 at gmail.com>
[mlir][arith] Gate min/max expansion in arith-expand behind include-min-max (#211884)
`arith-expand` expanded `arith.maximumf`/`minimumf`/`maxnumf`/`minnumf`
and the signed/unsigned integer max/min ops into `cmpf`/`cmpi` +
`select` sequences. These ops also have a direct arith-to-llvm lowering
to the `llvm.intr.maximum`/`minimum`/... intrinsics, which are a single
hardware instruction on many targets. Pipelines that run arith-to-llvm
after arith-expand (e.g. the GPU-to-XeVM pipeline) therefore paid a
large, avoidable overhead.
Add `include-min-max-f` (the float ops
maximumf/minimumf/maxnumf/minnumf) and `include-min-max-i` (the
signed/unsigned integer ops maxsi/maxui/minsi/minui) option (default `false`, changing the existing
behavior) that controls whether these min/max ops are expanded. The
min/max converters are factored into new
`populateExpandMinMaxFPatterns` and
`populateExpandMinMaxIPatterns`; the ceil/floor-div and scaling ext/trunc
expansions (which have no LLVM lowering) always run.
[2 lines not shown]
[MergeFunctions] Preserve import GUIDs when folding functions (#207003)
Computes the union of SamplePGO/ThinLTO import GUIDs from both inputs
and attaches it to the surviving function.
[CIR] Fix path generation to understand redecls (#213051)
Found this during a self-build, an additional decl was confusing the
check between getParent and the currentClass. This patch generalizes it
to use the 'isSameEntity' function which makes sure they are the same.
Also, as a drive-by, the test had some check-lines that andy submitted a
while back, that were accidentially not being tested, so fixed the
check-title to be actually checked.
[AMDGPU] Fix miscompile in performSelectCombine for fcmp one with NaN (#213029)
The fold `select (fcmp one x, K), y, K -> ..., x` returned x instead of
K when x was NaN, changing the result
[test] Migrate Arm thunk tests to the default thunk order (#212694)
Drop -z nosort-thunks (added by #211721 to keep creation order) and
update expectations to the default order: forward thunks sorted by
descending destination. Backward-only thunk sections keep creation
order and are unchanged.
In arm-thunk-re-add.s, retarget beq.w to imported2: sorting places
imported's pool thunk just within conditional-branch range, so
beq.w imported would reuse it instead of re-adding a thunk.
[test] Migrate AArch64 thunk tests to the default thunk order (#212691)
Drop -z nosort-thunks (added by #211721 to keep creation order) and
update expectations to the default order: forward thunks sorted by
descending destination.
[mlir][xegpu] Fix DPAS subgroup layout propagation for broadcast K dim (#211695)
This PR fixes an issue in getDpasSubgroupLayouts(): It failed to find
valid subgroup layouts for otherwise-legal DPAS ops. It treated the K
(contraction) dimension as distributed across subgroups — requiring
wgShape % sgLayout == 0 on every dim and gating on a
checkAlignedSgDataAB() equality — when K is actually broadcast: its full
extent stays in every subgroup. This rejected the only valid candidate,
causing layout propagation to bail out.
Update the two affected tests, which now propagate successfully instead
of failing.
assisted-by-claude
Co-authored-by: Claude Opus 4.8 <noreply at anthropic.com>
[AMDGPU] Add ptr.s.buffer.load intrinsic, use it from Clang (#209243)
This commit adds a version of the existing s_buffer_load intrinsic that
more accurately models the memory semantics of the s_buffer_load
instruction, namely that it is, in fact, a memory load.
To preserve the existing behavior that the "nomem" s.buffer.load
intrinsic was using, Clang and MLIR add !invariant.load metadata when
constructing the intrinsic (matching documented requirements on
scalarazable buffer loads) and a late codegen pass adds the metadata
just to be safe.
Tests that were "about" s.buffer.load have been copied to create
versions that use the new intrinsic, as was done for the other
*.ptr.buffer.* operations.
Other tests have been upgraded to use the new intrinsic. This has mainly
resulted in minor instruction ordering changes in prologues, if any
change at all. However, CodeGen/AMDGPU/dagcombine-fma-fmad.ll has seen a
[11 lines not shown]
ARM: Read float ABI from the "float-abi" module flag
Use the value from the module flag if present, otherwise
fall back on the legacy TargetOptions field until that is
removed.
Co-authored-by: Claude (Claude-Opus-4.8) <noreply at anthropic.com>
[AArch64][SME2] Add stridedorcontiguous pseudos for multi-vector-stores (#211551)
Select strided-or-contiguous psuedos for SME2 multi-vector stores,
allowing register allocation to choose the most profitable tuple layout.
The pseudos can then be expanded to choose the appropriate contiguous or
strided instruction after allocation.
[mlir][xegpu] Handle size-1 broadcast dim in insert_strided_slice layout setup (#211725)
This PR enhance `setupInsertStridedSliceResultLayout` to treat a size-1
source dim as a broadcast dim: keep its lane_data at 1 instead of
requiring divisibility.
Add regression tests for both the layout propagation setup and the
subgroup-to-lane distribution of such an op.
Assisted-by-claude
Co-authored-by: Claude Opus 4.8 <noreply at anthropic.com>
[AMDGPU] Fix fmul/fma legacy sign-of-zero miscompile (#203567)
V_MUL_LEGACY_F32 always returns +0.0 when either operand is ±0.0 or a
denormal treated as zero, while IEEE fmul XORs operand signs. This makes
results observably different when one operand is a runtime ±0.0:
fmul.legacy(-2.0, +0.0) = +0.0 (legacy zero clause forces +0.0)
fmul (-2.0, +0.0) = -0.0 (IEEE sign XOR: - * + = -)
The previous code folded whenever one operand matched m_FiniteNonZero()
or both were known never Inf/NaN, without accounting for sign-of-zero
divergence on the other operand.
Restructure the guard:
- Always safe: both operands are known never zero (legacy zero clause
cannot fire). Uses isKnownNeverLogicalZero to correctly account for
denormals flushed to zero under the function's f32 denormal mode.
[13 lines not shown]
[CIR] Add fenv attribute to cast operation (#212899)
This adds the optional #cir.fenv attribute to the cir.cast operation and
updates the verifier to enforce the attribute being present only when
the cast involves floating-point values.
Assisted-by: Cursor / various models
[mlir][tosa][tosa-to-linalg] Use 0 weights at skippable RFFT2D locations (#212293)
The TOSA specifications indicates that when computing the imaginary
output for locations (0,0), (0,W/2), (H/2,0), (H/2, W/2),
implementations may choose to skip the computation entirely and return
0, or compute each accumulation term as `-val_real * 0.0`.
The previous legalizations used `-val_real * sin(a)` but `sin(a)` may
not return exactly 0.0 due to representation error in the input to
`sin`.
This patch updates the legalization to instead check for indices where
`sin(a)` should be replaced by `0.0` and uses a `select` op to match the
expected behaviour according to the Specification when
tosa_extra_multiplies is set to `true`.
Signed-off-by: Ian Tayler Lessa <ian.taylerlessa at arm.com>
[lldb][docs] Document the script interpreter plugin build options (#212655)
Describe LLDB_ENABLE_DYNAMIC_SCRIPTINTERPRETERS and how it interacts
with LLDB_ENABLE_PYTHON_LIMITED_API, including the platform defaults for
both and why the Python Limited API requires dynamic script interpreter
plugins everywhere but Windows.
AMDGPU/GlobalISel: Switch more FP opcodes to extended LLTs (part 2) (#212909)
Migrate G_FPOW, G_FPOWI, G_INTRINSIC_FPTRUNC_ROUND, and FP<->INT
conversions opcodes to extended float LLTs.
Also update the relevant MIR tests.
[lldb][docs] Rewrite the MCP documentation for lldb-mcp (#212821)
The docs still describe lldb-mcp as a thin stdio-to-socket bridge that
auto-launches an LLDB and exposes a single lldb_command tool. It is now
a full featured multiplexer hosting its own sessions, with four tools
and pid-qualified URIs. Also restructure the documentation around how it
is used, with an explanation of the underlying architecture towards the
end.
