[APInt] Add a single-word fast path to smul_ov (#242542)
smul_ov detects overflow by dividing the product by one of the operands.
Callers on hot paths, such as BasicAA since #241379, pay for that
division.
For single-word values, multiply the sign-extended operands with
MulOverflow, which uses the compiler's overflow builtin where available,
and check that the product fits in the bit width. Wider values keep the
existing implementation.
Assisted-by: Claude Code
Assisted-by: OpenAI Codex
[CIR] Report side effects of volatile loads and memory asm operands
cir.load, cir.get_bitfield and cir.matrix.column_major_load declared only
a read of their address, so dead code elimination erased them when the
result was unused, even when the load was volatile or atomic. A cir.asm
with a memory operand such as "=*m" and no result was erased the same
way. A volatile or atomic load now also reports unknown read and write
effects, as the LLVM dialect does, and an inline asm reports a read and
a write of each memory operand. The new InlineAsmOp::isMemoryOperand
names the operand_attrs encoding of memory operands; the LLVM lowering
uses it too.
Assisted-by: Claude Code (claude-opus-5-5).
CodeGen: Merge shrinkToUses live-in segments in one pass
extendSegmentsToUses visits blocks in worklist order and inserted each
live-in segment into the middle of the segment vector, as did
createSegmentsForValues for the def segments. Sort both and merge them
in one pass instead, with at most one live-in segment per block. The
gain is largest in cases with large live ranges across many blocks.
llc instructions:u:
CTMark x86_64: -0.13%
CTMark arm64: -0.06%
AMDGPU sample: -0.28%
Largest AMDGPU improvements:
rocsolver potf2-specialized-kernels-d (gfx942): -4.40%
rocsolver getf2-small-d (gfx942): -2.62%
rocsolver getf2-small-d (gfx1100): -2.00%
rocprim warp_sort (gfx1100): -1.54%
Blender Cycles kernel (gfx1030/gfx1100/gfx1201): -0.9%
[2 lines not shown]
AMDGPU: Promote 16-bit arguments to i32 in the calling convention
Promote i16, f16 and bf16 arguments to i32 in the calling convention
when they are inreg or real true16 is not used, instead of special
casing the copy type in LowerFormalArguments. The manual call lowering
code can now rely on LocVT / ArgVT.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
AMDGPU: Copy 16-bit arguments in 32-bit registers as i32 and truncate (#242681)
A 16-bit argument passed in an SGPR, or in a VGPR without real true16,
lives in the low half of a full 32-bit register. Copy the argument as
i32 and truncate it rather than reading it with a 16-bit CopyFromReg of
the 32-bit live-in register. With real true16 a 16-bit VGPR value has
its own register class, so it is still copied directly.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[CIR] Give complex FP add/sub the fenv memory effects
cir.complex.fadd and cir.complex.fsub were Pure even with a constrained
fenv attribute, so they could be speculated or erased like unconstrained
ops. Derive them from CIR_FenvOp, as the other fenv ops are: constrained
ops read and write the FP environment and are only conditionally
speculatable, unconstrained ops stay effect-free.
Assisted-by: Claude Code (Opus 5.5).
[CIR] Reject the empty fenv attribute
`#cir.fenv<>` with no parameter set was accepted, printed as `fenv<>` and
lowered to a constrained intrinsic, so an unconstrained operation had two
spellings. Give `#cir.fenv` a verifier that requires at least one
parameter, leaving a missing `fenv` attribute as the only way to say an
operation is unconstrained. An absent parameter takes its value in the
default FP environment (tonearest, masked, non-strict). Speculation and
lowering share one FenvAttr::canIgnoreExceptions() check (masked and not
strict, lowered to fpexcept.ignore), so a strict op stays fpexcept.strict
even when masked. Reduce FPEnvConstrainedOpInterface to getFenvAttr() and
add a CIR_FenvOp base class carrying the fenv traits and argument, which
the fenv ops prepend their operands to.
Assisted-by: Claude Code (Opus 5.5).
DAG: Allow narrowing uint_to_fp source to the zext source type (#242668)
The narrowing fold skipped uint_to_fp (zext x) when the narrow type
matched x, to avoid undoing a target combine that widened the source.
AMDGPU was the target doing this, and stopped in 50d1a75d978c. This
prevented the fold in the most direct case, so uitofp (zext i32 to i64)
was expanded as a 64-bit conversion. This also hit with topological
DAG combine ordering, where zext i16 -> i64 is first rewritten to a
zext from i32.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
AMDGPU: Add more uitofp of zext from i32 tests (#242667)
These currently get the full i64 conversion expansion.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[Offload] Handle the unchecked error properly (#242521)
The assignment is not allow to overwrite an unchecked value. When a
process aborts, it might hide the real error.
This PR will resolve the test failure for `pgo_atomic_threads.c`,
`pgo_atomic_teams.c` and `d2d_memcpy.c`. Local run passes with
HSA_ENABLE_SDMA=0. But I haven't figure out why HSA_ENABLE_SDMA=0
exposes the errors.
