[MLIR] Allow recovering from semi-affine sets in FlatLinearValueConstraints
The IntegerSet constructor of FlatLinearValueConstraints asserts when
flattening fails, which happens for semi-affine integer sets. Since the
failure is signalled from inside a constructor, callers have no way to
detect the unsupported case ahead of time and no way to work around it;
they simply crash.
Add an optional `bool *error` parameter to the constructor. When it is
non-null, hitting the unimplemented case sets `*error` to true and
returns instead of asserting, letting callers bail out gracefully.
`*error` is set to false on success. When the pointer is null the
previous assert behavior is retained, so existing callers are unaffected.
[AMDGPU] Lower uniform uaddsat.i16 to SALU instructions
Promote uniform i16 uadd.sat to i32 in promoteUniformOpToI32 so it
lowers to SALU (s_add_i32 + s_min_u32) instead of VALU + readfirstlane.
The saturating add on zero-extended operands reduces to
umin(add(lhs, rhs), 0xffff).
Co-authored-by: Cursor <cursoragent at cursor.com>
[AMDGPU] Add tests for uniform uaddsat.i16 (NFC)
Precommit tests covering uniform (SALU) uadd.sat for i16 and vector
i16 (v2i16, v3i16, v4i16) using the amdgpu_ps calling convention so
results are returned in SGPRs. These currently lower to VALU
(v_add_u16 clamp / v_pk_add_u16 clamp + v_readfirstlane).
Co-authored-by: Cursor <cursoragent at cursor.com>
[Clang] [NFC] Update OptionalUnsigned ctor to only reject signed integers (#212321)
Currently, the `OptionalUnsigned(int) = delete;` constructor means that
constructing e.g. an `OptionalOrUnsigned<uint64_t>` from an `unsigned`
fails because overload resolution is ambiguous (because `unsigned` ->
`int` and `unsigned` -> `uint64_t` are both valid conversions). This
patch adds a constraint to make sure the deleted constructor only
catches signed integer types.
This is needed for #212319.
[OFFLOAD][L0] Fix zero argurment kernel launch (#212296)
PR #205224 changed the path for kernels with no arguments to not use
appendLaunchKernelWithArgs, while at the same time it removed
zeKernelSetGroupSize which resulted in incorrect sizing of the kernel.
This PR removes the alternate path and makes sure all launches go
through appendLaunchKernelWithArgs.
Alternatively we can restore zeKernelSetGroupSize on the alternate path.
[AMDGPU] Allow scalar operands in V_PK_LSHL_ADD_U64
Patch creates dummy source modifiers for this instruction to use
with OPSEL. Even though these do not support modifiers we can reuse
the same mechanics as with packed fp64 and other u64 to allow scalar
operands.
[BOLT] Fix strict mode bug in removal of block referenced by JT (#211940)
Summary:
BOLT can create a jump table object from a PC-relative operand reference
(e.g. leaq JT(%rip)) whenever the referenced memory looks like a PIC
jump table -- see
BinaryContext::handleAddressRef. This is independent of whether BOLT
recognizes the indirect jump dispatch itself. The jump table annotation
is only attached to the jmp instruction later, in
BinaryFunction::analyzeIndirectBranch, and only when the target-specific
analyzeIndirectBranch matcher recognizes the dispatch pattern.
If the matcher does not recognize the pattern, the annotation is never
attached to the jmp, even though the jump table object and its entries
exist and reference basic blocks in the function.
In strict mode BOLT still fully processes such a function: it stays
simple, the block is marked as having unknown control flow, and the jump
table object is kept intact. Because the terminator carries no jump
[11 lines not shown]
[BOLT] Release pseudo-probe decoder after emit (#211935)
PseudoProbeRewriter builds sizable data structures and then hold onto
them after they are needed.
PseudoProbeRewriter::postEmitFinalizer() parses the input .pseudo_probe
sections into an MCPseudoProbeDecoder whose address-to-probe and
GUID-to-function-desc maps can be very large (tens of GiB on big
binaries with many probes). This is not used again once probes have been
updated, yet they survive into the memory-heavy DWARF rewrite
(updateDebugInfo), directly inflating BOLT's peak RSS at the worst
possible time.
This frees them at the end of the postEmitFinalizer() calls, before
updateDebugInfo runs.
Stacked on top of the .dwo DIE diff, in large binaries you should
observe ~17% peak RSS wins if your
build uses pseudo probe maps.
[clang] Match MSVC ABI for over-aligned base tail padding on Arm64 (#210461)
When targeting aarch64-pc-windows-msvc, clang laid out a base following
an over-aligned, non-standard-layout base at the wrong offset. MSVC on
Arm64 reuses the over-aligned base's tail padding for the subsequent
base, but clang rounded the base up to a full slot, so the two disagreed
on member offsets, breaking interop between clang- and MSVC-built
binaries.
Fix: store each record's natural non-virtual alignment (excluding
`alignas`/`__declspec(align)` over-alignment) as
`getNonRequiredNVAlignment()`, and on Arm64 fold in a base's natural
alignment rather than its full alignment. This is gated to Arm64 only;
Arm64EC and x64 follow the x64 rule (no reuse) and are unchanged.
Validated against MSVC (Hostx64\arm64\cl.exe
/d1reportSingleClassLayout): clang now matches MSVC Arm64 exactly for
the reduced repro, the original polymorphic case (vftable + empty base +
template), and a range of probe cases covering natural vs.
[3 lines not shown]
[clang] Add __builtin_convert_to_arbitrary_fp
Expose the llvm.convert.to.arbitrary.fp intrinsic as a target-independent
builtin, the inverse of __builtin_convert_from_arbitrary_fp:
unsigned _BitInt(8) b =
__builtin_convert_to_arbitrary_fp(f, "Float8E4M3FN", "round.tonearest", 1);
The result type is derived from the format, so unlike the "from" direction this
is an ordinary CustomTypeChecking builtin with no parser or AST support needed.
It is unsigned _BitInt(N) for a scalar operand and an ext_vector_type of that
for a vector operand.
The rounding mode is a string literal validated with convertStrToRoundingMode,
matching what the IR verifier accepts, and the saturation flag must be an
integer constant expression equal to 0 or 1 since the intrinsic marks it ImmArg.
Clang only permits _BitInt vector elements of power-of-two width, so vector
operands are rejected for the 6-bit formats.
[3 lines not shown]
[HashRecognize] Use loop latch to determine step/start for conditional recurrence (#211916)
The function `matchConditionalRecurrence` iterates over both PHI inputs
and attempts to match each one to determine which is `Start` and which
is `Step`. However, some of the failure conditions in the loop `return
false`, which has the potential to classify some valid CRC loops as not
having a conditional recurrence. The loop here is not really needed at
all-- instead, use `L.getLoopLatch()` to determine `Start`/`Step`, since
the incoming value from the latch block will always be the `Step`. This
also avoids the aforementioned false negative classifications.
Assisted-by: Claude Opus 5
Outline widenUnitStridedLoadStore helper
Reuses it for both normal and speculative unit-strided memaccesses widening.
Apparently, I misread `createVectorPointer` interface (or maybe it changed
throughout the life of this PR), so this also fixes a bug in the new code.
[AMDGPU][GISel] Remove redundant AND on scalar shift amounts
A scalar shift only consumes the low log2(bitwidth) bits of its amount,
so an explicit (and amt, mask) feeding the amount is redundant whenever
mask has all of those low bits set. SelectionDAG already achieves this
via SimplifyDemandedBits on the shift-amount operand; this adds the
equivalent to the postlegalizer combiner for G_SHL/G_LSHR/G_ASHR.
Co-authored-by: Cursor <cursoragent at cursor.com>
[clang] Add __builtin_convert_from_arbitrary_fp
Expose the llvm.convert.from.arbitrary.fp intrinsic as a target-independent
builtin. It reinterprets an integer as the bits of a narrow floating-point
format that has no corresponding C type, and widens it to a native
floating-point type:
_Float16 h = __builtin_convert_from_arbitrary_fp(b, "Float8E4M3FN", _Float16);
Since the destination type cannot be derived from the arguments, it is passed
as a type argument and the builtin is parsed as a keyword, following the
__builtin_convertvector precedent.
The format is a string literal validated with
APFloatBase::isValidArbitraryFPFormat, so every format the IR verifier accepts
is accepted here; formats that no target lowers yet keep being reported by the
backend. The integer operand may be any integer type whose width matches the
format, which covers char/unsigned char for the 8-bit formats and _BitInt for
the narrower ones. Vector operands are supported with matching element counts.
[5 lines not shown]
[OpenMP] Propagate PRESENT to pointee entries in mapper codegen
Extend map-type-modifier propagation in emitUserDefinedMapper to the PRESENT
modifier, but only for entries that have an attach ptr (the pointee data, whose
storage differs from the struct's own). A present modifier on the outer clause
must require that pointee to be present on the device.
This is gated on a new PropagatePresentToPointee argument, which Clang sets from
CGM.getLangOpts().OpenMP >= 60. Before 6.0 the present modifier is treated as
not applying to the pointee: the spec committee confirmed the divergence
between the present motion modifier (to/from) and the present map-type modifier
(map) was unintentional, to be fixed as an OpenMP 6.0 erratum. Only propagation
is gated; present written directly in a mapper's own clause applies at all
versions.
A TODO notes PRESENT should also propagate to the struct's own members, which
is blocked while pointer members use PTR_AND_OBJ.
Update the present-check tests to their final 6.0-gated behavior.
[2 lines not shown]
