[clang][CodeGen] Do not use atomicrmw in compound assignments with FP conversions (#209114)
Per C11 6.5.16.2p1, when the left operand of operator `+=` has atomic
type, the right shall have arithmetic type. Do not try the `atomicrmw`
fast-path when FP conversion, arithmetic and truncation are required, as
a single atomic operation may not correctly capture the combined
semantics.
Fixes: https://github.com/llvm/llvm-project/issues/208997.
[offload] add translation of plugin-specific error codes to Offload API ECs (#207208)
liboffload plugins would not always exhaustively map the underlying
API's error values to ol_errc_t, resulting in unintended behavior - e.g.
olCreateProgram, given an invalid binary, would return OL_ERRC_UNKNOWN
instead of OL_ERRC_INVALID_BINARY.
This patch largely eliminates the most glaring cases of this by
assigning an ol_errc_t value to every error code of the underlying API
in every plugin. The proper granularity for this mapping is per-API
call, but that is a large rework that belongs in a separate patch.
Assisted-by: Claude
[VPlan] Relax address constraints during stride analysis (#209168)
VPlanTransforms::convertToStridedAccesses() only handled loads whose
address was a VPWidenGEPRecipe, missing pointer-induction loops where
the address is computed differently. Analyze the address VPValue
directly so these cases are also converted to strided loads.
Fixed #207171
AMDGPU: Convert tests to use subarch from triples in the file (#209869)
This set was using target triple in the source rather than command
line arguments.
Co-authored-by: Claude (Opus 4.8) <noreply at anthropic.com>
[Clang][OpenCL] Promote a few extensions to OpenCL 3.1 core (#204330)
Following 6 OpenCL extensions are promoted to core features in 3.1:
https://github.com/KhronosGroup/OpenCL-Docs/commit/9fff1a87a975
- cl_khr_extended_bit_ops
- cl_khr_integer_dot_product
- cl_khr_subgroup_extended_types
- cl_khr_subgroup_rotate
- cl_khr_subgroup_shuffle
- cl_khr_subgroup_shuffle_relative
A target claiming OpenCL C 3.1 conformance without supporting one of
these features is now diagnosed.
Updated release notes for the change.
Assisted-by: Claude
Reapply "[clang][bytecode] Diagnose pointer subtractions of elements … (#209998)
…of different arrays" (#209969)
This reverts commit 745b946cbbf30708044b4a2c4a7726d0c02ca0de.
Use only one `fold()` for the virtual base offset assertions.
The pointer subtraction _is_ invalid, but the current interpreter
doesn't diagnose it since the `LValueDesignator`s are invalid.
[flang][PFT-to-MLIR] Wrap unstructured Fortran constructs in scf.execute_region
Extend the PFT-to-MLIR (HLFIR/FIR) lowering so unstructured DO and IF
constructs are emitted inside scf.execute_region, hiding their multi-block
CFG behind a single op. OpenACC and OpenMP lowerings that reject
multi-block content (e.g. the "unstructured do loop in combined acc
construct" TODO in OpenACC.cpp) now see a structured op instead.
Flag: -mmlir --wrap-unstructured-constructs-in-execute-region (default on).
An evaluation is wrappable iff all of the following hold:
* wrap flag on
* eval is parser::DoConstruct or parser::IfConstruct
* eval.isUnstructured
* branchesAreInternal(eval) -- every controlSuccessor in the subtree
targets a nested eval or the constructExit
* !hasIncomingBranch(eval) -- no outside eval branches into the body
(PFT's synthetic IfConstruct around `if(c) goto X` absorbs label
[29 lines not shown]
[RISCV][VPlan] Cost vp.merges that are likely to be folded away as free (#209387)
A vp.merge is lowered to a PseudoVMERGE_VVM on RISC-V, and
RISCVVectorPeehole usually folds away PseudoVMERGE_VVMs when its true
operand is a binary op.
This accounts for this in the cost model and plumbs it through the loop
vectorizer. The following reduction loop was previously considered
unprofitable to vectorize, but after this change is now vectorized, and
folds away all its vp.merges:
void three_gathers(const long *a, const long *c, const long *d,
const long *b, long n, long *out) {
long s1 = 0, s2 = 0, s3 = 0;
for (long i = 0; i < n; i++) {
long j = b[i];
s1 += a[j];
s2 += c[j];
s3 += d[j];
[12 lines not shown]
[flang][PFT] record every target of multiway branches
The Evaluation node stored branch targets in a single `Evaluation *`
(`controlSuccessor`). `markBranchTarget` therefore silently dropped every
target after the first when lowering a computed `GO TO` or arithmetic `IF`,
so any analysis that consulted `controlSuccessor` to enumerate branch
targets saw only the first label.
Add `extraControlSuccessors` alongside
`controlSuccessor`. `markBranchTarget` still fills `controlSuccessor` with the
first target, but appends each subsequent distinct target to
`extraControlSuccessors`. Extend the PFT dumper to print the extra targets
after the first ("-> N, M, K"), so PFT dumps expose every branch target.
Co-Authored-By: Claude Sonnet 4.6 <noreply at anthropic.com>
AMDGPU: Use subarch triples in more unit tests
Avoid looking up the target by the cpu name with the legacy
amdgcn name.
Co-authored-by: Claude (Claude-Opus-4.8)
[DTLTO] Add remote compiler option forwarding coverage (#208589)
Add a cross-project DTLTO test that checks the existing LTO
configuration state serialized into the remote Clang command line. The
test uses the validate.py distributor to inspect the generated DTLTO
JSON.
Cover all the existing forwarded options.
Add paired negative checks for optional flags so the test also verifies
they are not emitted when the corresponding configuration state is not
set.
[CodeGen][ARM64EC] Copy first four arguments to FP registers in vararg exit thunks (#209581)
ARM64EC vararg functions receive all types of the first four arguments
in x0-x3. Because x86_64 expects floating-point arguments in FP
registers, always copy x0-x3 to d0-d3 in the exit thunks, matching
MSVC's behavior.
[DA] Rewrite BanerjeeMIV test with safe APInt interval arithmetic
The old banerjeeMIVtest computed inequality bounds using SCEV
arithmetic on 64-bit integers. Intermediate operations like
$(A^{-} - B^{+}) \times Iterations$ could overflow i64 even when all individual
coefficients and loop bounds fit, producing unsound results.
Replace the symbolic bound machinery with a self-contained APInt
interval arithmetic implementation. Key design decisions:
- BanerjeeInterval holds [Lower, Upper] signed-inclusive bounds.
Operations use APInt arithmetic at WideBits, chosen to guarantee no
intermediate overflow:
$$
WideBits = max(8, 2 \times BaseBits + MaxLevels + 8)
$$
This is provably sufficient, each term product needs at most `2 \times BaseBits + 1`
bits, and summing across MaxLevels terms needs at most ceil($\log_2 (MaxLevels)$)
extra bits.
[15 lines not shown]
AMDGPU: Reland: Codegen for v_dual_dot2acc_f32_f16/bf16 from VOP3
For V_DOT2_F32_F16 and V_DOT2_F32_BF16 add their VOPDName and mark
them with usesCustomInserter which will be used to add pre-RA register
allocation hints to preferably assign dst and src2 to the same physical
register. When the hint is satisfied, canMapVOP3PToVOPD recognises the
instruction as eligible for VOPD pairing by checking if it is VOP2 like:
dst==src2, no source modifiers, no clamp, and src1 is a register.
Mark both instructions as commutable to allow a literal in src1 to be
moved to src0, since VOPD only permits a literal in src0.
Original patch had a bug where it did not check if physical src
registers match register class of appropriate operand in fullVOPD
instructions, check is now done via isValidVOPDSrc.
[IR] Add elementwise modifier to atomic loads (#204556)
Add an `elementwise` modifier to atomic loads to represent
per-element atomic semantics for fixed-vector loads.
Without the modifier, a vector atomic load remains a whole-value
atomic operation. With `elementwise`, the load behaves as if it were
expanded into one scalar atomic load per fixed-vector element, without
providing atomicity for the vector value as a whole.
Discussion:
https://discourse.llvm.org/t/rfc-add-elementwise-modifier-to-atomic-loads-and-stores/91100
Revert "[X86] Remove redundant `cmp` before `adc` after `lzcnt` or `tzcnt`" (#210006)
Reverts #208392 - while we investigate a miscompile due to CF/ZF mismatch
AMDGPU: Validate VOPD/VOPD3 physical source registers against operand RC
Replace isVGPR checks with isValidVOPDSrc that validates physical source
registers against the actual combined VOPD/VOPD3 instruction's operand
register classes. Now we also validate operands for VOPD instructions.
[RISC-V] Make Short Forward Branches capable of matching Zibi instructions (#208637)
Add new SFB patterns to make `PseudoCCMOVGPR` capable of matching `beqi`
and `bnei`, so that it can be folded into `PseudoCC*` of `SFBALU_rr`
variants
[AMDGPU] Reject sub-dword format buffer loads and stores (#209703)
An i8 `buffer.{load,store}.format` has no corresponding hardware
instruction so diagnose it in SelectionDAG and fail legalization in
GlobalISel instead of emitting invalid format opcodes
AMDGPU: Convert tests to use subarch from triples in the file
This set was using target triple in the source rather than command
line arguments.
Co-authored-by: Claude (Opus 4.8) <noreply at anthropic.com>
