[AArch64] Don't merge branch conditions that both compare memory loads (#206504)
`shouldKeepJumpConditionsTogether` decides whether to fold two integer
branch conditions into a CMP/CCMP chain by pricing the RHS
dependency-chain latency. That ignores register pressure: when both
conditions compare loaded values, merging pins all the loaded operands
live at once to feed the chain instead of consuming them at each split
compare-and-branch. On a load-store target that extends their live
ranges across the region the branch dominates.
Decline to merge when both sides are integer compares of loaded values,
mirroring the machine CCMP pass (which won't speculate loads).
Reference:
https://github.com/llvm/llvm-project/pull/201486#issuecomment-4812845942
[lit] Add configurable slowest-test limit to --time-tests
Bare --time-tests still defaults to 20; --time-tests=N and --time-tests=all optionally control the slowest-test list shown in the histogram.
[OffloadBundler] Bound compressed bundles by header size, not magic scan (#206745)
When multiple offload bundles are concatenated, the unbundler
(clang-offload-bundler) and llvm-objdump --offloading located the end of
a compressed bundle, and the start of the next one, by scanning for the
next "CCOB" magic string starting right after the current header.
A zstd/zlib-compressed payload can legally contain those four bytes, so
the scan could stop in the middle of the compressed data and truncate
the bundle, corrupting the embedded code object. In practice this
produced a "decomposition" failure for hipBLASLt bf16 GEMMs on gfx942.
Use the authoritative total-size field recorded in the compressed bundle
header (format V2/V3) to compute the exact bundle boundary, and only
scan for the next magic past that point. Legacy bundles without a
recorded size (V1) keep the previous magic-scan fallback.
A skippable-frame fixture that embeds "CCOB" inside the compressed
payload is added to exercise the boundary logic from both
clang-offload-bundler and llvm-objdump --offloading.
[lit] Add configurable slowest-test limit to --time-tests
Bare --time-tests still defaults to 20; --time-tests=N and --time-tests=all optionally control the slowest-test list shown in the histogram.
Revert "[clang] Reland: fix getTemplateInstantiationArgs (#208285)" (#208426)
It caused various breakages; see comments on the PR.
This reverts commit 031b773b01700acf82f5977a5aa6024621b8211c.
Revert "[clang] Reject 'auto' storage class with type specifier in C++" (#208436)
Reverts llvm/llvm-project#166004.
Breaks stage 2 build, see comments on PR.
[LLVMABI] Implement the System V X86-64 ABI (#194718)
This PR implements the System V X86-64 ABI for the LLVM ABI Library
prototyped in https://github.com/llvm/llvm-project/pull/140112, and
wires it into clang's Codegen.
`X86_64TargetInfo` is a direct parallel to
`clang::CodeGen::X86_64ABIInfo`, but operates entirely on the
`llvm::abi` type system.
The AMD64 ABI classification spec, argument/return lowering , and the
supporting helpers are all reproduced against `llvm::abi::Type` and
FunctionInfo, keeping the ABI logic frontend-independent.
This, similar to the [BPF ABI implementaiton
](https://github.com/llvm/llvm-project/pull/194031/changes)is gated
under the `-fexperimental-abi-lowering` flag.
AMDGPU: Introduce amdgpu triple arch (#206480)
Move towards using the triple for representing incompatible
ISA changes. Use the subarch field to represent the various
incompatible cases. Previously we pretended a single triple arch
was universally compatible, and only distinguished by function
level subtargets. Move towards using distinct triples to enable
more sophisticated toolchain handling in the future, like proper
runtime library linking.
Introduce a new subarch per unique ISA, but also introduce
"major subarches" which are compatible by a set of covered
minor ISA versions. These map to the existing generic targets.
There are a few placeholder subarch entries, which currently
have missing backing generic arches for codegen.
This should be the preferred triple arch name going forward,
but is treated as an alias of amdgcn. This does not yet change
clang to emit the new triples.
Part of #154925
[X86] subvectorwise-store-of-vector-splat.ll - regenerate test checks to reduce diff in #189971 (#208419)
Add AVX1ORAVX2 check prefix to distinguish from AVX512 codegen
cmake: Remove cmake_crosscompiling check on LLVMSupport target export (#208420)
Revert exported target check added in
00b2f81418233397e601afaeea6d62c47a6c368a
to fix reported mingw cross compile regression. This is the quick fix
which restores the cmake warnings when building libc for amdgpu.
[orc-rt] Move the unit tests into test/unit (#208431)
Move unit tests from orc-rt/unittests into orc-rt/test/unit, completing
the consolidation of the runtime's tests under orc-rt/test (alongside
test/regression and test/tools).
[flang][MIF] Fix COSHAPE and THIS_IMAGE(coarray) type mismatch for non-i64 kinds
Both COSHAPE and THIS_IMAGE(coarray [,team]) return integer arrays whose
element type is determined by the Fortran KIND argument (default: i32).
The lowering code was ignoring resultType in both cases and hardcoding i64,
which caused downstream type mismatches:
- COSHAPE: SimplifyHLFIRIntrinsics assertion in PRODUCT(COSHAPE(y)) when
the product input type (i64) did not match the declared result type (i32).
- THIS_IMAGE(coarray): MLIR verifier error on arith.cmpi with mismatched
operand types (i64 vs i32) in comparisons like THIS_IMAGE(a) /= [5,0,-7].
Fix: derive eleTy from resultType in both genCoshape and genThisImage and
propagate it into the mif.* op's result type. In MIFOpConversion, extract
a shared convertI64SeqToEleTy helper that post-converts the i64 scratch
buffer written by the PRIF runtime into the declared-element-type buffer
when the two types differ. Both MIFCoshapeOpConversion and
MIFThisImageOpConversion call this helper.
Co-authored-by: Claude Opus 4.7 <noreply at anthropic.com>
[DirectX] Add `--dx-embed-debug` and `--dx-pdb-path` flags (#204166)
Add flags for DirectX in `llc`:
* `--dx-embed-debug` to embed debug info into ILDB part of DXContainer
* `--dx-pdb-path` to specify filename/path for PDB file output
This patch does not add flags for clang Driver, they will be implemented
later.
[orc-rt] Add regression test-tool infrastructure (#208398)
Set up the infrastructure for regression tests that build and run a
helper tool, in preparation for testing the logging backends.
- Add test/tools/ for test-support binaries, with a first tool,
orc-rt-smoke-check, wired into ORC_RT_TEST_DEPS and onto lit's PATH.
- Add test/regression/smoke-check.test, which runs the tool and matches
its output with FileCheck, exercising the tool-build and lit plumbing
end to end.
[flang][MIF] Fix COSHAPE and THIS_IMAGE(coarray) type mismatch for non-i64 kinds
Both COSHAPE and THIS_IMAGE(coarray [,team]) return integer arrays whose
element type is determined by the Fortran KIND argument (default: i32).
The lowering code was ignoring resultType in both cases and hardcoding i64,
which caused downstream type mismatches:
- COSHAPE: SimplifyHLFIRIntrinsics assertion in PRODUCT(COSHAPE(y)) when
the product input type (i64) did not match the declared result type (i32).
- THIS_IMAGE(coarray): MLIR verifier error on arith.cmpi with mismatched
operand types (i64 vs i32) in comparisons like THIS_IMAGE(a) /= [5,0,-7].
Fix: derive eleTy from resultType in both genCoshape and genThisImage and
propagate it into the mif.* op's result type. In MIFOpConversion, extract
a shared convertI64SeqToEleTy helper that post-converts the i64 scratch
buffer written by the PRIF runtime into the declared-element-type buffer
when the two types differ. Both MIFCoshapeOpConversion and
MIFThisImageOpConversion call this helper.
Co-authored-by: Claude Opus 4.7 <noreply at anthropic.com>
[ISEL] Fix x86-64 instruction selection bug leaking upper 32 bits (#205600)
The x86 backend had optimization patterns that matched:
`(or (and GR32:$dst, -256), (i32 (zextloadi8 addr:$src)))`
and lowered it to:
`(INSERT_SUBREG (i32 (COPY $dst)), (MOV8rm i8mem:$src), sub_8bit)`
INSERT_SUBREG for sub_8bit emits a movb instruction which preserves the
upper 56 bits. Now, if the GR32 dst came from a node that does not zero
the upper 32 bits (like IMPLICIT_DEF or EXTRACT_SUBREG), those upper 32
bits would be leaked into the resulting register without being zeroed.
This fixes it by ensuring the input operand satisfies def32 which
requires the upper 32 bits of the register to be set.
[Flang][MLIR][OpenMP] Move host op filtering to the omp dialect (#208189)
The MLIR pass that removes operations exclusively intended for the host
from OpenMP target offload modules is currently defined as part of
Flang. However, this is a feature that would benefit from being reusable
by other frontends, as removing such operations is a requirement for all
OpenMP target device modules prior to LLVM IR translation.
By moving the `omp-host-op-filtering` pass out of Flang, it had to be
updated to work on a lower-level LLVM dialect-based representation,
rather than FIR. This simplified some of the existing edge cases, such
as `fir.declare` ops and `fir.boxchar` type handling. In addition, new
function arguments are introduced as placeholders and return values from
host-only functions are removed, producing a cleaner result and
simplifying the pass as compared to previously.
As a result of a later execution of this pass, dynamic dispatch of host
functions via dispatch table using `fir.dispatch`, `fir.type_info` and
`fir.dt_entry` ops would break due to the removal of `fir.dt_entry`
[6 lines not shown]
[OpenACC] apply par dims to reductions in parallel regions (#208258)
For reductions that come from parallel constructs, explicitly set the
GPU parallel dimensions attribute to blockXDim on the acc.reduction_init
and acc.reduction_combine* ops since they will always be gang private
cmake: Gate host library targets on shared library support
This avoids cmake warnings about importing unsupported shared libraries (which
are also host build artifacts) when building libc for amdgpu.
00b2f81 guarded exported library targets and host find_package() calls behind
"if(NOT CMAKE_CROSSCOMPILING)". The intent was to avoid importing the always-shared
targets (LTO, Remarks) on targets which will warn due to not supportind shared
libraries. However CMAKE_CROSSCOMPILING is too broad since it is also set for
usable cross-built libraries.
Change to check if the consuming platform supports shared libraries instead. This
feels not quite precise though, since the found libraries could still be static.
Co-authored-by: Claude (Opus 4.8) <noreply at anthropic.com>
[Flang][MLIR][OpenMP] Fix declare_target globals visibility (#208188)
This patch introduces various changes to the handling of
`declare_target` global variables in Flang:
- Non-`declare_target` globals are unconditionally made "internal" when
compiling for an OpenMP offload target. This prevents potential symbol
redefinition issues related to globals that don't actually exist on the
device.
- Local SAVE variables handling for OpenMP offloading programs is fixed
to prevent their associated "internal" linkage from producing broken
device code for `declare_target enter(...)`.
- When globals are indirectly accessed from the target device (e.g.
`declare_target link(...)`), the associated and unused full-storage
global is marked with "internal" linkage to facilitate later removal.
- `declare_target device_type(host) enter(...)` variables are set to
external linkage when compiling for a target device, causing linker
errors if accessed. This mirrors Clang's behavior.
Fixes #195188, fixes #195468.
Assisted-by: Claude Opus 4.8.
cmake: Remove cmake_crosscompiling check on LLVMSupport target export
Revert exported target check added in00b2f81418233397e601afaeea6d62c47a6c368a
to fix reported mingw cross compile regression. This is the quick fix which
restores the cmake warnings when building libc for amdgpu.
AMDGPU: Fold inline immediates in peephole-opt
Currently the AMDGPU implementation of foldImmediate handles a small
set of special cases that require rewriting the instruction opcode
(COPY -> mov, and fma-like to fmac-like). Most general immediate folding
is handled in SIFoldOperands. Teach PeepholeOpt to fold all inline
immediates into target instructions which should always be an improvement.
This is to help alleviate some phase ordering problems in future changes.
I've also never liked how SIFoldOperands is written and it could use a
rewrite, and this will alleviate some of its responsibilties. It will
always be necessary since some operand folds require additional context
that foldImmediate won't have (namely, we have to make contextually aware
tradeoffs for which operands are worth folding to respect constant bus
restrictions, considering other users of the values).
Co-authored-by: Claude (Opus 4.8) <noreply at anthropic.com>
