[X86] subvectorwise-store-of-vector-splat.ll - regenerate test checks to reduce diff in #189971 (#208419)
Add AVX1ORAVX2 check prefix to distinguish from AVX512 codegen
cmake: Remove cmake_crosscompiling check on LLVMSupport target export (#208420)
Revert exported target check added in
00b2f81418233397e601afaeea6d62c47a6c368a
to fix reported mingw cross compile regression. This is the quick fix
which restores the cmake warnings when building libc for amdgpu.
[orc-rt] Move the unit tests into test/unit (#208431)
Move unit tests from orc-rt/unittests into orc-rt/test/unit, completing
the consolidation of the runtime's tests under orc-rt/test (alongside
test/regression and test/tools).
[flang][MIF] Fix COSHAPE and THIS_IMAGE(coarray) type mismatch for non-i64 kinds
Both COSHAPE and THIS_IMAGE(coarray [,team]) return integer arrays whose
element type is determined by the Fortran KIND argument (default: i32).
The lowering code was ignoring resultType in both cases and hardcoding i64,
which caused downstream type mismatches:
- COSHAPE: SimplifyHLFIRIntrinsics assertion in PRODUCT(COSHAPE(y)) when
the product input type (i64) did not match the declared result type (i32).
- THIS_IMAGE(coarray): MLIR verifier error on arith.cmpi with mismatched
operand types (i64 vs i32) in comparisons like THIS_IMAGE(a) /= [5,0,-7].
Fix: derive eleTy from resultType in both genCoshape and genThisImage and
propagate it into the mif.* op's result type. In MIFOpConversion, extract
a shared convertI64SeqToEleTy helper that post-converts the i64 scratch
buffer written by the PRIF runtime into the declared-element-type buffer
when the two types differ. Both MIFCoshapeOpConversion and
MIFThisImageOpConversion call this helper.
Co-authored-by: Claude Opus 4.7 <noreply at anthropic.com>
[DirectX] Add `--dx-embed-debug` and `--dx-pdb-path` flags (#204166)
Add flags for DirectX in `llc`:
* `--dx-embed-debug` to embed debug info into ILDB part of DXContainer
* `--dx-pdb-path` to specify filename/path for PDB file output
This patch does not add flags for clang Driver, they will be implemented
later.
[orc-rt] Add regression test-tool infrastructure (#208398)
Set up the infrastructure for regression tests that build and run a
helper tool, in preparation for testing the logging backends.
- Add test/tools/ for test-support binaries, with a first tool,
orc-rt-smoke-check, wired into ORC_RT_TEST_DEPS and onto lit's PATH.
- Add test/regression/smoke-check.test, which runs the tool and matches
its output with FileCheck, exercising the tool-build and lit plumbing
end to end.
[flang][MIF] Fix COSHAPE and THIS_IMAGE(coarray) type mismatch for non-i64 kinds
Both COSHAPE and THIS_IMAGE(coarray [,team]) return integer arrays whose
element type is determined by the Fortran KIND argument (default: i32).
The lowering code was ignoring resultType in both cases and hardcoding i64,
which caused downstream type mismatches:
- COSHAPE: SimplifyHLFIRIntrinsics assertion in PRODUCT(COSHAPE(y)) when
the product input type (i64) did not match the declared result type (i32).
- THIS_IMAGE(coarray): MLIR verifier error on arith.cmpi with mismatched
operand types (i64 vs i32) in comparisons like THIS_IMAGE(a) /= [5,0,-7].
Fix: derive eleTy from resultType in both genCoshape and genThisImage and
propagate it into the mif.* op's result type. In MIFOpConversion, extract
a shared convertI64SeqToEleTy helper that post-converts the i64 scratch
buffer written by the PRIF runtime into the declared-element-type buffer
when the two types differ. Both MIFCoshapeOpConversion and
MIFThisImageOpConversion call this helper.
Co-authored-by: Claude Opus 4.7 <noreply at anthropic.com>
[ISEL] Fix x86-64 instruction selection bug leaking upper 32 bits (#205600)
The x86 backend had optimization patterns that matched:
`(or (and GR32:$dst, -256), (i32 (zextloadi8 addr:$src)))`
and lowered it to:
`(INSERT_SUBREG (i32 (COPY $dst)), (MOV8rm i8mem:$src), sub_8bit)`
INSERT_SUBREG for sub_8bit emits a movb instruction which preserves the
upper 56 bits. Now, if the GR32 dst came from a node that does not zero
the upper 32 bits (like IMPLICIT_DEF or EXTRACT_SUBREG), those upper 32
bits would be leaked into the resulting register without being zeroed.
This fixes it by ensuring the input operand satisfies def32 which
requires the upper 32 bits of the register to be set.
[Flang][MLIR][OpenMP] Move host op filtering to the omp dialect (#208189)
The MLIR pass that removes operations exclusively intended for the host
from OpenMP target offload modules is currently defined as part of
Flang. However, this is a feature that would benefit from being reusable
by other frontends, as removing such operations is a requirement for all
OpenMP target device modules prior to LLVM IR translation.
By moving the `omp-host-op-filtering` pass out of Flang, it had to be
updated to work on a lower-level LLVM dialect-based representation,
rather than FIR. This simplified some of the existing edge cases, such
as `fir.declare` ops and `fir.boxchar` type handling. In addition, new
function arguments are introduced as placeholders and return values from
host-only functions are removed, producing a cleaner result and
simplifying the pass as compared to previously.
As a result of a later execution of this pass, dynamic dispatch of host
functions via dispatch table using `fir.dispatch`, `fir.type_info` and
`fir.dt_entry` ops would break due to the removal of `fir.dt_entry`
[6 lines not shown]
[OpenACC] apply par dims to reductions in parallel regions (#208258)
For reductions that come from parallel constructs, explicitly set the
GPU parallel dimensions attribute to blockXDim on the acc.reduction_init
and acc.reduction_combine* ops since they will always be gang private
cmake: Gate host library targets on shared library support
This avoids cmake warnings about importing unsupported shared libraries (which
are also host build artifacts) when building libc for amdgpu.
00b2f81 guarded exported library targets and host find_package() calls behind
"if(NOT CMAKE_CROSSCOMPILING)". The intent was to avoid importing the always-shared
targets (LTO, Remarks) on targets which will warn due to not supportind shared
libraries. However CMAKE_CROSSCOMPILING is too broad since it is also set for
usable cross-built libraries.
Change to check if the consuming platform supports shared libraries instead. This
feels not quite precise though, since the found libraries could still be static.
Co-authored-by: Claude (Opus 4.8) <noreply at anthropic.com>
[Flang][MLIR][OpenMP] Fix declare_target globals visibility (#208188)
This patch introduces various changes to the handling of
`declare_target` global variables in Flang:
- Non-`declare_target` globals are unconditionally made "internal" when
compiling for an OpenMP offload target. This prevents potential symbol
redefinition issues related to globals that don't actually exist on the
device.
- Local SAVE variables handling for OpenMP offloading programs is fixed
to prevent their associated "internal" linkage from producing broken
device code for `declare_target enter(...)`.
- When globals are indirectly accessed from the target device (e.g.
`declare_target link(...)`), the associated and unused full-storage
global is marked with "internal" linkage to facilitate later removal.
- `declare_target device_type(host) enter(...)` variables are set to
external linkage when compiling for a target device, causing linker
errors if accessed. This mirrors Clang's behavior.
Fixes #195188, fixes #195468.
Assisted-by: Claude Opus 4.8.
cmake: Remove cmake_crosscompiling check on LLVMSupport target export
Revert exported target check added in00b2f81418233397e601afaeea6d62c47a6c368a
to fix reported mingw cross compile regression. This is the quick fix which
restores the cmake warnings when building libc for amdgpu.
AMDGPU: Fold inline immediates in peephole-opt
Currently the AMDGPU implementation of foldImmediate handles a small
set of special cases that require rewriting the instruction opcode
(COPY -> mov, and fma-like to fmac-like). Most general immediate folding
is handled in SIFoldOperands. Teach PeepholeOpt to fold all inline
immediates into target instructions which should always be an improvement.
This is to help alleviate some phase ordering problems in future changes.
I've also never liked how SIFoldOperands is written and it could use a
rewrite, and this will alleviate some of its responsibilties. It will
always be necessary since some operand folds require additional context
that foldImmediate won't have (namely, we have to make contextually aware
tradeoffs for which operands are worth folding to respect constant bus
restrictions, considering other users of the values).
Co-authored-by: Claude (Opus 4.8) <noreply at anthropic.com>
AMDGPU: Constant fold instructions with inline immediate operands
Previously we would only try to perform constant folding and simplifications
when an immediate was folded into an instruction, not if the input was already
a folded constant.
Co-authored-by: Claude (Opus 4.8) <noreply at anthropic.com>
AMDGPU: Do not report inline immediates as legal for generic operands
This has one test change in an SI_CS_CHAIN_TC* test. Either this is fine
or the instruction definition should be changed.
[OffloadBundler] Bound compressed bundles by header size, not magic scan
When multiple offload bundles are concatenated, the unbundler
(clang-offload-bundler) and llvm-objdump --offloading located the end of a
compressed bundle, and the start of the next one, by scanning for the next
"CCOB" magic string starting right after the current header.
A zstd/zlib-compressed payload can legally contain those four bytes, so the
scan could stop in the middle of the compressed data and truncate the
bundle, corrupting the embedded code object. In practice this produced a
"decomposition" failure for hipBLASLt bf16 GEMMs on gfx942.
Use the authoritative total-size field recorded in the compressed bundle
header (format V2/V3) to compute the exact bundle boundary, and only scan
for the next magic past that point. Legacy bundles without a recorded size
(V1) keep the previous magic-scan fallback.
A skippable-frame fixture that embeds "CCOB" inside the compressed payload
is added to exercise the boundary logic from both clang-offload-bundler and
llvm-objdump --offloading.
[VPlan] Eliminate some vec temps with ArrayRef (NFC) (#207432)
The enabling change is e56187575 ([ArrayRef] Make iterator_range
constructor const-agnostic, #205183).
[libc++] Implement LWG4472: std::atomic_ref<const T> can be constructed from temporaries (#208131)
## Summary
- Implements LWG4472, i.e., adds a deleted `atomic_ref(T&&)` overload to
the primary template and three partial specializations of `atomic_ref`.
## Test
- Added `static_assert`s in `ctor.pass.cpp` asserting
`atomic_ref<T>`/`atomic_ref<const T>` reject construction from
`T&&`/`const T&&`.
Resolve #189840
[Offload] Make compressed offload bundle header little-endian (#206744)
The compressed offload bundle (CCOB) header integer fields (Version,
Method, FileSize, UncompressedFileSize, Hash) were serialized and read
in
host-native byte order. The on-disk format is little-endian, so on
big-endian hosts these fields were byte-swapped: writing produced a
malformed header, and reading misparsed the size, making
`llvm-objdump --offloading` crash/misbehave on s390x. This is also why
the
earlier bundle-size fix had to be reverted.
Make the header little-endian on every host:
- Read side: declare the `RawCompressedBundleHeader` fields as
`support::ulittle16_t` / `ulittle32_t` / `ulittle64_t`, so the bytes are
always interpreted as little-endian regardless of host.
- Write side: emit the header with
`support::endian::Writer(OS, endianness::little)` instead of host-native
[10 lines not shown]
[NVPTX] Add native `tanh.approx` support for f16/f16x2/bf16/bf16x2 (#203257)
Adds NVPTX backend support for the native PTX `tanh.approx` instructions on half-precision and bfloat types:
- `tanh.approx.f16` and `tanh.approx.f16x2` (PTX 7.0+, sm_75+)
- `tanh.approx.bf16` and `tanh.approx.bf16x2` (PTX 7.8+, sm_90+)
Adds a `FTANHInst` TableGen class with the new patterns in NVPTXInstrInfo.td and splits `ISD::FTANH` out of the unconditional `f16/bf16 -> f32` promotion loop in NVPTXISelLowering.cpp, marking it Legal when the target supports it (scalars promote, vectors expand on older targets). Also guards `tanh.approx.f32` behind sm_75 and adds the
missing `AddPromotedToType` for bf16.
PTX Spec Reference:
https://docs.nvidia.com/cuda/parallel-thread-execution/#half-precision-floating-point-instructions-tanh
Signed-off-by: Varad Rahul Kamthe <vkamthe at nvidia.com>
[SelectionDAG] Widen even 33-bit-magic udiv on free-zext targets (#207634)
On 64-bit targets, #181288 lowers a 32-bit unsigned division by a
constant with a 33-bit magic number (the `IsAdd` case) to a widened
64-bit high-multiply (`MULHU`/`UMUL_LOHI`), e.g. `x / 7` becomes a
single `mulq`/`umulh`/`mulhu`. That only reaches odd divisors. This
patch extends it to even divisors on targets where the `i32 -> i64`
zero-extension is free.
[AMDGPU] Clean up GFX13 feature list
GFX13 features were previously mostly inherited from GFX12/GFX1250.
- define an explicit gfx13 target parser feature list, including
gfx13-insts and cvt-pknorm-vop3-insts.
- remove transpose load F4/F6 support from gfx13 (previously copied from gfx12)
[VPlan] Add VPInstruction::Intrinsic opcode, use for scalar intrinsics. (#207541)
This patch adds a new Intrinsic opcode to VPInstruction, initially used
for generating calls to scalar intrinsics. The intrinsic ID as integer
is the last operand (i.e. the called function). Alternatively we could
also create the needed intrinsic declarations and pass the function
directly, but that would add potentially unused declarations, if we
decide to not vectorize.
The first patch migrates just VScale, but there are other opcodes
matching directly to intrinsics, which will be replaced in follow ups.
It also gives more flexibility going forward, e.g. allows emitting
min/max intrinsics when expanding SCEV min/max expressions.
PR: https://github.com/llvm/llvm-project/pull/207541
[flang][Driver] Add option for real sum reassociation (#207377)
Compiler driver option for #207371: -freal-sum-reassociation.
Disabled by default.
Assisted-by: Codex
