[flang] Add HLFIR-to-FIR pass pipeline extension points
The FIR optimizer extension points (FIROptEarly, FIRInliner, FIROptLast) all
run after HLFIR has been lowered to FIR, so the HLFIR intrinsic operations
(hlfir.sum, hlfir.matmul, ...) are gone by the time they run. Transformations
that need to see those operations have nowhere to attach.
Add two extension points to createHLFIRToFIRPassPipeline:
* HLFIROptEarly, at the start of the pipeline, before any HLFIR
simplification or inlining.
* HLFIROptLast, just before createLowerHLFIRIntrinsics.
Drivers register passes through registerHLFIROptEarlyEPCallbacks and
registerHLFIROptLastEPCallbacks on MLIRToLLVMPassPipelineConfig. The invoke
methods are const so they can be called on the const config the HLFIR pipeline
receives. With no callbacks registered the pipeline is unchanged.
Co-Authored-By: Claude Opus 5 <noreply at anthropic.com>
[flang] Export fir-opt symbols for MLIR dialect/pass plugins (#212152)
Lets plugins loaded with --load-dialect-plugin / --load-pass-plugin
resolve
MLIR and LLVM symbols against fir-opt, as mlir-opt already does.
---------
Co-authored-by: Claude Opus 4.8 <noreply at anthropic.com>
[CGProfile] Fix unhandled error crash on empty canonical function names (#201821)
A function whose entire name is a strippable suffix canonicalizes to an
empty name, making InstrProfSymtab::create return an error
The current solution with `(void)(bool)` does not really suppress the
error which leads to the crash
[libc] Add program_invocation(_short)_name and tweak err.h functions (#212448)
These GNU extensions hold the name of the program as invoked (argv[0])
and its short name (the basename after the last slash).
Both variables are initialized in the startup code. As with all of our
other variables, they are only available in full build mode.
The trickiest part of this patch are the error reporting functions from
<err.h>, which access this variable, and they are currently enabled in
overlay mode. To make them work, I add an #ifdef to select the right
version. I considered doing something more elaborate, like we have with
`errno`, but that seemed too heavy for a single occurrence.
I also drop the linux check in this function. The documentation says the
functions should print the "last component of the program name", which
"llvmlibc" is not. If someone wants to enable these functions on
non-linux, they can figure out what they want to print here and how.
Assisted by Gemini.
AMDGPU: Export the TargetParser feature bitset (#212946)
Previously this bitset was only used to populate the feature
name string map used by clang. Eventually this will replace
the current bitmask integer. AArch64 already has a similar
interface.
Co-authored-by: Claude (Claude-Opus-4.8)
[X86][AsmParser] Fix compiler crash on division by zero in MS inline asm (#213539)
This fixes issue #213415. If a user writes something like '1 / 0' or '1
% 0' in assembly, the compiler will now show a normal error message
instead of crashing completely.
Fixes #213415
Co-authored-by: 陈纪元 <chenjiyuan at chenjiyuandeMacBook-Air.local>
[GlobalsAA] Handle self-referencing stores in `AnalyzeUsesOfPointer` (#213631)
Correctly recognize that a global address does escape when it is stored
into itself. Such globals were previously incorrectly marked as
non-address-taken.
Fixes: https://github.com/llvm/llvm-project/issues/213232.
[libc] Add optional::value_or and clean up if_nameindex_test TODOs (#213682)
I went through the TODOs in if_nameindex_test.cpp:
- string::operator+=(string_view) was already present in string.h (added
in #210895), so I removed the append_bytes helper and switched to
operator+= directly.
- I added value_or (const & and && overloads) to cpp::optional and added
a test suite for it in optional_test.cpp.
- I replaced pop_front_or with pop_front returning optional<T> and
inlined the .value_or(...) calls in the fake network policy.
- Updated the CMake dependencies to account for the new optional usage.
Assisted by Gemini.
[libc] Add a C unit test framework wrapper and convert existing tests (#213657)
This removes the dependency on the host C library (hermetic tests),
makes sure the tests actually do something in release builds (where
assert() is a noop), and makes better and more consistent failure
messages.
This is just a thin wrapper over the existing framework which repackages
the C++ interface into something consumable by C code. I tried to keep
the interface consistent, but of course, many of the framework features
are C++ only. Registering more than one test function was tricky, so the
framework currently supports only one.
The main trick here was getting the static library linker to extract
LibcCTest.cpp.o from libLibcTest.unit.a. Since C test cases don't
instantiate static CTest objects in their own translation unit like C++
tests do (they cannot do that portably), nothing in the object file
referenced LibcCTest.cpp. I made this work by introducing
libc_c_test_anchor() and calling it explicitly inside the generated
libc_c_test_run() function.
AMDGPU: Export the TargetParser feature bitset
Previously this bitset was only used to populate the feature
name string map used by clang. Eventually this will replace
the current bitmask integer. AArch64 already has a similar
interface.
Co-authored-by: Claude (Claude-Opus-4.8)
AMDGPU: Validate generic processor features in TargetParser emitter (#213774)
Perform some initial validation that the feature set of generic
targets is consistent with the set of covered targets. For now, this
only performs this validation for the subset of frontend exported
features, so is limited to catching missed builtin support. In the future
arbitrary features should be validated, but this is complicated by workaround
features and size features which need to clamp to the common minimum.
Co-authored-by: Claude (Claude-Opus-4.8)
[libc++][pstl] Implementation of parallel std::is_sorted_until() based on std::adjacent_find() (#213445)
This PR adds implementation of a parallel `std::is_sorted_until()` based
on the parallel `std::adjacent_find()` and rebases the parallel
`std::is_sorted()` onto `std::is_sorted_until()`.
The implementation is effectively a one-liner:
```c++
// Find the first pair of adjacent elements that are not in sorted order,
// i.e. comp(rhs, lhs) is true.
auto res = AdjacentFind()(policy, std::move(first), last, [&](Ref lhs, Ref rhs) {
return comp(rhs, lhs);
});
```
Included tests check that:
- Semantics of the iterator-only version is correct.
- Semantics of the predicated version is correct.
- The functions correctly SFINAE out when the first argument is not an
[5 lines not shown]
[libc++][pstl] Implementation of parallel std::reverse() based on parallel for_each (#213487)
This PR implements a parallel version of `std::reverse()` based on the
parallel `__for_each()`.
The implementation walks the first half of the range in chunks, each
chunk is swapped with its mirrored counterpart via `std::swap_ranges()`
and `std::reverse_iterator<>`:
```c++
// Perform a chunked for_each on the first half of the range.
return __cpu_traits<_Backend>::__for_each(
first, first + (last - first) / 2, [first, last](ForwardIterator i, ForwardIterator j) {
// Derive the last position of the mirrored range.
ForwardIterator mirror_last = last - (i - first);
// Swap the elements in the range of the first half with their mirrored counterparts in the second half.
std::swap_ranges(i, j, std::reverse_iterator<ForwardIterator>(mirror_last));
});
```
[7 lines not shown]
[RISCV] Reduce spill/reload pairs when Xqcilo extension is enabled (#212807)
[RISCV] Reduce spill/reload pairs when Xqcilo extension is enabled
Currently, `SelectAddrRegImm26` calls `SelectAddrFrameIndex` first,
causing bare frame-index loads (offset 0) to select 48-bit loads/stores at
ISel. Due to `AddedComplexity=2` on the QC48LdPat patterns, the wide
opcode won over the standard LW/SW even though the resolved frame offset
typically fits simm12.
This led to more spills and reloads in functions which are under high
register pressure because 48-bit loads and stores are not marked easily
rematerializable. Also, simply adding 48-bit loads and stores to
`isLoadFromStackSlot/isStoreToStackSlot` doesn't solve the regression
for the multi call case and only by making Isel produce the plain
32/64-bit loads and store opcodes as the baseline does RA behave
identically.
Therefor this PR fixes the issue by:
[14 lines not shown]
[mlir][xegpu] Support batched matmul in VectorToXeGPU ContractionLowering (#211947)
Generalizes ContractionLowering in
mlir/lib/Conversion/VectorToXeGPU/VectorToXeGPU.cpp so that (batched)
N-D vector.contract ops lower to xegpu.dpas, not just plain 2D matmuls.
---------
Co-authored-by: Claude Opus 4.8 <noreply at anthropic.com>
[WebAssembly] Add funclet bundle to thread local wrapper calls (#213826)
When accessing a thread local variable, Clang generates a thread local
wrapper function that checks if the variable has been initialized, and
if it isn't, initializes it. This is a function call, so if this is
within a funclet (i.e., within a `catchpad` or `cleanuppad`), it needs
the funclet bundle argument, which was missing before. If it lacks a
funclet argument, it will be considered invalid and removed in
WinEHPrepare.
Fixes https://github.com/emscripten-core/emscripten/issues/27448.
[SimplifyCFG] Do not thread branches into uncontrolled convergent regions (#204958)
SimplifyCFG's foldCondBranchOnValueKnownInPredecessor redirects a
predecessor edge around a block whose branch condition is known on that
edge. When the bypassed block and the threading destination are on a
common cycle, this can change the cycle structure and with it the
dynamic instance of an uncontrolled convergent operation inside that
cycle.
On targets with branch divergence, this PR skips the candidate when the
destination can reach an uncontrolled convergent call on a path back to
the bypassed block. Operations using convergence control tokens are left
alone.
Fixes ROCM-26496.
workflows: Merge release-doxygen into release-documenation
These two workflows use the same script and have the same structure,
so it's easier just to have one job that builds both.
[mlir][xegpu] Lower 1D vector transfers to scattered load/store (#213469)
Both 1D vector.transfer_read / vector.transfer_write now lower to the
scattered xegpu.load / xegpu.store path instead of xegpu.load_nd /
xegpu.store_nd. We reserve the nd block path for rank ≥ 2 vectors.
Cleanup Test: Where both RUN configurations (with and without
--xevm-attach-target) produce identical IR for a function, the
duplicated LOAD-ND/LOAD-GATHER (and STORE-ND/STORE-SCATTER) check blocks
are collapsed into a single shared CHECK block.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply at anthropic.com>
