[orc-rt] Replace TaskDispatcher with Session-supplied wrapper-runner. (#204965)
TaskDispatcher was only used to run wrapper-function calls that
originated from the controller. Replace it with a callable type:
Session::RunWrapperCall = move_only_function<void(
orc_rt_SessionRef, uint64_t, orc_rt_WrapperFunctionReturn,
orc_rt_WrapperFunction, WrapperFunctionBuffer)>
Each call carries an outstanding ManagedCodeTaskGroup token; the runner
must eventually invoke Fn (which calls Return) or call Return directly
to bail out, otherwise Session shutdown blocks indefinitely.
Clients can supply any callable that satisfies the contract above. The
new QueueingRunner and ThreadPoolRunner classes (replacing
QueueingTaskDispatcher and ThreadPoolTaskDispatcher, respectively) are
provided as off-the-shelf options.
[LV] Avoid zero-width VF in computeVPlanOuterloopVF. (#204918)
RegSize / WidestType may be 0 for types wider than the vector register
size. Clamp VF to at least 1 (scalar), to avoid a crash. This matches
inner loop behavior.
Support for -fsplit-lto-unit option in flang driver (#204904)
Fix for buildbot failures in #202858
This commit fixes a regression introduced in commit
12aefe26cedd9a8f94546cc1f2be285cfddcc861 (Support for -fsplit-lto-unit
option in flang driver). When the compiler is built only for aarch64 one
of the testcase failed.
Added explicit check %if x86-registered-target for this testcase to
resolve the issue.
[tsan] fit Go/s390x mapping under QEMU (#204503)
QEMU linux-user first tries guest_base=0. In that identity-mapped mode,
fixed guest mappings use the same host addresses. On an x86-64 host
with four-level page tables, the Go/s390x meta shadow starts at
144 TiB, beyond the 128 TiB userspace limit, and its mmap fails with
ENOMEM during TSan initialization.
Move the meta shadow down by 32 TiB to
[0x700000000000, 0x780000000000), restoring the 16 TiB gap after the
shadow and placing all Go/s390x TSan regions below 2^47. Correct the
mapping comment's shadow size and ratio.
Failure report and native s390x comparison:
https://github.com/golang/go/issues/67881
QEMU identity guest-base selection:
https://github.com/qemu/qemu/blob/v10.2.3/linux-user/elfload.c#L1036-L1042
[9 lines not shown]
[orc-rt] Sink Session::sendWrapperResult into Session.cpp. NFC. (#204956)
This function is never called inline (except by Session::wrapperReturn,
which is also in Session.cpp), so there's no need for it to be in the
header.
[SimplifyCFG] Avoid threading loop-header branches in convergent functions
SimplifyCFG can fold a conditional branch when the condition is known from
a predecessor. When the destination is a loop header in a convergent function,
this can change the dynamic convergence structure of the loop even though the
scalar CFG rewrite is otherwise valid.
Skip this fold for loop-header branches in convergent functions so convergent
control flow is preserved.
Fixes ROCM-26496.
[clang] Add clang-format-check-format instead to CLANG_TEST_DEPS (#204908)
Ensure that clang-format doesn't break the existing format of its own
source.
Reverts #199169 and #199638.
[AMDGPU][VOPD] Cache load reachability checks in VOPDpairing (#204854)
#201930 causes significant compilation time regression when building
ROCm mathlibs.
Major regressions are caused by repeated queries to `DAG->IsReachable`
to detect possible scalarisation of loads when fusing a pair of
VOPD-capable instructions.
This patch caches the set of reachable loads for every potentially
hazardous load instruction to avoid the need to invoke
`DAG->IsReachable` at all.
[SelectionDAG] Keep split vector atomic store value in a vector register (#201566)
When the value of an ATOMIC_STORE has a vector type whose legalization
action is split (e.g. <4 x half>/<4 x bfloat> on X86 without F16C),
SplitVecOp_ATOMIC_STORE bitcast the value straight to a scalar integer
spanning the memory width. For a split vector that bitcast is expanded
element by element, reassembling the value in GPRs (a long pextrw/shl/or
sequence) before the store.
Instead, keep the value in a vector register when a legal vector form
exists: reinterpret it as a same-shaped integer-element vector (an FP
element type may have no legal vector form, e.g. bfloat on SSE2, while
the integer-of-element-size form does), widen that to a legal vector,
and extract the low integer element of the memory width. This issues the
store directly from a vector register (a single MOVQ/MOVD on X86),
matching the widen-path codegen already produced on AVX targets. Falls
back to the scalar bitcast when no suitable legal vector type exists.
Stacked on top of https://github.com/llvm/llvm-project/pull/197861; and
below of #197862.
[VPlan] Properly check predicates and types in canNarrowOps. (#204948)
Update canNarrowOps to properly check the types of all members match.
Similarly, for recipes with predicates, the predicates must match.
[llvm-objcopy][MachO] Use alignToPowerOf2 instead of alignTo (#204033)
During the review of #203680 I noticed that Mach-O objcopy files seems
to use `alignTo` and import `Alignment.h` to align some offsets to page
boundaries and similar requirements. However, the `alignTo` in
`Alignment.h`, while being intended for powers of 2, requires using an
alignment of type `llvm::Align`, and needs explicit conversion from
`uint64_t` and similar. Single `Alignment.h` includes `MathExtras.h`,
the `alignTo` being invoked ends up being a generic `alignTo` that does
not require powers of 2, and perform divisions and multiplications.
While some of those might be optimized by the compiler into efficient
power of 2 operations, there's an explicit `alignToPowerOf2` version
that is optimized and asserts the alignment is a power of 2 (with
asserts enabled). Since all the alignments should be power of 2 for the
Mach-O binary format, change from `alignTo` to `alignToPowerOf2` to make
the fact more visible (and get the extra safety net of the assertions).
As expected, the test suite of objcopy doesn't show any regressions, but
I have not done a performance benchmark around this either.
[llvm-objcopy][MachO] Align __LINKEDIT entries to pointer size (#203680)
Align Mach-O __LINKEDIT entries to the target pointer size when building
the tail layout. This matches the behavior of ld64 and lld-macho.
dyld on macOS 27 rejects loading dylibs with misaligned __LINKEDIT
entries.
See #203678 for details and the motivation of this fix.
AI Tool Use Disclosure:
Regarding the PR and the linked issue, I have personally wrote every
single part of the PR by myself, and have/ran/verified every single part
of the issue report as well without any AI tool usage.
I have used LLM-based coding agents only for debugging purposes, e.g. to
figure out why the dylib was not loading (from the original bug report),
and figuring out how to build, run, and test my local `llvm-objcopy`.
[VPlan] Skip shl->mul SCEV rewrite for out-of-range shift amounts. (#204921)
getSCEVExprForVPValue rewrites `shl x, c` as `x * (1 << c)` using
ScalarEvolution::getPowerOfTwo, which asserts that the power is less
than the type's bit width.
Only perform the rewrite when the shift amount is less than the
operand's bit width, to avoid assertion.
[FileCheck] Use default colors in input dumps
This patch makes two improvements to colors used in FileCheck input
dumps:
1. Without this patch, input line numbers and ellipses have a
foreground color of black, which is hard to see in a terminal with
a dark color theme. This patch changes that to the terminal's
default color.
2. Without this patch, the input text is accidentally set to bold when
neither `-v` or `-vv` is specified. Perhaps I never noticed
because I tend to always use `-vv`. This patch changes that to use
the terminal's default color.
Case 2 exposes a problem with LLVM's color implementation. Without
this patch, the call to `WithColor`'s constructor actually specifies
bold as `false`, but `WithColor` ignores that when the color is
`SAVEDCOLOR`. While it seems like that should be fixed, I am
concerned about the impact of such a fix on other tools that might
[12 lines not shown]
[AMDGPU] Use SchedModel latencies for Fence barrier edges (#204657)
For memory->fence dependencies, this PR sets the latency of the edge to
the instr latency of the predecessor memory instruction.
During lowering of these fences, we insert the necessary waitcnts, and
we end up waiting for any outstanding memory op at these fences. Thus,
the latency of the edges should be based on latency of the associated
load/stores.
[FileCheck] Use default colors in input dumps
This patch makes two improvements to colors used in FileCheck input
dumps:
1. Without this patch, input line numbers and ellipses have a
foreground color of black, which is hard to see in a terminal with
a dark color theme. This patch changes that to the terminal's
default color.
2. Without this patch, the input text is accidentally set to bold when
neither `-v` or `-vv` is specified. Perhaps I never noticed
because I tend to always use `-vv`. This patch changes that to use
the terminal's default color.
Case 2 exposes a problem with LLVM's color implementation. Without
this patch, the call to `WithColor`'s constructor actually specifies
bold as `false`, but `WithColor` ignores that when the color is
`SAVEDCOLOR`. While it seems like that should be fixed, I am
concerned about the impact of such a fix on other tools that might
[12 lines not shown]
[LoopCacheAnalysis] Generate tests by update_analyze_test_checks.py (#204807)
Since loop interchange has been enabled in the default pipeline,
development on LoopCacheAnalysis, which is used by LoopInterchange, is
becoming more active. So I think it's a good time to support automatic
test generation for LoopCacheAnalysis.
This patch does two things. First, it changes LoopCachePrinterPass from
a loop pass to a function pass to make it possible to use
update_analyze_test_checks.py. Second, it rewrites all the CHECK
directives in the existing LoopCacheAnalysis tests using the script.
[Verifier] Only accept noundef metadata on loads and update metadata tests (#204922)
noundef metadata has been accepted everywhere so far, which seems to be
an oversight. This patch rejects it everywhere except for load
instructions, which seem to be the only ones where it's supposed to be
supported. The other metadata tests are also updated so they are
somewhat similar to each other.
[VectorCombine] Add subvector reduction support to foldShuffleChainsToReduce (#199872)
Extends foldShuffleChainsToReduce to recognise subvector reductions
where the chain narrows through shuffles before extracting lane 0.
The matcher tracks per output lane attribution as the chain is walked.
Each lane carries a per source bitmask of contributing source lanes plus
a poison flag. Shuffles permute these records. Binops union them. At the
extract, lane 0's bitmasks rebuild the reduction as one or more partial
reduce intrinsics. The walk is capped at 32 chain nodes.
Also added new test file with 11 tests:
| Test | Reason |
| ------------------------------------------------------ |
--------------------------------------------- |
| `_add_v4i32`, `_add_v8i16`, `_add_v16i8`, `_add_v64i8` | basic
subvector reductions across types/sizes |
| `_mul_v16i8` | non-add reduction |
[20 lines not shown]
[clangd] Look for resource-dir relative to detected compiler path as a fallback (#203332)
If the standard resource directory (which is searched for relative to the clangd
executable) does not exist, look for one relative to the detected compiler as a
fallback. This handles some packaging schemes where clangd and clang are
installed in different prefixes and the resource directory is only located in the
latter.
Also print an error message to the log if the fallback didn't find an existing
directory either.
