[SandboxVectorizer] Implement topdown/bottomup vectorizers in unison (#205249)
Extend the Sandbox Vectorizer's `bottom-up-vec` pass so a single
implementation can vectorize in either direction, and add the top-down
strategy that walks def-use chains forward from a seed.
Direction selection
--------------------
The pass direction is chosen from the Region's auxiliary pass argument:
"bottom-up" (or empty, the default) and "top-down" map onto a
SchedDirection, and any other value is rejected with a fatal usage
error.
The vectorizer always runs in the same direction as the scheduler.
Top-down traversal
------------------
Bottom-up starts from a seed slice (e.g. stores to consecutive
addresses)
and recurses into operands. Top-down instead starts from a seed of
[40 lines not shown]
[SandboxVec][DAG] Fix notifyEraseInstr to skip scheduled neighbors (#212868)
Guard both loops with !PredN->scheduled() so scheduled neighbors are
left untouched, and add a unit test that erases a node with one
scheduled and one unscheduled predecessor to cover the fix.
[BOLT] Fix RELR addend updates for non-zero image bases (#213000)
This patch fixes an AArch64 BOLT failure exposed by LLVM commit
`fbba327208d7f6994f9a28a51dca3e913e3b444a`, which enabled packed
relative relocations (RELR) globally. BOLT incorrectly used a virtual
relocation address as a file offset when updating RELR addends,
corrupting instrumented executables when their image base was non-zero.
The observed CMake compiler-check failure was only a consequence of the
corrupted BOLT-instrumented Clang executable crashing; the defect itself
is in BOLT, not CMake.
The regression test links an AArch64 PIE with a non-zero image base and
verifies that BOLT updates the RELR addend at the correct file offset.
Assisted-by: Codex
[msan][test] Disable UB cases in vararg_shadow.cpp (#204246)
This patch comments out the tests for char and float, which have
undefined behavior: `warning: second argument to 'va_arg' is of
promotable type 'char'/'float'; this va_arg has undefined behavior
because arguments will be promoted to 'int'/'double' [-Wvarargs]`.
The non-UB way to rewrite the tests is to use `va_arg()` with int and
double types (when the original variables are char and float), but there
are already test cases for int and double.
Note also that the float test makes an assumption that
`cast<float>(cast<double>(float))` will maintain the original shadow,
which MSan need not guarantee. In particular,
https://github.com/llvm/llvm-project/pull/204197 breaks that assumption
(which led to a buildbot breakage and revert).
[libc][__support] Implement exact linear binning for TLSFFreeStoreImpl
Previously, size_to_bit_index used size >> UNIT_SIZE_LOG2 for linear bins,
which mapped size 24 to bin 1 and size 17 to bin 1, causing a mismatch
between allocation request sizes and physical block bucket sizes.
This change introduces exact mapping for linear bins:
- Sizes <= MIN_INNER_SIZE map to bin 0.
- Larger linear sizes use ((size - MIN_INNER_SIZE - 1) >> UNIT_SIZE_LOG2) + 1.
- Exponential bins remain unchanged (verified by z3).
TAG=agy
CONV=811f354e-5729-4a5f-af17-de4b5ca9e647
[NVPTX] Cleanup PTX virtual register encoding (NFC) (#212395)
Unify the scheme used to carry PTX virtual registers through
`MCOperand`s. The register class was encoded and decoded
by two hand-written switchtables, one in
`NVPTXAsmPrinter::encodeVirtualRegister` and one in
`NVPTXInstPrinter::printRegName`, kept in sync only by
comment. Both now share a `VirtualRegisterKind` enum and
`getVirtualRegisterPrefix` in `NVPTXBaseInfo.h`, which also
subsumes `getNVPTXRegClassStr` and drops the stale
`%f`/`%fd` cases left over from the typed float register
classes.
Co-Authored by Opus-5.
Revert "[github] Move BOLT reviewers to team (#211422)" (#213141)
We haven't really being tagged in new PRs anymore, so it looks like this
is not working.
Revert for now until we can fix this, so we don't miss PRs.
This reverts commit 04f124f6151e8c8410e28aba28dc38e590e34c58.
AMDGPU: Give v_cvt_sr_pk_bf16_f32 its own subtarget feature
v_cvt_sr_pk_bf16_f32 was gated on bf16-cvt-insts, but that feature is
also present on gfx950 where the (non-sr) v_cvt_pk_bf16_f32 was first
added. The stochastic-rounding v_cvt_sr_pk_bf16_f32 was only added
for gfx1250 and has no gfx950 encoding, so it would mis-select and
later hit the "Invalid opcode" assert. Introduce cvt-sr-pk-bf16-f32-inst,
currently added to gfx13 and 125*
Co-authored-by: Claude (Claude-Opus-4.8)
[clang-format] Prevent re-assigning type on finalized tokens (#210763)
Prevents a finalized token inside modifyContext from being reassigned
through the setType member function by checking if the token is
finalized.
This ensures ill-defined code like does not trigger an assertion failure
during reformatting.
Fixes #210509
[Offload][Test] Add llvm bin in the PATH for offload-unit suite (#213149)
This PR makes the offload-unit suite put llvm bin directory on PATH so
that tests need lld can find. It fixes the issue exposed in:
https://github.com/llvm/llvm-project/pull/212860
[SBVec] Add top-down vectorization to the unified Sandbox Vectorizer
Extend the Sandbox Vectorizer's `bottom-up-vec` pass so a single
implementation can vectorize in either direction, and add the top-down
strategy that walks def-use chains forward from a seed.
Direction selection
--------------------
The pass direction is chosen from the Region's auxiliary pass argument:
"bottom-up" (or empty, the default) and "top-down" map onto a
SchedDirection, and any other value is rejected with a fatal usage error.
The vectorizer always runs in the same direction as the scheduler.
Top-down traversal
------------------
Bottom-up starts from a seed slice (e.g. stores to consecutive addresses)
and recurses into operands. Top-down instead starts from a seed of
consecutive loads and recurses into *users*:
[32 lines not shown]
[SandboxIR] Fix notifyEraseInstr to skip scheduled neighbors
Guard both loops with !PredN->scheduled() / !SuccN->scheduled() so
scheduled neighbors are left untouched, and add a unit test that erases
a node with one scheduled and one unscheduled predecessor to cover the
fix.
[AMDGPU] Use a single SubtargetPredicate for fp8/bf8 -> f32 conversions (#212888)
Add FeatureCvtFP8SDWASrcSel for the form that takes the byte from the SDWA src0_sel field and FeatureCvtFP8ByteSel for the form that takes it from a byte_sel operand. Both imply FeatureFP8ConversionInsts, and a subtarget provides one of them, so the multiclass-generated HasCvtFP8SDWASrcSel and HasCvtFP8ByteSel replace the GFX9 and GFX11Plus predicates outright.
Assisted-By: Claude Opus 5
[flang] Add a pass to get OpenACC device ptr for CUDA kernel (#212299)
When a CUDA kernel is launched inside an OpenACC data region, it does
not properly get the device pointers and instead uses host data. This PR
adds a pass to get the device pointers set up by the OpenACC data
construct and pass them explicitly to the CUDA kernel.
Note: A possibly non-contiguous array argument is currently not supported. This pass will skip these cases.
---------
Co-authored-by: Yebin Chon <ychon at nvidia.com>
[mlir][OpenACC] Atomicize contended shared array reduction updates (#212971)
Example:
```fortran
!$acc parallel loop gang reduction(+:a)
do i = 1, N
!$acc loop worker reduction(+:a)
do j = 1, M
a(i) = a(i) + b(j,i)
end do
end do
```
In this code, the worker accumulator is block-shared, so every worker
updates
the same element with a plain read-modify-write and all but one partial
is lost.
Fix: make in-place updates of a block-shared array accumulator atomic,
[7 lines not shown]
