[mlir][x86] Accumulate AMX VNNI tiles in place (#233388)
Accumulate AMX VNNI contraction results in place instead of through
temporary storage when the accumulator and result support direct tile
accesses.
This reduces memory traffic and improves performance for contractions in
VNNI layout with a non-zero accumulator.
Assisted-by: Claude
MachineVerifier: Reject dead and live defs of the same register (#233110)
Previously an instruction could have a dead and a live def of the same
physical register, which has no consistent interpretation.
Some passes, namely register pressure tracking, implicitly rely on the
dead flag adjustments performed by running LiveVariables. In cases of
repeated defs, LiveVariables strips dead flags. When a super-register
is only partially used, it marks the super-register def dead and adds
a live implicit-def of the used sub-register.
Prior to #222627, pressure tracking treated live defs as winning. That
PR fixed some asserts by switching to a dead wins system where any
dead flag taints those regunits for the whole instruction. #225079
reverts the remainder of #222627, back closer to the interpretation of
this verifier check.
Only a dead def with a live def of the same register is
rejected. Mixed dead flags on overlapping defs of different registers
[6 lines not shown]
[clang][bytecode] Always check for initialization in RetValue (#233189)
Since we always call `toRValue()` here, all the fields we read need to
be initialized.
Fixes https://github.com/llvm/llvm-project/issues/232904
[llvm][clang] Store the VFS on LLVMContext (#232879)
Backend code occasionally has to resolve an input path: for example
`AsmPrinter` records an absolute path to the optimization record in the
remarks section, and the integrated assembler resolves `.include`
directives coming from inline asm.
Using `llvm::sys::fs` or `llvm::MemoryBuffer::getFile*()` APIs violates
the IO sandbox, which is currently on by default for Clang with
assertions enabled. To comply with the sandbox, this PR stores the
proper VFS on `LLVMContext`, making it available to the LLVM side.
Clients that do not enable the IO sandbox do not need to change, and
will default to using `llvm::vfs::getRealFileSystem()`.
NFC, since nothing uses the VFS yet.
[flang][FIRToMemRef] Propagate nontemporal to the memref ops (#233420)
rewriteLoadOp/rewriteStoreOp forwarded only the tbaa attribute, so the
nontemporal flag set on fir.load/fir.store was silently dropped when
they were replaced by memref.load/memref.store, which model it as well.
[clang][bytecode][NFC] Make string pointer check more explicit (#233360)
We already handled block pointers above, so the isReadablePointerType()
check here is equivalent to isStringPointer().
[IR] Compare the cleanup flag of landingpads (#233424)
The cleanup flag is stored in subclass data, so neither
Instruction::hasSameSpecialState nor FunctionComparator::cmpOperations
looked at it. A `cleanup catch` landingpad was treated as identical to a
plain `catch` which caused SimplifyCFG to merge such landing pads, and
MergeFunctions to merge functions that only differed in it. Either way the
cleanup code is skipped, or run for exceptions it should not see.
[mlir][tosa] Simplify pad shape inference and verification (#233064)
Treat constant padding amounts as ordinary integers during shape
inference. Remove the example assigning a dynamic-padding meaning to
negative values.
Keep PadOp verification local to operand and result types. Check padding
length through its shape type and drop checks of constant operand
contents and derived output extents.
Assisted-by: Codex
[MLIR] Default PDL off for dependency-only builds (#233108)
Default PDL support in PatternMatch to off whenever MLIR is configured
only as a dependency. Keep the existing default for explicitly enabled
MLIR builds and preserve user-provided PDL settings. This speeds up
clang build with CIR enabled.
Assisted-by: Codex
[SLP] Let targets choose the default for -slp-vectorize-once-used
instructions with a single user as standalone seeds after all other
roots are exhausted. It is on by default for every target. Our
downstream AMDGPU toolchain has disabled it since it was introduced,
because it does not pay off for AMDGPU, by changing the option's
default. That is a local divergence from upstream which has to be
carried, and re-checked, through every merge; the recent move of the
vectorizer options into TableGen (#231025) is one such case.
Make the default target-dependent instead, so the setting can live
upstream. Add TTI::preferSLPOnceUsedSeeds(), true by default, so each
target can pick the value that suits its performance, and have AMDGPU
return false. -slp-vectorize-once-used becomes an optional flag that,
when given, overrides the target, as -vectorizer-consider-reg-pressure
does for TTI::shouldConsiderVectorizationRegPressure().
Co-Authored-By: Claude <noreply at anthropic.com>
[LangRef] Relax the LLVM IR data race definition
The current LangRef rules categorize some surprising (to me) cases as data
races (more specifically: they define loads to return `undef` in surprising
situations).
At the core of the problem is the requirement that, if a read R may see more
than one write, R and all the writes it may see must be atomic, otherwise R
returns `undef`. That means that, if an atomic write W and an atomic read R
execute concurrently, a non-atomic write W' that happens-before both of them
causes R to return `undef`. We get that situation if W' initializes the
location in a sequential portion of the program, followed by proper
synchronization before a parallel portion of the program where W and R take
place.
Some litmus tests for the problem:
1. A non-atomic store that happens-before an atomic store and an atomic load:
```
[56 lines not shown]
[InstCombine] Fold exp2(uitofp x) to ldexp(1.0, x) with ninf
`exp2(uitofp iN x) -> ldexp(1.0, zext x)` currently requires N to be
narrower than `int`, because `ldexp` takes a signed `int` exponent.
When N equals the width of `int`, the fold is still correct under `ninf`.
Every LLVM FP type has $E_{\max} \le 16383 < 2^{15} \le 2^{N-1}$, so:
- $x < 2^{N-1}$: signed and unsigned $x$ agree, so $\operatorname{ldexp}(1.0, x) = 2^{x} = \operatorname{exp2}(x)$.
- $x \ge 2^{N-1}$: $\operatorname{exp2}(x) \ge 2^{2^{N-1}} > 2^{E_{\max}}$ overflows to $+\infty$, which is poison under `ninf`.
This only applies to the intrinsic, since the `exp2` libcall may set
`errno` on overflow.
This also enables `pow(2.0, uitofp x) -> ldexp` via the existing
`pow(2^n, x) -> exp2(n * x)` fold.
[InstCombine] Pre-commit tests for exp2(uitofp) -> ldexp with ninf. NFC (#227377)
Add baseline tests for folding `exp2(uitofp iN x)` to `ldexp(1.0, x)`
when N equals the width of C int, including negative tests (no ninf,
source wider than int, libcall).
[CIR] Lower array globals with many trailing zeros as packed structs (#232835)
A global such as `T x[NUM_ELEMENTS] = {1}` reaches LLVM lowering as a
`#cir.const_array` with one explicit element and NUM_ELEMENTS trailing
zeros.
When NUM_ELEMENTS is large that lowering becomes impractical and slow.
This PR fixes such cases by following OG:
1. when there are 8 or more explicit elements of the same type,
the array is lowered as a packed struct with a trailing zero-initialized
array.
2. When an array initializer ends in 8 or more zeros,
the array is lowered as a packed struct with the explicit elements
followed by one zero-initialized array.
`cxx11-initializer-aggregate.cpp` earlier the compilation was
not completing on my system. Now the test compiles
with -fclangir in about 0.1s, and its huge-array globals match classic
codegen. Several
existing tests now produce the same output as classic codegen for
[25 lines not shown]
[libc] Use a syscall wrapper for `sched_get_priority_min` (#233409)
This PR move `sched_get_priority_min` behind the centralized syscall
wrappers, following `libc/docs/dev/syscall_wrapper_refactor.md`.
[ClangLinkerWrapper] Emit extra job to generate -save-temps assembly (#224446)
Summary:
This is an oft-requested feature and unfortunately cannot be done
easily. The approach is just to make a redundant job that does the same
handling, but emits an `--lto-emit-asm` job in the background. Ideally
these get enough parallelism that the total compile time doesn't change
much.
I cannot find a cleaner way to return the 'old driver behavior' for
emitting assembly as was requested. THe LTO pipeline is not built to
generate intermediate files like this, and running this *then* consuming
the assembly on output would break thinLTO, partitioned LTO, or future
object linking.
[libc] Move setxid entrypoints to experimental (#233416)
POSIX requires functions that change credentials (setuid, setgid,
setreuid, setregid, setgroups, initgroups) to affect all threads in a
process. On Linux, the underlying kernel syscalls operate per-thread.
Propagating these changes across all threads requires setxid signal
machinery, which we do not have yet.
Move these entry points to experimental on Linux targets and add a TODO
to their implementations.
Assisted-by: Gemini
[lldb][test] Skip TestMultithreadedStopHook.py on all Linux (#233422)
Failures have been reported in x86_64 premerge jobs: ERROR: test
(TestMultithreadedStopHook.MultithreadedStopHookTestCase.test)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/packages/Python/lldbsuite/test/decorators.py",
line 221, in wrapper
return func(*args, **kwargs)
File
"/home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/API/api/multithreaded/stop_hook/TestMultithreadedStopHook.py",
line 19, in test
self.buildAndRunDriver("driver.cpp test_stop-hook.cpp", "inferior.cpp")
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/packages/Python/lldbsuite/test/lldbtest.py",
line 1877, in buildAndRunDriver
check_call(
[18 lines not shown]
