[AArch64] Reorder callee-saved spills to maximise load/store register groupings (#229349)
Currently, the callee-saved spills reorder to maximize ZPR pairs, but do
not reorder to maximize ZPR quads. Modify the reordering algorithm to
also maximize ZPR quads and pairs.
Note: Quad spills are not enabled yet but will implemented in a
subsequent PR.
The main idea behind the reordering is to align the starting offset of
the ZPR quads when we begin the spill due to the conditions required to
form a ZPR quad store/load. To do this, we calculate the starting offset
and use pairs/single spills to align the offset.
RISCV: Make GPRAll non-allocatable
GPRAll contains the reserved DUMMY_REG_PAIR_WITH_X0, which cannot be
spilled, so the class should not be allocatable. Nothing creates virtual
registers in it yet, but inline asm constraint lookup returns GPRAll for
cases like {x8} with i32 on riscv64.
This increases spilling in some tests due to RegisterClassInfo defects.
RegisterClassInfo::computePSetLimit subtracts the reserved registers of
the largest allocatable class in the GPR pressure set, which is now
GPRF16 instead of GPRAll. The reserved dummy unit is no longer
subtracted, raising the limit by one.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[AMDGPU] Removed unneeded entries from isRenamedInGFX9. NFC. (#231396)
The _sdwa forms are handled separately in pseudoToMCOpcode. The _e64_dpp
and _fake16_e64 forms only exist in GFX11+.
[RISCV] Support copies between GPR and FPR16 (#231382)
Handle GPR-to-FPR16 and FPR16-to-GPR copies in copyPhysReg using
`FMV.H.X` and `FMV.X.H`.
This fixes a GlobalISel crash when half-precision values cross register
banks, such as arguments and return values passed in GPRs under the
soft-float ABI.
Reproducer: https://godbolt.org/z/n8EaYjvhG
Follow up of #231097
[libc][stdlib] Optimize bsearch() by reordering comparison branches (#230823)
During a binary search, compare_result == 0 is triggered at most once,
or 0 times when the key is not found, whereas compare_result < 0 and
compare_result > 0 are triggered multiple times across loop iterations.
Check < 0 and > 0 before the equality case so the loop does not test for
== 0 first on every iteration.
With clang -O2, the loop body changes as follows:
x86-64:
```diff
movq %rbp, %rsi
callq *(%rsp)
testl %eax, %eax
- je .LBB0_7
js .LBB0_5
+ je .LBB0_7
notq %r13
[18 lines not shown]
[Passes] Run MergeICmps/ExpandMemCmps in LTO post-link pipeline (#230562)
Otherwise memcmp will never get expanded if running the combination of
ThinLTO pre-link and LTO post-link, which is what Rust does, and what
unified LTO does.
The pipeline position is chosen to roughly match what the module
optimization pipeline does. There is no compile-time impact.
For https://github.com/rust-lang/rust/issues/163677.
(cherry picked from commit 17379353858fbac934f573a22e6aa5acce6884d7)
[Mips] Remove experimental warning bits in a few places (#223230)
- Remove the warning message "warning: MIPS-I support is experimental" from
MipsSubtarget.cpp. The warning can interfere with build systems that do not
expect diagnostic output on stderr.
- Remove experimental / highly experimental from a few MIPS processors.
MIPS I is on track and the rest I wouldn't consider experimental anymore.
(cherry picked from commit 73641283fc038d819b5a0d3e5fd1315d501ff9e1)
[llvm-objcopy][COFF] Keep COMDAT section definition symbols when stripping (#228783)
--strip-unneeded and --discard-all removed the section definition symbol of
a COMDAT section if no relocation referenced it directly, which is the
common case since relocations target the COMDAT leader. The aux record of
that symbol carries the COMDAT selection; without it lld discards the
section as a leaderless COMDAT and silently drops relocations against it.
This broke e.g. .refptr.* sections in MinGW objects, producing executables
that load garbage instead of the referenced pointer.
Assisted-by: Claude Opus 5.5
(cherry picked from commit cf40a83cb398897d21bb54c5a7a2eecb8d0633b3)
release/23.x: [llvm-ar][GOFF] Implement symbol attributes for GOFF archives (#214528)
Backport 847fb5cfde68fdea490171bc24087ebb0cd90738
This backport has been modified slightly by updating the LIT tests to remove
the dependency on llvm-nm. llvm-nm added GOFF support after the LLVM 23.x release
cut, and its new functionality is not available in LLVM 23.
workflows/release-binaries: Improve Windows installer (#220414)
The Windows WiX installer was missing two things for its setup:
1. The "product ID" which is used to uninstall any previous installed
LLVM versions. This avoids having versions living side-by-side with each
new release. I've reused the same GUID as the current 23.1.0 Win64
installer, to continue that "line" of product. The AArch64 installation
and any prior LLVM version (NSIS) before 23.1.0 will need to be
uninstalled manually, if we want to avoid side-by-side installatin.
2. A "per machine" scope which avoids messy installations which are
installed globally, but cannot be uninstalled globally, and available
only for the current user. After this PR, the installation is available
/ uninstallable to all users on the machine.
Since LLVM currently requires cmake 3.20 for building, and hasn't been
bumped yet to 3.31, also provide a workaround for the WiX scope setup.
[4 lines not shown]
[Windows] Export Registry<T> members and accessors for plugins again (#221296)
Since #185141 reworked `llvm::Registry`, the registered-node list lives
behind the explicitly specialized
`llvm::detail::getRegistryLinkListInstance<R>()` accessor defined by
`LLVM_DEFINE_REGISTRY`, and the `Head`/`Tail` static data members that
used to be referenced across objects are gone (#163391 had special-cased
those).
`extract_symbols.py` handles neither. The accessor is a `??$` function
template specialization, which the script drops on the assumption that
it can be instantiated in the importing translation unit; here it
cannot, it is defined once by `LLVM_DEFINE_REGISTRY`. And `Registry<T>`
members are only exported when some object references the `Registry`
template, which nothing does anymore because every use goes through the
accessor. On a static clang.exe built with
`LLVM_EXPORT_SYMBOLS_FOR_PLUGINS`, the export table went from 11
`Registry<clang::PluginASTAction>` symbols on 22.1.8 to zero on 23.1.0,
and linking a plugin fails:
[31 lines not shown]
[Instrumentation] Define default value as float (#231383)
In #230746, the Instrumentation command line options were moved to
TableGen. The `icp-vtable-percentage-threshold` default value is now
defined as `0.995`, which is a double value, while the type for this
option is a float, triggering a C4305 warning with MSVC.
This fixes the issue by changing the literal to `0.995f`.
[mlir][bufferization] handle exit aliases in buffer-loop-hoisting (#226951)
As discussed in #223944, `buffer-loop-hoisting` rejects an allocation in
`scf.while` even though the buffer is never passed back to the region
containing the allocation. The allocation and the forwarded `after`
argument belong to different regions, so their common dominator is the
block containing the loop, which in turn makes `isLegalPlacement` return
false and reject the hoist.
This PR adds a fallback for `memref.alloc` directly in the `before`
region of `scf.while` when the existing common-dominator check rejects
the hoist. It keeps the full alias set and rejects hoisting if an alias
is carried back to `before`. It also checks uses of all aliases,
including loop results, for capture, deallocation and explicit
observations of allocation identity. The fallback requires the loop's
parent to stop further loop hoisting: it must either be a non-loop
operation or have the `HasParallelRegion` trait. This restriction avoids
exposing the allocation to further hoisting through enclosing loops with
incorrectly classified execution semantics, as in #226090.
[9 lines not shown]
[libcxx][CMake] Rework LLVM_ENABLE_ASSERTIONS interaction w. libcxx hardening (#229359)
[Libc++ hardening modes](libcxx/docs/Hardening.md) can be set in 2 ways:
- The default hardening mode is set when compiling libc++, this is
applied to the pre-compiled object files of libc++, and also sets the
default value for the hardening macros used in the headers of libc++.
- Users of libc++ can override the default in the headers for their
builds by setting a macro.
When LLVM_ENABLE_ASSERTIONS is set, and we are *using* libc++ to build
LLVM, we want to harden our binaries and tests (of other LLVM
components, NOT libc++ itself).
Additionally we also set the default hardening mode of libc++ itself
when performing an assertions build. This is in theory orthogonal to the
hardening of binaries and tests. It enables runtime hardening checks not
in the compiler itself, but in the code that links against libc++ (e.g.
applications compiled using the toolchain we just built).
[8 lines not shown]
[clangd][test] Add `clang` to `CLANGD_TEST_DEPS`. (#230061)
86a48da46ff5 (#193426) added `prebuilt_module_file_reuse.test`, which
invokes the `clang` binary directly to precompile a module file, but did
not add the binary to the dependencies of the corresponding CMake
target. This PR fixes that.
Signed-off-by: Ingo Müller <ingomueller at google.com>
[libunwind][AArch64] Avoid assuming 16-byte frame in RA_SIGN_STATE test (#230518)
Disabling the GlobalISel Localizer pass at -O0 (#177359) moves the
materialization of -1 constant argument of _Exit before the call to bar.
The value spills across the call, increasing the frame size of foo from
16 to 32 bytes.
The inline asm adds 16 to the current SP before signing LR. With the
larger frame it signs with the wrong SP and overwrites the spill instead
of the saved LR. Use the frame address to recover the entry SP in the
generated prologue and preserve the current SP in x16.
Assisted-by: codex
Attributor: Invalidate analyses when attributor-light only changes attributes
attributor-light only invalidated analyses for functions whose IR it
rewrote, but declared all function analyses preserved. Attribute changes
were not tracked, so e.g. a caller's MemorySSA could go stale after the
callee became readnone. Also invalidate functions whose attributes or
call site attributes changed, and their callers, like FunctionAttrs.
Keep the marker which stops the function simplification pipeline from
running again on CGSCC revisits, since attribute changes alone do not
need re-simplification. Without this, already simplified functions were
simplified again, making attributor-light much more expensive.
No compile time or output change on CTMark.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
TableGen: Emit GlobalISel combiner apply roots in a deterministic order (#231279)
Rules with more than one apply root emitted them in an order that
depended on pointer values, so the generated combiners for AArch64,
AMDGPU and RISC-V changed between runs. Use a SetVector instead of a
DenseSet.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[LLVM-C] Add bindings for querying and setting dso_local (#230659)
Add LLVMIsDSOLocal() and LLVMSetDSOLocal() to the C API, allowing users
to query and set the dso_local runtime preemption specifier on global
values such as global variables, functions and aliases.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[InstCombine] Limit recursion depth when evaluating in a different type (#217879)
canEvaluateTruncated(), canEvaluateZExtd() and canEvaluateSExtd()
recurse once
per level of the expression they are asked about, with nothing bounding
how far
they go, so a deep enough chain of eligible operations feeding a cast
overflows
the stack. Chains like that are not exotic: C and OpenCL promote char
and short
arithmetic to int, so an unrolled loop over a char array turns into a
long chain
of i32 operations rooted at a truncate.
Give up past MaxTypeEvalDepth levels. Depth is bounded rather than the
number of
values visited, because depth is what costs stack -- a wide but shallow
expression is still folded no matter how many values it has. Giving up
early
[6 lines not shown]
[clang] Replace PointerUnion::dyn_cast with llvm::dyn_cast (NFC) (#231373)
PointerUnion::dyn_cast has been soft-deprecated in favor of
llvm::dyn_cast and llvm::dyn_cast_if_present. This patch replaces the
former with llvm::dyn_cast where the operand is guaranteed to be
nonnull.
ClassTemplateSpecializationDecl::getSpecializedTemplateOrPartial()
always returns a nonnull pointer by construction regardless of whether
the specialization is from a primary template or a partial
specialization.
Assisted-by: Antigravity
[clang][CodeGen] Skip re-emission of already-defined globals in EmitDeferred
When a deferred decl was already emitted and its existing definition has a different LLVM type than the requested one (e.g. an inline variable whose initializer folded to an anonymous union type), EmitDeferred called GetAddrOfGlobal with ForDefinition, which replaced and erased the existing definition. Erasing the defined global dangles pointers held by callers, e.g. the AssociatedData of entries in llvm.global_ctors, leading to a use-after-free during EmitCtorList.
Check for an existing definition of the same entity before requesting a correctly typed value, and skip the re-emission. This keeps GetOrCreateLLVMGlobal's invariant that a ForDefinition request returns a global of exactly the requested type.
Add the OpenMP non-modules reproducer as a regression test alongside the header-unit test, and assert single emission via the position of the guard variable in both.
Co-authored-by: Michael Park <mcypark at gmail.com>
[SPIRV] Preserve AMDGPU metadata through NonSemantic.AuxData (#215510)
The PR implements an extension to `NonSemantic.AuxData` spec to add
`NonSemanticAuxDataInstructionMetadata`, which enables preservation of
metadata for instructions other than `GlobalVariable` and `Function`
objects for some AMDGPU metadata.
Part 2 of #213685 split, stacked on top of #215507.
[CIR][AMDGPU] Implement the clang::atomic statement attribute (#225364)
This patch implements support for the clang::atomic attribute for the
AMDGPU target. This attribute adjusts the atomic options in effect for
the statement it is attached to, which decides the AMDGPU metadata on
the resulting atomicrmw.
That also closes a gap the attribute exposed, as __hip_atomic_*
read-modify-writes carried no AMDGPU metadata at all. The "no.X"
metadata asserts the absence of a memory kind, so it is emitted when the
corresponding option is off, and amdgpu.no.fine.grained.memory,
amdgpu.no.remote.memory and amdgpu.ignore.denormal.mode now match
classic CodeGen.
Large parts of these changes correspond to CGAtomicOptionsRAII and
AMDGPUTargetCodeGenInfo::setTargetAtomicMetadata from OGCG.
Assisted-by: Claude Code Sonnet 5
[2 lines not shown]
