TableGen: Compute DAGISel pattern sort keys before sorting (#244009)
The pattern sort recomputed each pattern's complexity, cost and size on
every comparison. Compute them once per pattern instead.
RISCVGenDAGISel.inc tablegen run:
Gigainstructions Gigacycles CPU seconds
Before 61.5 31.4 9.8
After 57.1 (-7.1%) 27.8 (-11.4%) 8.7 (-10.9%)
AMDGPUGenDAGISel.inc tablegen run:
Gigainstructions Gigacycles CPU seconds
Before 94.6 46.1 14.3
After 91.4 (-3.4%) 44.6 (-3.3%) 13.9 (-3.1%)
All 376 tablegen runs in an LLVM build with all targets:
[4 lines not shown]
[DebugInfo][AT] Retire dead LiveIn/LiveOut state and avoid redundant RPO queuing (#234310)
AssignmentTrackingLowering and MemLocFragmentFill previously
pre-reserved and retained LiveIn/LiveOut maps for every block across the
entire pass, and re-queued forward RPO successors onto Pending even when
already scheduled on Worklist, scaling as O(Blocks * Vars).
Add BlockLivenessTracker and worklist helpers to:
- Skip storing LiveIn for acyclic blocks (unreachable from backedges),
moving a single predecessor's LiveOut when it has no other readers.
- Track remaining successor reads to erase dead predecessor LiveOut
entries eagerly, and clear LiveIn/LiveOut after traversal.
- Avoid queuing forward RPO successors (SuccRPO > RPO) already scheduled
on Worklist.
On a pathological case from glslang/SPIRV (9,049 blocks, 7,540
variables, 0 backedges; `llc -O2`):
- Peak RSS: 17.37 GB -> 354.5 MB (~49x reduction)
- Total `llc` wall time: 15.84s -> 4.97s (-68.6%)
[3 lines not shown]
[libc] Fix syscall under ASAN (#244038)
Followup to #243647 which broke the ASAN bot. Just mark the va_args
reads as ignored by the sanitizer.
Assisted-by: Automated tooling, human reviewed.
[clang] Shrink inline capacities of `Module` unresolved directives (#243156)
`Module` stores the module map directives that haven't been resolved yet
(`UnresolvedHeaders`, `UnresolvedExports`, `UnresolvedDirectUses`,
`UnresolvedConflicts`) and the headers that couldn't be found
(`MissingHeaders`) inline, with room for one or two elements each. These
lists are empty in modules deserialized from PCMs, and in modules parsed
from a module map usually only the unresolved exports (typically a
single `export *`) are non-empty. The dependency scanner creates a lot
of modules, so the inline storage adds up.
This patch drops the inline capacity of these lists:
- `UnresolvedHeaders`, `MissingHeaders`, `UnresolvedDirectUses`:
`SmallVector<T, 1/2>` -> `SmallVector<T, 0>`
- `UnresolvedExports`: `SmallVector<T, 2>` -> `SmallVector<T, 1>`, so
the common single `export *` still doesn't allocate
- `UnresolvedConflicts`: `std::vector<T>` (24 bytes) -> `SmallVector<T,
0>` (16 bytes)
[30 lines not shown]
CodeGen: Document best practices on target hooks (#243984)
Document that this is not a place to put controls for specific
transformations. This is a plea to agents to stop producing
fix-this-one-case or target virtual methods.
[Offload][NFC] Move plugin namespaces to llvm::offload and update include guards
Replace libomptarget namespaces and header guards with liboffload equivalents.
Mostly mechanical work by Claude.
Revert "[CodeGen][NFC] Factor MachineBlockPlacement::alignBlocks into reusable helper (#241995)" (#243994)
This reverts commit 6447d4dda5706f5d7946f86c1225fd964abfb39e (#241995)
at a reviewer's request while concerns about the change are addressed.
[Offload][Omp] Ensure correct teardown order
Use the new Offload API to ensure a proper libomptarget teardown
in case the process exits before reaching its `deinit` method.
Depending on different circumstances whether the shutdown happens
through `deinit` or through the liboffload `callback` everything
must be properly protected and allowed to be called safely multiple
times.
Assisted by Claude.
[Offload] Add shutdown callbacks to liboffload (#239628)
When Platforms are initialized liboffload sets an atexit handler that
ensures that is deinitialized before deinitialazing platforms.
Because this happens lazily, a library using offload that wants to
run some of its shutdown needs to properly setup its atexit handler
after the liboffload one (because they're called in reverse order).
This requires to know precisely the internal implementation of
liboffload.
Instead of requiring that, this PR adds an API that allows the users
of liboffload to register their own deinit code and be assured that it
will
be called before liboffload shutdowns independently of when platforms
are initialized.
Assisted by Claude.
[4 lines not shown]
[mlir][vector] Dispatch contraction lowering on iterator type (NFC) (#231337)
`ContractionOpLowering` chose between `lowerParallel` and
`lowerReduction` based on whether a dimension was contracting. Every
non-contracting dimension went to `lowerParallel`, which is what made
its name misleading, as noted in #228971 and #229824.
With this PR we dispatch on the iterator type instead:
* parallel dimensions go to `lowerParallel`, which no longer accepts an
iterator missing from the result map. It fails to match instead of
tolerating it.
* contracting dimensions go to `lowerReduction`, which now takes the LHS
and RHS indices of the dimension to lower instead of always using
iterator 0.
To honor what has been documented in #230458, this PR adds
`ContractionOp::getFreeDimMap` next to the existing batch and
contracting maps.
[8 lines not shown]
[clang][DebugInfo] Emit local types in their lexical block scopes (#242447)
With the background support of
https://github.com/llvm/llvm-project/pull/165032 in place, emit debug
info nodes for local types in their corresponding DILexicalBlock scopes.
This allows debug info consumers to distinguish between local types with
the same name declared in different lexical blocks. It is now done only
with `-debugger-tuning=gdb`, as lldb does not provide full support for
local types in lexical block scopes
(https://github.com/llvm/llvm-project/issues/54475).
Authored-by: Kristina Bessonova <kbessonova at accesssoftek.com>
[Offload][NFC] Move plugin namespaces to llvm::offload and update include guards
Replace libomptarget namespaces and header guards with liboffload equivalents.
Mostly mechanical work by Claude.
[Offload][NFC] Move plugins under liboffload (#242364)
Liboffload is the only library calling the plugins directly now, so move
the code inside it to reflect this change.
Also, adjust the CMake targets to also reflect this change by moving
from omptarget prefixes to LLVMOffload ones.
Assisted by Claude.
---
<sub>Stack created with <a
href="https://github.com/github/gh-stack">GitHub Stacks CLI</a> • <a
href="https://gh.io/stacks-feedback">Give Feedback 💬</a></sub>
[lldb] Suggest 'bugreport' after a failed expression (#243686)
A failed expression can be a typo or a bug in the expression evaluator,
and LLDB cannot tell the two apart. Follow every failed evaluation in
`expression` and `dwim-print` with the note from 25151270fb98.
Each debugger shows the note once per expression, whichever command
evaluates it. The key is the expression text. The error text embeds a
counter (`<user expression 0>`) that changes on every evaluation, so
using that would repeat the note on a retry.
The note goes to the command's error stream, so it also reaches
SBCommandReturnObject::GetError and the command transcript. The
diagnostics.suggest-bugreport setting turns it off.
Assisted-by: Claude
SelectionDAG: Track ScheduleDAGRRList live registers in a SparseSet
Schedule() allocated and zeroed two arrays of getNumRegs() + 1
pointers for every block, 333 KB on AMDGPU, even though only a few
registers are live at once. Keep the live registers in a SparseSet
instead. This also stops CheckForLiveRegDefMasked from skipping the
last register.
radeonsi shader-db compile time: -1.8%
CTMark instructions:u, geomean: O3 -0.04%
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[clang] Remove the temporary PCM file when writing fails (#243334)
When `writeImpl()` fails after creating the temporary file (writing to
it, querying its status, reading it back, or renaming it to the final
destination), it returns early and leaves the `*.pcm.tmp` file behind in
the module cache. This PR switches to `llvm::sys::fs::TempFile`, which
removes the file on all failure paths. It also removes it when the
process crashes, since it registers the file with `RemoveFileOnSignal`.
`writeImpl()` reports errors as `std::error_code`, so this PR also adds
`TempFile::createWithErrorCode()` and `TempFile::keepWithErrorCode()`.
These avoid converting `llvm::Error` back with `errorToErrorCode()`,
which aborts on inconvertible errors. The existing `create()` and
`keep(Name)` are now thin wrappers around them.
[Offload][NFC] Move plugin namespaces to llvm::offload and update include guards
Replace libomptarget namespaces and header guards with liboffload equivalents.
Mostly mechanical work by Claude.
[Offload][NFC] Move plugins under liboffload
Liboffload is the only library calling the plugins directly now, so move
the code inside it to reflect this change.
Also, adjust the CMake targets to also reflect this change by moving
from omptarget prefixes to LLVMOffload ones.
Assisted by Claude.
EliminateAvailableExternally: Preserve the function analysis manager proxy
Previously all cached function analyses were cleared When only
deleting function bodies, clear the analyses of those functions and
preserve the proxy.
CTMark instructions:u geomean: O3 -0.02% (kimwitu++ -0.12%, tramp3d-v4
-0.12%), ThinLTO -0.01%.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
JumpThreading: Register cloned assumptions in the assumption cache
JumpThreading clones assumes when threading blocks, but did not add the
clones to the assumption cache, leaving it stale after the pass.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[MustExecute] Define the block context helpers out of line
They use BasicBlock, which MustExecute.h only declares, so the header
did not compile without precompiled headers.
Co-Authored-By: Claude Opus 5.5 <noreply at anthropic.com>
MustExecute: Remember the summaries of the last program point queried
isInContextOf looked up the summaries of the program point for every
query, but the queries come in batches for the same program point, e.g.
for all uses of a value in followUsesInContext. Remember the summaries of
the last program point. NFC.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
MustExecute: Look up the blocks of contexts by their number
isInContextOf looked up the block of every queried instruction in a hash
map of the forward context and a hash set of the backward context, which
was a few percent of the full Attributor on large functions. The blocks of
a context are all included except for a prefix of the last block of a
forward context, so keep a bit vector indexed by block number instead,
and the end of the last block. NFC.
CTMark instructions:u with -attributor-enable=full:
O3: -0.74% (sqlite3 -2.22%)
ThinLTO: -1.27% (tramp3d-v4 -4.62%, sqlite3 -2.73%)
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[MustExecute] Fix the memory regression of block number contexts
The previous commit gave every block context a bit vector with a bit for
every block of the function, which takes memory quadratic in the number
of blocks: +110 MB for a switch with 30000 cases. Use a set until the
bit vector is not larger than the list of blocks of the context.
Also assert that the block numbers are not outdated, as DominatorTree
does, and drop the unused SetVector include.
Co-Authored-By: Claude Opus 5.5 <noreply at anthropic.com>
[MustExecute][NFC] Answer context queries with per-block summaries (#243532)
The Attributor explored the must-be-executed context of every program
point with its own iterator: `findInContextOf` walked the context until
it found the instruction, and `followUsesInMBEC` walked the whole
context to collect its conditional branches. In large functions with
many program points this is quadratic.
Add `isInContextOf` and `checkForAllTerminatorsInContext`, which answer
these queries from summaries shared by all program points of a function:
the instructions of each block that may not transfer execution, and the
forward and backward contexts of each block. Use them in the Attributor.
Output is unchanged on CTMark, CGBuiltin and the AMDGPU tests.
`opt` instructions on CTMark bitcode: sqlite3 -35% (attributor-light),
-78% (attributor-light-cgscc), -61% (attributor); tramp3d-v4 -46%
(attributor).
Co-authored-by: Matt Arsenault <Matthew.Arsenault at amd.com>
Assisted-by: Claude Opus 5.5