[clang-format] Fix a crash in UseBreakAfter on an unmatched brace (#240744)
With `BinPack: UseBreakAfter`, the loop that adds breaks between
arguments skips nested brackets by jumping to their `MatchingParen`.
When a nested bracket isn't closed on the same line, for example because
a `;` in an inactive `#else` branch splits the line, `MatchingParen` is
null and the loop dereferences it.
Stop the loop in that case. The tokens after the unclosed bracket are
nested inside it, so they aren't arguments of the outer bracket anyway.
Fixes #233301
AI-assisted; I reviewed and tested the change.
[ConstraintElim] Split signed min/max across their operands when checking a compare
The existing constraint systems only record the bounds that hold for both
operands of a min/max (smax(a, b) >= a and >= b, smin(a, b) <= a and <= b). The
other direction is a disjunction, which cannot be recorded as a fact. When a
compare of signed decompositions involves a min/max result, split it into its
operands: the decomposition is monotonic in that result, so it is the max or min
of its values on each side, and the relation holds for all or for either side
respectively.
https://alive2.llvm.org/ce/z/zsGkJ-
[CIR] Keep volatile and atomic return loads before cleanups (#242713)
When FlattenCFG routes a `cir.return` through a cleanup, it sinks a load
of an entry-block alloca that feeds the return into the block after the
cleanup. For a volatile or atomic load this moved the access past the
cleanup code. Such a load is now spilled to a temporary before the
cleanup, like any other return operand.
Assisted-by: Claude Code (claude-opus-5-5).
[AArch64] Simplify Apple runtime unroll calculation(NFC) (#242836)
- Name magic numbers, as constexpr
- Name line remainder calculations to make the semantics clear (includes
CSE for 2 calcs of InstsInLastLine)
- Use for instead of while and trim the first duplicate iteration
- Make SizeWithUC induction variable instead repeating a multiply
[libc][test] Bound mlockall memory locking and skip under sanitizers (#242736)
Shadow-memory sanitizers map terabytes of shadow memory, which is why
sanitizer_common intercepts mlockall as a no-op. Because
LIBC_NAMESPACE::mlockall invokes SYS_mlockall directly, it bypasses
sanitizer interceptors; calling mlockall(MCL_CURRENT) when
RLIMIT_MEMLOCK is unlimited or CAP_IPC_LOCK is held attempts to
populate and wire-lock the shadow region, exhausting physical RAM.
Define LIBC_HAS_SANITIZER in sanitizer.h for shadow-memory sanitizers
(ASan, MSan, TSan, and HWASan) and skip LlvmLibcMlockTest.MLockAll
under sanitizers. For unprivileged non-sanitizer builds (without
CAP_IPC_LOCK), also cap soft RLIMIT_MEMLOCK to 64 MiB inside MLockAll
so mlockall fails fast with ENOMEM if multi-gigabyte virtual arenas are
mapped on hosts with RLIM_INFINITY.
Assisted-by: Gemini
[GlobalMerge] Add -global-merge-reorder-only option
In reorder-only mode the pass makes the same grouping decisions but, instead
of merging, only places the affected globals next to each other so each keeps
its own symbol. Off by default.
rdar://192416322
[libc][math] Add do not optimize macros and guard some double-double functions.
Guard some double-double computations against -(a * b) -> a * (-b) optimizations from compilers.
[CIR][SystemZ][NFC] Add SystemZTargetCIRGenInfo (#237828)
Adds implementation for SystemZ TargetCIRGenInfo (following the
structure used in #215424). getTargetCIRGenInfo() had no SystemZ case so
it fell through to x86_64 target info.
The only hook in SystemZTargetCodeGenInfo in classic code gen that has a
CIR equivalent is setTargetAttributes. setTargetAttributes adds the
s390x-visible-vector-ABI module flag which CIR doesn't emit yet. This is
my intended next PR. For now, I add a MissingFeatures marker.
The other classic hook testFPKind has no counterpart in TargetCIRGenInfo
yet.
z/OS uses XPLINK ABI which has its own target info in classic codegen.
For now, z/OS uses the SystemZ ELF target info as a placeholder. I
marked it with targetCIRGenInfoOS() for follow up.
NFC.
[4 lines not shown]
[flang][parser] Parse the !DIR$ ALIGN compiler directive (#236444)
Parse a per-variable alignment directive in two spellings, `!DIR$
ALIGN(n) :: name-list` and the Intel-compatible `!DIR$ ATTRIBUTES ALIGN
: n :: name-list`, into one new `CompilerDirective::Align` node holding
the value and the names. Unparsing always emits the first spelling,
which becomes the canonical form (module files will use it). A bare
`!DIR$ ALIGN` or a token that merely starts with `ALIGN` still falls
through to the generic name/value directive parser, and `!DEC$` lines
remain comments.
Nothing acts on the directive yet. To make sure no revision silently
accepts and ignores it, name resolution warns that `!DIR$ ALIGN` is
recognized but not yet honored; semantics (applicability checks, value
validation, module files) and lowering follow in separate patches, and
the lowering patch removes the warning.
This is the first front-end patch of the directive proposed in the RFC
https://discourse.llvm.org/t/rfc-a-directive-to-set-the-alignment-of-variables-in-flang/92619;
[2 lines not shown]
CodeGen: hand assume_aligned tests to the care of update_cc_test_checks
My previous change already landed upstream for the two alloc_align
tests. Since a later patch's review feedback requested also checking
AssumeAlignedAttr in EmitReturnValueCheck, convert the two
corresponding assume_aligned tests (and refresh the alloc_align tests)
using update_cc_test_checks.py first so that the functional changes in
the next patch are obvious.
This patch is originally from 2023 and was rebased and revised with
assistance from an LLM. I have manually reviewed the diffs to the
limit of my abilities (I don't do much in LLVM, and this is my first
time touching clang in a meaningful way) and they appear sensible. The
commit messages are entirely written by me.
Differential Revision: https://reviews.llvm.org/D152400
SimplifyLibCalls: preserve allocation attributes in optimizeNew
When `LibCallSimplifier::optimizeNew` rewrites an `operator new` call into a
`__hot_cold_t` allocation call, forward the allocation attributes
(`allockind`, `allocsize`, `"alloc-family"`, and parameter `allocalign`) from
the original call onto the synthesized call instruction alongside its metadata.
This is a new patch since 2023 (unlike most of the rest of the stack)
and allows the next patch to remove more specialized C++ knowledge
from the LLVM layer, instead leaving it in clang where it seems more
sensible.
This patch was authored with the assistance of an LLM and was reviewed
by me before posting.
CGCall: emit allocptr attributes for operator delete
With this change, we're almost ready to drop the entire table out of
MemoryBuiltins.cpp and rely solely on attributes to detect allocator
functions.
Per review feedback on D123085, emit `allocptr` in
`CodeGenModule::ConstructAttributeList` so that both the function
declaration and all calls to replaceable global `operator delete` /
`operator delete[]` receive the attribute.
This patch is originally from 2023 and was rebased and revised with
assistance from an LLM. I have manually reviewed the diffs to the
limit of my abilities (I don't do much in LLVM, and this is my first
time touching clang in a meaningful way) and they appear sensible. The
commit messages are entirely written by me.
Differential Revision: https://reviews.llvm.org/D123085
clang: also check alloc_align and assume_aligned claims in return
Prior to this patch, we only check alloc_align and assume_aligned claims
in the caller. We are about to thread alloc_align information down into
LLVM, which would then have a chance to defeat the sanitizer checks by
performing optimizations. To avoid that, we also check alignment claims
in the called function prior to return.
Per review feedback on D121629:
- Also verify the alignment required by `AssumeAlignedAttr` in
`EmitReturnValueCheck`.
- Add a new `ReturnAlignmentAssumption` (`return_alignment_assumption`)
sanitizer handler in clang and compiler-rt that accepts the dynamic
return statement `SourceLocation` pointer loaded from `ReturnLocation`.
- Only emit the sanitizer check (and not a redundant `llvm.assume`) in
`EmitReturnValueCheck`.
This patch is originally from 2023 and was rebased and revised with
assistance from an LLM. I have manually reviewed the diffs to the
[5 lines not shown]
[NFC] clang: simplify emitAlignmentAssumptionCheck
Move the alignment check condition (`ptrtoint`, optional offset subtraction,
mask, and `icmp eq 0`) into `CodeGenFunction::emitAlignmentAssumptionCheck`
and emit the sanitizer check before `Builder.CreateAlignmentAssumption`.
This removes the need to pass `TheCheck` and `Assumption` into
`emitAlignmentAssumptionCheck` (or temporarily unlink and re-insert
`Assumption`) and prepares `emitAlignmentAssumptionCheck` to be called
directly without an accompanying `@llvm.assume`.
This patch is new since this line of work was originally authored in
2023. It was largely written by an LLM in response to review feedback
for the next patch, but I hand-reviewed the content and it looks
reasonable enough (with the same disclaimer as other patches that I
know nothing about clang.)
CGExprCXX: emit family and allockind attributes
This allows us to remove the special knowledge about C++ allocation from
LLVM's MemoryBuiltins.cpp, instead trusting attributes to communicate
the important properties.
Annoyingly, clang has to grow a little bit of hard-coded knowledge about
mangled symbol names in order to actually get the most-basic-matching
::operator::new when working out the alloc-family attribute. I spent
some time trying to do this "right" by looking up the available
operators new and finding the best one, but that's super tricky and this
has the virtue of working and being pretty obviously correct when
compared to the old code.
Happily, clang now emits allocator family attributes for custom
allocators like the `__hot_cold_t` overloads in `new_hot_cold.cpp`
(which `isReplaceableGlobalAllocationFunction` in clang recognizes),
allowing us to remove that special knowledge from LLVM.
[7 lines not shown]
clang: emit allocalign to LLVM for alloc_align attributes
LLVM is growing knowledge of this, so we should inform it when we have
the information available. As a bonus, since we now check return
alignments in the callee as of an earlier patch of mine, we can now drop
the redundant llvm.assume() that this previously emitted.
This patch is originally from 2023 and was rebased and revised with
assistance from an LLM. I have manually reviewed the diffs to the
limit of my abilities (I don't do much in LLVM, and this is my first
time touching clang in a meaningful way) and they appear sensible. The
commit messages are entirely written by me.
Differential Revision: https://reviews.llvm.org/D119271
[AMDGPU] Use bundle-aware iterator in matchVccMask (#243188)
The backward scan from the S_AND started at A.getReverseIterator(),
which is a not bundle-aware and can step into a preceding BUNDLE,
triggering the MachineInstrBundleIterator assertion. So, we shall start
from a MachineBasicBlock::reverse_iterator instead.
Fixes https://lab.llvm.org/buildbot/#/builders/10/builds/39523
(#241405).
Assisted-by: Cursor (Claude Opus 5.5)
[AMDGPU] Use isVALU(AllowLDSDMA=false) where LDS DMA cannot match. NFC. (#243039)
Switch calls to isVALU from AllowLDSDMA=true to AllowLDSDMA=false where
the result cannot differ for an LDS DMA instruction:
- The instruction is known not to be LDS DMA, e.g. wave reduction
opcodes
in SIISelLowering, or an else-if chain that already handled isLDSDMA.
- The check is combined with "defines register R". LDS DMA instructions
have no explicit defs and their only implicit defs are ASYNCcnt or
TENSORcnt. This covers all IsHazardDef callbacks passed to
getWaitStatesSinceDef.
- The check is ORed with isVMEM or IsMemOrExport, and the hazard only
applies to GFX90A+ GFX9 targets, where all LDS DMA instructions are
VMEM.
- The hazard only exists on GFX11, which has no LDS DMA instructions.
- shouldReadExec falls through to returning true anyway.
Also simplify "!isVALU(MI, true) || isLDSDMA(MI)" to
[2 lines not shown]
[clang][deps] Flush the in-process module cache on a background thread
The in-process module cache keeps every PCM built during the lifetime of the
dependency scanning service in memory, and flushes them to disk at the end.
Long-running builds that scan many variants of modules (e.g. with different
search paths or macro definitions) accumulate tens of gigabytes worth of
scanning PCMs this way. Moreover, the final flush is single-threaded and
happens after all scanning work is done.
This PR makes the service start a background thread that flushes each PCM to
disk as soon as it's written, and then replaces its in-memory buffer with the
memory-mapped file. Readers of the in-memory buffer keep it alive on their own
(shared ownership), and importers of the freshly built PCM never wait for the
IO, since the file gets written without holding the entry lock. Reads and
timestamps are still served from memory. The final flush is kept, but only
writes PCMs the background thread failed to write. With
`-no-flush-module-cache`, nothing changes.
I tried several approaches, all measured on a scan of the macOS SDK where
[52 lines not shown]
[clang] Extract built PCMs from their container without copying (#242384)
When a module is built implicitly, the importing instance extracts the
AST
file from the module cache's view of the written PCM and stores a copy
of it
in its `InMemoryModuleCache`. That's an extra heap copy of every built
PCM,
kept alive for the lifetime of the translation unit.
This PR resolves the FIXME by storing a buffer that keeps the written
PCM
alive and exposes just the AST file: the `__clangast` section for PCMs
wrapped in an object file (`-gmodules`), or the whole PCM otherwise. The
in-process module cache now shares the memory of its entry with the
importing instance, and the cross-process module cache gives it the
memory-mapped file.
For a regular implicit modules build importing Foundation and friends,
[5 lines not shown]
[lldb-dap] Don't parse launch-target args in runInTerminal launcher (#243096)
When the debug adapter requests the client to launch a program in the
terminal (runInTerminal), it provides a launcher command like:
```
lldb-dap --comm-file <file> --launch-target <program> <args>
```
Previously, if <args> contained flags matching lldb-dap's own options
(e.g., `--version` or `--help`), lldb-dap would consume them and act on
them (e.g. printing its version and exiting) instead of passing them to
the target program causing
the launch to fail.
The commit resolves the ambiguity by changing `--launch-target` to a
boolean flag and requiring a `--` separator before the target program
and its arguments.
The launcher command is now formatted as:
```
[2 lines not shown]
[lld][MachO] Batch command-line object files and archive member scanning in parallel
This is part of the ld64.lld performance improvements tracked by
Currently, command-line object files and archive member symbol tables
are still parsed and scanned one file at a time as Driver::processFile()
encounters them.
This change splits ObjFile::parseLazy() into scanLazy() and
registerLazy(), and queues command-line object files and archive members
via parseLater(). parseBatch() runs scanLazy() and parsePrepare() in
parallel across the batch before running registerLazy() and
parseFinish() serially in command-line order, flushing before any other
input type.
Locally, this shows an ~8% link-time performance improvement over the
baseline.
AI use: this change is based off a prototype developed by an LLM.
[2 lines not shown]
[lld][MachO] Split ObjFile::parse() and parse extracted archive rounds in parallel
This is part of the ld64.lld performance improvements tracked by
#222068 and #238938.
Currently, parsePendingExtracts() parses each extracted archive member
serially from start to finish, even though section creation, subsection
splitting, local symbol creation, and hashing external symbol names only
touch per-file state.
This change splits ObjFile::parse() into parsePrepare() and
parseFinish(), using makeThreadLocal() for per-file allocations.
parsePendingExtracts() now processes extracted members in rounds,
running parsePrepare() in parallel across each round and parseFinish()
serially in file order.
Locally, this shows a ~4% link-time performance improvement over the
baseline.
[3 lines not shown]
[lld][MachO][NFC] Accept CachedHashStringRef in SymbolTable insertion methods
This is part of the ld64.lld performance improvements tracked by
Currently, SymbolTable::insert() constructs a CachedHashStringRef on the
serial insertion path. This change adds CachedHashStringRef overloads
for insert(), addDefined(), addUndefined(), addCommon(), and
addLazyObject() so callers can pre-hash symbol names in parallel, while
keeping StringRef wrappers for existing callers.
AI use: this change is based off a prototype developed by an LLM.
This commit is output-preserving.
[AMDGPU] Fold OR of {SIGN|ANY}_EXTENDed SGPR booleans to SELECT
`or x, (sext cc)` with an SGPR boolean materializes the 0/-1 mask and
then ORs it in. `select cc, -1, x` does the same in one instruction.
After legalization, fold i32 `or x, (sext/anyext cc)` to
`select cc, -1, x`. The high bits of ANY_EXTEND are unspecified, so
they can be chosen to match sign extension. ZERO_EXTEND stays
excluded: `or x, (zext cc)` is `select cc, (or x, 1), x`, which keeps
the OR.
Two checks avoid code-size regressions. The extension must have one
use; otherwise it is still materialized and the select may need an
e64 cndmask. A divergent OR also skips the fold when its uniform
condition has other uses: the select needs the condition as a lane
mask, and the other use would then convert it back to SCC.
Split out of #230701, which extends the existing AND fold to
ANY_EXTEND. Both patches add the same isBoolSGPRExt helper.
[2 lines not shown]
CodeGen: Simplify def segment extension in shrinkToUses (#242869)
During extendSegmentsToUses, Segments holds only one def segment per
value, and none extend past their block. Live-in segments are kept
separately in LiveIns. To extend a def to a use, find the last segment
starting before the use, check that it starts in the use's block, and
move its end to the use. This replaces extendInBlock, whose general
search and merging of overlapped segments never applies here.
Also combine the per-block live-out and live-in tables into one struct.
llc instructions:u:
CTMark x86_64: -0.07%
CTMark arm64: -0.10%
AMDGPU sample: -0.27%
Largest AMDGPU improvements:
rocprim warp_sort (gfx1100): -1.89%
rocsolver stedc (gfx942): -1.70%
[4 lines not shown]