[CIR] Use static functions in cir-offload-merge
Keep the anonymous namespace for the struct only, and comment why the
bundler is used to check targets.
[PassBuilder] Run LoopUnrollAndJam before the vectorizers
LoopUnrollAndJam runs after LoopVectorize, which splits a vectorized
inner loop into a vector and a remainder loop. UnrollAndJam can't
handle that form, so most candidates are out of reach by the time it
runs. Move it to the start of addVectorPasses, which also covers the
full LTO pipeline. The pass is still only added under
-enable-unroll-and-jam.
With this change running over llvm-test-suites. We observe a speedup
over my Ryzen 9 9955HX in some cases (Shootout-C++-ary3 (-34.6%),
oopack_v1p8 (-6.0%)).
Assisted by: Claude # Testcases
[LoopUnrollAndJam] Jamming Outside Loop for stencil
Previously, we only tried to jam the loop if there was any
loop-invariant memory usage. This makes our UnrollAndJam quite
limited. GCC (gimple-loop-jam.cc) also handles stencils. What they do
is, for each MemRef pair, check the distance vector and allow jamming
if the vector is lexicographically positive.
Our safety checker (checkDependency) already allows these cases. To
match GCC's behavior, we try to calculate the actual step for each
memory usage using SCEV and find the maximum ReuseCount. Note that our
safety checker has already guaranteed that this is a forward edge, so
we don't need any additional safety checks here.
Assisted by: Claude # Testcases
[llvm][TableGen] Resolve intra-record field references to `?`
An inconsistency existed in TableGen prior to this PR:
* Referencing a template argument whose value was `?` resolved to and propagated `?`
* Referencing a record whose field is `?` resolved and propagated `?`
However, referencing a field of the record being built remained unresolved, even in the final resolution pass.
This meant that code like:
```tablegen
class Self {
string str = ?;
assert !if(!initialized(str), !ne(str, ""), 1), "str must not be empty";
}
def : Self;
```
would fail to compile as `!initialized(str)` was left unfolded, and `!if` never received a bit value: https://godbolt.org/z/oGh1PzP88
Similar issues arose with other code such as with `!dump` etc.
This PR fixes that issue by resolving a record field to `?` if the record is complete and this is the final resolve pass. As a follow-on effect, some places that previously relied on being kept unresolved now resolve to `?` which required implementing semantics for `?.field` accesses (instead of the current crash using `llvm_unreachable`). This PR therefore also defines `?.field` as resolving to `?`, similar to `?{0}` or `!cast`.
[IRNormalizer] Don't sink allocas (#238656)
`llvm.stackrestore` frees the allocas made after its `llvm.stacksave`,
but no SSA edge ties an alloca to either intrinsic. IRNormalizer can
sink an alloca past `llvm.stacksave`, so the matching
`llvm.stackrestore` frees it while it's still in use. Sinking it past
`llvm.stackrestore` instead leaves it allocated, so a loop can overflow
the stack.
Treat allocas as source nodes, like memory accesses, so they keep their
order relative to those instructions. Static allocas can still move up
to the leading allocas of the entry block, which only extends their
lifetime. As a tradeoff, inputs that differ only in alloca order may now
normalize differently.
Alive2 doesn't model `llvm.stackrestore` freeing allocas and also
accepts the old output, so the Before link writes that free as
`llvm.lifetime.end`:
Before (miscompiles 42 -> UB): https://alive2.llvm.org/ce/z/DJzU_h
[4 lines not shown]
[LangRef] Relax the LLVM IR data race definition (#233436)
The current LangRef rules categorize some surprising (to me) cases as data
races (more specifically: they define loads to return `undef` in surprising
situations).
At the core of the problem is the requirement that, if a read R may see more
than one write, R and all the writes it may see must be atomic, otherwise R
returns `undef`. That means that, if an atomic write W and an atomic read R
execute concurrently, a non-atomic write W' that happens-before both of them
causes R to return `undef`. We get that situation if W' initializes the
location in a sequential portion of the program, followed by proper
synchronization before a parallel portion of the program where W and R take
place.
Some litmus tests for the problem:
1. A non-atomic store that happens-before an atomic store and an atomic load:
```
[60 lines not shown]
[IndVarSimplify] Don't sink convergent calls out of the preheader (#239122)
`sinkUnusedInvariants` moves preheader instructions that are unused in
the loop into the exit block, and only rejects instructions that may
have side effects or read memory. A `memory(none)` convergent call is
therefore moved across the loop's exit branches, which can change the
set of threads that execute it together (see "Hoisting and sinking" in
`ConvergentOperations.md`). LICM, InstCombine and Sink already refuse to
sink convergent calls; skip them here as well.
On AMDGPU this sank a chain of WMMA intrinsics below a loop that was
later fully unrolled. Their load operands stayed in place, so they were
live across the whole unrolled body and VGPR pressure went up.
Assisted-by: Opus 5.5
raise the gem object size limit from 512M to 1.5G
volker@ reports that llama.cpp wants slightly more than 1G sized objects
when running on inteldrm (Raptor Lake).
[SPIR-V] Lower boolean switches to branches (#239575)
OpSwitch requires an integer selector, but i1 maps to OpTypeBool. Lower
boolean switches to branches before structurization, so that merge
instructions match the resulting CFG.
[mlir][Vector] Fix out-of-bounds access in StridedSliceBroadcast (#238219)
This fixes https://github.com/llvm/llvm-project/issues/237496.
`StridedSliceBroadcast` assumes slice attributes cover every source
dimension. When a slice specifies only a prefix of the dimensions,
adjusting a trailing unit dimension can access past the attribute arrays
and trigger an assertion.Bound the adjustment loop by the slice
attribute count, preserving unspecified trailing dimensions.
Assisted-by: OpenAI Codex
[mlir][llvm] Fix crash writing DIBasicTypeAttr bytecode without a name (#240240)
#185284 made the name parameter optional for DIBasicTypeAttr in
LLVMAttrDefs.td, but the LLVM dialect bytecode definition in
LLVMDialectBytecode.td still declared it as a required string. Writing
bytecode for a basic type without a name made the writer call getValue
on a null StringAttr, which crashed with a segmentation fault.
This patch makes name an OptionalAttribute<"StringAttr"> in the bytecode
definition, matching LLVMAttrDefs.td and the other debug info attributes
in the dialect. It changes the bytecode encoding of DIBasicTypeAttr. The
LLVM dialect does not implement bytecode versioning, and earlier changes
such as #217432 changed encodings of existing attributes in the same
way. Happy to take a different approach if reviewers prefer one.
Reproducer:
```mlir
llvm.func @f() attributes {di = #llvm.di_basic_type<tag = DW_TAG_base_type, sizeInBits = 32, encoding = DW_ATE_signed>} {
[16 lines not shown]
[DwarfDebug] Remove dead check for globals in abstract lexical blocks (NFC) (#235790)
DwarfCompileUnit::GlobalVarScopes records variable scopes from the
DICompileUnit's `globals:` list. skipLexicalScope() uses this
information to preserve abstract lexical blocks containing
function-local static variables (#193986).
Since #187927, function-local static variables are listed in their
subprograms' `retainedNodes:` instead, and the verifier rejects them in
the DICompileUnit's `globals:` list. So, the GlobalVarScopes check is no
longer needed.
arch.7: Mechanical fixes
+ Increase portability by never starting content lines with symbols
+ Improve search results by adding tunables to the variable database
+ Improve search result consistency by correcting description style
MFC after: 3 days
[clang][-Wunsafe-buffer-usage] Add per-family subgroups of -Wunsafe-buffer-usage-in-libc-call (#233604)
#127849 asks to turn off `strlen` while keeping `memcpy`, which isn't
possible while all unsafe libc calls share one diagnostic. Clang can't
create warning groups at run time, and one group per function would be
about 70 diagnostics. So each family of related libc functions gets a
subgroup of `-Wunsafe-buffer-usage-in-libc-call`: numeric-conversion,
string-manipulation, string-examination, memory, io, printf and scanf.
The parent flag, `-Werror=` included, still covers all of them, and the
warning text doesn't change. `strerror_s`, `strerror_r`, `bsearch` and
`qsort` have no family and stay under the parent only. `bcopy` and
`bzero` moved from "others" to memory. A call whose family is off is
dropped in the reporter, so a libc `printf` never moves to
`-Wunsafe-buffer-usage-in-format-attr-call`.
Fixes #127849.
AI usage: Claude Code helped with the test runs and test scaffolding. I
traced the issue, made the fix, and validated every change.
google-benchmark: updated to 1.10.0
1.10.0
exclude linking to lib rt for QNX to avoid linking error
fix(pyproject): Update license to SPDX identifier
Replicate generic hardware events on all CPU PMUs
2125: Fix error: format string is not a string literal
Revert "Remove redundant feature checks on re-run of CMake config ste…
Bump the version of python used for pip.
Add BENCHMARK_NAMED macro for named benchmarks without lambda
cmake: cache HAVE_* results to skip feature checks on re-runs
multiple fixes to avoid Windows x64 crashes
Refactor/modularize headers
refactor ToExponentAndMantissa to use StrFormat instead of std::stringstream
Fix CMake detection on pthread_affinity test on FreeBSD
Add WASI compatibility
docs: document benchmark_min_time CLI forms
feat: Add ScopedPauseTiming RAII helper
[56 lines not shown]
[InstCombine] Sink select into copysign with a constant magnitude (#235201)
The constant-operand guard in foldSelectIntoIntrinsicArg is a global
proxy for a per-intrinsic, per-operand property: whether a constant in
that position may enable cheaper lowering. Replace it with a
structured predicate that is conservative by default, and allow
sinking for the copysign magnitude operand, which has no cheaper
lowering. Canonicalize a negative constant magnitude before sinking,
as the corresponding visitCallInst fold no longer applies.
This restores the single-copysign form for the AMDGPU libcall
expansions that regressed in #232738 (review comment by jayfoad):
select c, copysign(0.0, x), copysign(v, x)
-->
copysign(select c, 0.0, v, x)
Fixes https://github.com/llvm/llvm-project/issues/235200
[AArch64][SME] Set inline incompatibility for SME using cost-model. (#233844)
Instead of maintaining a new switch-statement where we handle individual
intrinsics, rely on the TTI cost-model to check if an intrinsic can be
inlined into a streaming function.
[HEXAGON] Reject hardware loops with multiple exits (#239112)
findLoopControlBlock() can select a loop-control block whose exit leads
to an unreachable sink (e.g. a sanitizer trap), causing its branch
condition to be mistaken for the loop's trip count.
Detect this by checking the control block's actual CFG successors rather
than analyzeBranch()'s TB/FB targets, so implicit-fallthrough exits are
caught too. Only applies when the loop has more than one exiting block,
so single-exit and legitimate multi-exit loops are unaffected.
Adds hwloop-ubsan.ll and hwloop-ubsan-fallthrough.mir to cover the
explicit-branch and implicit-fallthrough trap cases.
