[LLVM] Use CreatePtrAddFrom(Signed|Unsigned)Offset helpers more (#239153)
Both to simplify code and make sure that the correct offset type
is always used (some places were hardcoding 64 or 32 bits, which
may not match the pointer index type).
[NVPTX] Support native 128-bit atomic load + store (#236423)
Lower i128 atomic loads and stores to ld/st.b128 on SM 70 and newer with
PTX 8.3, instead of emulating them with compare-and-swap or exchange.
Split values into pairs of i64 operands and preserve the memory ordering
and scope so instruction selection inserts the required fences.
Add coverage for i128 and fp128 loads and stores, including memory
ordering, address spaces, volatile accesses, and unsupported targets.
Update scalar and elementwise atomic CAS loop checks to use 128-bit
loads.
Assisted by: Codex
---------
Co-authored-by: Akshay Deodhar <adeodhar at nvidia.com>
[llvm-readobj] Avoid crashes for unsupported WebAssembly options (#237555)
Fix crashes when using --unwind, --stackmap, or --all on valid
WebAssembly object files. Before that WasmDumper used llvm_unreachable
for unwind information and stack maps, while they could be reached with
valid command line options. The fix replaces handlers with empty
functions to avoid crashing
AI assisted with understanding the issue, producing tests, and reviewing
implementation. I also ran the focused tests locally
Fixes #226398
[analyzer] Match standard libraries regardless `restrict` qualifier
Remove the `restrict` sensitivity from StdLibraryFunctionsChecker to
simplify the code and remove the risk of a failure to match.
c5dd283d2f2409dfca13628e15bbf0e526dfc3f0 demonstrates that UCRT does not
specify the `restrict` qualifiers leading to non-matches in
StdLibraryFunctionsChecker. There might be other non-standard-compliant
libc implementations that share this drawback.
The `restrict` sensitivity does not seem to improve analyzer precision
in any practical case, yet it increases code complexity and adds a risk
of FNs as happened with UCRT.
--
CPP-9480
[GlobalISel] Expand sdiv/srem by constant when G_SMULH splits into legal pieces (#235860)
`matchSDivOrSRemByConst` required `G_SMULH` to be legal for the exact
type, so wide vectors were scalarized into one `sdiv` per lane. This
also accepts types that the legalizer splits (`FewerElements`) into
legal pieces. Types where `G_SMULH` would be lowered instead, such as
`<2 x i64>`, are still rejected, as the expansion would be more
expensive than the scalar divisions there.
On AArch64 for `sdiv` by `<7, 6, 7, 6, ...>`:
| type | before | after | SelectionDAG |
|---|---|---|---|
| `<8 x i32>` | 26 (8 × `sdiv`) | 19 | 19 |
| `<16 x i16>` | 16 × `sdiv` | 14 | 13 |
| `<32 x i8>` | 32 × `sdiv` | 17 | 17 |
| `<2 x i64>` | 8 (2 × `sdiv`) | 8 | 15 |
[AMDGPU][SIMemoryLegalizer] Consider scratch operations as NV=1 if GAS is disabled
- Clarify that `thread-private` MMO flag is still useful.
- If GAS is not enabled (which is the default as of last patch), consider an op as `NV=1` if it's a `scratch_` opcode, or if the MMO is in the private AS.
- Add tests for the new cases.
- Update AMDGPUUsage GFX12.5 memory model
[Remarks][Support] Make YAML remark arguments round-trip
The remark serializer writes an argument value with more than one
newline as a literal block scalar. A block scalar has no escapes, so a
value that also held a control character was written with the raw byte,
which strict YAML readers reject. Output::blockScalarString now falls
back to a double-quoted scalar when a literal block cannot hold the
value: a non-printable character (C1 controls, and NEL, LS and PS,
which YAML 1.1 readers take as line breaks), a carriage return, or a
first line that starts with a space. Input accepts either form.
The remark parser took the raw scalar text and stripped only single
quotes, so double-quoted values came back with their quotes and
escapes. Decode scalars with ScalarNode::getValue and report escape
errors. Reject values that decode to a NUL, since the bitstream string
table stores NUL-terminated strings.
Output::blockScalarString always wrote "|", which keeps exactly one
trailing newline. Write "|-" or "|+" when needed. MIR output does not
[3 lines not shown]
[AMDGPU] Make globally-addressable-scratch opt-in
This feature is meant to be opt-in for more advanced users, not default-enabled.
It may reduce performance otherwise as we can't assume private AS is thread-local
when it is enabled.
Make it controllable via a `-amdgpu-globally-addressable-scratch` CL option and disable it by default.
[AArch64] Only fold single-use shifts into SSRA/USRA (#238937)
In the signed division by constant sequence, GlobalISel folded the wrong
shift into the accumulate, so the arithmetic shift was computed twice:
sshr v0.8b, v1.8b, #2
ushr v0.8b, v0.8b, #7
ssra v0.8b, v1.8b, #2
This only folds a shift into SSRA/USRA when the shift has one use, so
GlobalISel now picks USRA, like SelectionDAG:
sshr v0.8b, v0.8b, #2
usra v0.8b, v0.8b, #7
Also adds a GlobalISel run line and two tests for this to
`shift-accumulate.ll`.
[mlir][vector] Disallow pure reduction dims in vector.contract (#237709)
Require every reduction dimension in `vector.contract` to appear in both
LHS and RHS. This disallows one-sided ("pure") reduction dimensions and
aligns the restriction with `linalg.contract`.
For example, the following contraction has two reduction dimensions:
`r2` appears in both inputs, while `r1` appears only in LHS. This PR
makes such contractions fail verification:
```mlir
%res = vector.contract {
indexing_maps = [
affine_map<(r1, p, r2) -> (r1, r2)>,
affine_map<(r1, p, r2) -> (p, r2)>,
affine_map<(r1, p, r2) -> (p)>
],
iterator_types = ["reduction", "parallel", "reduction"],
kind = #vector.kind<add>
[17 lines not shown]
Reland "[AArch64] Make all fixed-width masked loads/stores legal on SVE targets" (#238786)
Relands #232757, which was reverted in #238371. I mistakenly thought my
PR was the cause of this buildbot failure:
https://lab.llvm.org/buildbot/#/builders/4/builds/13777
but it turned out to be introduced by #197391.
[CIR] Run SimplifyConstantLoad in the same driver as the other patterns (#238767)
CIRSimplify ran SimplifyConstantLoad in a second greedy driver because
the pattern cached a DominanceInfo that the ternary and switch rewrites
could leave stale. That split needed a second op list, pruned of the
loads the first driver erased (llvm/llvm-project#238505 fused the two
collection walks but kept the split and the pruning), and it missed
simplifications: a load inside a ternary branch only folded in the
second driver, after the ternary pattern had already rejected that
branch as non-trivial.
This change shares one DominanceInfo between the pass and the pattern
and keeps it coherent with a listener that drops a region's cached tree
whenever its block list changes, so all patterns run in one driver over
one worklist.
Assisted-by: Claude Code (claude-fable-5-1).
[GlobalISel] Remove obsolete RegisterBankInfo::getSizeInBits (NFC) (#238744)
With the getMinimalPhysRegClass cache obsolete (#238351) this helper is
equivalent to TargetRegisterInfo::getRegSizeInBits.
Assisted-by: codex
[ADT] Add LLVM_ATTRIBUTE_WARN_UNUSED to five ADT classes (#237802)
This patch adds LLVM_ATTRIBUTE_WARN_UNUSED to the following classes so
that compilers can diagnose unused variables of these types even though
they have non-trivial constructors and destructors:
- APInt
- DenseSet
- FoldingSetNodeID
- ImmutableMap
- ImmutableSet
Other ADT classes are left alone for now because adding the attribute
to them would trigger warnings in existing code.
Assisted-by: Antigravity
[mlir][llvm] Add MemProf metadata attributes and translation (#238988)
The LLVM dialect understands neither `!memprof` nor `!callsite`: both
are dropped on import and unrepresentable on export. Add
`#llvm.memprof_mib` (one MemInfoBlock: call stack, allocation-type tag,
optional context sizes) plus `#llvm.memprof_context_size`, with
`llvm.memprof`/`llvm.callsite` discardable attributes and import/export
translation.
Known limitation: the MLIR inliner carries both attributes over
unchanged, so contexts go stale across an MLIR-side inline (a
hint-quality issue, not a miscompile). The LLVM's own inliner rewrites
them.
[AMDGPU] Price misaligned flat and LDS accesses with the LDS misaligned bug
In WGP mode on subtargets with the LDS misaligned bug, a flat access is
split into pieces no wider than its alignment and no narrower than 4
bytes, and an LDS access aligned below 4 bytes uses 8-byte pieces as one
aligned to 4 does.
