[libc][regex] Add 256-bit ByteSet and '.' wildcard (#233561)
Representing wildcards and bracket expressions as a 256-way alternation
of Literal nodes would exhaust ExprPool and blow up derivative step
time. At the same time, storing a 32-byte cpp::bitset<256> inline inside
Expr would increase every AST node from 16 bytes to 40 bytes.
Add ExprKind::ByteSet backed by a deduplicated cpp::span<ByteSet>
side-table in ExprPool, keeping sizeof(Expr) at 16 bytes. Normalise
0-bit and 1-bit sets in ExprPool::make_byte_set to EmptySet and Literal
so single-byte character classes share the same canonical AST node as
plain literals, and wire the '.' wildcard in parse_ere and derivative.
Also simplify BucketProber and the ExprPool lazy-initialisation guards.
Against the external compliance suites in REG_NOSUB mode, this passes
21/456 tests (+1) in the Henry Spencer suite and 25/202 ERE tests (+3)
in Glenn Fowler's AT&T testregex basic.dat suite.
Assisted-by: Automated tooling, human reviewed.
[AMDGPU] Add a WAR stall for DS loads in the coexec scheduler
Add a WARVdst stall cost to the coexec scheduler for DS loads on gfx1250.
For each WMMA or VALU that reads a VGPR the load writes, found through the
load's anti-dependences, the load is stalled until four times the reader's
blocking cycles after the reader.
Co-authored-by: Jeffrey Byrnes <jeffrey.byrnes at amd.com>
[clang] Make SARIF better handle diagnostic about unknown argument (#233241)
`SARIFDiagnosticPrinter::HandleDiagnostic` optimistically assumes that
its diagnostic renderer (`SARIFDiag`) is always available, but the
assumption doesn't hold when driver is parsing arguments and emits
unknown argument diagnostic. There is a FIXME to handle diagnostics in
this limited mode, but the short-circuit `return` is placed too late,
which makes assertion builds crash.
See `TextDiagnosticPrinter::handleDiagnostic` for additional context:
https://github.com/llvm/llvm-project/blob/42ac09cebaad223f14f4c3052bcd5d6d25be0f3f/clang/lib/Frontend/TextDiagnosticPrinter.cpp#L131-L150
Fixes #233228
---------
Co-authored-by: Kazu Hirata <kazu at google.com>
Co-authored-by: Corentin Jabot <corentinjabot at gmail.com>
[libc++] Mark LWG4460 as Nothing To Do (#232252)
LWG4460 only adds a missing Throws: element to the last variant
constructor. libc++ already has the correct behaviour, so no code
changes are required.
Closes #189836
[clang][bytecode] Fix `Pointer::getLifetime()` for non-primitive array elements (#232605)
This code path is not reached for pointers to non-primitive array
elements in regular evaluation, but we do call `getLifetime()` for
random pointers in `__builtin_is_within_lifetime`.
[LoongArch] Compute known bits and sign bits for BSTRINS/BSTRPICK
BSTRINS and BSTRPICK are formed early by DAG combines, so the generic
known-bits and sign-bits analyses see them as opaque target nodes. When
such a value is live out of a block, its LiveOutInfo is lost and PHIs
merging it can no longer prove that the value is already sign- or
zero-extended, leaving redundant addi.w/bstrpick.d behind.
This is common in Rust code decoding UTF-8, e.g. `str::chars().rev()`,
where the multi-byte path assembles the code point with bstrins.d and is
merged with the ASCII byte loaded by ld.b before an i32 compare.
rustc-perf svg-parse-1 on LA664: instructions -2.4%, cycles -1.7%.
[libc] Keep __restrict on the Cortex-M memcpy for MVE targets (#233575)
With #226955 the SLP vectorizer versions the Cortex-M memcpy copy loops
with runtime alias checks on MVE targets (270 -> 412 bytes on
Cortex-M55).
Keeping __restrict on MVE targets lets it vectorize without versioning
(222 bytes). Non-MVE targets are unchanged.
Fixes #233426
[AMDGPU] Report packed F32 unpacking as a change
SIPreEmitPeephole::run did not set Changed after unpacking, so when
unpacking was the only change the new pass manager kept every cached
analysis and the legacy one saw an unchanged function.
Assisted-by: Claude Opus 5.5
[AMDGPU] Fix the packed F32 unpack window in SIPreEmitPeephole
collectUnpackingCandidates unpacked a v_pk_*_f32 only if its two halves
finished before NumPasses - 1, i.e. before the MFMA completes. A packed F32
cannot issue until NumPasses + 2 after an XDL (NumPasses + 3 after an XDL2x),
though, so a packed instruction placed between those points still stalled.
Unpacking pays off whenever the pair finishes before that cycle, so use
getPackedVALUIssueCycle - getVALUIssueCycle + 1 as the window.
Also:
- Skip SGEMM and DGEMM. They hold every VALU until NumPasses + 2, so
unpacking after them only adds an issue slot.
- Charge S_NOP its wait states when counting cycles. The larger window
would otherwise unpack instructions that issue long after the MFMA.
Timed with s_memtime on MI308X (peak TOPS limiter enabled) and MI350X, in
clocks per MFMA followed by four v_pk_fma_f32 (six for F8F6F4):
- XDL 16x16x16: 36 -> 32
- BF16 XDL2x 16x16x32: 40 -> 36
[12 lines not shown]
[AMDGPU] Add MFMA VALU and packed-VALU issue cycle helpers
Add getVALUIssueCycle and getPackedVALUIssueCycle. They return the first
cycle, in 4-clock issue slots counted from the MFMA's issue, at which a VALU,
or a packed VALU or DOT, can issue after an MFMA on gfx942 and gfx950.
The rules fit ATT thread traces taken on MI308X (gfx942, with the peak TOPS
limiter enabled) and MI350X (gfx950) for every MFMA family:
- An XDL holds VALU off until 1 + (widest A/B source in bytes) / 16.
- PK_F32 (v_pk_{add,fma,mul}_f32, v_pk_mov_b32) and DOT issue at
NumPasses + 2, one cycle more for XDL2x (an A source of 16 bytes or more).
Packed 16-bit ops issue like any other VALU.
- SGEMM and DGEMM hold every VALU until NumPasses + 2.
A scaled MFMA follows the same rules when it has to wait for an earlier
MFMA to free the pipe, as back-to-back MFMAs in a loop do: its scale load
overlaps that wait. Issued into an idle pipe, it starts one cycle late, which
delays VALU, PK_F32, DOT and the next MFMA by one cycle. The helpers model
the back-to-back case. Streams of scaled MFMAs run at the unscaled rate.
[7 lines not shown]
[AMDGPU] Don't copy TargetSchedModel in SIPreEmitPeephole (NFC)
run() and collectUnpackingCandidates copied the subtarget's scheduling
model, including its ResourceFactors vector, once per block and once per
MFMA. Bind references instead.
Assisted-by: Claude Opus 5.5
[AMDGPU] Precommit SIPreEmitPeephole unpack window tests (NFC)
Add cases where packed F32 instructions follow a legacy XDL, a BF16 XDL2x,
an F8F6F4 XDL2x (whose 32-byte A source holds VALU off one cycle longer),
an SGEMM, a DGEMM, and an XDL followed by long S_NOPs. Also add cases where
packed 16-bit instructions or DOTs, which the pass never unpacks, follow a
legacy XDL. The CHECK lines record what the pass does today.
Assisted-by: Claude Opus 5.5
[AMDGPU] Precommit SIPreEmitPeephole unpack dst flags test (NFC)
Add a case where the dst of an unpacked V_PK_FMA_F32 is renamable. The
CHECK lines record that the unpacked instructions lose the flag today.
Assisted-by: Claude Opus 5.5
[AMDGPU] Keep dst flags when unpacking packed F32 instructions
performF32Unpacking copied the new instructions' dst operands by value, so
its setIsUndef and setIsRenamable calls never reached the instructions. Take
the operands by reference.
Assisted-by: Claude Opus 5.5
[AMDGPU] Precommit SIPreEmitPeephole unpack invalidation test (NFC)
Add a case where unpacking a packed F32 instruction is the pass's only
change. Unpacking inserts an instruction, but the CHECK lines record that
SlotIndexesAnalysis stays cached afterwards today.
Assisted-by: Claude Opus 5.5
[AArch64] Add LocalStackSlotAllocation tests (NFC) (#233481)
AArch64 has little direct test coverage for LocalStackSlotAllocation.
Adapt the RISC-V local-stack-slot-allocation.ll test for AArch64.
Assisted-by: codex
[CIR][AMDGPU] Emit `!amdgpu.no.{fine.grained,remote}.memory` metadata on atomic xchg (#233440)
Add it to xchg too (not only atomic fetch ops), so IR matches what
classic codegen emits
[AMDGPU] Bias gfx1250 scheduling by co-execution slot flavor preferences
Add per-slot flavor preferences and avoidances to the WMMA co-execution window
model, and populate them in getCoExecInfo() for each WMMA variant.
Add CandidateHeuristics::tryCoexecSlot(), which uses them to pick between
candidates inside an active window, comparing each candidate at the stage it
would issue in. GCNHazardRecognizer exposes the current stage and window info
for this.
Each directive now matches exactly one diagnostic (Clang semantics), using the new OneDiagPerDirective option from the parent PR; added tests for duplicated diagnostics.
[AMDGPU] Model ordered XDL writes in expert scheduling
Use the VA_VDST ordering of qualifying XDL WMMA so expert scheduling can use
nonzero waits after out-of-order event mixes.
An XDL instruction with a destination register of 8 or more VGPRs decrements
VA_VDST after all instructions that were issued earlier.
This offers a significant benefit in compute bound kernels since we are not
required to pessimistically wait for WMMAs to write their results when
interleaving LDS.
```
VALU def r0
TRANS
VALU
TRANS
WMMA
va_vdst(1) # previously this would require va_vdst(0)
[2 lines not shown]
