[RISCV][GISel] Enable extended LLT
Pass -gisel-extended-llt to the GlobalISel TableGen backend and call
LLT::setUseExtended(true) in RISCVTargetMachine.
Update the GlobalISel tests: convert hand-written instruction-select
and knownbits MIR inputs from sN to iN/fN, and regenerate CHECK lines.
Some soft-float assembly output changes slightly (constants are
rematerialized instead of kept live across libcalls).
Co-Authored-By: Claude Sonnet 5.5 <noreply at anthropic.com>
[RISCV][GISel] Prepare for extended LLT
Make the RISC-V GlobalISel code correct when LLTs carry integer/float
kinds, ahead of enabling extended LLT:
- Legalize scalar G_BITCAST: legal up to XLen bits, narrowed above that
(i128/f128, or f64/i64 on RV32). Map it in RegisterBankInfo by type
kind and subtarget (FPR only if the FP extension holds that size in an
FPR), and select it as a COPY.
- Use f16/f32/f64/f128 in the FP legalizer rule sets so that types
created by mutations (e.g. f16 -> f32 widening) match the imported
TableGen patterns. Lower bf16 G_FCONSTANT via an integer constant.
- Add iXLen for building new instructions, keeping the any-scalar sXLen
for legality rules. Build mask vectors with i1 elements and use
integer types for other newly created values.
- Call lowering: bitcast f16/f32 values to integer before any-extending
them into wider locations, and bitcast back after truncating.
- RegisterBankInfo: an f64 on a Zilsd target without D lives in a GPR
pair, not an FPR.
[2 lines not shown]
[libc++] Mark LWG4480 as complete and test (#242789)
Resolves #189845
`<stdatomic.h>` already provides `ATOMIC_CHAR8_T_LOCK_FREE` via
`<atomic>`. This PR adds it to the synopsis along with a corresponding
test.
---------
Co-authored-by: A. Jiang <de34 at live.cn>
[MemProf] Assign contexts trimmed within a call's inlined stack ids to its node (#240193)
When updating the stack nodes for callsites while building the graph, a
call with inlined frames is assigned the contexts that include all of
its stack ids that have context nodes. Profile matching trims a
context's stack once its allocation type is unambiguous, so a context
can be trimmed at an earlier stack id in that sequence. When a later
stack id has a node because of contexts for a different allocation,
such a context was only assigned to calls ending at the earlier id
that was last in the trimmed context (e.g. an out-of-line copy of
the inlined callee), and the call with the inlined frames kept calling
the original callee even though all of its profiled contexts were cold.
Give each such call duplicates of the trimmed contexts' ids, created up
front from the unmodified graph so the main stack node update loop can
also hand them to the other calls those contexts flow through, except
calls ending at a node of the creating call's own sequence, which would
compete with it for the duplicates. Measured a small (0.5%) peak RSS
increase on a large internal target due to the additional duplicates,
[12 lines not shown]
Revert "[RISCV][GlobalISel] Combine byte-spreading operations to orc.b" (#243271)
Reverts llvm/llvm-project#240276
Test is failing when built without assertions.
Fixes #243242.
[CodeGen][NFC] Factor MachineBlockPlacement::alignBlocks into reusable helper (#241995)
Post-layout passes such as BasicBlockSections reorder and clone blocks
after MachineBlockPlacement (changing fallthrough predecessors) and
operate on Propeller block/edge counts rather than MBFIWrapper. Extract
the block alignment logic from MachineBlockPlacement::alignBlocks() into
llvm::alignMachineBlocks() with a lightweight BlockAlignmentProfile
interface so BasicBlockSections can recompute block alignments from
Propeller profiles without re-implementing the core alignment logic.
Assisted by: Gemini-4 Carbon
SimplifyLibCalls: preserve allocation attributes in optimizeNew
When `LibCallSimplifier::optimizeNew` rewrites an `operator new` call into a
`__hot_cold_t` allocation call, forward the allocation attributes
(`allockind`, `allocsize`, `"alloc-family"`, and parameter `allocalign`) from
the original call onto the synthesized call instruction alongside its metadata.
This is a new patch since 2023 (unlike most of the rest of the stack)
and allows the next patch to remove more specialized C++ knowledge
from the LLVM layer, instead leaving it in clang where it seems more
sensible.
This patch was authored with the assistance of an LLM and was reviewed
by me before posting.
CGExprCXX: emit family and allockind attributes
This allows us to remove the special knowledge about C++ allocation from
LLVM's MemoryBuiltins.cpp, instead trusting attributes to communicate
the important properties.
Annoyingly, clang has to grow a little bit of hard-coded knowledge about
mangled symbol names in order to actually get the most-basic-matching
::operator::new when working out the alloc-family attribute. I spent
some time trying to do this "right" by looking up the available
operators new and finding the best one, but that's super tricky and this
has the virtue of working and being pretty obviously correct when
compared to the old code.
Happily, clang now emits allocator family attributes for custom
allocators like the `__hot_cold_t` overloads in `new_hot_cold.cpp`
(which `isReplaceableGlobalAllocationFunction` in clang recognizes),
allowing us to remove that special knowledge from LLVM.
[7 lines not shown]
clang: emit allocalign to LLVM for alloc_align attributes (#239730)
LLVM is growing knowledge of this, so we should inform it when we have
the information available. As a bonus, since we now check return
alignments in the callee as of an earlier patch of mine, we can now drop
the redundant llvm.assume() that this previously emitted.
This patch is originally from 2023 and was rebased and revised with
assistance from an LLM. I have manually reviewed the diffs to the limit
of my abilities (I don't do much in LLVM, and this is my first time
touching clang in a meaningful way) and they appear sensible. The commit
messages are entirely written by me.
Differential Revision: https://reviews.llvm.org/D119271
[clang] Pass VFS to the ExpansionContext in expandResponseFiles (#243244)
expandResponseFiles() created the llvm::cl::ExpansionContext without a
file system and only set the provided VFS afterwards. The constructor
falls back to vfs::getRealFileSystem() when no file system is given, so
the real file system was always obtained even when the caller provided
a VFS. With the IO sandbox enabled, this is a sandbox violation, e.g.
when the dependency scanner builds a compilation for a by-name query.
Pass the VFS to the constructor instead.
[libc][math] Use multiply_sub and sub_multiply across math headers.
Replace negated multiply_add calls (multiply_add(a, b, -c), multiply_add(-b, c, a),
and multiply_add(b, -c, a)) across FPUtil and math implementation headers with
multiply_sub and sub_multiply.
[libc][math] Add multiply_sub and sub_multiply to multiply_add.h.
- Add fputil::multiply_sub (x * y - z) and fputil::sub_multiply (x - y * z)
alongside fputil::multiply_add (x * y + z).
- Use template-based hardware FMA dispatch and size-aware inline overloads so
explicit multiply_add<T> calls also use hardware FMA when available.
- Add NumberPair<T> add/sub overloads in double_double.h, and matching
multiply_sub and sub_multiply overloads for DoubleDouble and DyadicFloat.
- Add unit tests in libc/test/src/__support/FPUtil/multiply_add_test.cpp.
[SPIR-V] Legalize folded byte-offset GEPs for workgroup memory (#227104)
Fix `SPIRVLegalizePointerCast` crashes on constant-index workgroup array
accesses.
Rebuild folded byte-offset GEPs as typed logical access chains. For
example, byte offset `1024` into `[257 x i32]` becomes indices `0, 256`.
This keeps byte-address resource legalization unchanged and adds a
regression test for #227064.
Fixes #227064
Assisted by: Github Copilot
[X86] Enable SlotIndexes compaction by default
Compacting SlotIndexes before greedy allocation drops the entries of
erased instructions, so live range sizes and instruction distances
reflect the code being allocated rather than how much was erased
before it. AMDGPU has done this since #235806; do the same for X86.
Codegen changes only through allocation: no new crashes, verifier
errors or allocation failures in the lit suites, including
EXPENSIVE_CHECKS, and a stage-2 build of LLVM, clang and lld with
-verify-machineinstrs passes its own tests. On llvm-test-suite at -O3,
spills drop 0.3% and .text 0.06%, with compile time unchanged.
Most test updates are regenerated checks. misched-matmul now inserts
20 spills instead of 24. live-debug-vars-stale-slotindexes.mir gets
one more use of %27 so that accumulate_fit still reaches the stale
index path it was written for.
[X86][APX] Test the relocation of an NF add
test_regclass_not_updated_by_regalloc_2 only produced an NDD/NF add
with a GOT operand because the allocator happened to give the add a
destination different from its source. With SlotIndexes compaction it
picks the same register, so the add is compressed to the legacy form
and the CODE_6 TODO lost its test.
Add a function whose add keeps its source live, so it stays NDD and
becomes NF under any allocation.
[LAA] Check dependences between predicated stores to the same pointer. (#242902)
Vectorization may reorder stores to the same pointer across iterations
if some of them are predicated, e.g. the stores in one branch are all
executed before the stores in the other. This is only safe if the
dependences between them are safe, but they were not checked in two
cases:
* A write is only checked if there is an earlier write in its alias set,
so the dependences between stores to the first written pointer were
never checked.
* Loops with a single written pointer and no reads were treated as safe
without any checks, even if the pointer is written by multiple stores.
Check writes with multiple stores to the same pointer: if any of them is
predicated don't treat as safe. Also only skip the checks for loops with
a single store and no reads.
[AMDGPU] Select saturating FP8 converts on gfx1250
The f32 to FP8 and BF8 conversions on gfx1250 also clamp overflow and
infinity to the largest finite value while MODE.FP16_OVFL is set, so
enable the saturating selection there.
gfx1250 selects the True16 forms of these conversions, which need
their own saturating variants. WMMA and SWMMAC depend on FP16_OVFL
without reading the mode register, so restore the default before them
as well.
Change-Id: I2680d540f102adaa9b2293e21cdcbd3b9e4afd09
Co-Authored-By: Claude Fable 5.2 <noreply at anthropic.com>
[AMDGPU] Select saturating FP8 converts on gfx950
llvm.convert.to.arbitrary.fp with saturation was expanded generically
because v_cvt_pk_{fp8,bf8}_f32 do not saturate by default. For
E4M3FN, which has no infinity, the non-saturating form is poison on
overflow, so frontends had no way to get the hardware conversion with
defined overflow behaviour.
On gfx950 these conversions clamp overflow and infinity to the largest
finite value while MODE.FP16_OVFL is set. Select them for the
saturating form and let SIModeRegister set the bit, restoring the
default before anything else that depends on the mode and before
calls and returns.
Managing a second mode field exposed that a pending mode change could
be merged into an earlier insertion point without regard for the
instructions in between. Track the bits those instructions depend on.
Change-Id: I502af5acdc1d05de89189b7de63c8fe4e0d55107
Co-Authored-By: Claude Fable 5.2 <noreply at anthropic.com>
[HLSL][[202x] Do not emit implicit using directive if hlsl 202x and later (#243154)
Doesn't emit the implicit using namespace hlsl directive if 202x or
later.
Updates all tests which are 202x or later and relied on the implicit
directive with an explicit 'using namespace hlsl'.
Adds a new test showing for all versions of HLSL before 202x the
implicit using directive is included.
Closes #235100
Assisted by Copilot GPT-6.1 Sol