fix(SelectionDAG): isolate retry known bits
The LHS demanded-bits query already excludes bits masked by the RHS.
Its returned facts cannot safely reduce the RHS demand in turn.
Query LHS known bits independently, and preserve the original RHS facts
when a reduced-demand retry does not simplify. Defer the retry until
existing folds fail and skip constant or shared RHS operands.
Refresh the affected X86 atomic codegen checks.
Refs #230700
[SLP]Fix assert on gather of out-of-bounds extract
Failed shuffle checks left a partial mask that adjustExtracts still
consumed. Lanes absent from the mask (out-of-bounds, poison, undef
index) also widened the shuffle, so valid two-vector gathers were
rejected.
Reviewers:
Pull Request: https://github.com/llvm/llvm-project/pull/230847
[clang][CodeGen] Emit callee_type metadata for all indirect calls
CodeGenFunction::EmitCall only attached the call graph section callee_type
metadata to an indirect call when its caller asked for the emitted call
instruction through the callOrInvoke out parameter, because the emission
was nested in the block that stores it. Indirect calls emitted by callers
that do not pass callOrInvoke, such as the virtual destructor call of a
delete expression (emitVirtualObjectDelete -> EmitCXXDestructorCall), got
no callee_type metadata and therefore no indirect callee type identifiers
in the .llvm.callgraph section, so the call graph missed those edges.
Attach the metadata to the emitted call instruction regardless of whether
the caller asked for it.
Stacked on #230831.
Assisted-by: Claude
[clang][CodeGen] Identify virtual methods by their override chain root in the call graph section
A virtual call site is typed after the method named at the call site, but it
can reach any overrider of that method, and overriders may differ from it in
their covariant return type. In that case the `!callee_type` identifier of a
call through a base class pointer did not match the `!callgraph` identifier
of the overrider, and the call graph section missed the edge.
Identify every method of an override chain, at definitions, call sites and
thunks, by the type of the method that introduced the virtual function, so
that virtual calls through a pointer to any class of the hierarchy match the
overriders they can reach.
Stacked on #230838.
Assisted-by: Claude
[clang][CodeGen] Attach a single callgraph type identifier to function definitions
A function definition is annotated with its call graph section type
identifier twice, when its declaration is created (SetFunctionAttributes)
and when its body is emitted (SetLLVMFunctionAttributesForDefinition), so
every defined function carries two identical `!callgraph` attachments:
define void @foo() #0 !callgraph !3 !callgraph !3 {
Replace the attachment instead of adding a second one, so that a function
carries exactly one identifier. The `.llvm.callgraph` section is unaffected,
the AsmPrinter only uses the first identifier of a function.
Assisted-by: Claude
[libc][math] Speed up log2f subnormal normalization (#223718)
## Summary
Normalize positive subnormal inputs in `log2f` by converting the integer
significand to float exactly and accounting for the `2^-149` scale.
This replaces the existing floating-point multiplication by `2^23`.
Replacement prepared on main at
`681b57195f495c346c1d15229334bb0ffcbba957`.
The remaining diff changes only `libc/src/__support/math/log2f.h`.
The `log10f` table relocation is already covered upstream by #229173,
merged as `636bedc3b3c00266a95f139c4b79cb8b940c1a36`. It moved `LOG10_R`
and `LOG10_2` to namespace-scope inline constexpr storage. This update
preserves that implementation and drops our redundant table change,
leaving only the `log2f` subnormal optimization.
## Validation
[21 lines not shown]
[SLP]Fix crash on gather reduction root cast cost
Do not use gather nodes for cast context evaluation to prevent a crash.
Reviewers:
Pull Request: https://github.com/llvm/llvm-project/pull/230836
[clang][CodeGen] Emit callee_type metadata for all indirect calls
CodeGenFunction::EmitCall only attached the call graph section callee_type
metadata to an indirect call when its caller asked for the emitted call
instruction through the callOrInvoke out parameter, because the emission
was nested in the block that stores it. Indirect calls emitted by callers
that do not pass callOrInvoke, such as the virtual destructor call of a
delete expression (emitVirtualObjectDelete -> EmitCXXDestructorCall), got
no callee_type metadata and therefore no indirect callee type identifiers
in the .llvm.callgraph section, so the call graph missed those edges.
Attach the metadata to the emitted call instruction regardless of whether
the caller asked for it.
Stacked on #230831.
Assisted-by: Claude
[clang][CodeGen] Identify virtual methods by their override chain root in the call graph section
A virtual call site is typed after the method named at the call site, but it
can reach any overrider of that method, and overriders may differ from it in
their covariant return type. In that case the `!callee_type` identifier of a
call through a base class pointer did not match the `!callgraph` identifier
of the overrider, and the call graph section missed the edge.
Identify every method of an override chain, at definitions, call sites and
thunks, by the type of the method that introduced the virtual function, so
that virtual calls through a pointer to any class of the hierarchy match the
overriders they can reach. While here, attach `!callgraph` with `setMetadata`
so that a function carries exactly one identifier.
Assisted-by: Claude
[SLP]Drop wrap flags for negated lanes with constant expressions
Only ConstantInt operands are negated, so drop nsw/nuw on
a non-constant-int constants.
Follow-up to #230453.
Reviewers:
Pull Request: https://github.com/llvm/llvm-project/pull/230835
[clang][CodeGen] Emit callee_type metadata for all indirect calls
CodeGenFunction::EmitCall only attached the call graph section callee_type
metadata to an indirect call when its caller asked for the emitted call
instruction through the callOrInvoke out parameter, because the emission
was nested in the block that stores it. Indirect calls emitted by callers
that do not pass callOrInvoke, such as the virtual destructor call of a
delete expression (emitVirtualObjectDelete -> EmitCXXDestructorCall), got
no callee_type metadata and therefore no indirect callee type identifiers
in the .llvm.callgraph section, so the call graph missed those edges.
Attach the metadata to the emitted call instruction regardless of whether
the caller asked for it.
Stacked on #230831.
Assisted-by: Claude
[clang][CodeGen] Identify virtual methods by their override chain root in the call graph section
A virtual call site is annotated with the type of the method named at the
call site (e.g. `Base::clone` for `b->clone()` with `b` of type `Base *`),
but it can reach any overrider of that method. Overriders may differ from
the overridden method only in their covariant return type, in which case
their call graph section type identifier (`!callgraph`) did not match the
type identifier of the call site (`!callee_type`), so the call graph
missed the edge: `Derived *Derived::clone()` was identified as
`_ZTSFP7DerivedvE`, while calls through `Base *` were typed
`_ZTSFP4BasevE`.
Identify every method of an override chain, at the definition and at the
call site, by the type of the method that introduced the virtual function
(the root of the override chain), so that virtual calls through a pointer
to any class of the hierarchy match the overriders they can reach. A
method that overrides several virtual functions with different covariant
return types is identified after the first one; the return adjusting
thunks that fill the vtable slots created by the other overridden methods
[14 lines not shown]
[SLP]Count splats as single broadcast for in-loop trees
The loop-aware instruction-count check scaled the per-lane count of
splat gather entries by the trip count, rejecting profitable trees.
A splat is a single broadcast regardless of the loop scale. Fixes
4-6% RAJAPerf Apps_DIFFUSION3DPA regression on AArch64 introduced
by #211096.
Reviewers:
Pull Request: https://github.com/llvm/llvm-project/pull/230821
[ConstantFolding] Fold `bitinsert` and `bitextract` (#229115)
Constant fold `bitinsert` and `bitextract`.
- Bits of a `ConstantByte` are extracted or inserted, then reinterpreted
as if by `bitcast`.
- A `bitinsert` that overwrites every bit of the base becomes a
`bitcast` of the value.
- An out-of-range, `poison` source or `poison`/`undef` offset returns
`poison`.
```llvm
%a = bitextract i8, b32 512, i32 8 ; i8 2
%b = bitinsert b32 0, i8 1, i32 8 ; b32 256
%c = bitinsert b16 0, half 1.0, i32 0 ; b16 15360, the bits of half 1.0
%d = bitextract i8, b32 0, i32 25 ; i8 poison
```
