[llubi] Add support for vp.strided.load and vp.strided.store (#241715)
Building on top of https://github.com/llvm/llvm-project/pull/241710,
these are defined in terms of vp.gather and vp.scatter.
The base pointer must be aligned to the align attribute/default ABI
alignment, but subsequent pointers past the first lane are unaligned.
Assisted-by: Opus 5.5
[llubi] Add support for vp.gather and vp.scatter (#241710)
These follow the semantics of llvm.masked.gather/llvm.masked.scatter. We
need to just make sure that the implicit alignment is taken from the
element type, not the vector type.
Assisted-by: Opus 5.5
Update FP8 conversion test for x86 small-int ABI
b1c38d9b5839 (#236450) stopped emitting zeroext/signext on small
integer arguments for x86-64 SysV, so the autogenerated checks for
the i8 parameters no longer matched after rebasing onto main.
Regenerate the checks with update_cc_test_checks.py.
Change-Id: I2f8bb84a8d70888d0ec125e2dd1cf19dc39aaa08
Co-Authored-By: Claude Fable 5.2 <noreply at anthropic.com>
[scudo] Fix DoubleFreeFromSecondary to allow multiple messages. (#243238)
If the secondary is zeroed then a different error message will occur, so
allow that other message too.
Unwrap long lines in remaining new tests
Match sibling elementwise builtin tests, which keep RUN lines and
statements on one line.
Change-Id: I33de97f85093f2649c0035c22f99a33b1657486f
Co-Authored-By: Claude Opus 5.5 <noreply at anthropic.com>
Unwrap long lines in new tests
Test files are not clang-formatted; keep RUN lines and statements on
one line.
Change-Id: I95e21a076106bf2488e7246569609a8feb514d88
Co-Authored-By: Claude Opus 5.5 <noreply at anthropic.com>
Add tests for FP8 conversion builtin edge cases
Cover strictfp, _BitInt vectors, SVE and fixed-length RVV vectors,
__mfp8 codegen, f16/bf16 vector results, and 1- and 3-element
vectors, per review.
Change-Id: I1d3d32bca1d9a5aea4a6f9292230d80c5a62519d
Co-Authored-By: Claude Opus 5.5 <noreply at anthropic.com>
Address review feedback on FP8 conversion builtins
Reuse err_builtin_invalid_arg_type for all source operand errors
instead of builtin-specific diagnostics.
Accept integer constants that fit the format width, such as 0x38,
and std::byte, so common byte values need no explicit cast.
Drop the unused OpenCL fp64 path from checkFloatingPointTypeSupport.
Document floating-point environment and fast-math behavior; trim
implementation detail from the user docs.
Change-Id: I8676a49540bdbbb8a90827c83764803eeea850a5
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply at anthropic.com>
[clang] Add elementwise conversions from encoded FP8 values
Add nine builtins converting Float8E5M2, Float8E4M3FN, and Float8E5M3FNU
encodings to _Float16, __bf16, or float through
llvm.convert.from.arbitrary.fp.
Accept exactly 8-bit integer scalars and generic fixed-length vectors,
preserving vector kinds and element counts without integer promotions.
Also support scalar AArch64 __mfp8 containers. Apply destination type
availability checks, including deferred offload diagnostics, and reject
constant-expression use.
Document the interface and add semantic, template, language-mode,
target-specific, and IR-generation regression coverage.
[lldb][NFC] Move two scans of symbol table to Symtab methods (#241113)
This patchset changes two areas that were iterating over all Symbols
inside a Symtab, to instead call a method on Symtab.
`SymbolFileDWARFDebugMap::InitOSO` looks at every symbol to detect any
Mach-O debug map entries that are pointing to .o files with the dwarf in
them. It will now call `Symtab::HasDebugMap()`.
`SymbolFileSymtab::CalculateAbilities` searches for every symbol of type
eSymbolTypeSourceFile to add CompileUnits on to the capabilities flags
returned; that's now done by `Symtab::HasSourceFiles`. It also
initialized some lookups in SymbolFileSymtab eagerly: `m_func_indexes`,
`m_code_indexes`, `m_data_indexes`, `m_source_indexes`, and
`m_objc_class_name_to_index`. Three of those indexes are not used:
`m_func_indexes`, `m_data_indexes`, `m_objc_class_name_to_index`.
Move the calculation of `m_source_indexes` and `m_code_indexes` into a
new `SymbolFileSymtab::InitializeSymbolIndexes` method.
[19 lines not shown]
[Attributor] Don't derive interface attributes from non-exact definitions
The arguments and the return value of a function that might be replaced
by another definition, e.g. a linkonce_odr function, are not updated, but
AAs still took known information from the body in initialize, which was
then kept and propagated to the callers: the uses of the arguments in
followUsesInMBEC, assumptions in getAttrsFromAssumes, and the returned
values in AANonNull::isImpliedByIR. Add Attributor::canUseBodyFor and
skip the body for the interface of such functions.
Co-Authored-By: Claude Opus 5.5 <noreply at anthropic.com>
[Attributor] Precommit tests for attributes derived from non-exact definitions
The Attributor derives align for the arguments and nonnull for the return
value of linkonce_odr functions from their bodies, and propagates them to
the callers, although another definition might be used instead.
Co-Authored-By: Claude Opus 5.5 <noreply at anthropic.com>
[VPlan] Remove header masks whose tail lanes aren't used
Replace a use of header-mask && x with x if the user, transitively
through freeze, logical-and, not and mask.beforefirst(freeze ...), only
reaches masks that are and'ed with the header mask again. The header
mask is a prefix and lane i of mask.beforefirst only depends on lanes
0..i, so the lanes outside of the header mask are never used.
This avoids the i1 vp.merge that EVL tail folding needs for the inner
header mask, e.g. in the early exit mask.beforefirst(freeze(header-mask
&& c)) and in switch default edges.
[VPlan] Allow tail folding early exits with side effects
Continues the work in #179595.
This allows us to tail fold e.g. the following loop:
```c
extern char a[1024], b[1024], c[1024], y[1024];
void two_guards(void) {
for (int i = 0; i < 1000; i++) {
if (a[i] == 0)
break;
if (b[i] > 3) {
if (c[i] < 9)
y[i] = 1;
}
}
}
```
[30 lines not shown]
[VPlan] Support tail folding uncountable early exits
Enables the first step towards #179595. After #237254, the MaskedHandleExitInVectorLoop early exit style uses the CFG to handle masking for early exit loops. This allows it to compose with tail folding which is also handled via the CFG in #176143.
This allows us to tail fold the following loop:
```c
extern char x[256];
int f(char * restrict y) {
for (int i = 0; i < 256; i++) {
if (x[i] == 0)
return i;
}
return -1;
}
```
Into this on RISC-V:
[18 lines not shown]
[VPlan] Use vputils::findCanonicalIVIncrement in more places. NFC
In a future patch we want to enable more scenarios with early exits and tail folding. In these scenarios the terminator will no longer be a BranchOnCount/BranchOnCond but a BranchOnTwoConds, so a bunch of canonical IV matchers break. Use vputils::findCanonicalIVIncrement instead since it searches the users of the canonical IV instead of looking at the terminator.
[clang] Remove declarations without definitions (NFC) (#242170)
This patch removes function and method declarations that have no
definitions and no callers. This patch also explicitly deletes two
undefined default constructors so that accidental default construction
fails at compile time instead of link time.
Assisted-by: Antigravity
[VPlan] Allow early exit loops with side effects by masking in the vector loop.
When all blocks with side effects are dominated by every early exit, we can vectorize the loop if we mask off any lanes past the exit. We can do this with control flow:
```mermaid
flowchart TD
header --> exiting
exiting -->| exit mask | sideeffects --> latch --> latch.split
exiting -->| !exit mask | latch.split
```
For every early exiting block we add a branch to the latch on `!mask.beforefirst(exitcond)`. This is implemented in UncountableExitStyle::ReadOnly which is now renamed to MaskedHandleExitInVectorLoop.
This is a complementary exit style to MaskedHandleExitInScalarLoop. If any of the side effects aren't dominated by the early exit it still falls back to the scalar loop style, which is why none of the existing early exit side effect tests changed. Masking in loop is profitable for RISC-V and I'm assuming it's also profitable for AArch64 too, but if not then we can add a TTI hook for a preferred exit style.
[VPlan] Preserve SSA in early exit loops
MaskedCond is used to logically-and the condition to early exit with the current block in-mask. However because we use it in the latch block where the condition isn't defined on all paths, we have to workaround it not being in SSA in the verifier.
This fixes this by using reconstructSSA added in #212209 to preserve SSA in both the exit conditions and live outs in the exit blocks. It also means we get the right exit condition without MaskedCond and allows us to remove the MaskedCond exceptions in the verifier. MaskedCond is still kept since a followup PR will still use it.
One subtle point is that we need to freeze any branch conditions. Any lanes after and including the early exiting lane might contain poison and we can have branches on those poison lanes that didn't we didn't previously branch on in the scalar loop.
There's no miscompile on main today because we coincidentally don't keep SSA so exit conditions are read directly without being polluted by poison stemming from a branch past the early exit. But in this PR the exit conditions are polluted and we can end up with poison in the condition operand of a select, which causes an undefined result for AnyOf even if it freezes its operand. The poison in the branch will also cause problems in future if we decide to preserve uniform control flow and emit branches instead of blends. The exit_cond_merged_after_poison_branch test illustrates this.
[VPlan] Simplify BLEND X/M, false, ..., false -> M && X
If an i1 blend has false for all but one incoming value, then we can simplify this to the one non-false value. Blends by construction never have any overlapping mask values, so we don't need to worry about the order as the masks can't "shortcut" each other.
[VPlan] Move extract-lane(0, canWideIV) combine before unrolling
Some extract-lane combines can't be performed before unrolling, e.g. the one that combines it to Instruction::ExtractElement can end up extracting from a different part. But the extract-lane(0, canWideIV) can because at index 0 you're always extracting from the first part. Do this before unrolling, which improves codegen of unrolled code.