[LowerInvoke] Use createCallMatchingInvoke instead of rolling it ourselves (#200536)
Using createCallMatchingInvoke ensures we copy metadata from the invoke
onto the call.
This bug was found by a large run of Opus 4.7 looking for bugs in LLVM.
[IR] SubclassOptionalData is only optional for Instruction (#200419)
BasicBlock currently uses this for HasAddressTaken. ConstantExpr uses
this for flags and it should not be possible to modify constant
expressions after they have been created.
Therefore, update the documentation accordingly and remove the unsafe
clearSubclassOptionalData.
Remove hasSameSubclassOptionalData, as it is unused.
[libc][ci] Clean up libc-fullbuild-tests precommit CI. (#200520)
- Pin containers.
- Cleaner names for different targets and options.
- Add Build Test step.
- Skip shared tests and death tests.
[IR] Fix PoisoningVH relocation of a poisoned handle (#200540)
In LLVM_ENABLE_ABI_BREAKING_CHECKS builds, when poisoned (`deleted()` or
`allUsesReplacedWith()`), a PoisoningVH is removed from its use list but
keeps its raw value pointer for identity, so its PrevPtr/Next are left
stale.
PoisoningVH has no move constructor, so relocating a value that embeds
one
falls back to the copy constructor, where `setRawValPtr` relinks with
the stale pointers and corrupts the use list.
This is a latent bug for any relocation of a PoisoningVH handle
but becomes load-bearing for #199615 , which relocating erase exercises
it via ScalarEvolution's BackedgeTakenInfo (its ExitNotTakenInfo holds a
PoisoningVH<BasicBlock>).
Fix by special casing the `Poisoned` case.
Aided By Claude Opus 4.8
[IR][NFC] Inline Type::isIntegerTy(n) (#200471)
Although this gets inlined in LTO builds, non-LTO builds benefit from
having isIntegerTy(n) defined in a header.
Co-authored-by: Nikita Popov <npopov at redhat.com>
[libc] Tag cpp::byte with gnu::may_alias (#200462)
Clang's TBAA grants the [basic.lval]/11.3 char-aliasing privilege only
to the named ::std::byte type (Type::isStdByteType() requires the enum
to be declared in the std namespace). LIBC_NAMESPACE::cpp::byte lives in
libc's cpp namespace, so it gets its own TBAA node disjoint from char
even though it has the same shape as std::byte.
That mismatch lets the optimizer reorder typed loads past raw-byte
writes through cpp::byte *, miscompiling HeapSort on rv64/Release
(UnsortedThreeElementArray{1,2,3}, UnsortedTwoElementArray1 in
SortingTest.h). The same hazard is latent in every cpp::byte *-based
raw-aliasing site: memory_utils Ptr/CPtr, lsearch/lfind, block.h and
freelist_heap.h allocator metadata.
Tag the type with gnu::may_alias so accesses through cpp::byte * share
the universal char-aliasing TBAA node, fixing all of the above in one
place. This patch also reverts PR #194171, as the may_alias attribute
fixes it too.
[mlir][nfc] Fix assertion text in IndexingUtils (#181826)
Update the assertion text to match the actual code behavior.
Some functions enforce strictly positive values, whereas the error
message incorrectly mentioned "nonnegative".
[CIR][OpenCL] Lower kernel argument metadata to LLVM IR
Translate CIR OpenCL kernel argument metadata into the LLVM IR kernel_arg_* metadata attached to kernel functions. Preserve optional argument names so -cl-kernel-arg-info controls the LLVM metadata surface through the CIR attribute.
[CIR][OpenCL] Attach kernel argument metadata to CIR functions
Emit the CIR OpenCL kernel argument metadata attribute for kernel functions. Preserve CIR language address-space kinds until lowering and include argument names only when `-cl-kernel-arg-info` is enabled.
Depends on #199530.
[VPlan] Thread scalar type through VPBlend, VPExpression recipes. (NFC) (#200255)
Set the scalar type for VPBlendRecipe and VPExpressionRecipe at
construction time, instead of inferring it on demand via VPTypeAnalysis.
With this change, all VPValues have their scalar type set at
construction, so VPTypeAnalysis::inferScalarType becomes a thin wrapper
around VPValue::getScalarType.
To be removed in a follow-up:
https://github.com/llvm/llvm-project/pull/200256.
PR: https://github.com/llvm/llvm-project/pull/200255
[SLP] Fix extract-cost scale using NCD of all external-user sites
ExtractCostCalculated deduplicates by scalar so only the first
ExternalUser determines the scale, making the cost depend on IR block
ordering via LLVM's reverse-insertion use-list order.
Add a pre-pass computing ScalarToExtractBlock - the nearest common
dominator of all effective extract sites per scalar. For PHI users inside
a loop the effective site is the incoming block; for PHI users outside
all loops it is the PHI's own block (scale = 1). The extract cost is
then scaled by getLoopNestScale of the NCD block, which is fully
order-independent.
Fixes #199548
Reviewers: bababuck, RKSimon, hiraditya
Pull Request: https://github.com/llvm/llvm-project/pull/199962
[MLIR][GPU][NFC] Reformat GPU target attachment tests (#199339)
Reformat attach-targets.mlir so each GPU module has a labeled check
block, split target-attachment RUN lines, and keep comments tied to the
expected target-specific matches.
[SLP] Fix extract-cost scale for LCSSA-phi external users in nested loops
getScaleToLoopIterations() used U->getParent() for all PHI-node external
users. For an LCSSA phi at an inner-loop exit still inside an outer loop,
this gave outer-loop scale instead of inner*outer scale. Because
ExtractCostCalculated deduplicates by scalar, only the first ExternalUser
determines the scale, making the cost order-dependent on use-list ordering
(and thus on .ll block ordering).
Reviewers: hiraditya, RKSimon, bababuck
Pull Request: https://github.com/llvm/llvm-project/pull/199954
[SLP] Recompute copyable operand deps for duplicate copyable nodes
A bundle may duplicate a previously built node that has copyable elements
(same schedulable instructions, different copyable lane) while the parent
node also has copyable elements. An operand modeled as a copyable element
in the previous node is then used directly by the new node, which is not
registered in the tree yet. Recomputing that operand's direct
dependencies at this point misses the direct use, so the scheduler
decrements the operand more times than its dependency count and trips the
unscheduled-deps assertion.
Defer recomputation of such operand dependencies via
RecalcCopyableOperandDeps and redo it at the next bundle scheduling, when
the duplicate node is part of the tree. Also clear and recompute the
direct dependencies of bundles whose user is a gather node referenced
through EdgeIdx == UINT_MAX in scheduleBlock, so combined gather
sub-entries get correct dependencies against the full tree.
Reviewers:
Pull Request: https://github.com/llvm/llvm-project/pull/200564
[bazel] Fix compiler-rt:interception (#200561)
- Add `-DCOMPILER_RT_BUILD_PROFILE_ROCM=1`
- Prune `"lib/sanitizer_common/*.S"`, it means `*.inc.S`
- Add `-fvisibility=hidden`
Bazel doesn't add `-gline-tables-only` by default. Add flags to CMake
side to align the build to Bazel.
- `-DCOMPILER_RT_HAS_G_FLAG=OFF`
- `-DCOMPILER_RT_HAS_GLINE_TABLES_ONLY_FLAG=OFF`