[SandboxVectorizer] Vectorize partial store sub-bundles in LoadStoreVec
runOnRegion() previously required an entire store seed chain to vectorize
as one unit. A seed slice can legitimately fail that as a whole while a
sub-run within it is still fine, e.g. because it spans an address gap
(SeedBundle::getSlice sorts by address but doesn't guarantee contiguity)
or a sub-range fails to schedule. Add findLegalStoreRun()/isLegalStoreRun()
to search for the longest vectorizable run starting at a given position,
and have runOnRegion() call vectorizeStores() once per such run instead of
once for the whole chain. isLegalStoreRun() is purely an address/scheduling
check now -- no operand-eligibility check is needed since packOperands()
accepts any operand kind.
The search only ever shrinks a candidate length, never grows one:
Scheduler::trySchedule() permanently commits a successful multi-instruction
bundle (a later request that overlaps it and is not an exact match returns
AlreadyScheduled and fails), so starting from the longest candidate and
shrinking on failure is the only search order compatible with the
scheduler's contract.
[18 lines not shown]
[SandboxVectorizer] Make LoadStoreVec::vectorizeStores direction-agnostic
Remove the AllLoads/AllConstants operand-kind gate: vectorizeStores() no
longer requires a store chain's value operands to be all loads, all
constants, or neither. Instead it always builds the vector value via
packOperands(), which packs any mix of loads, constants, or arbitrary SSA
values via extractelement/insertelement -- direction-agnostic in the sense
that it doesn't care what kind of operand it's given, unlike the
load-specific and constant-specific paths it replaces.
vectorizeLoads()/createVectorLoad(), added by the previous commit for
load-kind seed slices, are unaffected: packOperands() only replaces
vectorizeStores()'s old all-loads fast path, which duplicated the same
vector-load construction createVectorLoad() already does.
Update load_store_vec.ll, load_store_vec_mixed_types.ll, and
AMDGPU/basic.ll for the new pack-based codegen.
check-llvm Transforms/SandboxVectorizer passes (31/31).
[SandboxVectorizer] Make tryEraseDeadInstrs tolerate multi-use load operands
vectorizeStores() currently only ever passes single-use load operands
to tryEraseDeadInstrs(), since isFoldableLoadOperand() guarantees this.
This is NFC today, but is prep for a follow-up commit that switches
vectorizeStores() over to packOperands(), which does not make that
guarantee.
[SandboxVectorizer] Add a direction-agnostic operand packer to LoadStoreVec
Add packOperands(), which packs a list of arbitrary Values into a single
vector value via extractelement/insertelement, regardless of what kind
of Value each one is (load, constant, or any other SSA value) -- unlike
vectorizeStores()'s current load-specific and constant-specific paths,
which require every operand to be uniformly one kind or the other.
Operands are combined at the granularity of their narrowest common
scalar element type (getCombinedElementType(), the same rule
VecUtils::getCombinedVectorTypeFor() uses for a mixed-type instruction
chain, generalized to arbitrary Values rather than only Instructions).
An operand wider than that granularity is split into multiple lanes via
a bitcast, matching how a vector-typed element is split into per-lane
extracts.
reinterpretSameWidth() picks bitcast, ptrtoint, or inttoptr as needed to
reinterpret an operand at the combined granularity: a plain bitcast can't
convert between pointer and non-pointer types, and inttoptr requires an
[9 lines not shown]
[SandboxVectorizer] Dispatch LoadStoreVec::runOnRegion on seed kind
runOnRegion() previously assumed its seed slice was always a store chain,
unconditionally casting Bndl[0] to StoreInst. This crashed (assertion in
areConsecutive<StoreInst>) whenever -sbvec-collect-seeds included "loads",
since a load-seeded region's Aux holds LoadInsts.
Add a symmetric top-level path for load-kind seed slices: createVectorLoad()
builds the vector load, and vectorizeLoads() -- unlike a load that merely
feeds a store -- has to handle arbitrary uses, so it replaces each original
load with an extract from the vector load (VecUtils::unpack()) rather than
just discarding it. runOnRegion() determines the seed kind from Bndl[0]
(asserting the slice is homogeneous, which SeedCollection guarantees) and
dispatches to vectorizeStores()/vectorizeLoads() accordingly.
No sub-run search yet: vectorizeStores()/vectorizeLoads() are each still
tried once over the whole seed slice, same as before this commit for
stores. Sub-bundle partitioning for both kinds is a separate follow-up.
[6 lines not shown]
Mk/Uses/npm.mk: Always use X' for denoting BLOB data type
SQLite's .dump always uses X' (capital X) for BLOB. So it's better we
use X' in the code for securing reproducibility of node modules
tarball created by pnpm.
Also, conversion from x' to X' in output dump file in case x' is
accidentally mixed in.
Reported by: pkg-fallout
[CodeGen] Derive the regalloc pipeline from the allocator, remove -optimize-regalloc (#215740)
-regalloc selects the allocator and -optimize-regalloc independently
selects the pipeline, but only two combinations are meaningful: a
non-fast allocator with -optimize-regalloc=0 is a fatal error, and
-regalloc=fast with the optimized pipeline runs the full preparation
just to rewrite a vacuous VirtRegMap.
Derive the pipeline from the allocator instead: an explicit -regalloc
implies its pipeline; the default follows the optimization level.
-regalloc=greedy now works at -O0 and -regalloc=fast always uses the
fast pipeline. Update tests to name the allocator; regenerate three
that checked the hybrid output.
Aided by Fable 5
[SandboxVectorizer] Dispatch LoadStoreVec::runOnRegion on seed kind
runOnRegion() previously assumed its seed slice was always a store chain,
unconditionally casting Bndl[0] to StoreInst. This crashed (assertion in
areConsecutive<StoreInst>) whenever -sbvec-collect-seeds included "loads",
since a load-seeded region's Aux holds LoadInsts.
Add a symmetric top-level path for load-kind seed slices: createVectorLoad()
builds the vector load, and vectorizeLoads() -- unlike a load that merely
feeds a store -- has to handle arbitrary uses, so it replaces each original
load with an extract from the vector load (VecUtils::unpack()) rather than
just discarding it. runOnRegion() determines the seed kind from Bndl[0]
(asserting the slice is homogeneous, which SeedCollection guarantees) and
dispatches to vectorizeStores()/vectorizeLoads() accordingly.
No sub-run search yet: vectorizeStores()/vectorizeLoads() are each still
tried once over the whole seed slice, same as before this commit for
stores. Sub-bundle partitioning for both kinds is a separate follow-up.
[6 lines not shown]
[Profcheck] Enable MergeFunc tests (#215881)
Remove MergeFunc from the profcheck exclusion list.
All MergeFunc tests pass with LLVM_ENABLE_PROFCHECK enabled, so no
expected-failure entries are needed.
Fix drift-repair apps test calling a nonexistent method
## Problem
`test_drift_repair_ix_apps` calls `docker.start_service`, which doesn't exist on this branch — the method is registered as `docker.state.start_service` on the private `DockerStateService`. The test was brought over from master, where the docker plugin's typesafe conversion collapsed that service into the public `docker` namespace. It has failed with "Method does not exist" on every CI run since it landed, so the drift-repair path it covers has never actually been exercised here, and it leaves `/mnt/.ix-apps` at 0755 because it loosens the perms before the call that would re-tighten them.
## Solution
Point the call at `docker.state.start_service`. Same function and same default `mount_datasets=False` — master's `docker.start_service` just delegates into it.
[ADT][docs] Document SortedVectorMap in ProgrammersManual (NFC) (#215930)
This patch adds a section for SortedVectorMap in the Programmer's
Manual, detailing its intended use cases (small maps, minimal memory
overhead vs DenseMap, iteration in sorted key order) and trade-offs
(O(log N) binary search lookup and O(N) insertion/deletion vs O(1) in
DenseMap).
We also add a cross-reference from the existing "A sorted 'vector'"
section.
[BOLT] Link pthread in Profile and Target libs to fix undefined pthread_rwlock_* symbols (#215748)
MCPlusBuilder.h uses llvm::sys::RWMutex, which aliases std::shared_mutex
on non-Apple platforms. Its inline lock/unlock calls pthread_rwlock_*
directly, so any shared library that instantiates these inline functions
must link pthread explicitly under -Wl,-z,defs.
This only shows up when building with -DBUILD_SHARED_LIBS=ON. With
static libs, the missing symbols get resolved elsewhere on the final
link line, so the problem stays hidden.
LLVMBOLTCore, LLVMBOLTRewrite, LLVMBOLTPasses, and LLVMBOLTUtils already
link ${LLVM_PTHREAD_LIB}; LLVMBOLTProfile and the LLVMBOLTTarget{X86,
AArch64,RISCV} libs were missing it, causing link failures.
[ORC] Move EPCGenericJITLinkMemoryManager to RTBridge proxies (#215797)
Reimplement EPCGenericJITLinkMemoryManager's reserve/initialize/release
calls using rt::Proxy objects rather than direct
ExecutorProcessControl::callSPSWrapperAsync calls, matching
EPCGenericMemoryAccess and EPCGenericDylibManager. The manager's
behavior is unchanged.
Details:
* SymbolAddrs (five ExecutorAddrs) becomes Bindings: the allocator
instance address plus rt::Proxy handles.
* The proxy types and SPS specs are hoisted into shared headers --
RTBridge/GenericMemoryManagerProxies.h and
RTBridge/SPS/GenericMemoryManagerProxySpecs.h -- since they now need
only Shared/SPS vocabulary. Deinitialize is included in the family for
completeness though this manager does not call it.
[9 lines not shown]
dpaa2: Apply if_flags and MAC filters in dpaa2_ni_init()
make sure interface flags and filters are reprogrammed during init().
The config isn't pushed into the hardware when the interface is down but
the flags are still being set, so we need to do the initial programming
ourselves. This fixes bridge and multicast behavior.
PR: 292006
Reported by: jhibbits
Approved by: adrian
Reviewed by: jhibbits
Differential Revision: https://reviews.freebsd.org/D58330
(cherry picked from commit dc12e3e0e72a73f1ad1a14d8d0fa4e2147151720)
Signed-off-by: Nick Price <nprice at FreeBSD.org>
[lldb] Guard against null dereference in GetCppObjectPointer (#215710)
ClangUserExpression::GetCppObjectPointer dereferenced the ValueObjectSP
returned by GetObjectPointerValueObject before checking it for null. Fix
by moving the existing check above the child lookups.
[Offload] Keep empty COFF offload entry ranges alive (#215390)
A Windows offload link can have no real offload entries. One example is
a HIP program, or a HIP-related host-only object, built with RDC
but with no kernels or registered device globals. The wrapper still
emits registration code that refers to the offload entry range.
On COFF this range is built from ordered sections. Empty start and stop
sections let `lld-link /opt:ref` discard them. The registration code
then has relocations against discarded `__start_llvm_offload_entries`
and `__stop_llvm_offload_entries` symbols, and the link fails.
Linux avoids this through the ELF section-retention path. The wrapper
emits a dummy `llvm_offload_entries` section entry and places it in
`llvm.used`, which gives the section a retain flag in the ELF object.
That keeps the section alive under `--gc-sections`, so the linker can
still synthesize the `__start` and `__stop` symbols.
The same fix does not map to COFF. COFF does not use ELF-style
[20 lines not shown]
[lldb] Change the Symbol rep for re-export symbols (#213356)
On Darwin system, we have re-export symbols. A library can have a symbol
table entry for function A() that is a re-export symbol; it's only data
is the name of the actual function to call, B(). When code calls A(),
the dynamic loader will resolve this to B() in some other library.
Previously, Symbol was using its AddressRange's Address object's offset
field to point to lldb memory where the name of the target function,
B(), was stored in the binary symbol table. In December Alex put up a PR
to stop abusing the Address object in this way, and store (1) the name
of the target function, and (2) once it has been looked up, the name of
the target function's library.
https://github.com/llvm/llvm-project/pull/172565
Alex originally added a ConstString target_name, FileSpec solib to
Symbol, which increased the size of this object, and lldb stores many of
them, so this was a problem.
[14 lines not shown]
[MachineModuleInfo] add a grouping mechanism to defer deletion MF (#214525)
Add a mechanism so that We can defer deleting the MF after a function is
finalized because there are cases in which we want to late-inline those
MF into some caller.
commit-id:d24afd34
