[OpenMP][DeviceRTL] Use the actual block size in the SPMD no-loop distribute path (#214073)
In the SPMD no-loop path the iteration index is `BId * NumThreads +
TId`, so `NumThreads` has to
be the size of the block the threads are actually in. `DistributeFor`
uses the caller-supplied
value as-is. If that value is larger than the block size, the index
strides past the end of each
block and the iterations in between are never run.
Fixes #198621.
Reproduced on gfx90a, `num_teams(4)`, 128 iterations, varying the value
passed as `NumThreads`
against the threads the block actually has:
| NumThreads passed | threads in block | iterations not run |
|---|---|---|
| 256 | 32 | 96 of 128 |
[38 lines not shown]
[mlir][tensor] Add a ValueBoundsOpInterface model for tensor.concat (#215346)
Attach ValueBoundsOpInterface model to tensor.concat. The result has the
same size as the inputs in every dimension except the concatenated one,
whose size is the sum of the input sizes.
The verifier makes all inputs equal in a dimension that is not
concatenated, so relate the result to all of them. Relating it to the
first input alone loses the bound when that input is the unbounded one,
which makes the result depend on operand order.
Code generated with Claude Code.
Signed-off-by: Víctor Pérez Carrasco <victor.pc.upm at gmail.com>
[mlir][tensor] Add a ValueBoundsOpInterface model for tensor.splat (#215344)
Attach ValueBoundsOpInterface model to tensor.splat, mapping each result
dimension to its size operand, through `getMixedValues`.
Code generated with Claude Code.
Signed-off-by: Víctor Pérez Carrasco <victor.pc.upm at gmail.com>
[offload] Fix compatibility for level_zero 25.22.33944 (#214215)
This patch changes a few of more recent level zero APIs to use
`apihelper::canCall`. Also I added a new template `callDefaulted` into
`APIHelpers.h` that can be used to call a function with all arguments
initialized using `{}` which can be used to call APIs to check if they
are implemented. It was used to implement
`INVALIDATE_LEVEL_ZERO_API(zeCommandListAppendLaunchKernelWithArguments);`
macro which checks if an API returns ZE_RESULT_ERROR_UNSUPPORTED_FEATURE
and based on that we can fallback those functions.
I also added a skip in our tests if `olLaunchHostFunction` returns not
supported. It made me think it all offload APIs should behave similarly
to level zero which returns ZE_RESULT_ERROR_UNSUPPORTED_FEATURE even if
arguments are invalid, so users can easily check availability of some
feature. In our level zero plugin for host tasks we return `not
supported` at the very end.
Tested on few version of level zero and cuda 12.4.1.
[3 lines not shown]
[CVP] Refine results of getValueAtUse for single users. (#212133)
As discussed in in https://github.com/llvm/llvm-project/pull/207522.
Update getValueAtUse's loop inspecting uses to intersect the at-use
value with the range.
This gives the minimum range for the value at the final use site.
Skipping the intersection would pessimize results in some cases, e.g.
when looking up the result in the use block returns overdefined, due to
the walk hitting the bound of traversed blocks, or when dealing with
cycles. https://github.com/dtcxzyw/llvm-opt-benchmark-nightly/pull/790
shows the regressions when just adjusting the context instruction for
the initial lookup (vs intersecting the result of getConstantRangeAtUse
https://github.com/llvm/llvm-project/pull/211668,
but should be comparable to this).
Triggers widely in real world programs, improving flag inference,
[14 lines not shown]
[CodeGen] Correctly classify/mark dead defs when adjusting lane liveness
Despite what the documentation of `adjustLaneLiveness` suggests, the
method never sets dead flags on dead def operands, even when missing
dead flags can later lead to machine verifier errors.
This makes the method identify dead definitions from definitions that
are initially thought to be alive, and makes it add a dead flag on the
last definition of a virtual register, matching the behavior expected
by the machine verifier (ref. "Instruction ending live segment on dead
slot has no dead flag").
`adjustLaneLiveness` and `detectDeadDefs` now also use the same
mechanism to identify dead definitions. It relies on comparing the
defined lanes of a definition with those that stay alive after it.
[libc++] Enforce the pinned benchmark suite version when submitting to LNT (#215570)
Instead of passing a benchmark suite version when dispatching jobs,
enforce that the version pinned in machines.json is used when submitting
to LNT. This will prevent bad data from making it into the LNT instance
by accident.
We still allow overriding the test suite when doing a workflow dispatch
of libcxx-benchmark-commit.yml since that is useful for e.g. dry-running
different versions of the test suite, but they shouldn't be submitted.
[AArch64] SME definitions for C1-Nano scheduling model (#207174)
Extend the C1-Nano scheduling model to add support for SME instructions.
These instructions differ from legacy scheduling model instruction
definitions in that they are sent to the CME co-processor when in
streaming mode. Modelling these instructions requires several changes to
the scheduling model
- definitions of CME processor resources. Instructions sent to this
co-processor are modelled as having latency derived from the SME
software optimization guide (SWOG),
- predicating the process resource groups for instructions sent to the
CME co-processor when in streaming mode, tests for all SME instructions
in the software optimization guide
Currently the C1-Nano SWOG lacks details of the following, so are not
currently implemented:
- instructions added by SME but don't require the processor to be in
streaming mode
[5 lines not shown]
[lldb] Refactor RegisterTypeBuilder (#213897)
This prepares it for emitting union types. Major changes:
* Entry function is now a dispatcher to builder functions for each type.
* Name mangling is standardised and the base name is generic.
* The register name parameter is no longer needed and so was removed.
* RegisterInfo is passed around until we need specific fields from it.
[SLP]Flatten alternate associative chains into one reassociated node
Each lane peels only chain links with its own opcode, keeping the
per-lane opcode on every combine level; emission linearizes into
main-opcode chain, alt-opcode chain, and a single select shuffle.
Reviewers: hiraditya, RKSimon, bababuck
Pull Request: https://github.com/llvm/llvm-project/pull/215098
[Flang][OpenMP] Support conditional lastprivate on host (#200086)
This patch lowers lastprivate(conditional:) on the host by leveraging
the existing user-defined reduction (UDR) infrastructure. A packed
struct is created where each thread tracks, for every lastprivate
variable, both the candidate value and the canonical iteration index of
its last update. The reduction combiner selects the value from the
sequentially later iteration (for do loops) or from the lexically later
section (for sections).
The implementation locates the enclosing omp.parallel and places the
shared struct before it so that all threads in the team reduce into the
same storage. For orphaned worksharing constructs — where no enclosing
parallel is visible at compile time — a module-scope global of the
struct type is used instead. This is correct for a single level of
parallelism, but concurrent nested teams executing the same orphaned
construct would race on the shared global. This limitation mirrors the
current Clang behavior, which also uses a single global and does not
support nested parallelism for conditional lastprivate.
Assisted-by : Claude Opus 4.6
[lldb][Windows] Expose each thread's TEB address as extended thread info (#213301)
A Windows thread's local storage is accessed through its Thread
Environment Block
([TEB](https://learn.microsoft.com/en-us/windows/win32/api/winternl/ns-winternl-teb)),
but nothing in the target exposes that address, so a language runtime
cannot resolve a variable local to the thread in the debuggee.
This patch publishes the thread's TEB from
`TargetThreadWindows::FetchThreadExtendedInfo` under the "teb_address"
key.
This is required for:
- https://github.com/swiftlang/llvm-project/pull/13597
workflows/release-binaries: Fixup Windows version string for release candidates (#215030)
The Wix installer does not support version strings with characters, so
we cannot use the -rc version strings on Windows. In order to work
around this, we change the version on the Windows builds from X.1.0-rcZ
to X.0.0.Z.
---------
Co-authored-by: Cullen Rhodes <cullen.rhodes at arm.com>
[NFC][SPIR-V] Fuse redundant full-traversal loops in SPIRVPrepareFunctions (#215490)
Split out of #211299 per review. Uses make_early_inc_range to rewrite
calls in place instead of collecting them into a vector first, and
builds the function worklist inside the existing per-function loop
instead of a separate pass over the module.
[SPIR-V] Avoid rescanning use lists to check for Block decoration (#211320)
isResourceType previously re-walked the use list of every struct type to
look for an OpDecorate Block, repeating the same scan whenever multiple
types shared a struct element
Record Block-decorated types at the point the decoration is emitted
instead
[SLP]Fix crash when the strided-load stride reuses an in-tree scalar
The runtime stride expansion may reuse an existing instruction with the
matching SCEV, which can be a part of the vectorized tree and get erased,
leaving the emitted stride computation with a poisoned operand. Detect
such scalars during the external uses analysis and register them as
externally used, so the stride operand is replaced with the extract and
the extract is counted in the tree cost.
Fixes https://github.com/llvm/llvm-project/pull/212579#pullrequestreview-4903760428
Reviewers:
Pull Request: https://github.com/llvm/llvm-project/pull/215552
[SelectionDAG] Apply target MMO flags to masked load/store (#212228)
`visitMaskedLoad` and `visitMaskedStore` ignore the target MMO flags.
For RISC-V, this manifests itself by LLVM generating `ntl.p1`
instruction regardless of the metadata attached to the masked load/store
IR instruction.
This change makes sure that `visitMaskedLoad`/`Store` properly take into
account the attached metadata and generate corresponding instruction.
Co-authored-by: Claude (Opus 4.8) <noreply at anthropic.com>
[NFC][SPIR-V] Fuse redundant store-collecting loop into main scan in SPIRVEmitIntrinsics (#215489)
Split out of #211299 per review: the AggrStores collection loop is
folded into the main instruction loop instead of running as a separate
pass
