[Clang][Sema] Fix lambda attribute processing order with clang optimize off (#217194)
This PR fixes a regression from #215173 where an `always_inline` lambda
on a `#pragma clang optimize off` will throw a compiler error. For
example,
```
#pragma clang optimize off
// 1. function:
__attribute__((always_inline)) void front_func() {}
// 2. lambda:
auto trailing_lambda = []() __attribute__((always_inline)) { return 42; };
```
causes only an error for the lambda, not function:
```
[16 lines not shown]
[X86] Remove write-only variable OffsetOperatorLoc (NFC) (#217231)
This patch removes OffsetOperatorLoc and getOffsetLoc in
IntelExprStateMachine. Nobody calls getOffsetLoc, so
OffsetOperatorLoc is write-only.
OffsetOperatorLoc and getOffsetLoc were introduced on Dec 30, 2019 in
commit 4a7aa252a32a94b1bb61b3dc7f027b4a27ae334f, but getOffsetLoc has
never been called.
Assisted-by: Antigravity
[mlir][IR] Add transient scope support for resettable MLIRContext. (#217320)
We have cases where we have many context being used to compile
independent modules. Creating a context per ends up being rather
expensive and the only reason to not reuse contexts more is bloat over
time. This changes adds a "transient" scope which can be used for such
cases. Initially I had tried a "cheap fork" approach for contexts, this
is complimentary to that (that one allows for independent contexts
cheaply), while this one is less invasive.
Introduce an overlay/layered state architecture in StorageUniquer and
MLIRContext to enable efficient scoping and rollback of transient types,
attributes, affine expressions, and distinct attributes added since
entering a transient scope.
When executing repeated compilation passes or running in long-lived
compiler services (e.g. JITs, servers, REPLs), allocating and destroying
an MLIRContext incurs significant overhead (~3.7 ms per context) to
re-register and load dialects. This change allows freezing an
[4 lines not shown]
[LV] Vectorize uncountable early exit store loops with combined conditions
Support the case where both the countable and uncountable exit conditions
have been combined by earlier passes.
[flang][FIRToSCF] Recompute a typed induction variable in closed form (#217051)
Example:
```fortran
do i = 1, n
a(i) = a(i) + 1
end do
```
In this code, `fir.do_loop` carries `i` in its own integer type, but
`fir-to-scf` normalizes the loop to a canonical `index` loop and passes `i`
through `iter_args`.
Fix: recompute `i` in closed form from the canonical induction variable, for
every step sign. No `iter_arg` and no loop result are added; the closed form
does not inherit `nsw`. Loops with a final value keep carrying it, since
their result is observable after the loop.
[GISel] Move IRTranslatorImpl into cpp file (#217349)
This is not needed in the header and including it in the header
regresses compile times for PassBuilder.cpp.
[mlir][vector] Verify non-unit strides on `masked/expand/compress` ops (#216485)
Closes the stride-verification gap left open by #204611 and #205869
(**Stacked on #211004**).
`vector.maskedload`/`maskedstore`/`expandload`/`compressstore` lower to
LLVM masked intrinsics that read/write N *consecutive* elements from a
single pointer (see [LangRef](https://llvm.org/docs/LangRef.htm)), but
none of them verified the memref's minor-dim stride, so `strided<[2]>`
verified successfully and silently miscompiled. This PR rejects
statically-known non-unit and dynamic strides.
---
The original PR (#210952, **identical to this one**) has been reverted
(#215224) due to `Linalg/CPU/ArmSME/matmul.mlir` and
`Linalg/CPU/ArmSME/multi-tile-matmul-mixed-types.mlir` e2e test failures
(see the
[buildbot](https://lab.llvm.org/buildbot/#/builders/121/builds/2588)).
[33 lines not shown]
[libc] Gate long double assertions that fail in long double = float64 targets (#217346)
Some assertions that involve long doubles fail if the type is
implemented with float64, such as in Arm 32 bits targets.
This patch gates these failing assertions under the proper macro.
Additionally, it fixes some uses of this same macro that were using an
incorrect spelling.
[clang] Fix crash on no-prototype indirect calls with -fexperimental-call-graph-section (#210846)
Prior to this change `-fexperimental-call-graph-section` will hit
unreachable code in CGCall.cpp for indirect calls through C style
function pointers declared with no argument list (no prototype). In this
change, we reconstruct the function prototype at the call site and generate
the type ID based on the constructed prototype.
Assisted-by: Gemini 3.1 Pro
[libsycl] Initial prefetch implementation for queue (#212104)
Adds the `prefetch` API for `queue` class as per SYCL 2020
specifications.
Adds `unit tests` for prefetch use cases with the appropriate mock
additions
AI has been used to help with slight logic adjustments in
`QueueImpl::prefetch` and for the unit tests
Align TOSA->EmitC integration tests with MLGO config (#216787)
Adapt existing TOSA->EmitC integration tests to use exactly the same
pipeline as the MLGO CMake config proposes in
https://github.com/llvm/llvm-project/pull/212650.
The first `RUN` directive tests the TOSA->EmitC lowering.
The second tests class-based C++ code generation for MLGO. For this, the
pipeline needs extra passes helping transform the model function into a
class:
- `promote-buffers-to-stack`: heap-based -> stack-based allocations
- `wrap-emitc-func-in-class`: standalone `emitc.func` -> `emitc.class`
- `mlgo-add-reflection-map`: adds a reflection map + helper method to
`emitc.class` for runtime field lookup.
Therefore, we're now emitting a `.cpp` file instead of `.c`.
The missing `buffer-deallocation-pipeline` lead to a skipped
canonicalization that created `arith` ops, which the current pipeline
couldn't actually lower, due to recent changes in the `arith-expand`
[2 lines not shown]
[clang][Sema] Fixed a crash when an `address_space` attribute with a dependent argument was written after the declarator-id (#216348)
Fixes #196982
Fixes #111463
When filling in source locations for a dependent `address_space` type,
we only looked at the declarator chunk being visited. But an attribute
written after the declarator-id appertains to the declared entity, so it
never lands on a chunk — it gets applied to the outermost type instead.
The search came up empty and we hit the `llvm_unreachable`. Nothing to
do with the malformed code in the bug report, by the way: plain template
`<int AS> void f() { void *p [[clang::address_space(AS)]]; }` crashes
too.
The lookup now searches every attribute list of the declarator (chunk,
declarator, decl-spec, declaration), passed as an `ArrayRef` by the
caller, so the attribute is always findable and the original
`llvm_unreachable` stays. Attributes that are invalid or malformed are
skipped, since they never produced a type. This matters when a malformed
[3 lines not shown]
[AMDGPU] Configure the software pipeliner policy
Set the pipeliner policy in overridePipelinerPolicy(): raise the maximum MII
and opt into the generic register-pressure detector.
MFMA latencies push the MII of otherwise pipelineable loops past the generic
limit of 27, so AMDGPU raises it to 256, which covers the II distributions
observed across Composable Kernels and Triton workloads on gfx950. Removing
the limit entirely pipelines no additional loops on those workloads and costs
around 23% more compile time, so the bound stays finite.
Enable the generic register-pressure detector so schedules exceeding the
target-provided pressure-set limits are rejected and retried at a higher II.
[MachinePipeliner] Let targets configure the maximum MII
The pipeliner rejects a loop whose minimum initiation interval exceeds
-pipeliner-max-mii. Its default of 27 suits targets with short instruction
latencies, but is too small for others: on AMDGPU a couple of MFMA
instructions already push the MII past it, so the loop never pipelines.
Move the limit into a MachinePipelinerPolicy that targets customize by
implementing TargetSubtargetInfo::overridePipelinerPolicy(). The generic
default is unchanged and an explicit -pipeliner-max-mii still wins over the
target's choice, so no target changes behavior here. Exercised by the AMDGPU
adoption in a following commit.
[AMDGPU] Add MachinePipeliner support for AMDGPU
Implement the target hooks to enable MachinePipeliner for AMDGPU. The
pass is off by default and can be enabled with -amdgpu-enable-pipeliner
at -O2 and above.
Only uniform, single-basic-block counted loops with a scalar (SCC)
back-edge are pipelined; loops with a divergent (VCC/EXEC) back-edge, or
containing calls or inline asm, are rejected. Code is generated by the
default modulo schedule expander (DFA, window scheduler, and MVE
expansion are disabled for AMDGPU).
Validated on gfx942 and gfx950 with Composable Kernel and Triton workloads.
[MachinePipeliner] Let targets enable the register-pressure detector
The generic MachinePipeliner register-pressure detector, added in #74807,
was only reachable through the global -pipeliner-register-pressure option.
Add MachinePipelinerPolicy::ShouldLimitRegPressure so targets can enable the
detector in TargetSubtargetInfo::overridePipelinerPolicy. An explicit command-
line option continues to override the target policy.
The policy defaults to false, preserving existing behavior for other targets.
It is exercised by the AMDGPU adoption in the following commit.
[mlir][acc] Introduce acc atomic to llvm patterns (#217142)
Add support for lowering the OpenACC atomic operations - atomic.read,
atomic.write, atomic.update, and atomic.capture - to LLVM dialect.
Co-authored-by: Matsu <kmatsumura at nvidia.com>
[CodeGen] Avoid register pressure limit underflow (#216372)
RegisterClassInfo::computePSetLimit() subtracts an estimated
reserved-register weight from the target-provided pressure-set limit. If
the weight exceeds the limit, the unsigned subtraction wraps to a very
large value, effectively disabling register-pressure checks. An equal
weight produces zero, which is the uncomputed-limit sentinel.
MachineSink and MachineLICM exposed this underflow after switching to
RegisterClassInfo limits.
Preserve the target-provided nonzero limit when the approximate
adjustment would exhaust it. Non-underflowing adjustments remain
unchanged. Add an AMDGPU regression test and update the affected checks.
[libc] Add support for wchar_t in StringConverter. (#211867)
This takes care of a TODO in LibcTest.cpp from #203355, allowing
`wstring_view` to be printed as UTF-8 in tests. It also enables updates
to __support/printf_core/ for implementing `swprintf`.
The change to `StringConverter` is enabled by extending the
`CharacterConverter` API when `wchar_t` is detected as being UTF-32.
The `StringConverter` unit test is updated so it no longer compiles to a
no-op on Windows.
Note: this change does not respect `-fwide-exec-charset`, which Clang still
does not support but GCC does. This is the existing libc behavior.
