[AMDGPU] Add synthetic apertures and use them for barriers (#209748)
Define what a synthetic aperture is, and adjust the barrier AS
to use this new system. This makes the barrier AS even safer to
use as now we can use all 32 bits of it without ever risking
hitting a valid address of any kind (LDS or outside LDS).
[X86] Enable preferZeroCompareBranch() (#219919)
This makes CodeGenPrepare reorganize code such as:
```
%c = icmp eq i32 %x, 10
br %c, then, else
%a = add i32 %x, -10
```
into
```
%a = add i32 %x, -10
%cmp = icmp eq %a, 0
%br %cmp, then, else
```
so that the zero-compare and branch gets lowered to a JCC based on the
[3 lines not shown]
[RFC][AMDGPU] Add BARRIER address space (#209746)
Add a new BARRIER address space that is used for global variables that
are used to represent the barrier IDs in GFX12.5.
These barrier addresses just have values corresponding 1-1 to barrier
IDs. They are still implemented on top of LDS, but the offsetting
happens during an addrspacecast to generic, not whenever the barrier GV
is used.
The motivation for this is to make the relation between LDS and barrier
GVs explicit in the compiler. It does add a bit more complexity, but
that complexity was already there, just hidden by pretending barrier GVs
were actual LDS.
[ADT] Give DenseMapPair its own members instead of a std::pair base. NFC (#221853)
std::pair declares a copy assignment operator, so it is not trivially
copyable. destroyAll and copyFrom therefore ask about KeyT and ValueT
separately. Hold first and second directly.
Conversion to std::pair is explicit; insert also takes a bucket, and a
converting constructor keeps range insert working across pair types.
Co-authored-by: Kazu Hirata <kazu at google.com>
[flang][OpenACC] Don't emit unstructured-loop TODO for wrappable loops
`genOpenACCLoopFromDoConstruct` raised an NYI for any unstructured DO inside `acc kernels`. That check ran before the wrapping in `genFIR(DoConstruct)`, so a loop whose CFG is self-contained aborted lowering even though the fall-back path would have wrapped it in an `scf.execute_region`.
Skip the TODO when `isWrappableConstruct()` accepts the loop. Such a loop is not attached to a directive; it is only nested inside the kernels region.
Correspondingly, a DO loop sitting directly in an `acc kernels` body is no longer wrappable, since it is the loop the directive parallelizes and hiding its CFG would leave the kernels region with nothing to partition.
[CodeGen][NPM] Port MachinePipeliner to NPM (#221676)
changes of note,
1. registers the pass with AMDGPUCodeGenPassBuilder::addPreRegAlloc()
2. Changes the core MachinePipeliner::run() method to return actual
"Changed" state
[IR] Do not inherit ABI attributes from the callee (#207173)
Usually attributes that are missing on the call-site are inherited form
the callee instead. This is fine for optional optimization attributes,
but problematic for ABI-affecting attributes.
ABI attributes generally must be placed at the call-site as well,
because there is no callee to inherit from for indirect calls. Failing
to specify ABI attributes at the call-site generally indicates a bug.
The motivation for being stricter about this (apart from just being the
right thing to do) is to properly support cases where the attributes at
the call-site and the callee actually differ. Two examples I'm aware of
are:
* During cross-language LTO, it's possible to have one side use a `ptr`
argument and the other a `ptr byval(ptr)` argument. Both represent the
same ABI using different IR encodings, but you can't just inherit the
`byval` attribute from one to the other (because the byval case requires
[22 lines not shown]
[flang][OpenACC] Don't emit unstructured-loop TODO for wrappable loops
`genOpenACCLoopFromDoConstruct` raised an NYI for any unstructured DO inside `acc kernels`. That check ran before the wrapping in `genFIR(DoConstruct)`, so a loop whose CFG is self-contained aborted lowering even though the fall-back path would have wrapped it in an `scf.execute_region`.
Skip the TODO when `isWrappableConstruct()` accepts the loop. Such a loop is not attached to a directive; it is only nested inside the kernels region.
[CIR] Reposition entry-block allocas ahead of control flow (#220496)
HoistAllocas returns early when an alloca was already in the entry
block, but being in the block is not the same as being ahead of the
control flow in it. The pass runs before FlattenCFG, which splits the
body at each structured control-flow op, so an alloca sitting after one
is left in a block that is no longer the entry block.
That matters because SROA only collects allocas from the entry block. An
alloca stranded behind a scope is invisible to it and never promoted, so
anything whose address it holds stays in memory.
[LoopSafetyInfo] Compute block colors lazily (NFC) (#221954)
Computing the block colors is fairly expensive, but only rarely actually
needed. We can delay computation to first use.
Fixes https://github.com/llvm/llvm-project/issues/221162.
[OffloadWrapper] Use getPtrAdd() instead of getGetElementPtr() (#222014)
One case has a redundant zero-offset GEP. The other two just add
a byte offset in a convoluted way.
[libcxx][libcxxabi][cmake] Deduplicate sanitizer flags function (#221757)
While experimenting with flang-rt builds I realised there were two
copies of this function.
We don't need it in flang-rt right now but I figure reducing duplication
is good anyway.
[SandboxVec][VecUtils] Introduce DeadInstrMorgue
Move dead instructions collector and erasor into VecUtils, so that
it is usable by both, BundleVec and LoadStoreVec, vectorizers. NFC
[C++20] [Modules] Keep DeductionGuideDecl in the general lookup table (#222246)
Close https://github.com/llvm/llvm-project/issues/222002
The root cause of the problem is, the deduction guide was not in the
general lookup table. The fix is to move them into the general lookup
table. This should be fine as the DeductionGuide may not pollute the
name lookup results.
[clangd] Place deduced return type hints after noexcept (#219829)
Deduced return type hints for functions and lambdas could appear before
a `noexcept` specifier, even though a trailing return type belongs after
the exception specification.
Use the end of the function type's local source range when it is
available. For lambdas without a function type location, keep the end of
the lambda introducer as a fallback.
Fixes clangd/clangd#2696
Assisted-by: Codex
[VPlan] Consolidate VPIWithType and VPI (NFC) (#203019)
The extra ResultType has been absorbed into VPInstruction, and the
recipe classes can now be consolidated. The patch also fixes missing
flags-printing in the case of WideIVStep, and includes a test to guard
against bad changes, discovered during the review process.
[AMDGPU] Rework pseudo-scalar transcendental inline constant restriction handling (#208459)
The f16 pseudo-scalar transcendental instructions do not support inline
constants on gfx12. The existing handling of this restriction is
incomplete. It involves checking the
hasNoF16PseudoScalarTransInlineConstants function in the right places.
For the SIInstrInfo::isInlineConstant function, for instance, this has
led to a situation where some overloads respect the restriction and
others do not. This approach also relies on completely disabling the
folding of constant operands. The machine code emission does still use
the inline constant encoding if it encounters a suitable operand.
Introduce a new "noinline" operand type and use it for the instruction
variant that is used on gfx12. This does also enable the use of literal
constants.
[ORC] Require symbol name kind in SymbolNameSpec constructor. (#222238)
Require clients to spell out the naming level that SymbolNameSpecs are
expressed in. (An implicit default is a footgun, and adds limited
convenience)
