[TableGen] Share identical super-register class masks (#224633)
For a register class and one sub-register index, the register info
emitter generates a bit mask with a bit set for every class whose
registers have that sub-register in this class. A class gets one mask
for each sub-register index for which such classes exist. These masks
were stored in the SubClassMasks array right after the class's sub-class
mask and read by striding from it, so the position of a mask determined
which class and sub-register index it belonged to. Identical masks of
different classes and sub-register indices therefore could not be
shared. AMDGPU has 15,145 such masks, only 1,035 of them distinct, and
one is repeated 370 times.
This patch emits each distinct mask once, into a pool that follows the
sub-class masks in the SubClassMasks array, and gives each class an
array of offsets into that pool, one per mask. The offsets are relative
to the MCRegisterClass object, which is the convention SubClassMaskOff
and SuperRegIndicesOff already use, so the storage remains a constant
initializer that needs no relocations.
[52 lines not shown]
[IndVarSimplify] Ensure the floating-point IV exit comparison runs at each loop iteration (#234133)
When converting a floating-point IV to an integer one, make sure that
the exit fcmp is executed on every iteration, as otherwise the IV may be
incremented past the exit value, leading the integer IV to overflow.
[InstCombine] Check stride preconditions in `foldCmpLoadFromIndexedGlobal` (#234187)
When folding a comparison of a load from a constant global into a
comparison on the index in `foldCmpLoadFromIndexedGlobal`, the
enumeration of offsets `ConstOffset + Idx * Stride` within the global is
assumed to be exhaustive. This holds as long as the stride is positive
for nusw GEPs, and it is a power of two for non-nusw GEPs, as `Idx *
Stride` may wrap. Neither assumption is guaranteed by the decomposed
linear expression, so make sure these are checked explicitly.
Fixes: https://github.com/llvm/llvm-project/issues/233092.
[AllocToken] Preserve operand bundles when replacing allocation calls (#234445)
With the non-fast ABI, `replaceAllocationCall()` replaces an allocation
call with a new call or invoke that takes an additional token ID
argument. It copies the calling convention, attributes, and metadata of
the original call, but not its operand bundles.
In exception handling (EH) based on catchpad`/`cleanuppad, as used by
the Windows MSVC ABI and WebAssembly, each call inside a catch or
cleanup block carries a "funclet" operand bundle that identifies the
enclosing EH pad. Without the bundle, the IR is invalid: the verifier
may reject it, or the backend may silently replace the call with
unreachable.
Preserve all operand bundles of the original call.
[Lanai] Rename -lanai-mem-alu-combiner to avoid a pass name collision (#234513)
opt registers each legacy pass name as an option, and
LanaiMemAluCombinerLegacy, named lanai-mem-alu-combiner, causes a
collision.
[X86] Support -mnop-mcount and -mrecord-mcount on x86 elf targets (#184641)
Support mnop-mcount and mrecord-mcount attributes when used with the
fentry-call=true attribute.
Similar to SystemZ, use the mnop-mcount attribute to place a nop instead
of a call to __fentry__.
Likewise, use the mrecord-mcount attribute to record the location of the
call to fentry (or nop if used with the above) to a section named
__mcount_loc.
This enables the use of -mnop-mcount -mrecord-mcount for x86 in Clang.
[LoongArch] Lower vector sign extension with VSLLWIL and VEXTH
vsllwil.{h.b/w.h/d.w} with a zero shift amount sign-extends the low
half of the source elements, and vexth.{h.b/w.h/d.w} the high half.
Use them for the custom SIGN_EXTEND legalization and for
SIGN_EXTEND_VECTOR_INREG, instead of computing the sign mask with vslti
and interleaving it with vilvl/vilvh, which takes one more instruction
per step. SIGN_EXTEND_VECTOR_INREG now also handles 4x extensions, which
previously fell back to a shuffle followed by a vslli/vsrai pair.
[Clang] Instantiate functions from constant evaluation. (#205557)
This PR is a rework of #173537, which was a rework of #115168; the sole
goal remains to address #73232. I used #173537 as a starting point (for
both code and tests), but attempted to follow the alternative design
requested in the comments therein.
A quick design overview:
- A `SemaProxy` interface is introduced to `AST`, which defines a set of
(possibly mutable) semantic operations that can be triggered by the
evaluation of certain expressions; this class finds its implementation
in `Sema/SemaEval.cpp`. `Sema` creates a unique `SemaProxy` to itself at
construction time, which is made available via
`Sema::getproxyForEval()`.
- Ordinary evaluation (e.g., via `Expr::EvaluateAsConstantExpr`) has no
access to `SemaProxy`, and therefore cannot act on the AST: instead,
variants of these functions (e.g.,
`Expr::EvaluateAsMandatedConstantExpr`) are introduced, which accept a
`SemaProxy &`. This makes clear from the call-side whether an operation
[36 lines not shown]
[IndVarSimplify] Skip exit predication past divergent effects (#183906) (#194202)
This PR makes `IndVarSimplify::predicateLoopExits` skip divergent
effects. As [nikic](https://github.com/nikic) 's suggestion, I reuse the
code about infinite loop detection in `LoopDeletion`
Close #183906
AMDGPU: Include callees in backend_stack_size PAL metadata
The stack size reported in the PAL metadata of a chain function should
include the stack sizes of any callees, since the consumers don't have
access to the call graph (this is especially important for whole wave
functions, because we don't even emit any PAL metadata for them). The
consumers are generally expected to allocate scratch based on the
maximum `.backend_stack_size` emitted for the functions in a pipeline.
They will also have to deal with recursion and indirect calls.
The `.stack_frame_size_in_bytes` metadata remains unchanged, i.e. it
represents just the stack size of the current function (without its
callees).
Assisted-by: AI
AMDGPU: Emit .backend_stack_size for entry points
Currently, the `.backend_stack_size` PAL metadata is only emitted for
functions, but some of the consumers expect it to exist for the entry
points too (without it, if the entry point uses more scratch than any
other functions in the pipeline then we will allocate too little
scratch).
The meaning is the same for entry functions as it is for non-entry
functions: the stack size of the current function plus any of its
non-chain callees.
Unlike `.scratch_memory_size`, this does not get rounded up to 16.
Assisted-by: AI
AMDGPU: Reset SP in entry point before chain call
Chain calls are always tail calls and they are expected to completely
take over the caller's stack frame. Chain functions receive a stack
pointer from their caller, but that is only in order to support
reserving space at the bottom of the stack for CWSR.
Currently, when an entry function has its own stack variables, the SP we
pass on to the chain function points after the stack of the caller. This
is wasteful, so the current patch introduces an epilogue for entry
functions if they contain a chain call. The epilogue will reset SP to
point to the bottom of the stack, either by copying it from FP if we've
been reserving space for CWSR or by setting it directly to zero.
Note that we also modify `requiresStackPointerReference` to return false
for tail calls. This means the prologue will only initialize SP if it's
really needed by the current function, relying on the epilogue to handle
it if we only need to set SP for the sake of our chain callee. This
avoids situations where we might unnecessarily initialize the SP with
[4 lines not shown]
