[AMDGPU][CodeGen] Allow remat with multiple users in multiple regions
This relaxes one of the constraints on rematerialization candidates in
the scheduler's `PreRARematStage`. The current implementation only allows
rematerializing a register if it has users in a single region. This
allows it when a register has multiple users in multiple regions.
In such cases the register is rematerialized as many times as there are
using regions, just before the first user in each using region. The cost
model for assessing rematerialization opportunities now takes into
account that mutliple new instructions may be created for each candidate.
[CodeGen] Correctly classify/mark dead defs when adjusting lane liveness
Despite what the documentation of `adjustLaneLiveness` suggests, the
method never sets dead flags on dead def operands, even when missing
dead flags can later lead to machine verifier errors.
This makes the method identify dead definitions from definitions that
are initially thought to be alive, and makes it add a dead flag on the
last definition of a virtual register, matching the behavior expected
by the machine verifier (ref. "Instruction ending live segment on dead
slot has no dead flag").
`adjustLaneLiveness` and `detectDeadDefs` now also use the same
mechanism to identify dead definitions. It relies on comparing the
defined lanes of a definition with those that stay alive after it.
[CodeGen] Overload `adjustLaneLiveness` to cleanly split use cases (NFC) (#215594)
When providing a non-null `AddFlagsMI` it makes no sense to pass a `Pos`
that is not `AddFlagsMI`'s own position. In such cases the position can
be queried from the MI directly, avoiding possible inconsistencies.
This splits `RegisterOperands::adjustLaneLiveness` into two overloads
whose behavior only differ in whether an MI's operand flags are updated
in the process.
AMDGPU/GlobalISel: Bitcasting G_TRUNC combine
Creating the G_TRUNC that changes type as well. This is really
the LLT::scalar trunc style that we inherited from switching to
extended LLTs, very common on non-true16 targets.
Affects inst-select pattern matching that were blocked by bitcast.
AMDGPU/GlobalISel: Fold (trunc (lshr x, 16)) to unmerge high half
Huge performance impact in some shaders with high register pressure.
What this really does is selects _hi16 register classes, and true16
machinery takes over later. Trunc + shift ends up in _lo16!
This requires allocating an extra register when the _lo16 half of the
original real 32-bit VGPR is still live.
clang/AMDGPU: Stop passing redundant -target-cpu to cc1
Now that the exact target is encoded in the triple's subarch field,
-target-cpu is redundant. This avoids polluting the resultant IR with
unwanted "target-cpu" attributes. The net result is the desired codegen
when compiling libraries for a major subarch and linking it into a
program compiled for a specific arch. e.g., compiling for "gfx9-generic"
would pollute the IR with "target-cpu"="gfx9-generic", so codegen
would ultimately be performed for the generic target even after
linking into the concrete gfx9 cpu. The specialization will now be
achieved by merging the triples without the linker or optimization
passes needing to fixup function attributes.
[libc++] Support file offsets larger than 2GB in basic_filebuf on 32-bit AIX and 32-bit glibc Linux (#215892)
Currently, opening files greater than 2GB on 32-bit AIX doesn't work,
even with `_LARGE_FILES` defined in the user program. A similar issue
exists with 32-bit glibc.
On AIX, the `_LARGE_FILES` macro enables programs to handle large files
(greater than 2GB). When `_LARGE_FILES` is defined, all data types,
structures, and subroutine names are mapped to their 64-bit versions
during preprocessing
([source](https://www.ibm.com/docs/en/aix/7.2.0?topic=volumes-writing-programs-that-access-large-files)).
Since the shared library is pre-compiled, defining the macro in user
programs doesn't affect fstream's open/seek/tell, causing failure with
large files on 32-bit.
Instead of building libc++ with `_LARGE_FILES` defined, which could
cause ODR violations, we can swap `fopen`/`fseeko`/`ftello` calls with
their 64-bit counter-parts.
[3 lines not shown]
[CIR] Implement fixed-point global literals (#217125)
These just have an integral representation. The implementation here is
identical to classic-codegen, so I just added some spot-check tests to
make sure we do the same thing.
[CodeGen] Give NewPM SplitCriticalEdge default values
To match the legacyPM version. This is needed for migrating
RegBankSelect without explicitly passing nullptr to these parameters,
which is not ideal given the LegacyPM version already sets them to
defaults.
Reviewers: arsenm, nikic
Pull Request: https://github.com/llvm/llvm-project/pull/217504
[CIR] Lower Fixed-point conversions to ints/floats/self (#217347)
As the next step in implementing fixed-point NYIs, this patch goes
through and implements the conversion operations. LLVM has a conversion
class for these that generates LLVM, so this duplicates that as
mechanically as possible to convert to CIR. The result is that we end up
with effectively identical IR.
I DID consider 'wiring' this through as its own type, however it is a
rarely used feature and I fear that doing so will result in lost
optimization opportunties vs converting it to 'int' early.
clang: Emit "target-abi" module flag for ARM
Previously only RISCV emitted the "target-abi" module flag.
We probably should just generally emit this for non-empty ABI names
but that's a broader behavior change. I'm also confused because the
clang side defines a non-empty value for many targets with no apparent
use in llvm.
Co-authored-by: Claude (Claude-Opus-4.8) <noreply at anthropic.com>
[AMDGPU][GISel] Rename amdgpu-reg-bank-legalize
Hyphenate to be consistent with other pass names like
amdgpu-reg-bank-select.
Reviewers: arsenm, vangthao95, aeubanks
Pull Request: https://github.com/llvm/llvm-project/pull/217488
ARM: Read the ABI from the "target-abi" module flag
This module flag is already used by RISCV, but ARM ignored it and
still exclusively relied on the -target-abi global option.
Co-authored-by: Claude (Claude-Opus-4.8) <noreply at anthropic.com>
[AMDGPU][GISel] Rename amdgpu-regbankselect amdgpu-reg-bank-select
To be more consistent with reg-bank-select and other passes that are
hyphenated rather than concatenated.
Reviewers: arsenm, vikramRH, aeubanks
Pull Request: https://github.com/llvm/llvm-project/pull/217487
[GISel] Rename regbankselect reg-bank-select
To be more consistent with other pass names, use hypens between the
words. Noticed this when I went to do a NewPM porting.
Reviewers: aeubanks, arsenm, vikramRH
Pull Request: https://github.com/llvm/llvm-project/pull/217486
[GlobalISel] Add G_EXTRACT_SUBVECTOR to computeNumSignBits (#216798)
Port the SDAG EXTRACT_SUBVECTOR computeNumSignBits handling to
GlobalISel. Offsets the demanded elements by the subvector index to
propagate sign bits from the source vector to the extracted subvector.
For scalable vectors, conservatively demands all elements.
Part of #150515.
TargetMachine: Verify option/module-flag consistency once per module
The -target-abi option / "target-abi" module flag conflict was reported from
the per-function getSubtargetImpl in RISCV and LoongArch, so a module with
multiple subtargets would repeatedly diagnose. Introduce a new TargetMachine
method to validate the module for the global options.
I wasn't sure the best place to actually perform this check. This is now
performing it in the earliest pass in codegen, MachineModuleInfo.
MachineModuleInfo is a glorified map from IR to MachineFunction, so I'm not
sure this is the right place. The other alternative I started with was the
AsmPrinter initialization.
Ideally we would eliminate the global options. In particular target-abi is
going to be a bit sticky, because it lives in MCOptions and is thus used by
non-codegen contexts which won't have an IR module to read a flag from.
Co-authored-by: Claude (Claude-Opus-4.8) <noreply at anthropic.com>
[AMDGPU] Fix eliminateFrameIndex clobbering a scavenged register (#217470)
Writing the offset into an undefined scavenged register let the
scavenger alias it with another temp and corrupt the address
