BSD Commit Log Search

AMDGPU/GlobalISel: Stop using changeTo in legalizer actions (#209203)

Use changeElementSizeTo or changeElementCountTo to preserve extended LLT.

[AArch64] NFC: Factor out code from FP_TO_INT (SVE). (#207200)

This just moves out some of the SVE lowering code from
LowerVectorFP_TO_INT into a separate function, so that we can reuse that
in LowerVectorFP_TO_INT_SAT.

AMDGPU/GlobalISel: Use extended LLTs in AMDGPUCombinerHelper

Here we also had to change comparisons to extended LLTs to avoid
matching bfloats. In the old scalar version, before the switch to extended
LLTs, s16 was treated as f16 and bf16 was combined as if it were f16.

AMDGPU/GlobalISel: Fix type mismatch in regbank combiner for applyD16Load

GlobalISel: Use extended LLTs in lshr narrow combine

GlobalISel: Use extended LLTs in extract lowering

AMDGPU/GlobalISel: Use integers for clamp i64 to i16 prelegalizer combine

GlobalISel: Use extended LLTs in f64 to f16 fptrunc lowering

AMDGPU/GlobalISel: Use integers for read-any-lane split type

AMDGPU/GlobalISel: Explicitly widen scalar to i32 for load and store

Affects f16 and bf16. Earlier, they were widened to f32 and s32 respectively.
The actual error was the artifact combiner creating a copy between f32/i32
which fails in the machine verifier. Maybe we could create a bitcast there.
However i32 is more efficient for us and matches well with how argument
lowering keeps f16 and bf16 in i32 copies to/from physical registers.
Also starting from f16 store, G_STORE %0(f16), %1(p1) :: (store (f16),
and doing widen scalar to 32 bit type, i32 makes more sense since store
will store 16 least significant bits G_STORE %0(i32), %1(p1) :: (store (f16)
compared to G_STORE %0(f32), %1(p1) :: (store (f16), which looks incorrect if
we assume input was really in f32 format.

AMDGPU/GlobalISel: Fix extending load narrow scalar

isAnyScalar is explicit LLT::scalar check but we want to narrow scalar
integer types as well.

AMDGPU/GlobalISel: Use integer as MMO type for loads and stores lowering

We could get away with just the type from MMO in most cases, but MMO splitting
creates MMO with LLT::scalar and we prefer integer.

AMDGPU/GlobalISel: Fix legalizer lowering for G_EXTRACT/INSERT_VECTOR_ELT

Use LLT::integer in bit twiddling lowering for extract/insert vector element.

AMDGPU/GlobalISel: Stop using changeTo in legalizer actions

Use changeElementSizeTo or changeElementCountTo to preserve extended LLT.

AMDGPU/GlobalISel: Fix G_MERGE_VALUES lowering for extended LLTs (#209202)

Use integer type for bit twiddling instead of scalar.

[X86] Fix swapped VPTERNLOG231_imm8/VPTERNLOG312_imm8 SDNodeXForm bodies (#209782)

Fixes #157929

VPTERNLOG231_imm8 and VPTERNLOG312_imm8 had their bodies swapped,
causing ISel to emit the wrong truth-table immediate whenever operand
reordering was needed (-O1+, not -O0).

Test changes:
- **avx512-vpternlog-commute.ll: regenerated with
update_llc_test_checks.py**
  (accounts for the large diff)
- Added 4 new cases: vpternlog231_rmik, vpternlog312_rmik,
  vpternlog231_rmbik, vpternlog312_rmbik

Tested: llvm-lit on llvm/test/CodeGen/X86 (no regressions), original
repro verified correct at -O0 through -O3, -Os, -Oz.

[AMDGPU] Accept extractelement of a widening cast when folding image ops to a16 (#208207)

canSafelyConvertTo16Bit() recognizes a scalar coordinate that is a
direct sext/zext/fpext from a 16-bit value (sext gated on AllowI16SExt).
Per-dimension coordinates can instead arrive as an extractelement of a
widening vector cast (extractelement((s|z|fp)ext <N x i16/half> Vec),
Idx). When the cast has more than one use, the extractelement(cast) ->
cast(extractelement) canonicalization does not fire, so the cast is left
in place and the coordinate is not recognized.

Strip a leading extractelement before the cast check so the same logic
handles scalar and per-lane coordinates, and mirror this in
convertTo16Bit() by re-extracting from the narrow vector, allowing the
widening cast to be removed once it has no other uses.

[VectorCombine] Fold bitcast(bitreverse.v8i8(bitcast(IntTy))) into bswap+bitreverse (#209037)

Add reverse direction to `foldBitOrderReverseAndSwap`:
`bitcast(bitreverse(<N x i8>)(bitcast(IntTy X)))` -->
`bitreverse(bswap(X))`

This avoids GPR<=>vector register crossings when integer ops are
cheaper. On AArch64 for example:

Before:
```asm
  fmov    d0, x0        ; GPR → vector crossing
  rbit    v0.8b, v0.8b
  fmov    x0, d0        ; vector → GPR crossing
```
After:
```asm
  rev     x8, x0        ; stays in integer registers
  rbit    x0, x8

    [8 lines not shown]

AMDGPU/GlobalISel: Use integers for clamp i64 to i16 prelegalizer combine

GlobalISel: Use extended LLTs in lshr narrow combine

AMDGPU/GlobalISel: Fix type mismatch in regbank combiner for applyD16Load

AMDGPU/GlobalISel: Use extended LLTs in AMDGPUCombinerHelper

Here we also had to change comparisons to extended LLTs to avoid
matching bfloats. In the old scalar version, before the switch to extended
LLTs, s16 was treated as f16 and bf16 was combined as if it were f16.

GlobalISel: Use extended LLTs in extract lowering

GlobalISel: Use extended LLTs in f64 to f16 fptrunc lowering

AMDGPU/GlobalISel: Use integers for read-any-lane split type

AMDGPU/GlobalISel: Fix extending load narrow scalar

isAnyScalar is explicit LLT::scalar check but we want to narrow scalar
integer types as well.

AMDGPU/GlobalISel: Use integer as MMO type for loads and stores lowering

We could get away with just the type from MMO in most cases, but MMO splitting
creates MMO with LLT::scalar and we prefer integer.

AMDGPU/GlobalISel: Explicitly widen scalar to i32 for load and store

Affects f16 and bf16. Earlier, they were widened to f32 and s32 respectively.
The actual error was the artifact combiner creating a copy between f32/i32
which fails in the machine verifier. Maybe we could create a bitcast there.
However i32 is more efficient for us and matches well with how argument
lowering keeps f16 and bf16 in i32 copies to/from physical registers.
Also starting from f16 store, G_STORE %0(f16), %1(p1) :: (store (f16),
and doing widen scalar to 32 bit type, i32 makes more sense since store
will store 16 least significant bits G_STORE %0(i32), %1(p1) :: (store (f16)
compared to G_STORE %0(f32), %1(p1) :: (store (f16), which looks incorrect if
we assume input was really in f32 format.

AMDGPU/GlobalISel: Fix legalizer lowering for G_EXTRACT/INSERT_VECTOR_ELT

Use LLT::integer in bit twiddling lowering for extract/insert vector element.

AMDGPU/GlobalISel: Stop using changeTo in legaizer actions

Use changeElementSizeTo or changeElementCountTo to preserve extended LLT.