[CIR] Make sure ptr-cast-to-vbase is guarded. (#225969)
It isn't clear how we missed this, but classic codegen does checks this,
so we should too.
Claude Helped diagnose/debug, but I did the copy/pasting :D
[clang] Inconsistent attributes for `dllexport __cxxabiv1:: __fundamental_type_info` with `-fvisibility=hidden` (#215859)
Fixes #207963
The issue is `__declspec(dllexport)` and `hidden` visibility cannot be
used together. LLVM rejects `globals` that have both attributes.
For normal globals, `CodeGenModule::setGlobalVisibility` already handles
this by giving `dllexport` priority over the visibility implied by
`-fvisibility=hidden`. RTTI follows a different code path, though.
`ItaniumRTTIBuilder::BuildTypeInfo` applies the visibility and DLL
storage class separately to the generated `_ZTI*` and `_ZTS*` globals,
so the conflict was not resolved.
As a result, marking `__cxxabiv1::__fundamental_type_info` as
`dllexport` while compiling with `-fvisibility=hidden` could generate
RTTI globals with both hidden visibility and `dllexport`, causing LLVM
to fail verification with Broken module found. The same problem can
occur for `dllexport`-ed polymorphic classes when targeting Windows with
[7 lines not shown]
[BOLT] Include indirect calls/branches in basic block count estimation (#225104)
**Before:** When BOLT adjusts basic block counts in LBR mode it
considers outgoing branch counts when `--fix-block-counts` is enabled.
This adjustment was made according to the counts of direct calls and
branches which are recorded in the `Count` annotation made in
`DataReader::convertBranchData`. Indirect calls and branches are also
recorded within the `CallProfile` annotation, however they do not
contribute to the adjustment of basic block counts.
**After:** When making this adjustment to basic block counts, also
consider the counts of indirect calls and branches within the
`CallProfile` annotation. Do this by summing the execution counts of the
targets from this call/branch and considering them in determining the
maximum execution count of the basic block.
Assisted by: Codex. This was to write the tests for `x86` and `RISCV`
targets.
[mlir][Affine] Prevent buffer hoisting out of affine.parallel (#225324)
I noticed that `buffer-loop-hoisting` can move allocations out of
`affine.parallel`, making a buffer that should be local to each
iteration shared between the parallel iterations.
`AffineParallelOp` implements `LoopLikeOpInterface` but was missing the
`HasParallelRegion` trait, so `isSequentialLoop` treated it as
sequential. This adds the missing trait to `affine.parallel`, allowing
the pass to recognize the parallel boundary without changing the
hoisting logic.
I’ve added a regression test based on the reproducer in #225149, which
checks that the allocation stays inside `affine.parallel` and that the
store and load still use that allocation.
Fixes #225149.
[Clang][RISCV] Add packed widening subtraction accumulate intrinsics (#225764)
Add Clang header intrinsics for the RISC-V P-extension packed widening
subtraction accumulate operations:
- __riscv_pwsuba_i16x4
- __riscv_pwsuba_i32x2
- __riscv_pwsubau_u16x4
- __riscv_pwsubau_u32x2
Each computes rd + (a - b). The header wrappers use generic LLVM IR,
so RV32 selects the direct `pwsuba.*` / `pwsubau.*` instructions while
RV64 lowers the generic IR to the decomposition specified by the
P-extension intrinsic spec.
[Clang][RISCV] Add packed subvector insert intrinsics (#225771)
Add __riscv_pset_i8x4_i8x8, __riscv_pset_u8x4_u8x8,
__riscv_pset_i16x2_i16x4 and __riscv_pset_u16x2_u16x4, which insert a
32-bit packed subvector into a 64-bit packed vector. The index selects
the low or high half and must be a constant, enforced with
__enable_if__ as for the element insert intrinsics.
[flang][cuda] Defer data-transfer conversion in OpenACC routines (#225906)
Add an option so cuf-convert can leave cuf.data_transfer in the
host copy of an OpenACC routine until host/device specialization.
After that, transfers in a specialized device body become
assignments. Place allocas inside IsolatedFromAbove offload
regions so later FIR lowering does not store into a temporary
defined outside acc.compute_region.
[RISCV][P-ext] Select PPAIREO.W when packing with the high word of a value (#225763)
The low-half packed subvector insert is a v2i32 whose low word is the
subvector and whose high word is the high word of the wide vector,
which is exactly `ppaireo.w` (the low word comes from `rs1`, the high
word from `rs2`). Select it directly instead of `srli` + `pack`.
Adds `rvp-ppaireo-w.ll`; `rvp-simd-64.ll` is updated because the scalar
low insert in `test_insert_vector_32` now selects `ppaireo.w`.
[SCCP] Call correct simplifyBinOp in FP operation
This simplify FPOps with correct FMF, which covers non-inlined function
that can not be simplified by InstComb pass.
Also, update the TODO as we have cover that case in simplifyBinOp. But
per-element optimization is not enable as we don't have structure-like
LatticeMap for vector.
[AMDGPU] Fold a constant add/sub into the sudot4/sudot8 accumulator
Fold a constant add into the accumulator operand of sudot4 and sudot8 when
clamping is disabled:
```
sudot(a, b, C1, false) + C2 -> sudot(a, b, C1 + C2, false)
```
Subtraction by a constant is canonicalized to addition of its negation.
[AMDGPU][InstCombine] Fold zero dot operands to accumulator
Fold AMDGPU dot intrinsics when either operand is zero.
`dot(a, 0) = 0` and `dot(0, b) = 0`, so replace the intrinsic with its accumulator.
This avoids unrelated clamp and add/sub reassociation cases.
[AMDGPU][InstCombine] Canonicalize dot constant operands (#225083)
Move constant dot product source operands to the right hand side
and add tests for signed and unsigned dot intrinsics.
[flang] Fold fir.if with a constant condition (#225542)
Use `populateRegionBranchOpInterfaceInliningPattern` to fold `fir.if`
when the condition is a constant. It inlines the region the condition
selects, replacing uses of the `fir.if` results with the operands of
that region's `fir.result`, and erases the `fir.result`. A false
condition with no else region leaves nothing behind.
[orc-rt] Make StandaloneMachOUnwindInfoRegistrar internal. (#225982)
StandaloneMachOUnwindInfoRegistrar depends on libunwind's
__unw_add_find_dynamic_unwind_sections, which is only available on
Darwin, and is only consumed within the runtime. Move its header to
orc-rt-internal/bedrock/sys/darwin/ (no longer installed), and its
sources and unit test to the corresponding sys/darwin/ directories.
The SPS CI adder header stays public, but is now installed only on Apple
platforms.
[RISCV][MC] Allow x0 as the destination of qc.inw (#225602)
According to the Xqciio specification, `qc.inw` does not impose any
restriction that excludes `x0` as the destination register. In
`qc.inw.yaml`, `rd` is defined as:
```
- name: rd
location: 11-7
```
This commit removes the incorrect `x0` restriction for `qc.inw`.
Link:
https://github.com/qualcomm/riscv-unified-db/blob/main/arch_overlay/qc_iu/inst/Xqci/qc.inw.yaml
[RISCV] Support getJumpTableIndex (#224197)
This patch implements getJumpTableIndex hook for RISC-V; it trace from
PseudoBRIND back to the %jump-table.N.
The li instruction in jumptable dispatch block exists because the phi
constant is emitted during instruction selection. LLVM chooses the
source block in which to place the phi constant, then postpones moving
it to MachineSink, which relocates the phi constant closer to its use
site. However, to sink it out of the jump table dispatch block,
MachineSink needs to split the critical edge, which requires updating
the jump table entry. It needs the getJumpTableIndex hook to know which
jump table to update.
This avoids unnecessary instructions in the hot path (the jump table
dispatch block).
[AMDGPU] Measure MFMA read hazards at each producer
Introduce more sophisticated traversal to avoid the following traps:
- order-dependent traversal and discarding seen BBs despite shorter path
- mis-matching distance and window of different producers
Record the best distance per BB instead of a visited flag and sweep the
arrivals in nondecreasing distance (bucket queue). This pairs producers
with their actual distance to a consumer in one go.
Fixed scenarios:
- MFMA reading an MFMA result as srcA, srcB or srcC
- VALU, memory or export instruction reading an MFMA result
rewrite-vgpr-mfma-to-agpr.ll gains an s_nop 2: a 4-pass XDL write that
partially overlaps the srcC read two slots later requires five wait
states, and none were emitted because the nearest producer wrote the
register in full.
AI-assisted.
[AMDGPU][NFC] Extract the MFMA read-window calculation
Move the wait states a consumer needs before reading an MFMA result out
of checkMAIHazards90A into getMFMAReadWaitStates, taking the producer as
an argument, so a caller can ask about a specific producer. The partial
srcC overlap half moves into getMFMAOverlappedSrcCWaitStates. The caller
passes the producer the walk recorded, so nothing changes.
AI-assisted.
[AMDGPU] Measure MFMA overwrite hazards at each instruction
Apply previously established processing to:
- VALU overwriting an MFMA result
- VALU overwriting a register an MFMA took as srcC
AI-assisted.
[RISCV][GlobalISel] Fix fptosi/fptoui from half to i64 on RV32 (#222316)
The fcvt.l[u].h patterns are RV64-only, so RV32 had no rule for
`G_FPTOSI/G_FPTOUI` with `{s64, s16}.` The magnitude of a half is at
most `65504`, so with Zfh the i32 result of `fcvt.w[u].h` never
overflows and can simply be extended to i64.
[AMDGPU] Use synthetic fixtures for generic feature validation tests (#225551)
Remove the classification test that includes the full AMDGPU target.
Real targets are validated when generating AMDGPUTargetParserDef.inc
during the build.
Extend the lightweight generic feature tests to cover inherited
classifications of backend-only features, support through member-side
implications, and unsupported generic-side implied features.
Addresses the timeout reported on #223179
