[DomTree] Remove support for unnumbered graphs (#212026)
After VPlan, all our in-tree users of the dominator tree have numbered
nodes. Remove the support for unnumbered graphs.
Also slightly simplify the numbering for pre-dominator trees to avoid an
extra +1 on every node number.
[SCEV] Use DenseMap for constant lookup, bypassing FoldingSet (NFC) (#212022)
Looking up/contructing SCEVConstants is a very frequent operation, with
a notable amount of compile-time spent in SCEV.
Add a new DenseMap, mapping IR constants to SCEVConstant. Constants are
already uniqued, so the pointer based lookup is notably faster than the
folding-set lookup.
The cache is used as fast early-exit, and the SCEVConstants are still
added to the folding set for completeness.
Compile-time impact
* stage1-O3: -0.23%
* stage1-ReleaseThinLTO: -0.21%
* stage1-ReleaseLTO-g: -0.19%
* stage1-aarch64-O3: -0.20%
* stage2-O3: -0.10%
[3 lines not shown]
[libc++] Implement `std::function_ref` (#186692)
Papers:
- P0792R14: function_ref: a type-erased callable reference
- P3948R1 constant_wrapper is the only tool needed for passing constant
expressions via function arguments
- P3961R1 Less double indirection in function_ref (RU-220)
LWG issues:
- LWG4256: Incorrect constraints for function_ref constructors from
nontype_t
- LWG4425: CTAD function_ref of data member pointer should produce
noexcept signature
Fixes #189606
Fixes #105376
Fixes #189604
Fixes #171321
Fixes #171387
[4 lines not shown]
[Sema] Skip expansion statements when determing local extern context (#210512) (#211745)
This bug showed up as a failed assertion that was asserting that if
not in a function or method context we should be in the global/file
context.
The root cause is that when determining the linkage context for a
decl we were failing to account for the existence of expansion
contexts. Ignoring the assertion failure, the functional effect of
this is that we would fail to detect incorrect local extern
declarations inside expansion contexts. The fix here is to make
sure that we use `getEnclosingNonExpansionStatementContext()` to
find the true DeclContext for the current scope.
The initial report only identified local extern function declarations
but the same bug occurred with extern var decls, but the path did
not lead to an assertion firing, just incorrect behaviour.
Thanks to Sirraide for explaining why this was going wrong, and
[5 lines not shown]
[DebugInfo][WebAssembly] Anchor stack locals to the frame base (#211826)
WebAssembly's stack pointer is not a register that can be described in
DWARF. When a function has no virtual frame base, because its stack
pointer is never explicitly referenced (e.g. a function whose only local
is dead), the frame base falls back to the __stack_pointer global and
the frame register is the physical SP.
addMachineReg failed for that register, so addMachineRegExpression
dropped the base and the caller emitted the frame offset with nothing
under it: a bare DW_OP_plus_uconst that underflows the DWARF stack when
the location is evaluated.
```
DW_AT_frame_base (DW_OP_WASM_location 0x3 0x0, DW_OP_stack_value)
DW_AT_location (DW_OP_plus_uconst 0xc) ;; before
DW_AT_location (DW_OP_fbreg +12) ;; after
```
[13 lines not shown]
[Pipeliner] Use VRMapPhi to generate phi in epilog (#211723)
When generating a phi in the epilog block corresponding to an existing
phi in the loop, the loop value should come from the new phi generated
in the kernel block (for a non-phi instruction in the loop) rather than
directly from the value map of the previous stage.
The new phis generated by generatePhis for non-phi instructions are
stored in VRMapPhi. Thread VRMapPhi through generateExistingPhis and add
a helper getMapPhiReg that returns the phi-generated register when
available, falling back to VRMap otherwise.
Without this fix, the epilog can pick up the initial prolog value of a
loop-carried register instead of the last kernel-iteration value,
producing incorrect results for pipelined loops with two or more stages
that carry values across iterations. This has been observed on Hexagon
with the modulo scheduled epilog of a right-shift-with-carry loop after
loop unrolling.
[3 lines not shown]
[VPlan][NFC] Number blocks (#212023)
Assign numbers to blocks for more efficient dominator tree construction.
The number is identical to the index in CreatedBlocks of the VPlan.
This removes the last user of unnumbered graphs for dominator trees.
Note that the compile-time change here is practically unmeasurable. The
motivation really is to remove the non-number code path from the
dominator tree.
[MC][NFC] Use EnumStrings for AtSpecifiers (#211827)
Store at specifiers without relocations using EnumStrings, reducing
.data.rel.ro by 4-5kiB. This also saves the more compactly at now 6B per
entry instead of 24B.
[SimplfiyCFG] Deduplicate edges in `hoistSuccIdenticalTerminatorToSwitchOrIf` (#212017)
Fix domtree update by deduplicating edges to be inserted. Tested by the
function `@dedup`.
[PGO][HIP] Support hipModuleLoad in offload PGO (#211875)
Offload PGO finds profile sections by inspecting the image passed to
hipModuleLoadData. hipModuleLoad only provides a file name, so its
device
profile counters were not collected.
After a successful hipModuleLoad, read the code object with the existing
profile file-buffer helper and register it through the same path used by
in-memory module loads.
[DomTreeConstr][NFC] Resize NodeInfos once on construction (#211998)
This avoids frequent size checks when accessing node infos. All numbered
graphs support getMaxNumber().
Also move away from BB->getParent() as preliminary work for numbering
VPlan blocks -- there, getParent() has a different meaning.
[AArch64] Add FMIN/FMAX clustering (#210906)
This patch adds a subtarget feature that controls scheduling FMIN/FMAX
instructions back to back. Enabled on Apple CPU.
