[AArch64] Fold zero-interleave shuffle into vector shift left (#210793)
A shuffle mask that interleaves zeros between every other byte element
is equivalent to a vector shift left on a wider element type. This
avoids generating a `tbl` instruction with a constant mask loaded from
memory, replacing it with a single `shl` instruction.
Before:
```asm
adrp x8, .LCPI0_0
ldr q1, [x8, :lo12:.LCPI0_0]
tbl v0.16b, { v0.16b }, v1.16b
```
After:
```asm
shl v0.8h, v0.8h, #8
```
[4 lines not shown]
[lldb][debugserver] Expedite the stopped frame's stack memory in jThreadsInfo (#212706)
Add `ReadFrameZeroStackMemory`, which expedites the innermost frame's stack
memory so a variables view on a stop is served from lldb's memory cache. When
frame 0's `$fp` looks usable, two windows are expedited:
* `[$fp + 2*ptr_size, $fp + 2*ptr_size + k_expedite_stack_arg_size)` for stack-passed
parameters, starting above the saved `{fp, lr}` pair the backchain already covers.
* `[$fp - below, $fp)`, `below = min($fp - $sp, k_expedite_stack_window - k_expedite_stack_arg_size)`,
for locals and spilled register arguments. A small frame gets all of `[$sp, $fp)`; a large one keeps the part nearest `$fp`, so the cost stays bounded.
If `$fp` fails validation (frameless leaf, or `$fp` used as a scratch GPR),
a single `[$sp, $sp + k_expedite_stack_window)` window is expedited instead.
Each window is a separate chunk, because lldb's L1 cache only serves reads
fully contained in one expedited chunk. Only the thread that stopped gets
these windows, so the stop reply does not grow with thread count.
`GetJSONThreadsInfo` now builds the `"memory"` array from both sources and
emits it whenever either produced an entry. Add `JSONGenerator::Array::empty`
for that check.
sys/socket.h: Fix AF_MAX
AF_MAX was always intended to be one more than the greatest allocated
value. Jeff broke this in 2013. Unfortunately, a bunch of people then
decided to adapt to the mistake instead of correcting it.
Fixes: 863c7e45628d (" - Reserve a special AF for SDP. The one we were incorrectly using before was taken by another AF.")
MFC after: 3 days
Sponsored by: Klara, Inc.
Sponsored by: NetApp, Inc.
Reviewed by: kevans, glebius
Differential Revision: https://reviews.freebsd.org/D58597
e1000: report UDP RSS hash type on igb/em
{em,igb}_determine_rsstype() mapped only the TCP and bare-IP RSS descriptor
types; the UDP types returned M_HASHTYPE_NONE.
The hardware does hash UDP, but with a NONE hashtype iflib skips its
flowid-based TX queue spread, so all forwarded UDP egressed on a single queue
and serialized transmit on one core.
Add the three UDP cases (IPV4_UDP, IPV6_UDP, IPV6_UDP_EX) so egress spreads
across all TX queues.
Reviewed by: kbowling, gallatin
Approved by: kbowling
MFC after: 1 week
MFC to: stable/14, stable/15
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D58513
(cherry picked from commit 285c749f575ed7f9e60555037f23ac673084c62a)
[lldb] Make RegisterFlagsDetector into RegisterTypesDetector
In future it may be generating things other than flags. Functionality
is the same, but the interface changes to use RegisterType.
hwpmc: fix event allocation on pre-Zen AMD CPUs
amd_allocate_pmc() chose the pmu-events code path whenever pmc_cpuid was
non-empty, and rejected any allocation lacking PMC_F_EV_PMU.
But pmc_cpuid is set for every AMD CPU, while the pmu-events tables only cover
Zen and later.
On older families (K8, Bobcat, Jaguar/16h, Bulldozer) libpmc finds no
pmu-events entry and falls back to the legacy path, which never sets
PMC_F_EV_PMU.
Reviewed by: mhorne
Approved by: mhorne
MFC after: 1 week
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D58468
(cherry picked from commit 6c4d9b9af1a3b247bf82a4228c835d106f535613)
e1000: report UDP RSS hash type on igb/em
{em,igb}_determine_rsstype() mapped only the TCP and bare-IP RSS descriptor
types; the UDP types returned M_HASHTYPE_NONE.
The hardware does hash UDP, but with a NONE hashtype iflib skips its
flowid-based TX queue spread, so all forwarded UDP egressed on a single queue
and serialized transmit on one core.
Add the three UDP cases (IPV4_UDP, IPV6_UDP, IPV6_UDP_EX) so egress spreads
across all TX queues.
Reviewed by: kbowling, gallatin
Approved by: kbowling
MFC after: 1 week
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D58513
(cherry picked from commit 285c749f575ed7f9e60555037f23ac673084c62a)
[X86] vector-reduce-*-bool.ll - update IR to match middleend (#213876)
InstCombine converts vXi1 logic reductions to bitcasted scalar integer
ops - we should be testing that, not llvm.vector.reduce.*.vXi1 calls
We were also failing to tag the i1 return values as zeroext
Shows a couple of hidden issues - poor handling of comparison results
from sub-128-bit vectors and handling of v32i1/v64i1 MOVMSK patterns on
pre-AVX2 targets
hwpmc: fix event allocation on pre-Zen AMD CPUs
amd_allocate_pmc() chose the pmu-events code path whenever pmc_cpuid was
non-empty, and rejected any allocation lacking PMC_F_EV_PMU.
But pmc_cpuid is set for every AMD CPU, while the pmu-events tables only cover
Zen and later.
On older families (K8, Bobcat, Jaguar/16h, Bulldozer) libpmc finds no
pmu-events entry and falls back to the legacy path, which never sets
PMC_F_EV_PMU.
Reviewed by: mhorne
Approved by: mhorne
MFC after: 1 week
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D58468
(cherry picked from commit 6c4d9b9af1a3b247bf82a4228c835d106f535613)
[VPlan] Process simplifyRecipes in a worklist
For now just push the new recipe onto the worklist, not its users.
This uses a post order traversal so we maintain the same simplification order as before.
I've gone through and checked every simplification we do is a canonicalisation that converges, and I checked on llvm-test-suite + SPEC CPU 2017 in various configurations that we don't hit any cycles.
[lldb] Store all XML register types in a single string map
We are assuming that their ID's are unique, so there's no need to keep
separate maps. We can do basic type checking by checking the kind of
the type pointed to.
A few more methods were added to the base RegisterType. GetSize()
returns 0 for enums because enums don't have a size until they are
used by a register. This is not ideal but it works for now.
[LV] Add vplan folds for urem(X, PowerOf2) -> and(X, PowerOf2 - 1) (#212198)
In this PR I've added support for the vplan fold:
urem(X, Y) -> and(X, Y - 1)
when Y is a power of 2. This should reduce the cost of the urem and
ensure the vplan is accurately costed. Such a change would normally
affect over 300 test files due to this being a common pattern in the
vector preheader. For now, I've limited the scope to only simplifying
occurences that are not in the vector preheader. In a follow-on PR I
will extend this to add support for
sub(X, urem(X, Y)) -> and(X, -Y)
as well permitting folds in the preheader.
[AArch64][SME]Refine memory effects for SME load/store intrinsics. (#205525)
Split SME load/store intrinsic definitions so loads and stores model
ArgMem, ZA, and ZT0 effects separately. Also mark ZA enable/disable as
side-effecting intrinsics with no memory access.
[lldb] Convert uses of RegisterTypeFlags into RegisterType
So we are using the generic interface that will work with
all future RegisterType derived classes.
Right now we'll only be asked to print RegisterTypeFlags, so
there's a few dyn_cast to that. Later we will switch on the
kind, and support rendering more types.
GlobalISel: Add type size guards in tryCombineMergeLike (#213702)
Bug in LegalizationArtifactCombiner when:
DstSize < UnmergeSrcSize case can create unmerge with one element.
DstSize > UnmergeSrcSize case can end up attempting to create merge
with one source element and hits assert(TmpVec.size() > 1).
GlobalISel: Add type size guards in tryCombineMergeLike
Bug in LegalizationArtifactCombiner when:
DstSize < UnmergeSrcSize case can create unmerge with one element.
DstSize > UnmergeSrcSize case can end up attempting to create merge
with one source element and hits assert(TmpVec.size() > 1).
AMDGPU/GlobalISel: Precommit tests for upcoming bug fix (#213701)
Here we have:
artifact combiner creating one element unmerge and
unmerge lowering of FP source using FP type for bit twiddling.
