sched_4bsd: Allocate runqueues only for present CPUs
4BSD has been allocating an array of MAXCPU runqueues, runq_pcpu[],
instead of one runqueue per actually present CPU. On amd64, MAXCPU is
1024 and 'struct runq' is 4128 bytes, causing runq_pcpu[] to take more
than 4 MiB of memory. On the vast majority of current systems, which
have at most 32 cores with SMT, this is a waste of memory.
Besides providing per-CPU runqueues, runq_pcpu[] has also been used to
determine the CPU ID of a given thread's associated runqueue through
pointer arithmetic.
Since per-CPU structures are only allocated for present CPUs, in order
to save space, move the runqueues to per-CPU fields and, for each thread
('struct ts_sched'), replace its runqueue pointer by the CPU ID of the
runqueue it is in (new 'ts_rqcpu' field). Set the thread's CPU ID to
the special NOCPU value when it is running on the global runqueue.
Drop the SKE_RUNQ_PCPU() macro as it is now simply equivalent to
[13 lines not shown]
sched_4bsd: Remove obsolete SMP scaling for ticks per priority level
The INVERSE_ESTCPU_WEIGHT scaling had been introduced by commit
b698380f33ef ("Quick fix for scaling of statclock ticks in the SMP
case. ...") to leave more discrimination room for multiple CPUs possibly
adding their ticks to the same 'struct ksegrp' (but also slightly
changing how CPU hogs are penalized).
Then, commit 8460a577a4b4 ("Make KSE a kernel option, ...") introduced
the current thread-based code, where tick accounting is only done on the
current thread, which renders this trick obsolete on !KSE.
Finally, when KSE was removed, the trick became generally obsolete.
The trick is actually even harmful because it changes the intended
behavior of priorizing more the CPUs that use the less ticks (and so,
impairs boosting "interactive" processes).
Remove it now. Clamping of 'ts_estcpu' and its relation to the
[8 lines not shown]
sched_4bsd: Fix conflating priority of differently-niced CPU-bound threads
We introduced (PRI_MAX_TIMESHARE - PRI_MIN_TIMESHARE) as part of
ESTCPULIM() in commit eebc148f25c3 ("sched_4bsd: ESTCPULIM(): Allow any
value in the timeshare range") in order to use more than a fixed number
(40) of all the available priority levels in the timeshare range (136
before the 256-queue runqueue work, 224 now) to take into account the
number of ticks a thread has run ('ts_estcpu').
In the computation of a new thread's priority (resetpriority()), in
addition to the "ticks running" contribution, the final priority also
includes a "nice" value contribution. The final value is clamped into
the [PRI_MIN_TIMESHARE; PRI_MAX_TRIMESHARE] range.
Problem is that the new "ticks running" contribution now can lead to
a computed priority value that exceeds PRI_MAX_TRIMESHARE, and is thus
finally clamped to PRI_MAX_TIMESHARE, which becomes an alias for all
out-of-bound values. In particular, this can conflate CPU-hungry
threads. With at least two of them competing on the same CPU, with an
[22 lines not shown]
sched_4bsd: Rename the global runqueue
In an upcoming change whose purpose is to stop having 4BSD always
allocate MAXCPU runqueues, wasting space on most machines, 'struct
td_sched' will store the CPU ID to which a thread is bound/pinned
instead of a pointer to the corresponding runqueue. As a consequence,
existing functions manipulating a thread's runqueue will need to point
to the inferred runqueue through a local variable. The name 'runq' is
the ideal one for these local variables, but before this change it
designated the global runqueue, also causing unnecessary ambiguity.
Thus, rename the global runqueue to the more explicit 'runq_global'.
Arguably, this should have been performed as part of commit e17c57b14ba9
("- Implement cpu pinning and binding. (...)").
No functional change (intended).
[olce: Massaged the commit message. Tested with source builds.]
[7 lines not shown]
sched_ule: Fix selecting lowest priority thread early in corner case
When transferring a thread with near 100% CPU statistics (but not 100%;
up to 57.5/59≈97.46%) to a CPU where the enqueue offset is ahead of at
least 2 from the dequeue one, which requires peculiar conditions to
happen (transfer triggered by a bind request or cpuset change, or during
balancing if a thread or more existed from a brief amount of time on the
origin CPU), the transferred thread can get placed after the dequeue
offset, effectively making it appear as a high priority one unduly,
causing latency increase for other threads.
The change here was missed when changing the enqueue and dequeue offsets
update mechanism to recover pre-256-queue-runqueue ULE anti-starvation
and fairness behavior. That change opened up the possibility that these
two offsets are apart by more than one.
Reviewed by: markj
Discussed with: Minsoo Choo <minsoo at minsoo.io>
Fixes: 6792f3411f6d ("sched_ule: Recover previous nice and anti-starvation behaviors")
[5 lines not shown]
pf: allow unspecified addressed for certain MLD messages
As per RFC 3590 MLD Report and Done messages are permitted to use the
unspecified address as a source address (e.g. during duplicate address
detection for the first IPv6 address). Allow this, but only this.
Reported by: Alexander Leidinger <Alexander at Leidinger.net>
Reviewed by: bms
See also: OpenBSD, sashan <sashan at openbsd.org>, 60036e8507
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D59334
pf: Prevent pf dropping TCP state with crafted reset packet.
Revision 1.1212 of pf.c weakened the TCP reset check in stateful
connection tracking to let legitimate resets pass in the backwards
window. Such a reset is accepted only if its acknowledgment number
matches perfectly. But as a workaround for broken stacks, pf
replaces an acknowledgment number of 0 in a reset with the tracked
sequence of the peer. Then the perfect match always succeeds, and
an attacker can spoof resets more easily than intended. Use the
acknowledgment number from the wire, before the workaround has
modified it.
discovered by Minghao Zhang; OK sashan@
Obtained from: OpenBSD, bluhm <bluhm at openbsd.org>, 1e0a1f4b82
Sponsored by: Rubicon Communications, LLC ("Netgate")
pw(8): Brace the name-or-id synopsis so it matches the code
An unbraced pipe list made -n look like it could take a uid (or gid).
Use braces for a single choice of name or id, and keep -u newuid /
-g newgid on the name invocation only. Drop the USER/GROUP OPTIONS
sentences that said -n could be a numeric id.
PR: 269193
MFC after: 3 days
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D59472
EC2: Add desktop flavour
Desktop AMIs have xrdp enabled and boot to a KDE desktop; they are
as compatible as possible with EC2 Windows AMIs, setting a random
password and printing it to the console in encrypted format to be
retrieved using the EC2 GetPasswordData API.
Two rc.d scripts are included in this commit which will not exist
in the long term: ec2_addpass will become part of the ec2-scripts
package, and ec2_desktop_extras will go away once its functionality
is included elsewhere.
MFC After: 1 month
Relnotes: yes
Sponsored by: Amazon
e1000: Serialize 82579 CSR writes with the Management Engine
The 82579 PCIm2PCI arbiter can acknowledge a host MAC CSR write while
the Management Engine is accessing another CSR. The host write can be
lost; subsequent target accesses may no longer be claimed by the MAC and
can hang the system.
For 82579 controllers with valid management firmware, wait for the ME
CSR access indication before every MAC CSR write. Keep the wait bounded
and use DELAY because writes occur in interrupt and datapath contexts.
Verify every transmit and receive tail write. If a tail does not hold
the requested value, disable its datapath direction and request a full
iflib reset.
Keep the ordinary register-write path as a direct MMIO write behind a
predicted per-device gate. Contain the wait and tail recovery in the
82579 slow path rather than adding tail-specific accessors and state to
the rest of the e1000 family.
[12 lines not shown]
e1000: Serialize 82579 CSR writes with the Management Engine
The 82579 PCIm2PCI arbiter can acknowledge a host MAC CSR write while
the Management Engine is accessing another CSR. The host write can be
lost; subsequent target accesses may no longer be claimed by the MAC and
can hang the system.
For 82579 controllers with valid management firmware, wait for the ME
CSR access indication before every MAC CSR write. Keep the wait bounded
and use DELAY because writes occur in interrupt and datapath contexts.
Verify every transmit and receive tail write. If a tail does not hold
the requested value, disable its datapath direction and request a full
iflib reset.
Keep the ordinary register-write path as a direct MMIO write behind a
predicted per-device gate. Contain the wait and tail recovery in the
82579 slow path rather than adding tail-specific accessors and state to
the rest of the e1000 family.
[12 lines not shown]
iavf: Probe the Hyper-V VF device ID
The shared code already recognizes IAVF_DEV_ID_VF_HV and handles it
through the regular iavf register and virtchnl paths, but the PCI probe
table omits it. Add the missing entry so the driver attaches.
PR: 239849
MFC after: 1 week
apple_bce: require usb(4) for vhci functionality
The vhci portion of the driver relies on usb(4) for both constants and
functionality. Add an explicit dependency on usb(4) in `files.amd64` so
it builds properly 100% of the time in a static kernel config.
Reviewed by: seuros
Differential Revision: https://reviews.freebsd.org/D59469
cat: add -A, -E and -T flags
Support cat -A, -E and -T, which are commonly used by Linux shell
scripts. -E prints a "$" at the end of each line, -T renders tabs
as ^I, and -A is equivalent to -vET.
MFC After: 1 week
Discussed with: jrtc27
Reviewed by: jrtc27, ziaee
Differential Revision: https://reviews.freebsd.org/D59250
(cherry picked from commit d2cfdf4c12a76db3612c83575403150811f5aa9a)
nuageinit: root should be allowed to log in when disable_root is false
PermitRootLogin is "no" by default and that stopped root from logging in
even though disable_root was set to false during initialization.
Reviewed by: bapt
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D59101
(cherry picked from commit a209aac86882d3062318dea4971b2f728463c7e2)
nuageinit: fix ssh_pwauth string handling
Treat "no"/"unchanged" correctly instead of any non-nil value as yes.
(cherry picked from commit 8d4d5f2e8e56e7b24b50706acbdc0c8c750c6fed)
nuageinit: support allow_public_ssh_keys
Skip importing datasource public keys when set to false.
(cherry picked from commit 8f378c83c6b8b052433834ca0b53677459dc0e85)
grep(1): optimize -w/--word-regexp word boundary check
The -w option checks word boundaries before and after each potential
match by decoding the adjacent character. This was done via the
heavyweight sscanf(3) with "%lc", which goes through the full scanf
parser and locale-aware mbrtowc(3) machinery even for simple ASCII.
Replace with a three-tier fast path:
1. ASCII bytes (< 0x80): simple isalnum(3) / '_' comparison
2. UTF-8 continuation bytes (0x80-0xBF): interior bytes of a multi-byte
character are always word characters -> no further decoding needed
3. Multi-byte start bytes (>= 0xC0): decode with mbrtowc(3) directly
instead of sscanf(3)/%lc, avoiding scanf parser overhead
Benchmark with ministat(1) (10 runs each):
Worst-case ASCII (100k lines of 100 'a' chars, -w 'a'):
Difference at 95.0% confidence: -15.3% +/- 3.1%
[16 lines not shown]
grep: periodic timer-based fflush instead of unconditional per-line flush
Replace the unconditional fflush(stdout) in grep_printline and
procmatches with a periodic timer that flushes at most once every
100ms. This preserves interactive responsiveness (grep | tee,
grep | tail -f) while avoiding 1M+ write(2) syscalls when
processing large inputs.
The flush interval is tracked via clock_gettime(CLOCK_MONOTONIC)
and a static timespec. --line-buffered continues to flush
immediately via setlinebuf(3), as before.
Benchmark on 1M lines (37MB output to file):
unconditional fflush: 1.90s (sys 1.22s)
periodic 100ms timer: 0.49s (sys 0.007s)
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D57528
(cherry picked from commit ffe47c424e0a45f5d8d20a5944477821bd946eef)
contrib/lutok: remove autotools generated files
These files provide no value in the FreeBSD tree proper and change
frequently, depending on what machine I generate the release tarball on
(and what versions of autotools are on the host). Nuke the autogenerated
files to avoid bloating commit history/the tree.
MFC after: 3 days
Requested by: Benjamin Jacobs <freebsd at dev.thsi.be>
Revert "contrib/lutok: remove autotools and doxygen generated files"
I messed up the commit title. Redo it with a correct summary.
This reverts commit 3dbe1800848ca71f5cc083f1d5c5b4759b601fae.
contrib/lutok: remove autotools and doxygen generated files
These files provide no value in the FreeBSD tree proper and change
frequently, depending on what machine I generate the release tarball on
(and what versions of autotools are on the host). Nuke the autogenerated
files to avoid bloating commit history/the tree.
MFC after: 3 days
Requested by: Benjamin Jacobs <freebsd at dev.thsi.be>
Introduce nfsdtop(8)
Display top-like NFS server I/O using dtrace(1).
Also supports JSON output for time-series.
Relnotes: yes
Reviewed by: ziaee, bcr, adrian
Differential Revision: https://reviews.freebsd.org/D59438
