depend-cleanup.sh: Remove everything before the last build epoch.
We were deleting the .a files for llvm when it was build in bootstrap,
but this caused us to rebuild all the bootstrap things since the
bootstrap compiler changed, making the build time 25 minutes instead of
77 seconds on one of my systems. Items before the last rebuild from
scratch epoch are no longer relevant because you have to rebuild
entirely, so the incremental conditions that lead up to them being
needed are no longer true.
Sponsored by: Netflix
Reviewed by: jrtc27, emaste
Differential Revision: https://reviews.freebsd.org/D53693
rm(1): remove whiteouts when forcibly removing directories
Commit 2ed053cde5 changed UFS' VOP_RMDIR() behavior to no longer
ignore whiteouts when determining whether a directory is empty,
unless explicitly requested by the caller. However, this also
necessitates a change to rm(1) to avoid breaking the expected
behavior when forcibly removing directory hierarchies via `rm -fr`.
I neglected to make this follow-on change despite discussing it
in the review for the breaking commit (D45987).
Finally address the breakage by making `rm -fr` imply FTS_WHITEOUT
when rm(1) reads directory contents via fts_read(3). While here,
also fix a logic error which produces a spurious 'No error' warning
message on stdout for each deleted whiteout.
Reported by: csjp
Reviewed by: csjp, kib, olce
Differential Revision: https://reviews.freebsd.org/D53640
(cherry picked from commit 8b92a6ad597e224b616a8b1d6983f3b55c85308e)
vmimage.subr: Don't log missing files from pkgs
When installing "extra" packages (aka those built from the ports tree),
we record everything being installed in METALOG.pkg; the contents of
that file is appended to METALOG before we generate the filesystem.
There are two cases when files recorded in METALOG.pkg will no longer
exist by the time we create the final disk image:
1. If a pkg bug results in false dependencies being installed which
are later removed by "pkg autoremove", and
2. If the pkg we build and install from /usr/ports is older than the
pkg on pkg.freebsd.org, and pkg gets upgraded automatically as part of
installing extra packages.
The ultimate issue in both cases is that there's no mechanism for
removing entries from METALOG when we run 'pkg delete'.
Address this build breakage by checking, line by line, if filesystem
objects mentioned in METALOG.pkg exist before appending them to METALOG.
[5 lines not shown]
vmimage.subr: Enable FreeBSD-base repo if pkgbase
When creating a VM image using pkgbase, create a configuration file in
/usr/local/etc/pkg/repos/FreeBSD.conf which enables the FreeBSD-base
repository. (This repository is defined in /etc/pkg/FreeBSD.conf as
being disabled by default.)
Reported by: Mark Millard
MFC after: immediately (needed for 15.0-RC1)
(cherry picked from commit e0c41af9256b5c5a6c97c85d468ff734e29f1bd5)
vmimage.subr: Log pkg/local.sqlite if pkgbase
We were doing this in vm_extra_install_packages but VM images without
any extra packages installed would not get this installed. This
results in a pkgbase system which thinks it doesn't have any packages
installed (even though all the files are right there).
Add a "metalog_add_data ./var/db/pkg/local.sqlite" call to the pkgbase
install code path, and make the call from vm_extra_install_packages
conditional on !PKGBASE.
Reported by: Michael Dexter
MFC after: immediately (needed for 15.0-RC1)
(cherry picked from commit 10a4f2d016dccb5cfa03800bebc09a9b421c4df7)
bsdinstall: Enable FreeBSD-base repo when pkgbase
When performing a pkgbase install, create a configuration file in
/usr/local/etc/pkg/repos/FreeBSD.conf which enables the FreeBSD-base
repository. (This repository is defined in /etc/pkg/FreeBSD.conf as
being disabled by default.)
Reported by: Mark Millard
Reviewed by: markj
MFC after: immediately (needed for 15.0-RC1)
Differential Revision: https://reviews.freebsd.org/D53777
(cherry picked from commit a96230e0329b32cbee0999813787a86eab521da4)
GCE: Specify the architecture of images
Without a specified architecture, a user can attempt to create an
arm64 instance with an amd64 image or vice versa. With the change
the API will prevent that mismatch.
(cherry picked from commit 0a8ecca4e3156bcd4ebbfcb24d968e67a3a09434)
GCE: Add TARGET and FS to image family
GCE image family is meant to be unique per set of image characteristics
so that a user can create instances using the image family instead of the
image name to reliably get a similar image with updated software, but no
other changes.
Without this change, the instances create API would select the most recent
non-deprecated image matching the name, regardless of architecture or
filesystem.
(cherry picked from commit fc83e6c5e1f20087314dc52b63e485db87a98b86)
vmimage.subr: autoremove -y
Running 'pkg autoremove' without -y results in VM image builds failing
when (bogusly installed) packages are removed.
Fixes: 6a13aeac3c1f ("vmimage.subr: pkg autoremove after pkg install")
MFC after: immediately (needed for 15.0-RC1)
(cherry picked from commit 509dfd369046ee45b724d119c9eed43228fbb894)
GCE: Don't .error on unsupported targets
We ingest Makefile.gce even when we're not trying to create GCE images
so we don't want to .error here. Instead, set GCE_ARCH to a dummy
value which should make the problem clear to anyone who attempts to
create GCE images on an unsupported architecture.
Reported by: Jenkins
Fixes: 0a8ecca4e315 ("GCE: Specify the architecture of images")
(cherry picked from commit 787d09753f70bb382a7cbfba742a612fa54069e6)
pkg: Add FreeBSD-base to /etc/pkg/FreeBSD.conf
On -CURRENT and -STABLE this gets bits from pkg.freebsd.org; but we get
base system bits from pkgbase.freebsd.org for BETA/RC/RELEASE.
Note that this repository is disabled by default, but can still be used
by explicitly specifying it, e.g. "pkg upgrade -r FreeBSD-base".
With hat: re
MFC after: 8 hours (needed in 15.0-RC1)
(cherry picked from commit 8e0b1a1c03a35a5db13e370cb3e62585d3db68e3)
bsdinstall: Use pkgbase.f.o for BETA/RC/RELEASE
For BETA/RC/RELEASE builds, fetch files from the appropriate repository
on pkgbase.freebsd.org, using the appropriate signing keys. Note that
there is a separate repository for each BETA and RC; this ensures that
someone installing from e.g. a 15.0-RC1 ISO will get 15.0-RC1 bits and
not whatever happens to be the most recent build from releng/15.0.
With hat: re
MFC after: 8 hours (needed in 15.0-RC1)
(cherry picked from commit 51184e38c932b7acedfc26fef3a0aad4c04f8959)
bsdinstall: Don't install FreeBSD-base.conf
When installing e.g. 15.0-RC1, we want to get files from the 15.0-RC1
pkgbase repository; but running 'pkg upgrade' after installation should
get the latest bits build from releng/15.0.
With hat: re
MFC after: 8 hours (needed in 15.0-RC1)
(cherry picked from commit bdfc223c7a64369ed133a179c40067435a9f2cb3)
bsdinstall: Copy in all keys
The keys used for pkgbase signing are going to be placed in
/usr/share/keys/pkgbase-N where N is the FreeBSD major version
number; as such it's not sufficient to copy /usr/share/keys/pkg
into the install chroot, but instead we need /usr/share/keys/*.
With hat: re
MFC after: 8 hours (needed in 15.0-RC1)
Differential Revision: https://reviews.freebsd.org/D53753
(cherry picked from commit 6b0909f94099f92992fbd58eee7ce0f9289cb05e)
Add pkgbase signing keys for FreeBSD 15
These keys were generated by cperciva@ using the AWS Key Management
Service. They will not be used for signing anything other than
FreeBSD 15.x pkgbase repositories.
Keys will be generated for FreeBSD 16 at a later date, and likely in
a different way.
With hat: re
MFC after: 8 hours (needed in 15.0-RC1)
Differential Revision: https://reviews.freebsd.org/D53768
(cherry picked from commit b05f38262b58c1e82ad5f076f2b0b9fe08ddc98b)
vmimage.subr: Don't log missing files from pkgs
When installing "extra" packages (aka those built from the ports tree),
we record everything being installed in METALOG.pkg; the contents of
that file is appended to METALOG before we generate the filesystem.
There are two cases when files recorded in METALOG.pkg will no longer
exist by the time we create the final disk image:
1. If a pkg bug results in false dependencies being installed which
are later removed by "pkg autoremove", and
2. If the pkg we build and install from /usr/ports is older than the
pkg on pkg.freebsd.org, and pkg gets upgraded automatically as part of
installing extra packages.
The ultimate issue in both cases is that there's no mechanism for
removing entries from METALOG when we run 'pkg delete'.
Address this build breakage by checking, line by line, if filesystem
objects mentioned in METALOG.pkg exist before appending them to METALOG.
[3 lines not shown]
vmimage.subr: autoremove -y
Running 'pkg autoremove' without -y results in VM image builds failing
when (bogusly installed) packages are removed.
Fixes: 6a13aeac3c1f ("vmimage.subr: pkg autoremove after pkg install")
MFC after: immediately (needed for 15.0-RC1)
vmimage.subr: Enable FreeBSD-base repo if pkgbase
When creating a VM image using pkgbase, create a configuration file in
/usr/local/etc/pkg/repos/FreeBSD.conf which enables the FreeBSD-base
repository. (This repository is defined in /etc/pkg/FreeBSD.conf as
being disabled by default.)
Reported by: Mark Millard
MFC after: immediately (needed for 15.0-RC1)
bsdinstall: Enable FreeBSD-base repo when pkgbase
When performing a pkgbase install, create a configuration file in
/usr/local/etc/pkg/repos/FreeBSD.conf which enables the FreeBSD-base
repository. (This repository is defined in /etc/pkg/FreeBSD.conf as
being disabled by default.)
Reported by: Mark Millard
Reviewed by: markj
MFC after: immediately (needed for 15.0-RC1)
Differential Revision: https://reviews.freebsd.org/D53777
vmimage.subr: Log pkg/local.sqlite if pkgbase
We were doing this in vm_extra_install_packages but VM images without
any extra packages installed would not get this installed. This
results in a pkgbase system which thinks it doesn't have any packages
installed (even though all the files are right there).
Add a "metalog_add_data ./var/db/pkg/local.sqlite" call to the pkgbase
install code path, and make the call from vm_extra_install_packages
conditional on !PKGBASE.
Reported by: Michael Dexter
MFC after: immediately (needed for 15.0-RC1)
pf: handle divert packets
In a divert setup pf_test_state() may return PF_PASS, but not set the state
pointer. We didn't handle that, and as a result crashed immediately afterwards
trying to dereference that NULL state pointer.
Add a test case to provoke the problem.
PR: 260867
MFC after: 2 weeks
Submitted by: Phil Budne <phil.budne at gmail.com>
Sponsored by: Rubicon Communications, LLC ("Netgate")
arm64/vmm: Fix handling of MDCR_EL2.TDE
TDE (make EL2 the target EL for debug exceptions) is set both when
setting guest breakpoints and when single-stepping the guest. In some
cases we may configure both capabilities, and when subsequently
disabling one of them we need to take care to avoid clearing TDE if the
other is still configured.
MFC after: 3 days
Fixes: 75cb949228bb ("arm64/vmm: Add breakpoint and single-stepping support")
Sponsored by: CHERI Research Centre (EPSRC grant UKRI3001)
(cherry picked from commit b6f25aca11c98acc49f111f2899ecd4e96c7debd)
arm64/vmm: Don't set MDSCR_EL1.KDE when enabling single-stepping
When VHE mode is enabled, this results in a hang on the host. In
particular, when MDSCR_EL2.KDE is set to 1 and the CPU is executing at
EL_D, i.e., EL2, debug exceptions are enabled. In non-VHE mode, we call
into the guest by trapping to EL2, which implicitly masks debug
exceptions by setting PSTATE.D. However, in VHE mode, PSTATE.D remains
clear, so when the guest's MDSCR_EL1 value is loaded, we immediately
begin single-stepping.
In non-VHE mode there is no need to set KDE either, so just stop setting
it.
Reviewed by: andrew
MFC after: 3 days
Sponsored by: CHERI Research Centre (EPSRC grant UKRI3001)
Differential Revision: https://reviews.freebsd.org/D48965
(cherry picked from commit ac0032344ca256f758a5eeb0fd6089dd647b0496)
