EC2: Use stream-optimized VMDK format
Once enabled on all of the branches, this will reduce bandwidth
consumption from uploading weekly snapshot builds from ~500 GB to
~100 GB, as well as significantly speeding up the process.
MFC after: 2 months
Sponsored by: Amazon
EC2: Pass --vmdk to bsdec2-image-upload if needed
Starting with version 1.5.0, bsdec2-image-upload supports uploading
stream-optimized VMDK files. If we build images in that format, we
need to upload them appropriately.
MFC after: 2 weeks
Sponsored by: Amazon
mkimg: Add support for stream-optimized VMDK
FreeBSD VM/cloud images tend to be highly compressible: The generic VM
images compress roughly 3.5:1, and cloud images typically even more
since they have significant unused space in their virtual disks. This
generally doesn't matter for users who can download compressed release
images and extract them locally, but for EC2 in particular relying on
uncompressed image formats is painful: We now upload over 500 GB/week
to AWS.
This commit adds the "stream-optimized" version of the VMDK format,
which compresses each 64 kB "grain" individually (and omits grains
which are all zeroes). Compared to uncompressed formats, this can
produce much smaller images; experiments indicate that using this
format for EC2 image uploads will reduce the weekly traffic to under
100 GB/week.
Co-authored-by: Claude Opus 5.5
MFC after: 2 weeks
[2 lines not shown]
__FreeBSD_version: Bump for mkimg -f vmdks
Also, adjust bootstrap tools code in Makefile.inc1 to reflect this; in
addition to rebuilding mkimg if it is out of date, we now need to build
libz, which was not previously needed by mkimg.
Reviewed by: kevans
MFC after: 2 weeks
Sponsored by: Amazon
Differential Revision: https://reviews.freebsd.org/D60153
mkimg: Add tests for vmdks format
This is the 'stream-optimized' version of the VMDK format.
MFC after: 2 weeks
Sponsored by: Amazon
Differential Revision: https://reviews.freebsd.org/D60154
mkimg: Make vmdk's desc_fmt slightly more generic
This will allow it to be reused in upcoming work.
Suggested by: Claude Opus 5.5
MFC after: 2 weeks
Sponsored by: Amazon
Differential Revision: https://reviews.freebsd.org/D60150
mkimg: Add image_buffer_region
This is like image_copyout_region, but copies into a buffer rather than
writing out to a file; it will be used by future support for compressed
images (and potentially other circumstances where image data must be
manipulated before being written out to disk).
Reviewed by: jrm
No objection from: Christos Komis (author)
MFC after: 2 weeks
Sponsored by: Google LLC (GSoC 2025)
Differential Revision: https://reviews.freebsd.org/D60149
mkimg: Check for realloc failure
If realloc fails, clean up and return ENOMEM; don't just copy data
into NULL.
Reviewed by: emaste
Reported by: Claude Opus 5.5
MFC after: 2 weeks
Sponsored by: Amazon
Differential Revision: https://reviews.freebsd.org/D60147
mkimg: Avoid leaking a page of mmap
The image_file_map function adjusts the provided file offset to be
page-aligned, with a resulting increase in the size of the mapped
region; the increased size needs to be used when unmapping as well.
Reported by: Claude Opus 5.5
Fixes: baf4abfc39b2 ("Allow building mkimg as cross-tool")
MFC after: 2 weeks
Sponsored by: Amazon
Differential Revision: https://reviews.freebsd.org/D60148
release: Avoid generating .pkgsave files in OCI images
This also installs the pkg key from the current source tree instead of
using the one currently installed on the host.
Reviewed by: dch
MFC after: 1 day
Differential Revision: https://reviews.freebsd.org/D52615
(cherry picked from commit e21e6e96b662dcbb2f0e37ab356c9dded62a586e)
virtio_console: Handle VIRTIO_CONSOLE_RESIZE
Multiport devices report console size per port by control event,
which was ignored and hence their ttys could never have a window
size.
Reviewed by: markj
Approved by: fuz (mentor)
Differential Revision: https://reviews.freebsd.org/D60094
virtio: Validate host-supplied used lengths
virtio_console used the host-supplied used lengths without
validation, so a host could report a length larger than
the buffer, causing the receive and control paths to read
past the end of it. Clamp to the buffer size.
vtnet already rejected used lengths larger than the buffer,
but performed the check after converting the length to an
int, so a length larger than INT_MAX would become negative
and bypass the check. Reject such frames early before the
conversion, and count them in ierrors and rx_frame_too_large.
Reviewed by: markj
Approved by: fuz (mentor)
Differential Revision: https://reviews.freebsd.org/D60092
virtio_console: Negotiate VIRTIO_CONSOLE_F_SIZE
The driver reads cols/rows and sets the tty window size but
never offered the feature, so that code has been dead since
the very beginning; also read size under the configuration
generation count.
Reviewed by: markj
Approved by: fuz (mentor)
Differential Revision: https://reviews.freebsd.org/D60089
Merge commit c4ce37507537 from llvm-project (by ShengYi Hung):
[Clang][Sema] Create LocalScope for Variable Template (#228280)
A variable template should create its own LocalScope, as it should be
opaque to other instantiations. This can occur when there are multiple
instantiations in the same lexical scope. The correct behavior is that
these instantiations should not be chained together.
Assisted-by: Claude # Test ReleaseNote
Fixes: #134148
This fixes an assertion while building the devel/glaze port.
PR: 276265
MFC after: 3 days
vt_fb: Do not perform VT switch in vd_init hook
After 40c20fc29cad it is done by vt_core as soon as vt lock is dropped
after vd_init() has been executed to avoid sleeping with non-sleepable
lock held.
Reviewed by: quentin.thebault_defenso.fr, vexeduxr
Differential Revision: https://reviews.freebsd.org/D59632
bhyveload: do not hang on EOF from console input
Currently, when bhyveload(8) fails to boot the guest,
it drops into the loader prompt waiting for user input.
This behaviour is inconvenient when using bhyveload(8) from
scripts.
Make it exit when it receives EOF from console input.
PR: 286289
Reviewed by: markj
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D59226
(cherry picked from commit 8c20260bc55d7b4be0cbcf0a940505ae15ab41a9)
bhyveload: do not hang on EOF from console input
Currently, when bhyveload(8) fails to boot the guest,
it drops into the loader prompt waiting for user input.
This behaviour is inconvenient when using bhyveload(8) from
scripts.
Make it exit when it receives EOF from console input.
PR: 286289
Reviewed by: markj
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D59226
(cherry picked from commit 8c20260bc55d7b4be0cbcf0a940505ae15ab41a9)
bhyve.8: add details on using TPM with UEFI
Add a note that UEFI VMs using TPM devices should be configured
to use a varfile. Some UEFI boot loaders, such as shim, update
persistent boot variables and then reset the system when a TPM is
present. Without a writable varfile, the VM may be reset repeatedly.
Add a TPM device example to the examples list.
While here, add a missing "\" to the "uefivm" example, and add ".Pp"
before the vCPU pinning examples for consistency with other examples.
PR: 287326
Reviewed by: michaelo, ziaee
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D60181
