Consolidate the managed-dataset registries and guard every mutator
## Problem
"Is this a dataset middleware manages?" was answered by five separate registries with four membership sets and no shared matching rule: `INTERNAL_PATHS` in `plugins/zfs/utils.py`, `INTERNAL_DATASETS` in `plugins/pool_/dataset_query_utils.py`, `internal_datasets_filters` in `plugins/pool_/dataset.py`, inline literals in `alert/source/datasets.py`, and `INVALID_DATASETS` in `plugins/zettarepl.py`. None had unit coverage, and they had drifted: `<pool>/ix-applications` was creatable and then permanently invisible because one carried a trailing slash the others lacked, and the replication registry did not know about the apps datasets at all. A sixth spelling in `docker/fs_manage.py` tested `startswith("boot-pool/")`, which misses `freenas-boot` entirely and also misses the bare string `boot-pool` -- so whenever the boot pool's own root dataset was what sat at the apps mountpoint, it was reported as the mounted apps dataset.
Two of the five matched by substring, which is the cheapest way to spell "is under" in a filter DSL and not a matching policy anyone chose. It confiscated names the user is entitled to. `<pool>/ix-apps-data` is a user dataset that merely starts with a managed name, and `<pool>/.systembackup` is what a competent admin would call a backup of the system dataset; both were hidden from `pool.dataset.query` forever, which left them unmanageable, since the lookups behind `get_instance`, `update` and `delete` all run through that listing. The same reach refused their creation and dropped their ZFS events. Nothing middleware creates lives at those names: `.system`, `ix-apps`, `ix-applications` and `.truenas_containers` are literals created at a fixed depth of one below the pool root, so everything a substring caught beyond that was, by construction, someone else's dataset.
"May this caller change it?" was answered by a `bypass` field on the snapshot request models and by `exclude_internal_paths` on `ZFSResourceQuery`. Both are `Private`, which the API layer already refuses from the wire, so neither was reachable by a caller. `exclude_internal_datasets` was the exception, and a real one: it was read out of `pool.dataset.query`'s free-form `extra` dict, where the model's `extra="forbid"` never reaches, because that governs unknown model fields rather than keys inside a dict value. Anyone holding `DATASET_READ` could set it and enumerate the boot pool, the system dataset and the apps datasets.
Separately, eleven public entry points had no protection at all and none of them funnelled through a guarded implementation: `pool.dataset.promote`, `rename`, `set_quota`, `get_quota`, `lock`, `change_key` and `inherit_parent_encryption_properties`, the three `zfs.tier` mutators, and `pool.snapshot.rename`. Ten of those mutate; `get_quota` only reads, but it reported the quota accounting of datasets that are not part of the user-facing surface. `promote` was the worst of them, since managed children are frequently clones and promoting one reparents its origin snapshot. `pool.snapshot.rename` turned out to be unreachable in any case: it passed a snapshot id to `zfs.resource.rename`, which rejects any name containing `@`, so the endpoint could not succeed for any valid input and had no test coverage.
The same whole-component test was also being asked about snapshots. `zfs.resource.snapshot.query` and `zfs.resource.snapshot.count` fed it names like `tank/.system at snap`, where the suffix lands on the component being compared, so the answer came back False for exactly the shapes the filter existed for. The per-snapshot filter and the opt-out that turns it off were both dead for the `.system` and apps snapshot names, and the count's direct-snapshot branch consulted neither. Nothing leaked, because a working dataset-level check filters the parent before any snapshot below it is reached. That is the problem rather than the reassurance: it leaves a dead guard sitting behind a live one, with nothing to fail if the live one is ever moved or dropped as redundant.
## Solution
- **One module, one predicate per question.** `utils/zfs/managed_datasets.py` replaces all five registries, with one function per caller decision -- `hidden_from_zfs_listing`, `hidden_from_snapshot_listing`, `hidden_from_dataset_listing`, `blocked_from_mutation`, `excluded_from_zfs_events`, `excluded_from_replication` -- each spelling out its own membership where you can read it against the rule at once. One rule covers everything: compare the component directly below the pool root against a name, exactly, which matches that dataset and everything under it the way ZFS's own name algebra does. The boot pools are a separate disjunct, because they are matched at component zero instead. `hidden_from_snapshot_listing` is the one predicate with no rule of its own -- it drops a snapshot suffix and asks `hidden_from_zfs_listing`, the way `deny_protected_snapshot` does for `deny_protected_path`. The refusal helpers `deny_protected_path` and `deny_protected_snapshot` sit in the same file directly beneath the predicate they gate, so answering "is this ours?" and "may this caller change it?" is one place to look rather than two files to choose between, and they replace the hand-rolled `if not bypass and has_internal_path(...)` that every call site previously spelled out for itself, snapshot-suffix stripping included. Callers that must accumulate into `ValidationErrors` mid-pass, or that want to name the containing dataset rather than the path they were handed, ask the predicate directly; the helpers are a convenience for the one common message shape, not a boundary.
- **The rule is shared; the membership carries the divergence.** Every predicate matches the same way, so where two of them disagree it is because they manage a different set of names, and that is product policy you can read in one line. The apps datasets are deliberately left out of the replication membership, which means replication offers them, because replication is the supported way to back them up. `.truenas_containers` takes part in the product listing only, reproducing today's behaviour while containers are addressed separately; that leaves it destroyable through `zfs.resource.destroy` while `pool.dataset.delete` answers ENOENT, which is written down rather than quietly fixed. The event view stays narrower than the product listing for the same reason -- a container dataset is hidden from the listing but its destruction still has to be published, or the cleanup never runs.
[12 lines not shown]
[flang][OpenMP] Lower allocator-backed storage for allocate clauses (#211621)
Part of #211620.
This is the first of two stacked changes implementing OpenMP `allocate`
clause lowering for fixed-size intrinsic scalar `private` and
`firstprivate` items on host `omp.parallel`.
It carries each allocate item’s private-storage mapping through the
OpenMP dialect, allocates with the requested allocator (using the
runtime default for an omitted or null handle), and releases the storage
during region finalization.
The `align` modifier is handled by the stacked follow-up.
Assisted-by: Copilot
[lldb][Mach-O] Fix load-command loops spinning on cmdsize = 0 (#205134)
Every function in `ObjectFileMachO` and `ObjectContainerMachOFileset`
that iterates over load commands advances the file offset by
`lc.cmdsize` after reading each command. A malformed command with
cmdsize smaller than `sizeof(load_command)` (in particular cmdsize = 0)
does not make forward progress, so the loop spins for ncmds iterations.
With `ncmds` close to `INT_MAX` the function never returns in practice.
Factor the read-and-validate step into a static template helper
`ReadMachOCommand<T>` in each plugin's translation unit. It reads the
8-byte cmd/cmdsize header and returns false on EOF or on a cmdsize that
is too small to make forward progress. All load-command loops now use
this helper, replacing the previously duplicated GetU32 + cmdsize
check. `T` may be `llvm::MachO::load_command` or any of its richer
variants (uuid_command, dylib_command, thread_command, ident_command,
encryption_info_command, ...). The helper only touches the leading
cmd/cmdsize fields, leaving the rest of `T` for the caller to fill in.
[15 lines not shown]
vendor/ncurses: Add a trailing nul for magic-string in putwin.
This was flagged by GCC 12.5's -Wstringop-overread.
Patch is from upstream, therefore fixing on the vendor branch.
filesystems/tclvfs: add PORTEPOCH
The last version bump went back in time, 1.5.0 < 20080503_1.
While at it, use tabs instead of spaces after variable names.
* Avoid calling assume_default_colors because this in turn, calls
_change_pair which dirties lines resulting in a blank screen iff
stdsrc is not being used.
* Restore line dirtying code to __change_pair since makech() no longer
indirectly calls it.
[OpenMP][NFC] Align pointee-entry terminology with HasAttachPtr docs
Drop the "pointee (attach-ptr) entries" phrasing in favor of the wording
already used by the landed HasAttachPtr documentation: name the flag
("entries with HasAttachPtr") where the comment sits next to a
HasAttachPtr/hasAttachPtr() test, and use plain "pointee entries" in
prose.
"attach-ptr entries" reads as the entries for the attach pointer itself,
which is exactly the case HasAttachPtr excludes -- it is false on the
ATTACH entry and on an entry mapping the pointer as an object in its own
right, and true only for the pointee storage reached through that ptr.
Comments only; no functional change.
[flang] Add HLFIR-to-FIR pass pipeline extension points
The FIR optimizer extension points (FIROptEarly, FIRInliner, FIROptLast) all
run after HLFIR has been lowered to FIR, so the HLFIR intrinsic operations
(hlfir.sum, hlfir.matmul, ...) are gone by the time they run. Transformations
that need to see those operations have nowhere to attach.
Add two extension points to createHLFIRToFIRPassPipeline:
* HLFIROptEarly, at the start of the pipeline, before any HLFIR
simplification or inlining.
* HLFIROptLast, just before createLowerHLFIRIntrinsics.
Drivers register passes through registerHLFIROptEarlyEPCallbacks and
registerHLFIROptLastEPCallbacks on MLIRToLLVMPassPipelineConfig. The invoke
methods are const so they can be called on the const config the HLFIR pipeline
receives. With no callbacks registered the pipeline is unchanged.
Co-Authored-By: Claude Opus 5 <noreply at anthropic.com>
[flang] Export fir-opt symbols for MLIR dialect/pass plugins (#212152)
Lets plugins loaded with --load-dialect-plugin / --load-pass-plugin
resolve
MLIR and LLVM symbols against fir-opt, as mlir-opt already does.
---------
Co-authored-by: Claude Opus 4.8 <noreply at anthropic.com>
[CGProfile] Fix unhandled error crash on empty canonical function names (#201821)
A function whose entire name is a strippable suffix canonicalizes to an
empty name, making InstrProfSymtab::create return an error
The current solution with `(void)(bool)` does not really suppress the
error which leads to the crash
[libc] Add program_invocation(_short)_name and tweak err.h functions (#212448)
These GNU extensions hold the name of the program as invoked (argv[0])
and its short name (the basename after the last slash).
Both variables are initialized in the startup code. As with all of our
other variables, they are only available in full build mode.
The trickiest part of this patch are the error reporting functions from
<err.h>, which access this variable, and they are currently enabled in
overlay mode. To make them work, I add an #ifdef to select the right
version. I considered doing something more elaborate, like we have with
`errno`, but that seemed too heavy for a single occurrence.
I also drop the linux check in this function. The documentation says the
functions should print the "last component of the program name", which
"llvmlibc" is not. If someone wants to enable these functions on
non-linux, they can figure out what they want to print here and how.
Assisted by Gemini.
AMDGPU: Export the TargetParser feature bitset (#212946)
Previously this bitset was only used to populate the feature
name string map used by clang. Eventually this will replace
the current bitmask integer. AArch64 already has a similar
interface.
Co-authored-by: Claude (Claude-Opus-4.8)
[X86][AsmParser] Fix compiler crash on division by zero in MS inline asm (#213539)
This fixes issue #213415. If a user writes something like '1 / 0' or '1
% 0' in assembly, the compiler will now show a normal error message
instead of crashing completely.
Fixes #213415
Co-authored-by: 陈纪元 <chenjiyuan at chenjiyuandeMacBook-Air.local>
[GlobalsAA] Handle self-referencing stores in `AnalyzeUsesOfPointer` (#213631)
Correctly recognize that a global address does escape when it is stored
into itself. Such globals were previously incorrectly marked as
non-address-taken.
Fixes: https://github.com/llvm/llvm-project/issues/213232.
Consolidate the managed-dataset registries and guard every mutator
## Problem
"Is this a dataset middleware manages?" was answered by five separate registries with four membership sets and three matching algorithms: `INTERNAL_PATHS` in `plugins/zfs/utils.py`, `INTERNAL_DATASETS` in `plugins/pool_/dataset_query_utils.py`, `internal_datasets_filters` in `plugins/pool_/dataset.py`, inline literals in `alert/source/datasets.py`, and `INVALID_DATASETS` in `plugins/zettarepl.py`. None had unit coverage, and they had drifted: `<pool>/ix-applications` was creatable and then permanently invisible because one carried a trailing slash the others lacked, and the replication registry did not know about the apps datasets at all. A sixth spelling in `docker/fs_manage.py` tested `startswith("boot-pool/")`, so it missed `freenas-boot` entirely and reported the boot pool's own root dataset as the mounted apps dataset on any system upgraded from a FreeNAS-era install.
Three of the five matched by substring, which is the cheapest way to spell "is under" in a filter DSL and not a matching policy anyone chose. It confiscated names the user is entitled to. `<pool>/ix-apps-data` is a user dataset that merely starts with a managed name, and `<pool>/.systembackup` is what a competent admin would call a backup of the system dataset; both were hidden from `pool.dataset.query` forever, which left them unmanageable, since the lookups behind `get_instance`, `update` and `delete` all run through that listing. The same reach refused their creation and dropped their ZFS events. Nothing middleware creates lives at those names: `.system`, `ix-apps`, `ix-applications` and `.truenas_containers` are literals created at a fixed depth of one below the pool root, so everything a substring caught beyond that was, by construction, someone else's dataset.
"May this caller change it?" was answered by a `bypass` field on the snapshot request models and by `exclude_internal_paths` on `ZFSResourceQuery`. Both are `Private`, which the API layer already refuses from the wire, so neither was reachable by a caller. `exclude_internal_datasets` was the exception, and a real one: it was read out of `pool.dataset.query`'s free-form `extra` dict, where the model's `extra="forbid"` never reaches, because that governs unknown model fields rather than keys inside a dict value. Anyone holding `DATASET_READ` could set it and enumerate the boot pool, the system dataset and the apps datasets.
Separately, ten public entry points had no protection at all and none of them funnelled through a guarded implementation: `pool.dataset.promote`, `rename`, `set_quota`, `get_quota`, `lock`, `change_key` and `inherit_parent_encryption_properties`, both `zfs.tier` mutators, and `pool.snapshot.rename`. Nine of those mutate; `get_quota` only reads, but it reported the quota accounting of datasets that are not part of the user-facing surface. `promote` was the worst of them, since managed children are frequently clones and promoting one reparents its origin snapshot. `pool.snapshot.rename` turned out to be unreachable in any case: it passed a snapshot id to `zfs.resource.rename`, which rejects any name containing `@`, so the endpoint could not succeed for any valid input and had no test coverage.
The same whole-component test was also being asked about snapshots. `zfs.resource.snapshot.query` and `zfs.resource.snapshot.count` fed it names like `tank/.system at snap` and `boot-pool at snap`, where the suffix lands on the component being compared, so the answer was always False -- the per-snapshot filter and the opt-out that turns it off were both inert, and the count's direct-snapshot branch consulted neither. Nothing leaked, because a working dataset-level check filters the parent before any snapshot below it is reached. That is the problem rather than the reassurance: it leaves a dead guard sitting behind a live one, with nothing to fail if the live one is ever moved or dropped as redundant.
## Solution
- **One module, one predicate per question.** `utils/zfs/managed_datasets.py` replaces all five registries, with one function per caller decision -- `hidden_from_zfs_listing`, `hidden_from_snapshot_listing`, `hidden_from_dataset_listing`, `blocked_from_mutation`, `excluded_from_zfs_events`, `excluded_from_replication` -- each spelling out its own membership where you can read it against the rule at once. One rule covers everything: compare the component directly below the pool root against a name, exactly, which matches that dataset and everything under it the way ZFS's own name algebra does. The boot pools are a separate disjunct on every predicate, because they are matched at component zero. `hidden_from_snapshot_listing` is not a second rule -- it drops a snapshot suffix and asks `hidden_from_zfs_listing`, the way `deny_protected_snapshot` does for `deny_protected_path`. The refusal helpers `deny_protected_path` and `deny_protected_snapshot` sit in the same file directly beneath the predicate they gate, so answering "is this ours?" and "may this caller change it?" is one place to look rather than two files to choose between, and they replace the hand-rolled `if not bypass and has_internal_path(...)` that every call site previously spelled out for itself, snapshot-suffix stripping included. Callers that must accumulate into `ValidationErrors` mid-pass, or that want to name the containing dataset rather than the path they were handed, ask the predicate directly; the helpers are a convenience for the one common message shape, not a boundary.
- **The rule is shared; the membership carries the divergence.** Every predicate matches the same way, so where two of them disagree it is because they manage a different set of names, and that is product policy you can read in one line. Replication deliberately omits the apps datasets, because replication is the supported way to back them up. `.truenas_containers` takes part in the product listing only, reproducing today's behaviour while containers are addressed separately; that leaves it destroyable through `zfs.resource.destroy` while `pool.dataset.delete` answers ENOENT, which is written down rather than quietly fixed. The event view stays narrower than the product listing for the same reason -- a container dataset is hidden from the listing but its destruction still has to be published, or the cleanup never runs.
[14 lines not shown]
