FreeNAS/freenas 03b6ad9src/middlewared/middlewared/plugins account.py, src/middlewared/middlewared/plugins/account_ constants.py

NAS-135784 / 25.04.2 / Avoid TOCTOU issues in account and iscsi.target (by anodos325) (#16468)

This commit changes the behavior of get_next_uid and get_next_gid such
that we keep track of uids/gids handed out by this endpoint and prevent
them from being reused in concurrent calls. These reservations of uid /
gids time out after one hour.

During investigation it was discovered that additional simpler toctou
issues existed related to iscsi.target and nvmet namespaces. These ones
were remedied by adding locking.

Original PR: https://github.com/truenas/middleware/pull/16460
Jira URL: https://ixsystems.atlassian.net/browse/NAS-135784

Co-authored-by: Andrew Walker <awalker at ixsystems.com>
DeltaFile
+58-28src/middlewared/middlewared/plugins/account.py
+58-0tests/unit/test_reserved_ids.py
+30-0src/middlewared/middlewared/utils/reserved_ids.py
+16-5src/middlewared/middlewared/plugins/iscsi_/targets.py
+1-0src/middlewared/middlewared/plugins/account_/constants.py
+163-335 files

FreeNAS/freenas cd2b60dsrc/middlewared/middlewared/plugins account.py, src/middlewared/middlewared/plugins/account_ constants.py

NAS-135784 / 25.10 / Avoid TOCTOU issues in account and iscsi.target (#16460)

This commit changes the behavior of get_next_uid and get_next_gid such
that we keep track of uids/gids handed out by this endpoint and prevent
them from being reused in concurrent calls. These reservations of uid /
gids time out after one hour.

During investigation it was discovered that additional simpler toctou
issues existed related to iscsi.target and nvmet namespaces. These ones
were remedied by adding locking.
DeltaFile
+58-28src/middlewared/middlewared/plugins/account.py
+58-0tests/unit/test_reserved_ids.py
+30-0src/middlewared/middlewared/utils/reserved_ids.py
+16-5src/middlewared/middlewared/plugins/iscsi_/targets.py
+9-6src/middlewared/middlewared/plugins/nvmet/namespace.py
+1-0src/middlewared/middlewared/plugins/account_/constants.py
+172-396 files

FreeNAS/freenas f2eb63bsrc/middlewared/middlewared/utils reserved_ids.py

Avoid modifying dict while iterating keys

This bug was introduced while addressing review.
DeltaFile
+1-1src/middlewared/middlewared/utils/reserved_ids.py
+1-11 files

FreeNAS/freenas bf94666src/middlewared/middlewared/plugins/virt attachments.py global.py, src/middlewared/middlewared/pytest/unit/plugins/virt test_attachment_delegate.py

Handle incus service itself in attachment delegate
DeltaFile
+109-0tests/api2/test_virt_attachment_delegate.py
+75-29src/middlewared/middlewared/pytest/unit/plugins/virt/test_attachment_delegate.py
+83-17src/middlewared/middlewared/plugins/virt/attachments.py
+8-1src/middlewared/middlewared/plugins/virt/global.py
+3-2src/middlewared/middlewared/test/integration/assets/pool.py
+1-1src/middlewared/middlewared/test/integration/assets/virt.py
+279-506 files

FreeNAS/freenas c9d575bsrc/middlewared/middlewared/plugins kerberos.py etc.py, src/middlewared/middlewared/plugins/directoryservices_ datastore.py activedirectory_join_mixin.py

Fix
DeltaFile
+16-6src/middlewared/middlewared/plugins/kerberos.py
+9-5src/middlewared/middlewared/plugins/directoryservices_/datastore.py
+2-2src/middlewared/middlewared/plugins/directoryservices_/activedirectory_join_mixin.py
+2-2src/middlewared/middlewared/plugins/directoryservices_/health.py
+2-1src/middlewared/middlewared/plugins/directoryservices_/activedirectory_health_mixin.py
+0-2src/middlewared/middlewared/plugins/etc.py
+31-186 files

FreeNAS/freenas 5f3dfb8src/middlewared/middlewared/plugins ldap.py, src/middlewared/middlewared/plugins/ldap_ ldap_client.py

NAS-135792 / 25.04.2 / Fix mutual TLS authentication for LDAP (#16466)

This commit fixes a longstanding validation bug in mutual tls
authentication for the LDAP plugin. Tests will be added once CI pipeline
added with this authentication type and an LDAP identity provider. This
commit is not targeted for master since the underlying fix will be
rolled into the overall larger rewrite project.
DeltaFile
+3-2src/middlewared/middlewared/plugins/ldap.py
+4-1src/middlewared/middlewared/plugins/ldap_/ldap_client.py
+7-32 files

FreeNAS/freenas 2087dfasrc/middlewared/middlewared/plugins ldap.py, src/middlewared/middlewared/plugins/ldap_ ldap_client.py

Fix mutual TLS authentication for LDAP

This commit fixes a longstanding validation bug in mutual tls
authentication for the LDAP plugin. Tests will be added once
CI pipeline added with this authentication type and an LDAP
identity provider. This commit is not targeted for master since
the underlying fix will be rolled into the overall larger rewrite
project.
DeltaFile
+3-2src/middlewared/middlewared/plugins/ldap.py
+4-1src/middlewared/middlewared/plugins/ldap_/ldap_client.py
+7-32 files

FreeNAS/freenas df9ab7bsrc/middlewared/middlewared/alembic/versions/25.10 2025-05-09_20-47_storj_endpoint.py 2025-05-07_20-47_storj_endpoint.py

fix migration
DeltaFile
+30-0src/middlewared/middlewared/alembic/versions/25.10/2025-05-09_20-47_storj_endpoint.py
+0-30src/middlewared/middlewared/alembic/versions/25.10/2025-05-07_20-47_storj_endpoint.py
+30-302 files

FreeNAS/freenas c11a407tests/api2 test_account.py

fix tests
DeltaFile
+1-1tests/api2/test_account.py
+1-11 files

FreeNAS/freenas 94c6c80src/middlewared/middlewared/plugins account.py, src/middlewared/middlewared/utils reserved_ids.py

Make the lock type clearer
DeltaFile
+11-7src/middlewared/middlewared/plugins/account.py
+0-4src/middlewared/middlewared/utils/reserved_ids.py
+11-112 files

FreeNAS/freenas 82a0e5csrc/middlewared/middlewared/api/v25_04_2 app.py filesystem.py, src/middlewared/middlewared/utils/account authenticator.py

Merge branch 'master' of https://github.com/truenas/middleware into NAS-135606
DeltaFile
+650-0src/middlewared/middlewared/utils/account/authenticator.py
+392-0src/middlewared/middlewared/api/v25_04_2/app.py
+369-0src/middlewared/middlewared/api/v25_04_2/filesystem.py
+345-0tests/unit/test_utmp.py
+332-0src/middlewared/middlewared/api/v25_04_2/acl.py
+313-0src/middlewared/middlewared/api/v25_04_2/virt_instance.py
+2,401-0178 files not shown
+10,209-653184 files

FreeNAS/freenas d41b8d6src/middlewared/middlewared main.py, src/middlewared/middlewared/alembic/versions/25.10 2025-05-08_11-00_periodic_snapshot_task_state.py

Merge branch 'master' of https://github.com/truenas/middleware into NAS-133835
DeltaFile
+30-13src/middlewared/middlewared/plugins/zettarepl_/state.py
+38-4src/middlewared/middlewared/api/base/handler/version.py
+32-0src/middlewared/middlewared/alembic/versions/25.10/2025-05-08_11-00_periodic_snapshot_task_state.py
+12-13src/middlewared/middlewared/api/base/handler/model_provider.py
+2-22src/middlewared/middlewared/main.py
+11-1src/middlewared/middlewared/plugins/mail.py
+125-5310 files not shown
+146-5816 files

FreeNAS/freenas 399d7cfsrc/middlewared/middlewared/plugins service.py account.py, src/middlewared/middlewared/plugins/failover_ event.py

Revert "NAS-131416 / 25.10 / Convert service management plugin to jobs" (#16465)

Reverts truenas/middleware#16380
DeltaFile
+69-101src/middlewared/middlewared/plugins/service.py
+14-35src/middlewared/middlewared/plugins/iscsi_/alua.py
+18-18src/middlewared/middlewared/plugins/failover_/event.py
+10-11tests/api2/test_015_services.py
+9-9src/middlewared/middlewared/plugins/account.py
+8-8tests/api2/assets/websocket/service.py
+128-18297 files not shown
+348-424103 files

FreeNAS/freenas ef3a328src/middlewared/middlewared/api __init__.py, src/middlewared/middlewared/api/base/handler model_provider.py

flake8
DeltaFile
+4-4src/middlewared/middlewared/api/base/handler/model_provider.py
+4-0src/middlewared/middlewared/api/base/server/ws_handler/base.py
+1-0src/middlewared/middlewared/api/__init__.py
+9-43 files

FreeNAS/freenas ed2727fsrc/middlewared/middlewared/plugins account.py

Fix typo
DeltaFile
+2-2src/middlewared/middlewared/plugins/account.py
+2-21 files

FreeNAS/freenas f26d3d1src/middlewared/middlewared/plugins ldap.py, src/middlewared/middlewared/plugins/ldap_ ldap_client.py

Fix mtls auth LDAP
DeltaFile
+3-2src/middlewared/middlewared/plugins/ldap.py
+4-1src/middlewared/middlewared/plugins/ldap_/ldap_client.py
+7-32 files

FreeNAS/freenas 78f1155src/middlewared/middlewared/utils reserved_ids.py

Address review
DeltaFile
+5-1src/middlewared/middlewared/utils/reserved_ids.py
+5-11 files

FreeNAS/freenas 00a3aa2src/middlewared/middlewared/plugins account.py, src/middlewared/middlewared/utils reserved_ids.py

Address review
DeltaFile
+19-13src/middlewared/middlewared/plugins/account.py
+0-6src/middlewared/middlewared/utils/reserved_ids.py
+19-192 files

FreeNAS/freenas 367cc48src/middlewared/middlewared/api/base/handler model_provider.py

don't raise `KeyError` in `_create_model`
DeltaFile
+28-16src/middlewared/middlewared/api/base/handler/model_provider.py
+28-161 files

FreeNAS/freenas 19de1dasrc/middlewared/middlewared/api/v25_10_0 directory_services.py, src/middlewared/middlewared/plugins/directoryservices_ datastore.py connection.py

Fix
DeltaFile
+20-18src/middlewared/middlewared/plugins/directoryservices_/datastore.py
+12-9src/middlewared/middlewared/plugins/smb_/util_smbconf.py
+6-5src/middlewared/middlewared/api/v25_10_0/directory_services.py
+7-3src/middlewared/middlewared/utils/directoryservices/credential.py
+3-1src/middlewared/middlewared/utils/directoryservices/ad.py
+1-1src/middlewared/middlewared/plugins/directoryservices_/connection.py
+49-376 files

FreeNAS/freenas 02c95c6src/middlewared/middlewared/api/v25_10_0 virt_instance.py, src/middlewared/middlewared/plugins/virt instance.py

Allow running containers in privileged mode
DeltaFile
+17-3src/middlewared/middlewared/api/v25_10_0/virt_instance.py
+11-4src/middlewared/middlewared/plugins/virt/instance.py
+28-72 files

FreeNAS/freenas 6dc9231src/middlewared/middlewared/plugins network.py

fix leaving stale db entries on interface delete
DeltaFile
+4-5src/middlewared/middlewared/plugins/network.py
+4-51 files

FreeNAS/freenas b4382c9src/middlewared/middlewared/test/integration/utils client.py

NAS-135790 / 25.04.2 / Fix test client with py_exceptions=False (#16463)

DeltaFile
+10-1src/middlewared/middlewared/test/integration/utils/client.py
+10-11 files

FreeNAS/freenas 01d40c2src/middlewared/middlewared/plugins service.py account.py, src/middlewared/middlewared/plugins/failover_ event.py

NAS-131416 / 25.10 / Convert service management plugin to jobs (#16380)

DeltaFile
+101-69src/middlewared/middlewared/plugins/service.py
+35-14src/middlewared/middlewared/plugins/iscsi_/alua.py
+18-18src/middlewared/middlewared/plugins/failover_/event.py
+11-10tests/api2/test_015_services.py
+9-9src/middlewared/middlewared/plugins/account.py
+8-8tests/api2/assets/websocket/service.py
+182-12897 files not shown
+424-348103 files

FreeNAS/freenas 58c1b67src/middlewared/debian control

NAS-135788 / 25.10 / remove comment from debian/control (#16462)

Nightly builds are failing because of this comment line.
DeltaFile
+0-1src/middlewared/debian/control
+0-11 files

FreeNAS/freenas 758a186src/middlewared/debian control

remove comment from debian/control
DeltaFile
+0-1src/middlewared/debian/control
+0-11 files

FreeNAS/freenas 537c768src/middlewared/middlewared/plugins network.py

fix leaving stale db entries on interface delete
DeltaFile
+6-5src/middlewared/middlewared/plugins/network.py
+6-51 files

FreeNAS/freenas 59a364esrc/middlewared/middlewared main.py, src/middlewared/middlewared/api/base/handler version.py model_provider.py

`APIVersion` raises `APIVersionDoesNotContainModelException` instead
DeltaFile
+18-19src/middlewared/middlewared/api/base/handler/version.py
+2-1src/middlewared/middlewared/api/base/handler/model_provider.py
+1-1src/middlewared/middlewared/api/base/server/legacy_api_method.py
+1-1src/middlewared/middlewared/main.py
+22-224 files

FreeNAS/freenas d182027src/middlewared/middlewared/api/base/handler model_provider.py version.py

polymorphic method `register_model`
DeltaFile
+46-1src/middlewared/middlewared/api/base/handler/model_provider.py
+4-27src/middlewared/middlewared/api/base/handler/version.py
+50-282 files

FreeNAS/freenas c0824dftests/unit test_reserved_ids.py

Adjust unit test

Make it so that unit test exercises concurrent attempts to get
next uid / gid.
DeltaFile
+18-7tests/unit/test_reserved_ids.py
+18-71 files