NAS-133371: Add complete WebShare service implementation
- Add WebShare service plugin with dataset management
- Add API endpoints and database migrations
- Add systemd service definitions for webshare and truesearch
- Add integration and unit tests
- Fix circular dependency in service startup by adding skip_reload parameter
NAS-136763 / 26.04 / Allow no args on some API methods (#16774)
The `single_argument_arg` decorator makes the single dictionary argument
required, even if all of its fields have defaults. This PR makes the
following methods so that they can be called without passing any
arguments instead of passing an empty dictionary.
* `filesystem.acltemplate.by_path`
* `catalog.apps`
* `core.arp`
* `core.debug`
* `group.get_group_obj`
* `sharing.smb.share_precheck`
* `user.get_user_obj`
http://jenkins.eng.ixsystems.net:8080/job/tests/job/api_tests/5370/
NAS-136921 / 25.10-RC.1 / plug a slow memory leak in service.* api (by yocalebo) (#16846)
Investigating the peculiar scenario outlined in
https://github.com/truenas/middleware/pull/16836, I saw that the parent
process was consuming ~19GB of resident memory. The loop was spinning
running `core.bulk` api calls, more specifically `service.restart`. I
created a very simple script that runs `service.restart <service>` in a
`while True` loop. Monitoring the parent process in top, the resident
memory would grow sporadically in chunks. A bit more investigation using
`tracemalloc` and `resource` modules, I saw that there were some
left-over references to objects from `pystemd` module. I found that
upstream had a memory leak related to the `Unit` class but that was
fixed back in 2018. I also confirmed that our version doesn't suffer
from that memory leak but I was still suspicious.
I ended up forcefully `del unit` and `del job_object` items and ran my
script again. With my changes, there are still some allocations taking
up memory in the parent process, however, they stabilized and did not
produce the same behavior from before my changes.
[9 lines not shown]
NAS-136921 / 26.04 / plug a slow memory leak in service.* api (#16842)
Investigating the peculiar scenario outlined in
https://github.com/truenas/middleware/pull/16836, I saw that the parent
process was consuming ~19GB of resident memory. The loop was spinning
running `core.bulk` api calls, more specifically `service.restart`. I
created a very simple script that runs `service.restart <service>` in a
`while True` loop. Monitoring the parent process in top, the resident
memory would grow sporadically in chunks. A bit more investigation using
`tracemalloc` and `resource` modules, I saw that there were some
left-over references to objects from `pystemd` module. I found that
upstream had a memory leak related to the `Unit` class but that was
fixed back in 2018. I also confirmed that our version doesn't suffer
from that memory leak but I was still suspicious.
I ended up forcefully `del unit` and `del job_object` items and ran my
script again. With my changes, there are still some allocations taking
up memory in the parent process, however, they stabilized and did not
produce the same behavior from before my changes.
[3 lines not shown]
NAS-136941 / 25.04.2.1 / Fix display device password/port validation (by Qubad786) (#16843)
Automatic cherry-pick failed. Please resolve conflicts by running:
git reset --hard HEAD~1
git cherry-pick -x 8a5acc7c88b1761c8de39334eff876dff1a3596f
If the original PR was merged via a squash, you can just cherry-pick the
squashed commit:
git reset --hard HEAD~1
git cherry-pick -x 362936694803252db897818607dac614182ad741
## Problem
We have users who had old VMs where `web_port` could be 5800 and also
`passoword` as an optional field. Around bluefin, we changed this
behaviour to make sure that ports below 5900 cannot be used for new
display devices and that display devices require a password.
[10 lines not shown]
NAS-136941 / 25.10 / Fix display device password/port validation (#16839)
## Problem
We have users who had old VMs where `web_port` could be 5800 and also
`passoword` as an optional field. Around bluefin, we changed this
behaviour to make sure that ports below 5900 cannot be used for new
display devices and that display devices require a password.
However with the change to pydantic, these validations got moved to
query as well which breaks these existing consumers.
## Solution
Make sure we do not validate this data when we are querying these
devices as we don't want to break existing consumers.
Fix validation and handling of MTLS cert for LDAP
This commit fixes several bugs related to handling of MTLS certificates
in the LDAP / SSSD config as well as the datastore plugin. CI tests will
be added in separate PR once we have a google domain for jenkins
testing.
NAS-136920 / 24.10.2.4 / fix busy loop on HA wrt to service.* actions (by yocalebo) (by bugclerk) (#16840)
On an internal system, an HA system was exposed to a network
segmentation between the controllers. This caused both controllers to
become MASTER. This caused a busy-loop of core.bulk service.restart
calls to be sent back and forth to each controller endlessly. To remedy
this situation I've added an extra check to ensure that
`failover.disabled.reasons` needs to be empty before forwarding the
remote call.
Original PR: https://github.com/truenas/middleware/pull/16836
Original PR: https://github.com/truenas/middleware/pull/16837
---------
Co-authored-by: Caleb <yocalebo at gmail.com>
NAS-136949 / 25.10 / Ignore libvirt service actions in service_remote on HA systems (#16838)
This is the "belt and suspenders" approach that compliments the changes
introduced here https://github.com/truenas/middleware/pull/16830. With
those changes this shouldn't be possible, however, it's better to be
safe than sorry. While I'm here, I went ahead and added some comments to
explain what each if-condition does and it's context.
NAS-136920 / 25.04.2.1 / fix busy loop on HA wrt to service.* actions (by yocalebo) (#16837)
On an internal system, an HA system was exposed to a network
segmentation between the controllers. This caused both controllers to
become MASTER. This caused a busy-loop of core.bulk service.restart
calls to be sent back and forth to each controller endlessly. To remedy
this situation I've added an extra check to ensure that
`failover.disabled.reasons` needs to be empty before forwarding the
remote call.
Original PR: https://github.com/truenas/middleware/pull/16836
---------
Co-authored-by: Caleb <yocalebo at gmail.com>
NAS-136920 / 25.10 / fix busy loop on HA wrt to service.* actions (#16836)
On an internal system, an HA system was exposed to a network
segmentation between the controllers. This caused both controllers to
become MASTER. This caused a busy-loop of core.bulk service.restart
calls to be sent back and forth to each controller endlessly. To remedy
this situation I've added an extra check to ensure that
`failover.disabled.reasons` needs to be empty before forwarding the
remote call.
