OPNSense/core 9ade41bsrc/opnsense/scripts/captiveportal cp-background-process.py, src/opnsense/scripts/captiveportal/lib ipfw.py arp.py

Captive Portal: re-introduce hash lookup for accounting purposes (#10275)

(cherry picked from commit 3c2780e2fc66b196ae99c52e64e08d59b1a901b9)
(cherry picked from commit 00b1481f57bd29ce5f318d8698ba9f6519929a97)
DeltaFile
+70-25src/opnsense/scripts/captiveportal/lib/ipfw.py
+26-0src/opnsense/service/templates/OPNsense/IPFW/ipfw.conf
+14-9src/opnsense/scripts/captiveportal/cp-background-process.py
+3-1src/opnsense/scripts/captiveportal/lib/arp.py
+113-354 files

OPNSense/core 7c8a426src/opnsense/mvc/app/models/OPNsense/Interfaces/FieldTypes DUIDField.php

interfaces: put back DUID validity for faulty setup

Since DUID is part of interface settings it's better to not invalidate
existing values and/or rewrite them later on.
DeltaFile
+5-1src/opnsense/mvc/app/models/OPNsense/Interfaces/FieldTypes/DUIDField.php
+5-11 files

OPNSense/core cedf816src/opnsense/mvc/app/views/OPNsense/Firewall nat_rule.volt

Firewall: NAT: Fix inverted source/destination cosmetic issue in SNAT and One-to-One NAT grids (#10215)

(cherry picked from commit ca2fe06203500f708e28fcad09c0f5e1510353fd)
DeltaFile
+2-1src/opnsense/mvc/app/views/OPNsense/Firewall/nat_rule.volt
+2-11 files

OPNSense/core bc82e19src/opnsense/mvc/app/controllers/OPNsense/Kea/forms dialogReservation4.xml, src/opnsense/mvc/app/models/OPNsense/Kea KeaDhcpv4.xml KeaDhcpv4.php

Services: Kea DHCPv4: Add client-id to reservations (#10288)

* Services: Kea DHCPv4: Add client-id to reservations

* Should be client_id in the row

* Add client_id to dhcpv4 config generator

* client-id is stored differently in the running configuration and the lease endpoint, it must be normalized here so we can return a correct match in is_reserved

* Fix typo in client_id

* Use MAC address instead of Ether address in validation message, fix missing back reference in DHCPv6 reservation validation

* Update src/opnsense/mvc/app/models/OPNsense/Kea/KeaDhcpv4.php

Co-authored-by: Franco Fichtner <franco at opnsense.org>

* Update src/opnsense/mvc/app/models/OPNsense/Kea/KeaDhcpv4.php

    [7 lines not shown]
DeltaFile
+16-1src/opnsense/mvc/app/models/OPNsense/Kea/KeaDhcpv4.xml
+11-1src/opnsense/mvc/app/views/OPNsense/Kea/leases4.volt
+7-0src/opnsense/mvc/app/models/OPNsense/Kea/KeaDhcpv4.php
+4-2src/opnsense/scripts/kea/get_kea_leases.py
+6-0src/opnsense/mvc/app/controllers/OPNsense/Kea/forms/dialogReservation4.xml
+2-2src/opnsense/mvc/app/models/OPNsense/Kea/KeaDhcpv6.php
+46-62 files not shown
+50-68 files

OPNSense/core 414dd8fsrc/opnsense/mvc/app/controllers/OPNsense/Firewall/Api FilterController.php, src/opnsense/mvc/app/library/OPNsense/Base UIModelGrid.php

mvc: regression in c81417f26747a9e4e46f608c2791bbae805e79fd, missing descriptions.

(cherry picked from commit ae326511d16ff6f35fee7f7dd22bf1d761b48f96)
DeltaFile
+6-1src/opnsense/mvc/app/controllers/OPNsense/Firewall/Api/FilterController.php
+6-1src/opnsense/mvc/app/library/OPNsense/Base/UIModelGrid.php
+12-22 files

OPNSense/core c4aefc0src/opnsense/mvc/app/controllers/OPNsense/Firewall/Api FilterController.php

Firewall: Rules [new] - refactor searchRuleAction() to use the same filtering and sorting logic on MVC and legacy data.

The current implementation is applying our filter logic twice for MVC records, first it uses the default searchBase() construct, which it then needs to pipe through searchRecordsetBase() again. There are a couple of downsides here, it's more expensive (although the user likely won't notice), but also requires duplication of filter logic.

With the logic introduced in https://github.com/opnsense/core/commit/c81417f26747a9e4e46f608c2791bbae805e79fd , we can extract the exact same content from our model so we can merge all at once and push it through our filtering and sorting logic.

The idea is to be able to "smarten" searchRecordsetBase() a bit so we can apply some additional logic based on types at some point in time, which requires all data to passthrough the same pipeline at least.

This commit should be backwards compatible with the previous code.

(cherry picked from commit dad956e3143e7aa26f6be140e4c0ff48d1758e22)
(cherry picked from commit c2dcb5ba2fb4716d0e7880fac0c62680293d1b21)
DeltaFile
+53-87src/opnsense/mvc/app/controllers/OPNsense/Firewall/Api/FilterController.php
+53-871 files

OPNSense/core 90b0605src/opnsense/mvc/app/library/OPNsense/Base UIModelGrid.php

mvc: minor cleanup in UIModelGrid, remove flatten() method as getFlatNodes() is almost the same.

(cherry picked from commit c81417f26747a9e4e46f608c2791bbae805e79fd)
(cherry picked from commit 5e64948059c1fb4e57a4b15ca9402eaa3c92d7d0)
DeltaFile
+6-13src/opnsense/mvc/app/library/OPNsense/Base/UIModelGrid.php
+6-131 files

OPNSense/core 0adc0b5src/etc/inc/plugins.inc.d captiveportal.inc, src/opnsense/service/templates/OPNsense/Captiveportal lighttpd-zone.conf

Captive Portal: adjust redirection for modern portal support
DeltaFile
+83-42src/etc/inc/plugins.inc.d/captiveportal.inc
+11-7src/opnsense/service/templates/OPNsense/Captiveportal/lighttpd-zone.conf
+94-492 files

OPNSense/core 869c992src/etc/inc interfaces.inc, src/etc/inc/plugins.inc.d captiveportal.inc

Merge remote-tracking branch 'origin/master' into kea-dynamic-poc
DeltaFile
+70-25src/opnsense/scripts/captiveportal/lib/ipfw.py
+44-47src/etc/inc/plugins.inc.d/captiveportal.inc
+63-24src/etc/inc/interfaces.inc
+67-16src/opnsense/mvc/app/views/OPNsense/Diagnostics/traffic.volt
+65-2src/opnsense/scripts/kea/get_kea_leases.py
+35-22src/opnsense/mvc/app/controllers/OPNsense/Interfaces/Api/VipSettingsController.php
+344-13623 files not shown
+530-24529 files

OPNSense/core 59fc27dsrc/www interfaces.php

interfaces: make this clearer
DeltaFile
+3-3src/www/interfaces.php
+3-31 files

OPNSense/core da663d7src/opnsense/mvc/app/controllers/OPNsense/Kea/forms dialogSubnet4.xml dialogSubnet6.xml, src/opnsense/mvc/app/models/OPNsense/Kea KeaDhcpv6.php KeaDhcpv4.php

kea: add subnet vltime

PR: https://github.com/opnsense/core/issues/7592

Co-authored-by: @bgilesmca
(cherry picked from commit cb9c4a2ac827813ce8ace908b2958755cebf42bd)
DeltaFile
+9-0src/opnsense/mvc/app/controllers/OPNsense/Kea/forms/dialogSubnet4.xml
+9-0src/opnsense/mvc/app/controllers/OPNsense/Kea/forms/dialogSubnet6.xml
+4-0src/opnsense/mvc/app/models/OPNsense/Kea/KeaDhcpv6.php
+4-0src/opnsense/mvc/app/models/OPNsense/Kea/KeaDhcpv4.php
+1-0src/opnsense/mvc/app/models/OPNsense/Kea/KeaDhcpv6.xml
+1-0src/opnsense/mvc/app/models/OPNsense/Kea/KeaDhcpv4.xml
+28-06 files

OPNSense/core 7e75075src/etc/inc console.inc

shell: safe iteration for VLAN/LAGG in port assginment
DeltaFile
+10-14src/etc/inc/console.inc
+10-141 files

OPNSense/core 9f8431fsrc/opnsense/mvc/app/library/OPNsense/Firewall Util.php, src/opnsense/mvc/app/models/OPNsense/Kea KeaDhcpv6.php KeaDhcpv4.php

firewall: make getRealInterface() a static utility function #10251 (#10258)

Align implementation with get_real_interface() except that we shall
not give a fallback of $device = $interface since we want better
integrity in MVC code (and don't deal with devices in configuration
data there).

(cherry picked from commit dc357ece1a5e6f9b5713b7cc1e986f6483c91931)
DeltaFile
+47-0src/opnsense/mvc/app/library/OPNsense/Firewall/Util.php
+2-33src/opnsense/mvc/app/models/OPNsense/Routing/Gateways.php
+7-11src/opnsense/mvc/app/models/OPNsense/Kea/KeaDhcpv6.php
+4-7src/opnsense/mvc/app/models/OPNsense/Kea/KeaDhcpv4.php
+60-514 files

OPNSense/core c6ab599src/opnsense/mvc/app/controllers/OPNsense/Kea/Api LeasesController.php, src/opnsense/mvc/app/views/OPNsense/Kea leases6.volt leases4.volt

Services: Kea DHCPv4/6: Build reservation status from control socket output, so it matches the scope of individual subnet (#10276)

* Services: Kea DHCPv4/6: Build reservation status from control socket output, so it matches the scope of individual subnets as well. Add client-id since it's relevant for IPv4 leases as well in default configuration.

We return an array now, change frontend detection if it's dynamic or static lease

Missed a closing bracket

Typo in client_id

Remove unused imports in LeasesController

Add comment to build_reserved_matches() to explain why the subnet-id logic exists now

* Add state as well, helpful for troubleshooting

* Add a state formatter to convert number status into their documented meaning

* Some data-width micro management

    [2 lines not shown]
DeltaFile
+63-2src/opnsense/scripts/kea/get_kea_leases.py
+0-37src/opnsense/mvc/app/controllers/OPNsense/Kea/Api/LeasesController.php
+16-5src/opnsense/mvc/app/views/OPNsense/Kea/leases6.volt
+15-3src/opnsense/mvc/app/views/OPNsense/Kea/leases4.volt
+94-474 files

OPNSense/core a67a25esrc/opnsense/mvc/app/controllers/OPNsense/Kea/Api Dhcpv4Controller.php Dhcpv6Controller.php

Services: Kea DHCPv4/6: Enable internalModelSafeDelete due to increased model relation field usage (#10287)

(cherry picked from commit fe8c0f27cab78631130d27ddc3e73ee4efc4ebbb)
DeltaFile
+1-0src/opnsense/mvc/app/controllers/OPNsense/Kea/Api/Dhcpv4Controller.php
+1-0src/opnsense/mvc/app/controllers/OPNsense/Kea/Api/Dhcpv6Controller.php
+2-02 files

OPNSense/core fd7401asrc/etc/inc interfaces.inc

interfaces: move this inside the actual block

Nothing to sort in the else case.
DeltaFile
+2-2src/etc/inc/interfaces.inc
+2-21 files

OPNSense/core ff0393bsrc/etc/inc interfaces.inc

interfaces: adjust and annotate interface_dhcpv6_id() #7647

This does not change the output unless an unknown or empty interface
is passed which is mostly to protect future GUI use from returning a
wrong IAID.  This means from a functional perspective this is risk-
free.

Also note that this is the "default_id" and not always the resulting
ID due to what was already worked on for this ticket.
DeltaFile
+17-15src/etc/inc/interfaces.inc
+17-151 files

OPNSense/core da9068dsrc/etc/inc filter.lib.inc

firewall: openvpn legacy safe iteration from master
DeltaFile
+2-1src/etc/inc/filter.lib.inc
+2-11 files

OPNSense/core 8edd6eesrc/opnsense/mvc/app/controllers/OPNsense/Kea/forms dialogReservation4.xml, src/opnsense/mvc/app/models/OPNsense/Kea KeaDhcpv4.xml KeaDhcpv4.php

Services: Kea DHCPv4: Add client-id to reservations (#10288)

* Services: Kea DHCPv4: Add client-id to reservations

* Should be client_id in the row

* Add client_id to dhcpv4 config generator

* client-id is stored differently in the running configuration and the lease endpoint, it must be normalized here so we can return a correct match in is_reserved

* Fix typo in client_id

* Use MAC address instead of Ether address in validation message, fix missing back reference in DHCPv6 reservation validation

* Update src/opnsense/mvc/app/models/OPNsense/Kea/KeaDhcpv4.php

Co-authored-by: Franco Fichtner <franco at opnsense.org>

* Update src/opnsense/mvc/app/models/OPNsense/Kea/KeaDhcpv4.php

    [5 lines not shown]
DeltaFile
+16-1src/opnsense/mvc/app/models/OPNsense/Kea/KeaDhcpv4.xml
+11-1src/opnsense/mvc/app/views/OPNsense/Kea/leases4.volt
+7-0src/opnsense/mvc/app/models/OPNsense/Kea/KeaDhcpv4.php
+6-0src/opnsense/mvc/app/controllers/OPNsense/Kea/forms/dialogReservation4.xml
+4-2src/opnsense/scripts/kea/get_kea_leases.py
+2-2src/opnsense/mvc/app/models/OPNsense/Kea/KeaDhcpv6.php
+46-62 files not shown
+50-68 files

OPNSense/core fe8c0f2src/opnsense/mvc/app/controllers/OPNsense/Kea/Api Dhcpv4Controller.php Dhcpv6Controller.php

Services: Kea DHCPv4/6: Enable internalModelSafeDelete due to increased model relation field usage (#10287)
DeltaFile
+1-0src/opnsense/mvc/app/controllers/OPNsense/Kea/Api/Dhcpv4Controller.php
+1-0src/opnsense/mvc/app/controllers/OPNsense/Kea/Api/Dhcpv6Controller.php
+2-02 files

OPNSense/core a43360dsrc/www interfaces.php

interafaces: amendment to previous to use correct default
DeltaFile
+9-11src/www/interfaces.php
+9-111 files

OPNSense/core 18b63ebsrc/etc/inc interfaces.inc, src/www interfaces.php

interfaces: bring back interface_dhcpv6_id() and improve use a bit #7647

Keep this as $default_id but allow the overrides already implemented.
Do not emit a default IAID when the interface is not set or faulty so
that we can use it in the GUI safely for a placeholder.

PR: https://forum.opnsense.org/index.php?topic=51148.0
DeltaFile
+36-11src/etc/inc/interfaces.inc
+11-11src/www/interfaces.php
+47-222 files

OPNSense/core db0e57bsrc/etc/inc interfaces.inc, src/www interfaces.php

interfaces: bring back interface_dhcpv6_id() and improve use a bit #7647

Keep this as $default_id but allow the overrides already implemented.

PR: https://forum.opnsense.org/index.php?topic=51148.0
DeltaFile
+36-11src/etc/inc/interfaces.inc
+11-11src/www/interfaces.php
+47-222 files

OPNSense/core 0f78f69src/opnsense/mvc/app/views/OPNsense/Kea leases4.volt

Fix typo in client_id
DeltaFile
+2-2src/opnsense/mvc/app/views/OPNsense/Kea/leases4.volt
+2-21 files

OPNSense/core ba9ed8bsrc/etc/inc filter.inc interfaces.inc

backend: use bridges/bridged safe iteration

(cherry picked from commit 9cd352e408a0f2bb874ad3bd0035fb932f305c7b)
DeltaFile
+7-6src/etc/inc/filter.inc
+3-9src/etc/inc/interfaces.inc
+10-152 files

OPNSense/core 14eadfcsrc/etc/inc interfaces.inc

interafces: safe gres/gifs iteration

(cherry picked from commit 3d7cbd8079ddeb1febbad9cd1bd0467c6931e23e)
DeltaFile
+4-16src/etc/inc/interfaces.inc
+4-161 files

OPNSense/core d5f7b99src/etc/inc interfaces.inc

interfaces: use safe iteration in backend code

(cherry picked from commit 295c05cad94a746b8dd24dbaeac0d5a325259ef1)
DeltaFile
+5-5src/etc/inc/interfaces.inc
+5-51 files

OPNSense/core 295c05csrc/etc/inc interfaces.inc

interfaces: use safe iteration in backend code
DeltaFile
+5-5src/etc/inc/interfaces.inc
+5-51 files

OPNSense/core 6246b48src/etc/inc/plugins.inc.d captiveportal.inc

Captive Portal: remove redirection on HTTPS, ditch non-functional pass statement as well

In theory, clients only use HTTP to detect the presence of a portal.
If they were to use HTTPS, the 302 redirect would in most cases
not be accessible, as the certificate presented is most likely not
valid, cutting off the communication before any redirect to a login
page can happen.

The portal itself can and should remain accessible on HTTPS, as this
is the URL the redirect is pointing to. This may be attached to a
valid certificate as well, but the key point is that access to
this URL doesn't strictly need redirection for everything on port
443.

This should prevent clients opening bogus connections to the
captive portal, which consumes a lot of TLS traffic on the network
stack, bogging down lighttpd in bigger setups and creating
a lot of established states in pf.
DeltaFile
+44-47src/etc/inc/plugins.inc.d/captiveportal.inc
+44-471 files

OPNSense/core 6991129src/opnsense/scripts/kea get_kea_leases.py

client-id is stored differently in the running configuration and the lease endpoint, it must be normalized here so we can return a correct match in is_reserved
DeltaFile
+4-2src/opnsense/scripts/kea/get_kea_leases.py
+4-21 files