libpfctl: add pfctl_get_rule_h()

Add a handle variant of pfctl_get_rule(). This converts us from using
the nvlist variant to the netlink variant, and also moves us closer to a
world where all libpfctl functions take the handle.

While here have pfctl use the new function.

Sponsored by: Rubicon Communications, LLC ("Netgate")

libpfctl: fix file descriptor leak

pfctl_get_rules_info() opened a netlink socket, but failed to close it again.
Fix this by factoring out the netlink-based function into a _h variant that
takes struct pfctl_handle, and implement pfctl_get_rules_info() based on that,
remembering to close the fd.

While here migrate all in-tree consumers to the _h variant.

MFC after: 3 days
Sponsored by: Rubicon Communications, LLC ("Netgate")

pf: convert DIOCGETSTATUS to netlink

Introduce pfctl_get_status_h() because we need the pfctl_handle. In this variant
use netlink to obtain the information.

Sponsored by: Rubicon Communications, LLC ("Netgate")

snmp_pf: use libpfctl's pfctl_get_rules_info() rather than DIOCGETRULES

Prefer libpfctl functions over direct access to the ioctl whenever
possible.

Sponsored by: Rubicon Communications, LLC ("Netgate")

snmp_pf: use libpfctl's pfctl_get_status() rather than DIOCGETSTATUS

Prefer libpfctl functions over direct access to the ioctl whenever
possible. This will allow subsequent removal of DIOCGETSTATUS (in 15) as
there already is an nvlist-based alternative.

MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D41650

Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

bsnmp: Fix incorrect use of libpfctl

When we introduced libpfctl in bnsmpd this was done incorrectly, and a
GETRULE call was replaced by pfctl_add_rule().
Change it to pfctl_get_rule().

Reviewed by: donner
MFC after: 3 days
Sponsored by: Rubicon Communications, LLC ("Netgate")

pf: Allow multiple labels to be set on a rule

Allow up to 5 labels to be set on each rule.
This offers more flexibility in using labels. For example, it replaces
the customer 'schedule' keyword used by pfSense to terminate states
according to a schedule.

Reviewed by: glebius
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D29936

libpfctl: Switch to pfctl_rule

Stop using the kernel's struct pf_rule, switch to libpfctl's pfctl_rule.
Now that we use nvlists to communicate with the kernel these structures
can be fully decoupled.

Reviewed by: glebius
MFC after: 4 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D29644

bsnmp: Use libpfctl

Reviewed by: glebius
MFC after: 4 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D29642

Fix the following -Werror warning from clang 10.0.0 in bsnmpd:

usr.sbin/bsnmpd/modules/snmp_pf/pf_snmp.c:1661:4: error: misleading indentation; statement is not part of the previous 'else' [-Werror,-Wmisleading-indentation]
return (-1);
^
usr.sbin/bsnmpd/modules/snmp_pf/pf_snmp.c:1658:5: note: previous statement is here
} else
^

The intent was to group the return statement with the previous syslog()
call.

MFC after: 3 days

Unbreak snmp_pf(3) after the changes introduced in r338209

PR: 237011
Event: Vienna Hackathon 2019
MFC after: 2 weeks

Extended pf(4) ioctl interface and pfctl(8) to allow bandwidths of
2^32 bps or greater to be used. Prior to this, bandwidth parameters
would simply wrap at the 2^32 boundary. The computations in the HFSC
scheduler and token bucket regulator have been modified to operate
correctly up to at least 100 Gbps. No other algorithms have been
examined or modified for correct operation above 2^32 bps (some may
have existing computation resolution or overflow issues at rates below
that threshold). pfctl(8) will now limit non-HFSC bandwidth
parameters to 2^32 - 1 before passing them to the kernel.

The extensions to the pf(4) ioctl interface have been made in a
backwards-compatible way by versioning affected data structures,
supporting all versions in the kernel, and implementing macros that
will cause existing code that consumes that interface to use version 0
without source modifications. If version 0 consumers of the interface
are used against a new kernel that has had bandwidth parameters of
2^32 or greater configured by updated tools, such bandwidth parameters
will be reported as 2^32 - 1 bps by those old consumers.

All in-tree consumers of the pf(4) interface have been updated. To
update out-of-tree consumers to the latest version of the interface,
define PFIOC_USE_LATEST ahead of any includes and use the code of
pfctl(8) as a guide for the ioctls of interest.

PR: 211730
Reviewed by: jmallett, kp, loos
MFC after: 2 weeks
Relnotes: yes
Sponsored by: RG Nets
Differential Revision: https://reviews.freebsd.org/D16782

Update bsnmp to version 1.13. This does not bring user-visible changes.
For developers gensnmptree can now generate functions for enums to convert
between enums and strings and to check the validity of a value.
The sources in FreeBSD are now in sync with the upstream which allows to
bring in IPv6 modifications.

various: general adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.

style(9): clean up whitespace

MFC after: 3 days

style(9): clean up trailing whitespace

MFC after: 3 weeks

Merge the projects/pf/head branch, that was worked on for last six months,
into head. The most significant achievements in the new code:

o Fine grained locking, thus much better performance.
o Fixes to many problems in pf, that were specific to FreeBSD port.

New code doesn't have that many ifdefs and much less OpenBSDisms, thus
is more attractive to our developers.

Those interested in details, can browse through SVN log of the
projects/pf/head branch. And for reference, here is exact list of
revisions merged:

r232043, r232044, r232062, r232148, r232149, r232150, r232298, r232330,
r232332, r232340, r232386, r232390, r232391, r232605, r232655, r232656,
r232661, r232662, r232663, r232664, r232673, r232691, r233309, r233782,
r233829, r233830, r233834, r233835, r233836, r233865, r233866, r233868,
r233873, r234056, r234096, r234100, r234108, r234175, r234187, r234223,
r234271, r234272, r234282, r234307, r234309, r234382, r234384, r234456,
r234486, r234606, r234640, r234641, r234642, r234644, r234651, r235505,
r235506, r235535, r235605, r235606, r235826, r235991, r235993, r236168,
r236173, r236179, r236180, r236181, r236186, r236223, r236227, r236230,
r236252, r236254, r236298, r236299, r236300, r236301, r236397, r236398,
r236399, r236499, r236512, r236513, r236525, r236526, r236545, r236548,
r236553, r236554, r236556, r236557, r236561, r236570, r236630, r236672,
r236673, r236679, r236706, r236710, r236718, r237154, r237155, r237169,
r237314, r237363, r237364, r237368, r237369, r237376, r237440, r237442,
r237751, r237783, r237784, r237785, r237788, r237791, r238421, r238522,
r238523, r238524, r238525, r239173, r239186, r239644, r239652, r239661,
r239773, r240125, r240130, r240131, r240136, r240186, r240196, r240212.

I'd like to thank people who participated in early testing:

Tested by: Florian Smeets <flo freebsd.org>
Tested by: Chekaluk Vitaly <artemrts ukr.net>
Tested by: Ben Wilber <ben desync.com>
Tested by: Ian FREISLICH <ianf cloudseed.co.za>

Catch up with struct pfi_kif change.

In bsnmpd(1) add support for SNMPv3 message processing model, including message authentication, packet encryption & view-based access control (RFC 3412, 3414, 3415).

Sponsored by: The FreeBSD Foundation
Reviewed by: philip@ (mostly)
Approved by: philip@

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Now actually implement reading/refreshing/returning data from the pfTablesAddrTable
and modify the BEGEMOT-PF-MIB to add support for IPV6 address' statistics in the PF
tables via pfTablesAddrNetType and pfTablesAddrNet. While here, upgrade the
pf_tree.def file to the new format that includes enumerated values. Also make sure
to return SNMP_ERR_NOSUCHNAME for ALTQ objects, if ALTQ is disabled, so that the agent
will know to skip the pfAltq subtree when servicing GETNEXT requests from SNMP clients
(otherwise snmpwalk on begemotPf would stop at the pfAltq subtree with bsnmpd returning
SNMP_ERR_GENERR).

Make sure the snmp_pf module will first refresh its entires if necessary,
then find a specific entry, and get the requested value. So far, it found
the specific entry, refreshed the entry list if necessary, and got the
requested value from the found entry. The problem is that refreshing nukes
all old entries and replaces them with new ones and the obtained entry
pointer was no longer valid after the refresh.

Reviewed by: bz, philip
MFC after: 1 week

Add support for retrieving labeled pf filter rule counters.

PR: bin/132847
Submitted by: Szalai Andras <szalai (dot) bandi (at) gmail.com>

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

MFC r200122

Make sure enough memory is allocated for a struct pft_entry when
refreshing the list of pf tables.

Make sure enough memory is allocated for a struct pft_entry when
refreshing the list of pf tables.

OKed by: philip
MFC after: 1 week

- Import the HEAD csup code which is the basis for the cvsmode work.

Use INSERT_OBJECT_INT_LINK_INDEX macro instead of TAILQ_INSERT_TAIL when
filling the table of ALTQ queues retrieved from the kernel.

It is possible for the kernel to return the queues not by pa.altq.qid order.
When this happens, pf_snmp would only partially fill its table.

PR: bin/120974
Submitted by: Mykola Dzham <i -at- levsha.org.ua>
MFC after: 3 days

Link pf 4.1 to the build:
- move ftp-proxy from libexec to usr.sbin
- add tftp-proxy
- new altq mtag link

Approved by: re (kensmith)

Check for altq presence during module init, and allow module to work even if
altq is not present (just disable the altq bits in that case).

PR: 89601
Submitted by: Juraj Lutter <otis -at- sk.freebsd.org>
MFC after: 3 days

Fix unhandled malloc errors.

PR: bin/83467
Submitted by: Dan Lukes <dan -at- obluda.cz>
Prodded by: glebius
MFC after: 3 days

Catch up with the bsnmpd daemon's 64bit time ticks.

time_t is signed, so s/uintmax_t/intmax_t/g. Has anyone got an extra pointy
hat for me? I'm running out of them.

Spotted by: des
Pointy hat to: philip

Cast time_t to uintmax_t so they can be printed as %ju without breaking on
various architectures. This should appease tinderbox on alpha and not break
it anywhere else.

Suggested by: harti

Make this compile properly on 64bit machines.

Pointy hat to: philip

Add snmp_pf, a bsnmpd module for monitoring (not managing!) pf firewalls.
Currently implemented are most things related to states and interfaces, todo
are address tables and altq. I've tried to keep this roughly in sync with a
project implementing this for net-snmpd.

Reviewed by: harti, dhartmei
MFC after: 1 month