MFC r334876:

pf: Fix deadlock with route-to

If a locally generated packet is routed (with route-to/reply-to/dup-to) out of
a different interface it's passed through the firewall again. This meant we
lost the inp pointer and if we required the pointer (e.g. for user ID matching)
we'd deadlock trying to acquire an inp lock we've already got.

Pass the inp pointer along with pf_route()/pf_route6().

PR: 228782

MFC r329950:

pf: Cope with overly large net.pf.states_hashsize

If the user configures a states_hashsize or source_nodes_hashsize value we may
not have enough memory to allocate this. This used to lock up pf, because these
allocations used M_WAITOK.

Cope with this by attempting the allocation with M_NOWAIT and falling back to
the default sizes (with M_WAITOK) if these fail.

PR: 209475
Submitted by: Fehmi Noyan Isi <fnoyanisi AT yahoo.com>

MFC r316355

pf: Fix leak of pf_state_keys

If we hit the state limit we returned from pf_create_state() without cleaning
up.

PR: 217997
Submitted by: Max <maximos@als.nnov.ru>

MFC 315529

pf: Fix rule evaluation after inet6 route-to

In pf_route6() we re-run the ruleset with PF_FWD if the packet goes out
of a different interface. pf_test6() needs to know that the packet was
forwarded (in case it needs to refragment so it knows whether to call
ip6_output() or ip6_forward()).

This lead pf_test6() to try to evaluate rules against the PF_FWD
direction, which isn't supported, so it needs to treat PF_FWD as PF_OUT.
Once fwdir is set correctly the correct output/forward function will be
called.

PR: 217883
Submitted by: Kajetan Staszkiewicz
Sponsored by: InnoGames GmbH

MFC r304152:

pf: Add missing byte-order swap to pf_match_addr_range

Without this, rules using address ranges (e.g. "10.1.1.1 - 10.1.1.5") did not
match addresses correctly on little-endian systems.

PR: 211796
Obtained from: OpenBSD (sthen)

MFC r290521:

pf: Fix broken rule skip calculation

r289932 accidentally broke the rule skip calculation. The address family
argument to PF_ANEQ() is now important, and because it was set to 0 the macro
always evaluated to false.
This resulted in incorrect skip values, which in turn broke the rule
evaluations.

MFC 300501, 300508

pf: Fix ICMP translation

Fix ICMP source address rewriting in rdr scenarios.

pf: Fix more ICMP mistranslation

In the default case fix the substitution of the destination address.

PR: 201519
Submitted by: Max <maximos@als.nnov.ru>

MFC r284777, r284814, r284863 and r298088:

ALTQ FAIRQ discipline import from DragonFLY.

Differential Revision: https://reviews.freebsd.org/D2847
Obtained from: pfSense
Sponsored by: Rubicon Communications (Netgate)

MFC 296932:
pf: Improve forwarding detection

When we guess the nature of the outbound packet (output vs. forwarding) we need
to take bridges into account. When bridging the input interface does not match
the output interface, but we're not forwarding. Similarly, it's possible for the
interface to actually be the bridge interface itself (and not a member interface).

MFC r290161:

pf: Fix IPv6 checksums with route-to.

When using route-to (or reply-to) pf sends the packet directly to the output
interface. If that interface doesn't support checksum offloading the checksum
has to be calculated in software.
That was already done in the IPv4 case, but not for the IPv6 case. As a result
we'd emit packets with pseudo-header checksums (i.e. incorrect checksums).

This issue was exposed by the changes in r289316 when pf stopped performing full
checksum calculations for all packets.

Submitted by: Luoqi Chen

MFC r289316:

pf: Fix TSO issues

In certain configurations (mostly but not exclusively as a VM on Xen) pf
produced packets with an invalid TCP checksum.

The problem was that pf could only handle packets with a full checksum. The
FreeBSD IP stack produces TCP packets with a pseudo-header checksum (only
addresses, length and protocol).
Certain network interfaces expect to see the pseudo-header checksum, so they
end up producing packets with invalid checksums.

To fix this stop calculating the full checksum and teach pf to only update TCP
checksums if TSO is disabled or the change affects the pseudo-header checksum.

PR: 154428, 193579, 198868
Relnotes: yes
Sponsored by: RootBSD

MFC r287376

pf: Fix misdetection of forwarding when net.link.bridge.pfil_bridge is set

If net.link.bridge.pfil_bridge is set we can end up thinking we're forwarding
in pf_test6() because the rcvif and the ifp (output interface) are different.
In that case we're bridging though, and the rcvif the the bridge member on
which the packet was received and ifp is the bridge itself.
If we'd set dir to PF_FWD we'd end up calling ip6_forward() which is
incorrect.

Instead check if the rcvif is a member of the ifp bridge. (In other words, the
if_bridge is the ifp's softc). If that's the case we're not forwarding but
bridging.

PR: 202351

MFC r285945, r285960:

Respect pf rule log option before log dropped packets with IP options or
dangerous v6 headers

Reviewed by: gnn, eri
Approved by: gnn, glebius
Obtained from: pfSense
Sponsored by: Netgate
Differential Revision: https://reviews.freebsd.org/D3222

Merge r285944: fix typo: delete nsn if we were the last reference.

Merge r283061, r283063: don't dereference NULL is pf_get_mtag() fails.

PR: 200222

Merge 280169: always lock the hash row of a source node when updating
its 'states' counter.

PR: 182401

Merge r281536

pf: Fix forwarding detection

If the direction is not PF_OUT we can never be forwarding. Some input packets
have rcvif != ifp (looped back packets), which lead us to ip6_forward() inbound
packets, causing panics.

Equally, we need to ensure that packets were really received and not locally
generated before trying to ip6_forward() them.

Differential Revision: https://reviews.freebsd.org/D2822
Reviewed by: gnn

Merge r278843, r278858

In the forwarding case refragment the reassembled packets with the same
size as they arrived in. This allows the sender to determine the optimal
fragment size by Path MTU Discovery.

Roughly based on the OpenBSD work by Alexander Bluhm.

Differential Revision: https://reviews.freebsd.org/D2816
Reviewed by: gnn

Merge r278831, r278834

Update the pf fragment handling code to closer match recent OpenBSD.
That partially fixes IPv6 fragment handling.

Differential Revision: https://reviews.freebsd.org/D2814
Reviewed by: gnn

MFC 266852,270223:
- Fix pf(4) to build with MAXCPU set to 256. MAXCPU is actually a count,
not a maximum ID value (so it is a cap on mp_ncpus, not mp_maxid).
- Bump MAXCPU on amd64 from 64 to 256. In practice APIC only permits 255
CPUs (IDs 0 through 254). Getting above that limit requires x2APIC.

MFC r279910:
Reset mbuf pointer to NULL in fastroute case to indicate that mbuf was
consumed by filter. This fixes several panics due to accessing to mbuf
after free.

MFC: 272906

Change the PF hash from Jenkins to Murmur3. In forwarding tests
this showed a conservative 3% incrase in PPS.

Original Differential Revision: https://reviews.freebsd.org/D461
Submitted by: des
Reviewed by: emaste

MFC r263710, r273377, r273378, r273423 and r273455:

- De-vnet hash sizes and hash masks.
- Fix multiple issues related to arguments passed to SYSCTL macros.

Sponsored by: Mellanox Technologies

Merge r270928: explicitly free packet on PF_DROP, otherwise a "quick"
rule with "route-to" may still forward it.

PR: 177808
Approved by: re (gjb)

Fix ABI broken in r270576. This is direct commit to stable/10.

Reported by: kib

Merge r270023 from head:
Do not lookup source node twice when pf_map_addr() is used.

PR: 184003
Submitted by: Kajetan Staszkiewicz <vegeta tuxpowered.net>
Sponsored by: InnoGames GmbH

Merge r270022 from head:
pf_map_addr() can fail and in this case we should drop the packet,
otherwise bad consequences including a routing loop can occur.

Move pf_set_rt_ifp() earlier in state creation sequence and
inline it, cutting some extra code.

PR: 183997
Submitted by: Kajetan Staszkiewicz <vegeta tuxpowered.net>
Sponsored by: InnoGames GmbH

Merge 270010 from head:
Fix synproxy with IPv6. pf_test6() was missing a check for M_SKIP_FIREWALL.

PR: 127920
Submitted by: Kajetan Staszkiewicz <vegeta tuxpowered.net>
Sponsored by: InnoGames GmbH

Merge r269998 from head:
- Count global pf(4) statistics in counter(9).
- Do not count global number of states and of src_nodes,
use uma_zone_get_cur() to obtain values.
- Struct pf_status becomes merely an ioctl API structure,
and moves to netpfil/pf/pf.h with its constants.
- V_pf_status is now of type struct pf_kstatus.

Submitted by: Kajetan Staszkiewicz <vegeta tuxpowered.net>
Sponsored by: InnoGames GmbH

MFC r264689:
De-virtualize UMA zone pf_mtag_z and move to global initialization part.

The m_tag struct does not know about vnet context and the pf_mtag_free()
callback is called unaware of current vnet. This causes a panic.

PR: kern/182964

MFC r264220:
Execute pf_overload_task() in vnet context. Fixes a vnet kernel panic.

Reviewed by: trociny

Merge r262763, r262767, r262771, r262806 from head:
- Remove rt_metrics_lite and simply put its members into rtentry.
- Use counter(9) for rt_pksent (former rt_rmx.rmx_pksent). This
removes another cache trashing ++ from packet forwarding path.
- Create zini/fini methods for the rtentry UMA zone. Via initialize
mutex and counter in them.
- Fix reporting of rmx_pksent to routing socket.
- Fix netstat(1) to report "Use" both in kvm(3) and sysctl(3) mode.

Bulk sync of pf changes from head, in attempt to fixup broken build I
made in r263029.

Merge r257186,257215,257349,259736,261797.

These changesets split pfvar.h into several smaller headers and make
userland utilities to include only some of them.

Merge r261882, r261898, r261937, r262760, r262799:
Once pf became not covered by a single mutex, many counters in it became
race prone. Some just gather statistics, but some are later used in
different calculations.

A real problem was the race provoked underflow of the states_cur counter
on a rule. Once it goes below zero, it wraps to UINT32_MAX. Later this
value is used in pf_state_expires() and any state created by this rule
is immediately expired.

Thus, make fields states_cur, states_tot and src_nodes of struct
pf_rule be counter(9)s.

Merge r261029: remove NULL pointer dereference.

Merge r258478, r258479, r258480, r259719: fixes related to mass source
nodes removal.

PR: 176763

Merge several fixlets from head:

r257619: Remove unused PFTM_UNTIL_PACKET const.
r257620: Code logic of handling PFTM_PURGE into pf_find_state().
r258475: Don't compare unsigned <= 0.
r258477: Fix off by ones when scanning source nodes hash.

MFC r287376

pf: Fix misdetection of forwarding when net.link.bridge.pfil_bridge is set

If net.link.bridge.pfil_bridge is set we can end up thinking we're forwarding
in pf_test6() because the rcvif and the ifp (output interface) are different.
In that case we're bridging though, and the rcvif the the bridge member on
which the packet was received and ifp is the bridge itself.
If we'd set dir to PF_FWD we'd end up calling ip6_forward() which is
incorrect.

Instead check if the rcvif is a member of the ifp bridge. (In other words, the
if_bridge is the ifp's softc). If that's the case we're not forwarding but
bridging.

PR: 202351

MFC r285945, r285960:

Respect pf rule log option before log dropped packets with IP options or
dangerous v6 headers

Reviewed by: gnn, eri
Approved by: gnn, glebius
Obtained from: pfSense
Sponsored by: Netgate
Differential Revision: https://reviews.freebsd.org/D3222

Merge r285944: fix typo: delete nsn if we were the last reference.

Merge r283061, r283063: don't dereference NULL is pf_get_mtag() fails.

PR: 200222

Merge 280169: always lock the hash row of a source node when updating
its 'states' counter.

PR: 182401

Merge r281536

pf: Fix forwarding detection

If the direction is not PF_OUT we can never be forwarding. Some input packets
have rcvif != ifp (looped back packets), which lead us to ip6_forward() inbound
packets, causing panics.

Equally, we need to ensure that packets were really received and not locally
generated before trying to ip6_forward() them.

Differential Revision: https://reviews.freebsd.org/D2822
Reviewed by: gnn

Merge r278843, r278858

In the forwarding case refragment the reassembled packets with the same
size as they arrived in. This allows the sender to determine the optimal
fragment size by Path MTU Discovery.

Roughly based on the OpenBSD work by Alexander Bluhm.

Differential Revision: https://reviews.freebsd.org/D2816
Reviewed by: gnn

Merge r278831, r278834

Update the pf fragment handling code to closer match recent OpenBSD.
That partially fixes IPv6 fragment handling.

Differential Revision: https://reviews.freebsd.org/D2814
Reviewed by: gnn

MFC 266852,270223:
- Fix pf(4) to build with MAXCPU set to 256. MAXCPU is actually a count,
not a maximum ID value (so it is a cap on mp_ncpus, not mp_maxid).
- Bump MAXCPU on amd64 from 64 to 256. In practice APIC only permits 255
CPUs (IDs 0 through 254). Getting above that limit requires x2APIC.

MFC r279910:
Reset mbuf pointer to NULL in fastroute case to indicate that mbuf was
consumed by filter. This fixes several panics due to accessing to mbuf
after free.

MFC: 272906

Change the PF hash from Jenkins to Murmur3. In forwarding tests
this showed a conservative 3% incrase in PPS.

Original Differential Revision: https://reviews.freebsd.org/D461
Submitted by: des
Reviewed by: emaste

MFC r263710, r273377, r273378, r273423 and r273455:

- De-vnet hash sizes and hash masks.
- Fix multiple issues related to arguments passed to SYSCTL macros.

Sponsored by: Mellanox Technologies

Merge r270928: explicitly free packet on PF_DROP, otherwise a "quick"
rule with "route-to" may still forward it.

PR: 177808
Approved by: re (gjb)

Fix ABI broken in r270576. This is direct commit to stable/10.

Reported by: kib

Merge r270023 from head:
Do not lookup source node twice when pf_map_addr() is used.

PR: 184003
Submitted by: Kajetan Staszkiewicz <vegeta tuxpowered.net>
Sponsored by: InnoGames GmbH

Merge r270022 from head:
pf_map_addr() can fail and in this case we should drop the packet,
otherwise bad consequences including a routing loop can occur.

Move pf_set_rt_ifp() earlier in state creation sequence and
inline it, cutting some extra code.

PR: 183997
Submitted by: Kajetan Staszkiewicz <vegeta tuxpowered.net>
Sponsored by: InnoGames GmbH

Merge 270010 from head:
Fix synproxy with IPv6. pf_test6() was missing a check for M_SKIP_FIREWALL.

PR: 127920
Submitted by: Kajetan Staszkiewicz <vegeta tuxpowered.net>
Sponsored by: InnoGames GmbH

Merge r269998 from head:
- Count global pf(4) statistics in counter(9).
- Do not count global number of states and of src_nodes,
use uma_zone_get_cur() to obtain values.
- Struct pf_status becomes merely an ioctl API structure,
and moves to netpfil/pf/pf.h with its constants.
- V_pf_status is now of type struct pf_kstatus.

Submitted by: Kajetan Staszkiewicz <vegeta tuxpowered.net>
Sponsored by: InnoGames GmbH

MFC r264689:
De-virtualize UMA zone pf_mtag_z and move to global initialization part.

The m_tag struct does not know about vnet context and the pf_mtag_free()
callback is called unaware of current vnet. This causes a panic.

PR: kern/182964

MFC r264220:
Execute pf_overload_task() in vnet context. Fixes a vnet kernel panic.

Reviewed by: trociny

Merge r262763, r262767, r262771, r262806 from head:
- Remove rt_metrics_lite and simply put its members into rtentry.
- Use counter(9) for rt_pksent (former rt_rmx.rmx_pksent). This
removes another cache trashing ++ from packet forwarding path.
- Create zini/fini methods for the rtentry UMA zone. Via initialize
mutex and counter in them.
- Fix reporting of rmx_pksent to routing socket.
- Fix netstat(1) to report "Use" both in kvm(3) and sysctl(3) mode.

Bulk sync of pf changes from head, in attempt to fixup broken build I
made in r263029.

Merge r257186,257215,257349,259736,261797.

These changesets split pfvar.h into several smaller headers and make
userland utilities to include only some of them.

Merge r261882, r261898, r261937, r262760, r262799:
Once pf became not covered by a single mutex, many counters in it became
race prone. Some just gather statistics, but some are later used in
different calculations.

A real problem was the race provoked underflow of the states_cur counter
on a rule. Once it goes below zero, it wraps to UINT32_MAX. Later this
value is used in pf_state_expires() and any state created by this rule
is immediately expired.

Thus, make fields states_cur, states_tot and src_nodes of struct
pf_rule be counter(9)s.

Merge r261029: remove NULL pointer dereference.

Merge r258478, r258479, r258480, r259719: fixes related to mass source
nodes removal.

PR: 176763

Merge several fixlets from head:

r257619: Remove unused PFTM_UNTIL_PACKET const.
r257620: Code logic of handling PFTM_PURGE into pf_find_state().
r258475: Don't compare unsigned <= 0.
r258477: Fix off by ones when scanning source nodes hash.