if_ovpn: cope with loops

User misconfiguration may lead to routing loops where we try to send the tunnel
packet into the tunnel. This eventually leads to stack overflows and panics.

Avoid this using if_tunnel_check_nesting(), which will drop the packet if we're
looping or we hit three layers of nested tunnels.

MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")

sockets: don't malloc/free sockaddr memory on getpeername/getsockname

Just like it was done for accept(2) in cfb1e92912b4, use same approach
for two simplier syscalls that return socket addresses. Although,
these two syscalls aren't performance critical, this change generalizes
some code between 3 syscalls trimming code size.

Following example of accept(2), provide VNET-aware and INVARIANT-checking
wrappers sopeeraddr() and sosockaddr() around protosw methods.

Reviewed by: tuexen
Differential Revision: https://reviews.freebsd.org/D42694

if_ovpn: clear mbuf flags on rx

When we receive a packet and remove the encapsulating layer we should
also clear out protocol flags and any mbuf tags.

If we do not we risk confusing firewalls filtering the tunneled packet.

See also: https://redmine.pfsense.org/issues/14682#change-69073
Sponsored by: Rubicon Communications, LLC ("Netgate")

if_ovpn: atomic_set -> atomic_store

The intent is to set the value to UINT32_MAX, not to |= UINT32_MAX.
Happily the intent (ensure that we do not send further packets) is
achieved either way.

Reported by: markj
Sponsored by: Rubicon Communications, LLC ("Netgate")

if_ovpn: ensure we never re-use sequence numbers

if_ovpn already notified userpsace when there was a risk of sequence
number re-use, but it trusted userspace to actually rotate the key.

Convert the internal sequence number counter to 64 bits so we can detect
overflows and then refuse to send packets.

Event: BSDCan 2023
Reviewed by: Leon Dang <ldang@netgate.com>
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D40187

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

if_ovpn: notify userspace when we've used half of the sequence numbers

OpenVPN uses the sequence number (as well as a userspace supplied nonce)
to build the IV. This means we should avoid re-using sequence numbers.
However, userspace doesn't know how many packets we've sent (and thus
what sequence number we're up to).

Notify userspace when we've used half of the available sequence numbers
to tell it that it's time for a key renegotiaton.

Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D39570

if_ovpn: ovpn_find_peer_by_ip() is unused without INET

Don't define ovpn_find_peer_by_ip() if INET is not set, and do the same
for ovpn_find_peer_by_ip6() and INET6.

Reported by: mjg
Sponsored by: Rubicon Communications, LLC ("Netgate")

ifnet/API: Move struct ifnet definition to a <net/if_private.h>

Hide the ifnet structure definition, no user serviceable parts inside,
it's a netstack implementation detail. Include it temporarily in
<net/if_var.h> until all drivers are updated to use the accessors
exclusively.

Reviewed by: glebius
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D38046

Revert "if_ovpn: allow peer lookup by vpn4/vpn6 address"

This reverts commit 92f0cf77db18502cac9a731cd2f6e8f3cc8a9369.

This change was incorrect, at least because it uses ovpn_kpeer's tree
for multipbe RB_TREEs.

This is a performance change, not a functional one, so we can revert
this until it can be fixed.

Reported by: Gert Doering <gert@greenie.muc.de>
Sponsored by: Rubicon Communications, LLC ("Netgate")

if_ovpn: fix LINT-NOIP build

Reported by: mjg
Sponsored by: Rubicon Communications, LLC ("Netgate")

if_ovpn: cleanup offsetof() use

Move the use of the `offsetof(struct ovpn_counters, fieldname) /
sizeof(uint64_t)` construct into a macro.
This removes a fair bit of code duplication and should make things a
little easier to read.

Reviewed by: zlei
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D37607

if_ovpn: include peer counters in a OVPN_NOTIF_DEL_PEER message

When we remove a peer userspace can no longer retrieve its counters. To
ensure that userspace can get a full count of the entire session we now
include the counters in the deletion message.

Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D37606

if_ovpn: allow peer lookup by vpn4/vpn6 address

Introduce two more RB_TREEs so that we can look up peers by their peer
id (already present) or vpn4 or vpn6 address.
This removes the last linear scan of the peer list.

Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D37605

if_ovpn: implement OVPN_GET_PEER_STATS

Allow userspace to retrieve per-peer traffic stats.

Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D37604

if_ovpn: start tracking per-peer packets/bytes in/out

OpenVPN will introduce a mechanism to retrieve per-peer statistics.
Start tracking those so we can return them to userspace when queried.

Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D37603

if_ovpn: remove OVPN_SEND_PKT

OpenVPN userspace no longer uses the ioctl interface to send control
packets. It instead uses the socket directly.
The use of OVPN_SEND_PKT was never released, so we can remove this
without worrying about compatibility.

Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D37602

if_ovpn: extend notifications with a reason

Extend peer deleted notifications (which are the only type right now) to
include the reason the peer was deleted. This can be either because
userspace requested it, or because the peer timed out.

Reviewed by: zlei
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D37583

if_ovpn: remove peer limit

Replace the fixed-sized array by an RB_TREE. This should both speed up
lookups and remove the 128 peer limit.

Reviewed by: zlei
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D37524

if_ovpn: pass control packets through the socket

Rather than passing control packets through the ioctl interface allow
them to pass through the normal UDP socket flow.
This simplifies both kernel and userspace, and matches the approach
taken (or the one that will be taken) on the Linux side of things.

Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D37317

if_ovpn: ensure we're in vnet context when calling sorele()

We reference count to ensure we don't release the socket while we still
have data in flight. That means that we can end up releasing the socket
from ovpn_encrypt_tx_cb().

We must have a vnet context set when calling sorele() (which asserts
this from within sofree()), so move the CURVNET_SET()/CURVNET_RESTORE()
to ensure this is the case.

While here also add a couple of assertions to make this more obvious,
and to ease future debugging.

Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D37326

if_ovpn: fix AES-128-GCM support

We need to explicitly list AES-128-GCM as an allowed cipher for that
mode to work. While here also add AES-192-GCM. That brings our supported
cipher list in line with other openvpn/dco platforms.

Sponsored by: Rubicon Communications, LLC ("Netgate")

if_ovpn: avoid netisr_queue name conflicts

Rename the netisr_queue variable in if_ovpn.c to avoid naming conflicts.

Sponsored by: Rubicon Communications, LLC ("Netgate")

if_ovpn: add sysctls for netisr_queue() and crypto_dispatch_async()

Allow the choice between asynchronous and synchronous netisr and crypto
calls. These have performance implications, but depend on the specific
setup and OCF back-end.

Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D37017

if_ovpn(4): implement ioctl() to set if_flags

Fully working openvpn(8) --iroute support needs real subnet config
on ovpn(4) interfaces (IFF_BROADCAST), while client-side/p2p
configs need IFF_POINTOPOINT setting. So make this configurable.

Reviewed by: kp

if_ovpn: fix use-after-free

ovpn_encrypt_tx_cb() calls ovpn_encap() to transmit a packet, then adds
the length of the packet to the "tunnel_bytes_sent" counter. However,
after ovpn_encap() returns 0, the mbuf chain may have been freed, so the
load of m->m_pkthdr.len may be a use-after-free.

Reported by: markj
Sponsored by: Rubicon Communications, LLC ("Netgate")

if_ovpn: remove an incorrect assertion

netisr_dispatch() can fail, especially when under high traffic loads.
This isn't a fatal error, so simply don't check the return value.

Sponsored by: Rubicon Communications, LLC ("Netgate")

if_ovpn: ensure we're in net_epoch when calling ovpn_encap()

If the crypto callback is asynchronous we're no longer in net_epoch,
which ovpn_encap() (and ip_output() it calls) expect.

Ensure we've entered the epoch.

Do the same thing for the rx path.

Sponsored by: Rubicon Communications, LLC ("Netgate")

if_ovpn: fix address family check when traffic class bits are set

When the tunneled (IPv6) traffic had traffic class bits set (but only >=
16) the packet got lost on the receive side.

This happened because the address family check in ovpn_get_af() failed
to mask correctly, so the version check didn't match, causing us to drop
the packet.

While here also extend the existing 6-in-6 test case to trigger this
issue.

PR: 266598
Sponsored by: Rubicon Communications, LLC ("Netgate")

if_clone: migrate some consumers to the new KPI.

Convert most of the cloner customers who require custom params
to the new if_clone KPI.

Reviewed by: kp
Differential Revision: https://reviews.freebsd.org/D36636
MFC after: 2 weeks

if_ovpn: fix memory leak on unload

When we're unloading the if_ovpn module we sometimes end up only freeing
the softc after the module is unloaded and the M_OVPN malloc type no
longer exists.

Don't return from ovpn_clone_destroy() until the epoch callbacks have
been called, which ensures that we've freed the softc before we destroy
M_OVPN.

Sponsored by: Rubicon Communications, LLC ("Netgate")

if_ovpn: remove incorrect rounding up of packet sizes

The ciphers used by OpenVPN (DCO) do not require data to be block-sized.
Do not round up to AES_BLOCK_LEN, as this can lead to issues with
fragmented packets.

Reported by: Gert Doering <gert@greenie.muc.de>
Sponsored by: Rubicon Communications, LLC ("Netgate")

Add getsock

All but one consumers of getsock_cap only pass 4 arguments.
Take advantage of it.

protosw: refactor protosw and domain static declaration and load

o Assert that every protosw has pr_attach. Now this structure is
only for socket protocols declarations and nothing else.
o Merge struct pr_usrreqs into struct protosw. This was suggested
in 1996 by wollman@ (see 7b187005d18ef), and later reiterated
in 2006 by rwatson@ (see 6fbb9cf860dcd).
o Make struct domain hold a variable sized array of protosw pointers.
For most protocols these pointers are initialized statically.
Those domains that may have loadable protocols have spacers. IPv4
and IPv6 have 8 spacers each (andre@ dff3237ee54ea).
o For inetsw and inet6sw leave a comment noting that many protosw
entries very likely are dead code.
o Refactor pf_proto_[un]register() into protosw_[un]register().
o Isolate pr_*_notsupp() methods into uipc_domain.c

Reviewed by: melifaro
Differential revision: https://reviews.freebsd.org/D36232

if_ovpn: reject non-UDP sockets

We must ensure that the fd provided by userspace is really for a UDP
socket. If it's not we'll panic in udp_set_kernel_tunneling().

Reported by: Gert Doering <gert@greenie.muc.de>
Sponsored by: Rubicon Communications, LLC ("Netgate")

if_ovpn: fix unused functions with NOINET / NOINET6

ovpn_find_peer_by_ip() is not used if INET is not defined. Do not
define the function in that case. Same for ovpn_find_peer_by_ip6().

Fix these warnings:

/usr/src/sys/net/if_ovpn.c:1580:1: warning: unused function 'ovpn_find_peer_by_ip' [-Wunused-function]
ovpn_find_peer_by_ip(struct ovpn_softc *sc, const struct in_addr addr)
^
/usr/src/sys/net/if_ovpn.c:1599:1: warning: unused function 'ovpn_find_peer_by_ip6' [-Wunused-function]
ovpn_find_peer_by_ip6(struct ovpn_softc *sc, const struct in6_addr *addr)
^

Reported by: mjg
Sponsored by: Rubicon Communications, LLC ("Netgate")

if_ovpn: fix mbuf leak

If the link is down or we can't find a peer we do not transmit the
packet, but also don't fee it.

Remember to m_freem() mbufs we can't transmit.

Sponsored by: Rubicon Communications, LLC ("Netgate")

if_ovpn: deal with short packets

If we receive a UDP packet (directed towards an active OpenVPN socket)
which is too short to contain an OpenVPN header ('struct
ovpn_wire_header') we wound up making m_copydata() read outside the
mbuf, and panicking the machine.

Explicitly check that the packet is long enough to copy the data we're
interested in. If it's not we will pass the packet to userspace, just
like we'd do for an unknown peer.

Extend a test case to provoke this situation.

Sponsored by: Rubicon Communications, LLC ("Netgate")

if_ovpn: handle m_pullup() failure

Ensure we correctly handle m_pullup() failing in ovpn_finish_rx().

Reported by: Coverity (CID 1490340)
Sponsored by: Rubicon Communications, LLC ("Netgate")

if_ovpn: deal with v4 mapped IPv6 addresses

Openvpn defaults to binding to IPv6 sockets (with
setsockopt(IPV6_V6ONLY=0)), which we didn't deal with.
That resulted in us trying to in6_selectsrc_addr() on a v4 mapped v6
address, which does not work.

Instead we translate the mapped address to v4 and treat it as an IPv4
address.

Sponsored by: Rubicon Communications, LLC ("Netgate")

if_ovpn: static probe points

Sprinkle a few SDTs around if_ovpn to ease debugging.

Sponsored by: Rubicon Communications, LLC ("Netgate")

ovpn: Introduce OpenVPN DCO support

OpenVPN Data Channel Offload (DCO) moves OpenVPN data plane processing
(i.e. tunneling and cryptography) into the kernel, rather than using tap
devices.
This avoids significant copying and context switching overhead between
kernel and user space and improves OpenVPN throughput.

In my test setup throughput improved from around 660Mbit/s to around
2Gbit/s.

Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D34340