mlx5: Use software enum in APIs instead of PRM

Users of the steering APIs shouldn't use the PRM directly.
Create an software enum to be used instead.

Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Sponsored by: NVidia networking
MFC after: 1 week

mlx5: Refactor flow actions into a struct

Create a struct to hold flow actions to be used when creating
a flow rule.

Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Sponsored by: NVidia networking
MFC after: 1 week

sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

mlx5: Make mlx5_del_flow_rule() NULL safe.

This change factors out repeated NULL checks.

No functional change intended.

MFC after: 1 week
Sponsored by: NVIDIA Networking

Mechanically convert mlx5en(4) to IfAPI

Reviewed by: zlei
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D38595

mlx5en: Make sure the NIC IP addresses are written to firmware on link up.

Fixes e059c120b4223fd5ec3af9def21c0519f439fe57 .

PR: 261746
MFC after: 1 day
Sponsored by: NVIDIA Networking

mlx5en: Create flowtables in correct order.

Because it affects how the flow tables may re-direct traffic.

MFC after: 1 week
Sponsored by: NVIDIA Networking

mlx5: Implement flow steering helper functions for TCP sockets.

This change adds convenience functions to setup a flow steering rule based on
a TCP socket. The helper function gets all the address information from the
socket and returns a steering rule, to be used with HW TLS RX offload.

MFC after: 1 week
Sponsored by: NVIDIA Networking

mlx5en: Create and destroy all flow tables and rules when the network interface attaches and detaches.

Previously flow steering tables and rules were only created and destroyed
at link up and down events, respectivly. Due to new requirements for adding
TLS RX flow tables and rules, the main flow steering table must always be
available as there are permanent redirections from the TLS RX flow table
to the vlan flow table.

MFC after: 1 week
Sponsored by: NVIDIA Networking

mlx5en: Only delete installed VxLAN rules.

MFC after: 1 week
Sponsored by: NVIDIA Networking

mlx5en: Fix inverted logical assignment.

MFC after: 1 week
Sponsored by: NVIDIA Networking

mlx5en: idiomatic use of preprocessor, in particular paths

MFC after: 1 week
Sponsored by: NVIDIA Networking

mlx5en: normalize use of the opt_*.h files

MFC after: 1 week
Sponsored by: NVIDIA Networking

mlx5en: add ASCII art providing an overview of flow tables organization

Reviewed by: hselasky
Sponsored by: Mellanox Technologies/NVidia Networking
MFC after: 1 week

mlx5en: Add missing error case when creating VXLAN flow tables.

While at it remove redundant checks for IFCAP_VXLAN_HWCSUM flag.

MFC after: 1 week
Reviewed by: kib
Sponsored by: Mellanox Technologies // NVIDIA Networking

mlx5en: Honor IFCAP_VXLAN_HWCSUM

In particular, avoid creating TIR or installing flow rules for VXLAN
if the capability is disabled.

Reported and reviewed by: hselasky
Sponsored by: Mellanox Technologies/NVidia Networking
MFC after: 1 week

mlx5en: remove all dynamic vxlan steering rules on close and reinstall on open

Reviewed by: hselasky
Sponsored by: Mellanox Technologies/NVidia Networking
MFC after: 1 week

mlx5en: add mlx5e_add_vxlan_rule_from_db() helper

Reviewed by: hselasky
Sponsored by: Mellanox Technologies/NVidia Networking
MFC after: 1 week

mlx5en: remove unneeded protocol and port parameters from mlx5e_add_vxal_rule

the values are already passed in the db element

Reviewed by: hselasky
Sponsored by: Mellanox Technologies/NVidia Networking
MFC after: 1 week

mlx5en: register vxlan start/stop handlers

Handlers maintain flow rules and inform hardware about non-standard VxLAN
port in use. The database of the vxlan end points is maintained.

Reviewed by: hselasky
Sponsored by: Mellanox Technologies/NVidia Networking
MFC after: 1 week

mlx5: Integrate eswitch and mpfs management code.

Reviewed by: hselasky
Sponsored by: Mellanox Technologies
MFC after: 2 weeks

Convert to if_foreach_llmaddr() KPI.

Reviewed by: hselasky

Unify prints in mlx5en(4).

All prints in mlx5en(4) should use on of the macros:
mlx5_en_err/dbg/warn

Submitted by: slavash@
MFC after: 3 days
Sponsored by: Mellanox Technologies

Add support for Multi-Physical Function Switch, MPFS, in mlx5en.

MPFS is a logical switch in the Mellanox device which forward packets
based on a hardware driven L2 address table, to one or more physical-
or virtual- functions. The physical- or virtual- function is required
to tell the MPFS by using the MPFS firmware commands, which unicast
MAC addresses it is requesting from the physical port's traffic.
Broadcast and multicast traffic however, is copied to all listening
physical- and virtual- functions and does not need a rule in the MPFS
switching table.

Linux commit: eeb66cdb682678bfd1f02a4547e3649b38ffea7e
MFC after: 3 days
Sponsored by: Mellanox Technologies

Implement macro for asserting priv lock in mlx5en.

MFC after: 3 days
Sponsored by: Mellanox Technologies

Ensure the flowtable rules are not freed twice in mlx5en(4).
This can happen when re-loading the driver.

MFC after: 3 days
Sponsored by: Mellanox Technologies

Undo previous steps upon returning failure in mlx5en(4).

Else flowtable resources may not be properly freed.

MFC after: 3 days
Sponsored by: Mellanox Technologies

Make sure the flow destination structure does not use values off the stack
in mlx5en(4).

MFC after: 3 days
Sponsored by: Mellanox Technologies

Add missing newline.

MFC after: 1 week
Sponsored by: Mellanox Technologies

ifnet: Replace if_addr_lock rwlock with epoch + mutex

Run on LLNW canaries and tested by pho@

gallatin:
Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5
based ConnectX 4-LX NIC, I see an almost 12% improvement in received
packet rate, and a larger improvement in bytes delivered all the way
to userspace.

When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1,
I see, using nstat -I mce0 1 before the patch:

InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree
4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32
4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32
4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32
4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32
4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32
4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32
4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32

After the patch

InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree
5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51
5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51
5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51
5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51
5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52
5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52

Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch

Reviewed by: gallatin
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D15366

mlx5en: Always allow VLAN id 0.

According to the 802.1Q-2014 9.6 VLAN Tag Control Information, VID value 0
means that there is no VLAN tag assigned to the packet, and only PCP and
DEI values from the tag are meaningful. Current flow table programming
filter out such packets.

When programming VLAN filter for flow table, unconditionally add rule which
accept packets with VLAN id 0. The packets are already handled correctly
by the network stack.

Reviewed by: hselasky, slavash
Sponsored by: Mellanox Technologies
MFC after: 1 week

Refactor the flowsteering APIs used by mlx5en(4). This change is needed by
the coming ibcore and mlx5ib updates in order to support traffic redirection
to so-called raw ethernet QPs.

Remove unused E-switch related routines and files while at it.

Sponsored by: Mellanox Technologies
MFC after: 1 week

Remove useless NULL checks.

NULL is not returned when allocating memory passing the M_WAITOK flag.

Submitted by: trasz @
Differential Revision: https://reviews.freebsd.org/D5772
Sponsored by: Mellanox Technologies
MFC after: 1 week

Update the MLX5 core module:
- Add new firmware commands and update existing ones.
- Add more firmware related structures and update existing ones.
- Some minor fixes, like adding missing \n to some prints.

Sponsored by: Mellanox Technologies
MFC after: 1 week

Add SR-IOV guest support to the mlx5en driver.

This patch adds the missing pieces needed for device setup using the
mlx5en driver inside a virtual machine which is providing hardware
access through SR-IOV.

Sponsored by: Mellanox Technologies
MFC after: 1 week

Style changes, mostly automated.

Differential Revision: https://reviews.freebsd.org/D4179
Submitted by: Daria Genzel <dariaz@mellanox.com>
Sponsored by: Mellanox Technologies
MFC after: 3 days

Add mlx5 and mlx5en driver(s) for ConnectX-4 and ConnectX-4LX cards
from Mellanox Technologies. The current driver supports ethernet
speeds up to and including 100 GBit/s. Infiniband support will be
done later.

The code added is not compiled by default, which will be done by a
separate commit.

Sponsored by: Mellanox Technologies
MFC after: 2 weeks