ipfw: Use nitems(foo) instead of sizeof(foo)/sizeof(foo[0])

Pull Request: https://github.com/freebsd/freebsd-src/pull/888
Signed-off-by: Elyes Haouas <ehaouas@noos.fr>

dummynet: add simple gilbert-elliott channel model

Have a simple Gilbert-Elliott channel model in
dummynet to mimick correlated loss behavior of
realistic environments. This allows simpler testing
of burst-loss environments.

Reviewed By: tuexen, kp, pauamma_gundo.com, #manpages
Sponsored by: NetApp, Inc.
Differential Revision: https://reviews.freebsd.org/D42980

Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

ipfw: use unsigned int for dummynet bandwidth

This allows the maximum value of 4294967295 (~4Gb/s) instead of previous
value of 2147483647 (~2Gb/s).

Reviewed by: np, scottl
Obtained from: pfSense
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D31582

[ipfw] quieten maybe-uninitialized errors in ipfw when compiled under mips-gcc-6.3.0.

This is mostly an exercise to set variables to NULL/0 when declared, but
one was ensuring a string variable was set before printing it.
We should never see "<unknown>" in a printed rule; if we do then this code
definitely has some bugs that need addressing.

ipfw(8): Fix most warnings with the default WARNS level.

- Add missing const and static qualifiers.
- Avoid shadowing the global "co" by renaming it to "g_co".
- Avoid mixing signedness in loop bound checks.
- Leave -Wcast-align warnings disabled for now.

Reviewed by: ae, melifaro
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D25456

Use strlcat().

MFC after: 2 weeks

Revert r327005 - SPDX tags for license similar to BSD-2-Clause.

After consultation with SPDX experts and their matching guidelines[1],
the licensing doesn't exactly match the BSD-2-Clause. It yet remains to be
determined if they are equivalent or if there is a recognized license that
matches but it is safer to just revert the tags.

Let this also be a reminder that on FreeBSD, SPDX tags are only advisory
and have no legal value (but IANAL).

Pointyhat to: pfg
Thanks to: Rodney Grimes, Gary O'Neall

[1] https://spdx.org/spdx-license-list/matching-guidelines

SPDX: These are fundamentally BSD-2-Clause.

They just omit the introductory line and numbering.

ipfw: dummynet: Add 'G' and 'g' suffix for bandwidth configuration/display

MFC after: 2 weeks
Sponsored by: Gandi.net

sbin/ipfw: strcpy, strncpy => strlcpy

Reported by: Coverity
CID: 1356162, 1356166
MFC after: 3 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D10662

In parse_range() validate both range values instead of checking
the top value twice.

PR: 202295
MFC after: 1 week

Flush buffer after output. This fixes adding new data to already
printed flows.

PR: 210882
MFC after: 3 days

Import Dummynet AQM version 0.2.1 (CoDel, FQ-CoDel, PIE and FQ-PIE).

Centre for Advanced Internet Architectures

Implementing AQM in FreeBSD

* Overview <http://caia.swin.edu.au/freebsd/aqm/index.html>

* Articles, Papers and Presentations
<http://caia.swin.edu.au/freebsd/aqm/papers.html>

* Patches and Tools <http://caia.swin.edu.au/freebsd/aqm/downloads.html>

Overview

Recent years have seen a resurgence of interest in better managing
the depth of bottleneck queues in routers, switches and other places
that get congested. Solutions include transport protocol enhancements
at the end-hosts (such as delay-based or hybrid congestion control
schemes) and active queue management (AQM) schemes applied within
bottleneck queues.

The notion of AQM has been around since at least the late 1990s
(e.g. RFC 2309). In recent years the proliferation of oversized
buffers in all sorts of network devices (aka bufferbloat) has
stimulated keen community interest in four new AQM schemes -- CoDel,
FQ-CoDel, PIE and FQ-PIE.

The IETF AQM working group is looking to document these schemes,
and independent implementations are a corner-stone of the IETF's
process for confirming the clarity of publicly available protocol
descriptions. While significant development work on all three schemes
has occured in the Linux kernel, there is very little in FreeBSD.

Project Goals

This project began in late 2015, and aims to design and implement
functionally-correct versions of CoDel, FQ-CoDel, PIE and FQ_PIE
in FreeBSD (with code BSD-licensed as much as practical). We have
chosen to do this as extensions to FreeBSD's ipfw/dummynet firewall
and traffic shaper. Implementation of these AQM schemes in FreeBSD
will:
* Demonstrate whether the publicly available documentation is
sufficient to enable independent, functionally equivalent implementations

* Provide a broader suite of AQM options for sections the networking
community that rely on FreeBSD platforms

Program Members:

* Rasool Al Saadi (developer)

* Grenville Armitage (project lead)

Acknowledgements:

This project has been made possible in part by a gift from the
Comcast Innovation Fund.

Submitted by: Rasool Al-Saadi <ralsaadi@swin.edu.au>
X-No objection: core
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D6388

Merge buffer-printing changes from from projects/ipfw as preparation
for branch merge.

Requested by: luigi

Suppord showing named tables in ipfw(8) rule listing.

Kernel changes:
* change base TLV header to be u64 (so size can be u32).
* Introduce ipfw_obj_ctlv generc container TLV.
* Add IP_FW_XGET opcode which is now used for atomic configuration
retrieval. One can specify needed configuration pieces to retrieve
via flags field. Currently supported are
IPFW_CFG_GET_STATIC (static rules) and
IPFW_CFG_GET_STATES (dynamic states).
Other configuration pieces (tables, pipes, etc..) support is planned.

Userland changes:
* Switch ipfw(8) to use new IP_FW_XGET for rule listing.
* Split rule listing code get and show pieces.
* Make several steps forward towards libipfw:
permit printing states and rules(paritally) to supplied buffer.
do not die on malloc/kernel failure inside given printing functions.
stop assuming cmdline_opts is global symbol.

ECN marking implenetation for dummynet.
Changes include both DCTCP and RFC 3168 ECN marking methodology.

DCTCP draft: http://tools.ietf.org/html/draft-bensley-tcpm-dctcp-00

Submitted by: Midori Kato (aoimidori27@gmail.com)
Worked with: Lars Eggert (lars@netapp.com)
Reviewed by: luigi, hiren

Fix some compile errors at high WARNS, including one
for an uninitialized variable.

unused parameters and variables are annotated with
(void)foo; /* UNUSED */
instead of __unused, because this code needs to build
also on linux and windows.

remove some write-only variables.
There is another block of code that is now useless as the computation
is done in the kernel.

More whitespace fixes.

Checked with: md5, diff -x -w

Whitespace fixes.

Checked with: md5, diff -w

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

expand_number(3) takes a uint64_t * now.

MFC after: 3 weeks

better printing of headers when listing flows

Slightly different handling of printf/snprintf for unaligned uint64_t,
which should improve readability, and also to ease the port to
platforms that do not support %llu

MFC after: 3 days

MFC of a large number of ipfw and dummynet fixes and enhancements
done in CURRENT over the last 4 months.
HEAD and RELENG_8 are almost in sync now for ipfw, dummynet
the pfil hooks and related components.

Among the most noticeable changes:
- r200855 more efficient lookup of skipto rules, and remove O(N)
blocks from critical sections in the kernel;
- r204591 large restructuring of the dummynet module, with support
for multiple scheduling algorithms (4 available so far)
See the original commit logs for details.

Changes in the kernel/userland ABI should be harmless because the
kernel is able to understand previous requests from RELENG_8 and
RELENG_7. For this reason, this changeset would be applicable
to RELENG_7 as well, but i am not sure if it is worthwhile.

accept lower case m as a synonym for Mega (bit/s or bytes/s).

+ implement (two lines) the kernel side of 'lookup dscp N' to use the
dscp as a search key in table lookups;

+ (re)implement a sysctl variable to control the expire frequency of
pipes and queues when they become empty;

+ add 'queue number' as optional part of the flow_id. This can be
enabled with the command

queue X config mask queue ...

and makes it possible to support priority-based schedulers, where
packets should be grouped according to the priority and not some
fields in the 5-tuple.
This is implemented as follows:
- redefine a field in the ipfw_flow_id (in sys/netinet/ip_fw.h) but
without changing the size or shape of the structure, so there are
no ABI changes. On passing, also document how other fields are
used, and remove some useless assignments in ip_fw2.c

- implement small changes in the userland code to set/read the field;

- revise the functions in ip_dummynet.c to manipulate masks so they
also handle the additional field;

There are no ABI changes in this commit.

implement listing of a subset of pipes/queues/schedulers.
The filtering of the output is done in the kernel instead of userland
to reduce the amount of data transfered.

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

make the listing of queues/pipes/schedulers handle the case of
data size increasing while we fetch the info.

remove stale comment

Bring in the most recent version of ipfw and dummynet, developed
and tested over the past two months in the ipfw3-head branch. This
also happens to be the same code available in the Linux and Windows
ports of ipfw and dummynet.

The major enhancement is a completely restructured version of
dummynet, with support for different packet scheduling algorithms
(loadable at runtime), faster queue/pipe lookup, and a much cleaner
internal architecture and kernel/userland ABI which simplifies
future extensions.

In addition to the existing schedulers (FIFO and WF2Q+), we include
a Deficit Round Robin (DRR or RR for brevity) scheduler, and a new,
very fast version of WF2Q+ called QFQ.

Some test code is also present (in sys/netinet/ipfw/test) that
lets you build and test schedulers in userland.

Also, we have added a compatibility layer that understands requests
from the RELENG_7 and RELENG_8 versions of the /sbin/ipfw binaries,
and replies correctly (at least, it does its best; sometimes you
just cannot tell who sent the request and how to answer).
The compatibility layer should make it possible to MFC this code in a
relatively short time.

Some minor glitches (e.g. handling of ipfw set enable/disable,
and a workaround for a bug in RELENG_7's /sbin/ipfw) will be
fixed with separate commits.

CREDITS:
This work has been partly supported by the ONELAB2 project, and
mostly developed by Riccardo Panicucci and myself.
The code for the qfq scheduler is mostly from Fabio Checconi,
and Marta Carbone and Francesco Magno have helped with testing,
debugging and some bug fixes.

mfc r200056
use qsort_r instead of heapsort;
staticize two functions.

fix argument type in the call to expand_number

Submitted by: gcc 4.3
MFC after: 3 days

use qsort_r instead of heapsort;
staticize two functions.

MFC after: 3 days

MFC r199626:
Fix minor resource leak in a function.

Reviewed by: luigi

Fix minor resource leak in a function.

Reviewed by: luigi
MFC after: 1 week

- fix dummynet 'fast' mode for WF2Q case.
- fix printing of pipe profile data.
- introduce new pipe parameter: 'burst' - how much data can be sent through
pipe bypassing bandwidth limit.

Permit the specification of bandwidth values within
"profile" files (bandwidth is mandatory when using a
profile, so it makes sense to have everything in one place).

Update the manpage accordingly.

Submitted by: Marta Carbone

remove a printf that was only useful for debugging.

MFC after: 3 days

Add emulation of delay profiles, which lets you model various
types of MAC overheads such as preambles, link level retransmissions
and more.

Note- this commit changes the userland/kernel ABI for pipes
(but not for ordinary firewall rules) so you need to rebuild
kernel and /sbin/ipfw to use dummynet features.

Please check the manpage for details on the new feature.

The MFC would be trivial but it breaks the ABI, so it will
be postponed until after 7.2 is released.

Interested users are welcome to apply the patch manually
to their RELENG_7 tree.

Work supported by the European Commission, Projects Onelab and
Onelab2 (contract 224263).

remove duplicate #include

put the altq-related functions into a separate file.
Minor cleanup of the includes used by the various source files,
including annotations of why certain headers are used.

fix printing of uint64_t values, so we can use WARNS=2

fix wrong variable usage...

Put dummynet-related code in a separate file.
To this purpose, add prototypes for global functions in ipfw2.h
and move there also the list of tokens used in various places in the code.