netgraph: provide separate malloc type for nodes that are missing it

The kernel option NG_SEPARATE_MALLOC helps to debug memory leaks in
netgraph(4). Several nodes were missing the support.

sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

ng_pptpgre(4): Fix a typo in a source code comment

- s/mimimum/minimum/

MFC after: 3 days

net: clean up empty lines in .c and .h files

Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked).
Use it in preparation for a general review of all nodes.

This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.

Mark all obvious cases as MPSAFE. All entries that haven't been marked
as MPSAFE before are by default marked as NEEDGIANT

Approved by: kib (mentor, blanket)
Commented by: kib, gallatin, melifaro
Differential Revision: https://reviews.freebsd.org/D23718

Make ng_pptpgre(8) netgraph node be able to restore order for packets
reordered in transit instead of dropping them altogether.
It uses sequence numbers of PPtPGRE packets.

A set of new sysctl(8) added to control this ability or disable it:

net.graph.pptpgre.reorder_max (1) defines maximum length of node's
private reorder queue used to keep data waiting for late packets.
Zero value disables reordering. Default value 1 allows the node to restore
the order for two packets swapped in transit. Greater values allow the node
to deliver packets being late after more packets in sequence
at cost of increased kernel memory usage.

net.graph.pptpgre.reorder_timeout (1) defines time value in miliseconds
used to wait for late packets. It may be useful to increase this
if reordering spot is distant.

MFC after: 1 month

sys/netgraph: spelling fixes in comments.

No functional change.

sys: extend use of the howmany() macro when available.

We have a howmany() macro in the <sys/param.h> header that is
convenient to re-use as it makes things easier to read.

Mechanically substitute flags from historic mbuf allocator with
malloc(9) flags within sys.

Exceptions:

- sys/contrib not touched
- sys/mbuf.h edited manually

Remove duplicate check.

Submitted by: Dmitry Luhtionov <dmitryluhtionov@gmail.com>

Node constructor methods are supposed to be called in syscall
context always. Convert nodes to consistently use M_WAITOK flag
for memory allocation.

Reviewed by: julian

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

MFC r206021, r206032, r206049, r206050:
Remove some more alignment constraints.

Remove alignment constraints.

Retire the MALLOC and FREE macros. They are an abomination unto style(9).

MFC after: 3 months

- Import the HEAD csup code which is the basis for the cvsmode work.

Some minor code and math optimizations.

Rewrite node to support multiple hooks, alike to ng_l2tp, to use one pair
of pptpgre and ksocket nodes for all calls between two peers. This patch
modifies node's API by adding new "session_%04x" hook names support, while
keeping backward compatibility.

Together with appropriate user-level support (by latest mpd5) it gives
huge performance benefits for case of multiple active calls between
two peers because of avoiding data duplication and extra socket processing.
On my benchmarks I have got more then 10 times speedup for the 200
simultaneous PPTP calls between two peers.
In conclusion, it allows now to build effective "clients <=> PAC <=> PNS"
setups.

Quoting Alexander:
Formulas described in RFC require high precision of floating point.
Formulas of integer math implemented in ng_pptpgre give mistake in range
of +0-7ms on RTT and +0-3ms on deviation. This leads to significant
underestimation of real packet RTT.

I have made a very simple patch to reduce mistake to +4-3ms on RTT and
+2-1ms on deviation. Mistake in RTT is not good, but gets covered by
deviation. To cover worst possible negative mistake in deviation I have
added 2ms to it. Also this 2 ms cover the case when measured deviation
is so small (about zero) that it can interfere with process scheduling
delays or weather on Mars.

My tests show decreasing of packet losses on 20ms RTT link from 2.5% to
0.3% while speed increased un 1/3.

Reviewed by: archie

Rework locking, that I have introduced recently, since it was incorrect:

First, mutexed callouts are incompatible with netgraph nodes, because
netgraph(4) can guarantee that the function will be called with mutex
held.

Second, nodes should not send data to their neighbor holding their
mutex. A node does not know what stack can it enter sending data in
some direction. May be executing will encounter a place to sleep.

New locking:

- ng_pptpgre_recv() and ng_pptpgre_xmit() must be entered with mutex held.
- ng_pptpgre_recv() and ng_pptpgre_xmit() unlock mutex before
sending data and then return unlocked.
- callout routines acquire mutex themselves.

Lock down PPTP node, since it has many data structures, that won't survive
parallel ng_pptp_rcvdata():

- Add a per-node mutex.
- Acquire mutex during all ng_pptp_rcvdata() method.
- Make callouts protected by mutex. Now callouts count as
netgraph writers, but there are plans to allow reader callouts
for nodes, that have internal locking.
- Acquire mutex in ng_pptp_reset(), which can be triggered
by a message or node shutdown.

PR: kern/80035
Tested by: Deomid Ryabkov <myself rojer.pp.ru>
Reviewed by: Deomid Ryabkov <myself rojer.pp.ru>

Utilize callout_pending() macro

/* -> /*- for license, minor formatting changes

Use ng_callout() and ng_uncallout() instead of home-grown
implementation.

Tested by: Savchuk Taras
Reviewed by: archie
Approved by: julian (mentor)

Increase PPTP_MAX_TIMEOUT up to 3 seconds. 10 prooved too much for high packet
loss links, and 1 second appeared to be too small for high latency links.

If we will receive more complaints, we should make this parameter configurable.

PR: kern/69536
Approved by: archie, julian (mentor)
MFC after: 3 days

Switch to using C99 sparse initialisers for the type methods array.
Should make no binary difference.

Submitted by: Gleb Smirnoff <glebius@cell.sick.ru>
Reviewed by: Harti Brandt <harti@freebsd.org>
MFC after: 1 week

Add 'enableWindowing' configuration knob to the ng_pptpgre(4) netgraph node.

Submitted by: Michael Bretterklieber <mbretter@a-quadrat.at>
MFC after: 2 weeks

Lower the maximum ACK timeout for GRE packets from 10 to 1 second.
In practice it seems that in situations of high packet loss the ACK
timeout seems to hit this maximum (perhaps inappropriately, but the
estimation algorithm is not perfect, so apparently it happens). In
any case, 10 seconds is way too high a value so lower to 1 second.

MFC after: 3 days

Don't use ovbcopy().

Back out M_* changes, per decision of the TRB.

Approved by: trb

Remove M_TRYWAIT/M_WAITOK/M_WAIT. Callers should use 0.
Merge M_NOWAIT/M_DONTWAIT into a single flag M_NOWAIT.

o Untangle the confusion with the malloc flags {M_WAITOK, M_NOWAIT} and
the mbuf allocator flags {M_TRYWAIT, M_DONTWAIT}.
o Fix a bpf_compat issue where malloc() was defined to just call
bpf_alloc() and pass the 'canwait' flag(s) along. It's been changed
to call bpf_alloc() but pass the corresponding M_TRYWAIT or M_DONTWAIT
flag (and only one of those two).

Submitted by: Hiten Pandya <hiten@unixdaemons.com> (hiten->commit_count++)

Relax checking of incoming PPTP GRE packets a bit: ignore a bogus payload
length field when there's no payload indicated by the header 'S' bit.
This works around semi-brokenness in the Mac OS X PPTP client.

Const'ify variables to make it clear we're not writing to the mbuf data.

Reviewed by: julian, brian
MFC after: 1 week

Fix GCC warnings caused by initializing a zero length array. In the process,
simply things a bit by getting rid of 'struct ng_parse_struct_info' which
was useless because it only contained one field.

MFC after: 2 weeks

Use 'struct callout' instead of 'struct callout_handle' to avoid
exhausting the kernel timeout table. Perform the usual gymnastics to
avoid race conditions between node shutdown and timeouts occurring.

Also fix a bug in handling ack delays < PPTP_MIN_ACK_DELAY. Before,
we were ack'ing immediately. Instead, just impose a minimum ack delay
time, like the name of the macro implies.

MFC after: 1 week

Update to C99, s/__FUNCTION__/__func__/,
also don't use ANSI string concatenation.

Don't reference a node after we dropped a reference to it
(same as in previous checkin, but in a different function).

netgraph.h:
Change a prototype.
Add a function version of ng_ref_node() when debugging so
a breakpoint can be set on it.
ng_base.c:
add 'node' as an argument to ng_apply_item so that it is up
to the caller to take over and release the item's reference on
the node. If the release reports back that the node went away
due to the reference going to 0, the caller should cease referencing
the now defunct node. (e.g. the item was a 'kill node' message).
Alter ng_unref_node to report back the residual references as a result.
ng_pptpgre.c:
Don't reference a node after we dropped a reference to it.
(What if it was the last?)
Fixes a node leak reported by Harti Brandt <brandt@fokus.gmd.de>
which was due to an incorrect earlier attempt to fix the
"accessing node after dropping the last reference" problem.

Fix potential crash caused by packets with bogus ACK's.

Reported by: Fabien THOMAS <fabient@netasq.com>

Part 2 of the netgraph rewrite.
This is mostly cosmetic changes, (though I caught a bug or two while
makeing them)
Reviewed by: archie@freebsd.org

Rewrite of netgraph to start getting ready for SMP.
This version is functional and is aproaching solid..
notice I said APROACHING. There are many node types I cannot test
I have tested: echo hole ppp socket vjc iface tee bpf async tty
The rest compile and "Look" right. More changes to follow.
DEBUGGING is enabled in this code to help if people have problems.

Divorce the kernel binary ABI version number from the message
format version number. (userland programs should not need to be
recompiled when the netgraph kernel internal ABI is changed.

Also fix modules that don;t handle the fact that a caller may not supply
a return message pointer. (benign at the moment because the calling code
checks, but that will change)

Reviewed by: Archie@freebsd.org
This clears out my outstanding netgraph changes.
There is a netgraph change of design in the offing and this is to some
extent a superset of soem of the new functionality and some of the old
functionality that may be removed.

This code works as before, but allows some new features that I want to
work with and evaluate. It is the basis for a version of netgraph
with integral locking for SMP use.

This is running on my test machine with no new problems :-)

Add the use of M_ZERO to netgraph.

Submitted by: josh@zipperup.org
Submitted by: Robert Drehmel <robd@gmx.net>
Submitted by: archie
Approved by: archie

Since neither archie nor I work at Whistle any more, change our email
addresses to be the more usefu @freebsd.org ones
so we can keep getting bug-reports.
- man pages to follow..

Calling untimeout(9) leads to a race window where memory could be leaked.
Close this window by simply not calling untimeout(9).

Remove unnecessary #include's as reported by phk's script.

Allocate all memory (including within node constructors) with M_NOWAIT
instead of M_WAITOK, to allow for maximum flexibility.

Followup to previous commit..
- It's worthwhile to use untimeout(9), even though we must still protect
against "false" timeouts, because most of the time it saves having to
handle a dummy timeout event.
- Slight tweaks to the delayed ACK algorithm paramters.

Several fixes:
- Fix slowness when operating over fast connections, where the timeout(9)
granularity is on the same order of magnitude as the round trip time.
timeout(9) can happen up to 1 tick early, which was causing receive
ack timeouts to happen too early, causing bogus "lost" packets.
- Increase the local time counter to 64 bits to avoid roll-over.
- Keep statistics on memory allocation failures.
- Add a new option to always include the ack when sending data packets.
Might be useful in high packet loss situations. Might not.

Fix incorrectly implemented receive ACK timeout algorithm:
instead of bumping the recvAck counter by one, pretend that
all outstanding xmit packets are acknowleged, and restart
transmitting anew, with an empty (but halved) transmit window.

Put a lower bound on the adaptive timeout value.

- Start sequence numbers at zero instead of one; the rest of the
world seems to interpret the spec this way
- Initialize transmit window to two instead of one; helps get things
going initially when the first packet may get dropped
- Really fix the shutdown + timeout race condition this time

Fix a couple of bugs:
- Properly handle 32 bit sequence numbers when they wrap around
- Don't drop GRE packets with stale ACK numbers, just ignore the ACK
- Close race between node being shutdown and timer going off
Also add support for lots of statistics, and control message ASCIIfication

Two simple changes to the kernel internal API for netgraph modules,
to support future work in flow-control and 'packet reject/replace'
processing modes.

reviewed by: phk, archie

Remove ~25 unneeded #include <sys/conf.h>
Remove ~60 unneeded #include <sys/malloc.h>

Fix race condition caused by missing splnet()'s.

New netgraph node type 'pptpgre': this performs GRE encapsulation
for the PPTP protocol as specified in RFC 2637.