Copy stable/10@r272459 to releng/10.1 as part of
the 10.1-RELEASE process.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Merged /head:r256510,256512,257376,257469,260052:

Make 'netstat -i' use getifaddrs(3) instead of kvm(3).

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Introduce new structure sfstat for collecting sendfile's statistics
and remove corresponding fields from struct mbstat. Use PCPU counters
and SFSTAT_INC() macro for update these statistics.

Discussed with: glebius

Show the number of times we block waiting for mbufs.

Machines can stall out because mbufs are low, however sometimes we won't
see "requests denied", instead we see user land processes or kernel threads
blocking waiting for mbufs because they set M_WAIT. These consumers do not
see errors, only stalling.

Unfortunately until now, netstat did not export this information
so you could have experienced an mbuf shortage and have no way of
seeing it unless you happen to run netstat at the exact time of the
shortage and see "in use" = "max".

By exporting the number of times processes are blocked, we can
effectively see how often non-interrupt context threads are effectively
"denied".

MFC after: 2 weeks

Revert most part of 200420 as requested, as more review and polish is
needed.

Remove unneeded header includes from usr.bin/ except contributed code.

Tested with: make universe

style(9)
+ kread is not a boolean, so check it as such
+ fix $FreeBSD$ Ids
+ denote copyrights with /*-
+ misc whitespace changes.

Restore netstat -M functionality for most statistics on core dumps. In
general, when support was added to netstat for fetching data using sysctl,
no provision was left for fetching equivalent data from a core dump, and
in fact, netstat would _always_ fetch data from the live kernel using
sysctl even when -M was specified resulting in the user believing they
were getting data from coredumps when they actually weren't. Some specific
changes:
- Add a global 'live' variable that is true if netstat is running against
the live kernel and false if -M has been specified.
- Stop abusing the sysctl flag in the protocol tables to hold the protocol
number. Instead, the protocol is now its own field in the tables, and
it is passed as a separate parameter to the PCB and stat routines rather
than overloading the KVM offset parameter.
- Don't run PCB or stats functions who don't have a namelist offset if we
are being run against a crash dump (!live).
- For the inet and unix PCB routines, we generate the same buffer from KVM
that the sysctl usually generates complete with the header and trailer.
- Don't run bpf stats for !live (before it would just silently always run
live).
- kread() no longer trashes memory when opening the buffer if there is an
error on open and the passed in buffer is smaller than _POSIX2_LINE_MAX.
- The multicast routing code doesn't fallback to kvm on live kernels if
the sysctl fails. Keeping this made the code rather hairy, and netstat
is already tied to the kernel ABI anyway (even when using sysctl's since
things like xinpcb contain an inpcb) so any kernels this is run against
that have the multicast routing stuff should have the sysctls.
- Don't try to dig around in the kernel linker in the netgraph PCB routine
for core dumps.

Other notes:
- sctp's PCB routine only works on live kernels, it looked rather
complicated to generate all the same stuff via KVM. Someone can always
add it later if desired though.
- Fix the ipsec removal bug where N_xxx for IPSEC stats weren't renumbered.
- Use sysctlbyname() everywhere rather than hardcoded mib values.

MFC after: 1 week
Approved by: re (rwatson)

Achieve WARNS=2 by using uintmax_t to pass around 64-bit quantities,
including to printf(). Using uintmax_t is also robust to further
extensions in both the C language and the bitwidth of kernel counters.

Tested on: i386 amd64 ia64

Improved description for packet zone statistics.

Sponsored by: TCP/IP Optimization Fundraise 2005
MFC after: 3 days

Print statistics on mbuf+clusters in packet zone.

Sponsored by: TCP/IP Optimization Fundraise 2005
MFC after: 3 days

Print statistics on jumbo mbuf clusters.

Sponsored by: TCP/IP Optimization Fundraise 2005
MFC after: 3 days

Sort sfbuf allocation counters with other sfbuf information.

MFC after: 3 days

Un-#if 0 the printing of allocation failure counts for mbufs, clusters,
and packets in netstat -m.

MFC after: 3 days

Remove two lines of debugging output that accidentally snuck into the
commit to fix up kvm support for netstat -m.

Modify netstat -mb to use libmemstat when accessing a core dump or live
kernel memory and not using sysctl. Previously, libmemstat was used
only for the live kernel via sysctl paths.

This results in netstat output becoming both more consistent between
core dumps and the live kernel, and also more information in the core
dump case than previously (i.e., mbuf cache information).

Statistics relating to sfbufs still rely on a kvm descriptor as they
are not currently exposed via libmemstat. netstat -m operating on a
core is still unable to print certain sfbuf stats available on the live
kernel.

MFC after: 1 week

Since libmemstat(3) now supports its own error management mechanism,
use that instead of trying to use errno, in order to produce a
sensible error message.

MFC after: 1 day

Modify "netstat -mb" to use libmemstat(3) when acting on a live system,
with a number of positive benefits:

- Start using UMA(9) statistics for mbufs and clusters, which avoids
using the mbuf allocator statistics which suffer from races under
load on SMP. This should eliminate "negative" mbuf counts in
netstat -mb.

- We are now able to track cached (free) mbufs and clusters and count
it towards memory allocated by the network stack.

- We are now also able to track memory allocated to mbuf tags since
libmemstat(3) can also query malloc(9). We don't print this except
as part of the total (for now - #if 0).

- We are now able to track mbuf/cluster/packet allocation failures,
although they are not currently printed (#if 0).

- Don't print out sfbuf statistics when running on a kernel core, as
currently that code is able only to query sysctl for statistics.

MFC after: 1 week

Add __FBSDID. Replace local variable sin by sockin to not conflict with sin(3).
Use warnx() instead of warn() when error message is not of any interest. Add
prototypes.

Bring in mbuma to replace mballoc.

mbuma is an Mbuf & Cluster allocator built on top of a number of
extensions to the UMA framework, all included herein.

Extensions to UMA worth noting:
- Better layering between slab <-> zone caches; introduce
Keg structure which splits off slab cache away from the
zone structure and allows multiple zones to be stacked
on top of a single Keg (single type of slab cache);
perhaps we should look into defining a subset API on
top of the Keg for special use by malloc(9),
for example.
- UMA_ZONE_REFCNT zones can now be added, and reference
counters automagically allocated for them within the end
of the associated slab structures. uma_find_refcnt()
does a kextract to fetch the slab struct reference from
the underlying page, and lookup the corresponding refcnt.

mbuma things worth noting:
- integrates mbuf & cluster allocations with extended UMA
and provides caches for commonly-allocated items; defines
several zones (two primary, one secondary) and two kegs.
- change up certain code paths that always used to do:
m_get() + m_clget() to instead just use m_getcl() and
try to take advantage of the newly defined secondary
Packet zone.
- netstat(1) and systat(1) quickly hacked up to do basic
stat reporting but additional stats work needs to be
done once some other details within UMA have been taken
care of and it becomes clearer to how stats will work
within the modified framework.

From the user perspective, one implication is that the
NMBCLUSTERS compile-time option is no longer used. The
maximum number of clusters is still capped off according
to maxusers, but it can be made unlimited by setting
the kern.ipc.nmbclusters boot-time tunable to zero.
Work should be done to write an appropriate sysctl
handler allowing dynamic tuning of kern.ipc.nmbclusters
at runtime.

Additional things worth noting/known issues (READ):
- One report of 'ips' (ServeRAID) driver acting really
slow in conjunction with mbuma. Need more data.
Latest report is that ips is equally sucking with
and without mbuma.
- Giant leak in NFS code sometimes occurs, can't
reproduce but currently analyzing; brueffer is
able to reproduce but THIS IS NOT an mbuma-specific
problem and currently occurs even WITHOUT mbuma.
- Issues in network locking: there is at least one
code path in the rip code where one or more locks
are acquired and we end up in m_prepend() with
M_WAITOK, which causes WITNESS to whine from within
UMA. Current temporary solution: force all UMA
allocations to be M_NOWAIT from within UMA for now
to avoid deadlocks unless WITNESS is defined and we
can determine with certainty that we're not holding
any locks when we're M_WAITOK.
- I've seen at least one weird socketbuffer empty-but-
mbuf-still-attached panic. I don't believe this
to be related to mbuma but please keep your eyes
open, turn on debugging, and capture crash dumps.

This change removes more code than it adds.

A paper is available detailing the change and considering
various performance issues, it was presented at BSDCan2004:
http://www.unixdaemons.com/~bmilekic/netbuf_bmilekic.pdf
Please read the paper for Future Work and implementation
details, as well as credits.

Testing and Debugging:
rwatson,
brueffer,
Ketrien I. Saihr-Kesenchedra,
...
Reviewed by: Lots of people (for different parts)

Fixed misspellings of 0 as NULL.

Use floating point instead of unsigned long longs in percentage
calculations. Long longs should never be used, since they break compiling
with C90 compilers and don't necessarily work any better than longs for
avoiding overflow.

Print percentages with another digit of precision since they can be small
and this is easy to do now that the format is floating point.

Restored some more of the old -m output:
Print the percentage of allocated memory that is in use. This is the
amount of memory in active mbufs and mbuf clusters relative to the
total amount of memory soft-allocated for mbufs and mbuf clusters.

Print the percentage of allocated memory that is wired (cached). The
old mbuf allocator never freed memory so printing this value wasn't
useful. A previous version of netstat for the new allocator printed
the in-use amount as a percentage of the wired amount.

Fixed some nearby style bugs (excessive parenthesization and a redundant
return).

Reviewed by: alfred

Teach netstat about the new sendfile statistics.

Clean up the style of the previous commit, and fix a few
type mismatches as well.

Suggested by: bde

Teach netstat to read and display the new sfbuf statistics.

Fix percentages by using long long to hold values for 'space',
overflow was breaking a bunch of the stats, specifically the
percentage displayed for wired memory.

Fix the output for current/peak/max lines, I forgot to output the types.
161/320/51200 (current/peak/max):
-to-
639/25696/51200 mbufs in use (current/peak/max):

I asked Bosko Milekic for help with 'peak' reporting, and he suggested
using the old 'cached' value but reporting it as 'cached'.

I've decided to report the 'cached' as 'peak', why? Well because
it is the peak, the peak of what is actually allocated. 'cached'
doesn't make sense to me as a user.

Restore old netstat -m output.
A new flag '-c' can be used to ask for the cache stats.

Make the mb_alloc low-watermark sysctl-tunable read-only and make
netstat(1) not display it for now because its effects are not yet
completely implemented and we're about to cut 5.2-RELEASE.
This is temporary.

Approved by: re (scottl, rwatson)

o Allow "buckets" in mb_alloc to be differently sized (according to
compile-time constants). That is, a "bucket" now is not necessarily
a page-worth of mbufs or clusters, but it is MBUF_BUCK_SZ, CLUS_BUCK_SZ
worth of mbufs, clusters.
o Rename {mbuf,clust}_limit to {mbuf,clust}_hiwm and introduce
{mbuf,clust}_lowm, which currently has no effect but will be used
to set the low watermarks.
o Fix netstat so that it can deal with the differently-sized buckets
and teach it about the low watermarks too.
o Make sure the per-cpu stats for an absent CPU has mb_active set to 0,
explicitly.
o Get rid of the allocate refcounts from mbuf map mess. Instead,
just malloc() the refcounts in one shot from mbuf_init()
o Clean up / update comments in subr_mbuf.c

Warns cleanups for netstat:
1) Include arpa/inet.h for ntohs.
2) Constness fixes.
3) Fix shadowing except for "sin" which shouldn't be in scope.
4) Remove register keyword.
5) Add missing initialsers to user defined structs.
5) Make prototype of netname6 globally visable.
6) Use right macros for printing syncache stats (even though entrie isn't
a word).

Re-enable mbtypes statistics in the mbuf allocator. I disabled these
when I changed the allocator bits. This implements per-CPU mbtypes
stats by keeping net number of decrements/increments of a given mbtype
per-CPU and then summing all of the per-CPU mbtypes to produce the total
net number of allocated mbufs of the given mbtype.
Counters are carefully balanced to avoid/prevent underflows/overflows.

mbtypes stats are re-enabled with the idea that we may occasionally
(although very rarely) observe slight inconsistencies in the stat
reporting. Most of the time, we should be fine, though.

Also make appropriate modifications to netstat(1) and systat(1) to do
the necessary reporting.

Submitted by: Jiangyi Liu <jyliu@163.net>

Add a "X KBytes of wired memory reserved" metric that represents
approximately the amount of memory allocated from the mbuf maps
and sitting in the mbuf allocator's cache containers, and display
in parantheses the percentage of said memory that is actually
in use at the given time `netstat -m' is executed.

Suggested by: mjacob

- Do not handle the per-CPU containers in mbuf code as though the cpuids
were indices in a dense array. The cpuids are a sparse set and treat
them as such, setting up containers only for CPUs activated during
mb_init().

- Fix netstat(1) and systat(1) to treat the per-CPU stats area as a sparse
map, in accordance with the above.

This allows us to properly boot with certain CPUs disactivated. However, if
we later decide to re-activate said CPUs, we will barf until we decide to
implement CPU spinon/spinoff callback hooks to allow for said CPUs' per-CPU
containers to get configured on their activation.

Reported by: mjacob
Partially (sys/ diffs) Submitted by: mjacob

- Fix space allocation for mbstat structure
- Make sure to try hw.ncpu if kern.smp.cpus doesn't exist (i.e. on UP) to
get number of CPUs.

Introduce numerous SMP friendly changes to the mbuf allocator. Namely,
introduce a modified allocation mechanism for mbufs and mbuf clusters; one
which can scale under SMP and which offers the possibility of resource
reclamation to be implemented in the future. Notable advantages:

o Reduce contention for SMP by offering per-CPU pools and locks.
o Better use of data cache due to per-CPU pools.
o Much less code cache pollution due to excessively large allocation macros.
o Framework for `grouping' objects from same page together so as to be able
to possibly free wired-down pages back to the system if they are no longer
needed by the network stacks.

Additional things changed with this addition:

- Moved some mbuf specific declarations and initializations from
sys/conf/param.c into mbuf-specific code where they belong.
- m_getclr() has been renamed to m_get_clrd() because the old name is really
confusing. m_getclr() HAS been preserved though and is defined to the new
name. No tree sweep has been done "to change the interface," as the old
name will continue to be supported and is not depracated. The change was
merely done because m_getclr() sounds too much like "m_get a cluster."
- TEMPORARILY disabled mbtypes statistics displaying in netstat(1) and
systat(1) (see TODO below).
- Fixed systat(1) to display number of "free mbufs" based on new per-CPU
stat structures.
- Fixed netstat(1) to display new per-CPU stats based on sysctl-exported
per-CPU stat structures. All infos are fetched via sysctl.

TODO (in order of priority):

- Re-enable mbtypes statistics in both netstat(1) and systat(1) after
introducing an SMP friendly way to collect the mbtypes stats under the
already introduced per-CPU locks (i.e. hopefully don't use atomic() - it
seems too costly for a mere stat update, especially when other locks are
already present).
- Optionally have systat(1) display not only "total free mbufs" but also
"total free mbufs per CPU pool."
- Fix minor length-fetching issues in netstat(1) related to recently
re-enabled option to read mbuf stats from a core file.
- Move reference counters at least for mbuf clusters into an unused portion
of the cluster itself, to save space and need to allocate a counter.
- Look into introducing resource freeing possibly from a kproc.

Reviewed by (in parts): jlemon, jake, silby, terry
Tested by: jlemon (Intel & Alpha), mjacob (Intel & Alpha)
Preliminary performance measurements: jlemon (and me, obviously)
URL: http://people.freebsd.org/~bmilekic/mb_alloc/

remove K&R support

revert removal of warning and K&R support

Requested by: bde

remove most of the warnings

Restore -M -N support for -m.

PR: 20808

Change the "in use" percentage metric to actually display a real
"in use" percentage. In other words, show how much of mb_map is in use.

Replace the mbuf external reference counting code with something
that should be better.

The old code counted references to mbuf clusters by using the offset
of the cluster from the start of memory allocated for mbufs and
clusters as an index into an array of chars, which did the reference
counting. If the external storage was not a cluster then reference
counting had to be done by the code using that external storage.

NetBSD's system of linked lists of mbufs was cosidered, but Alfred
felt it would have locking issues when the kernel was made more
SMP friendly.

The system implimented uses a pool of unions to track external
storage. The union contains an int for counting the references and
a pointer for forming a free list. The reference counts are
incremented and decremented atomically and so should be SMP friendly.
This system can track reference counts for any sort of external
storage.

Access to the reference counting stuff is now through macros defined
in mbuf.h, so it should be easier to make changes to the system in
the future.

The possibility of storing the reference count in one of the
referencing mbufs was considered, but was rejected 'cos it would
often leave extra mbufs allocated. Storing the reference count in
the cluster was also considered, but because the external storage
may not be a cluster this isn't an option.

The size of the pool of reference counters is available in the
stats provided by "netstat -m".

PR: 19866
Submitted by: Bosko Milekic <bmilekic@dsuper.net>
Reviewed by: alfred (glanced at by others on -net)

Make mbstat.m_mtypes seperate and viewable via sysctl, also
expand the size from short to ulong

Submitted by: Ian Dowse <iedowse@maths.tcd.ie>
PR: kern/19809

Add display of maximum allowed mbuf count to match mbuf cluster count.

Submitted by: Bosko Milekic <bmilekic@dsuper.net>

$Id$ -> $FreeBSD$

Fix "make world" breakage because MT_RTABLE was still referenced here.

Quick hack to allow netstat to compile once more: ifdef out MT_SOOPTS

Change a variable from int to size_t.

Fixed printf format errors.

mbuf, inet, and unix modules no longer read kvm.

Output mbuf clusters as current/peak/max. Requires new kern.ipc.nmbclusters
sysctl variable (previously committed).

Add (unsigned) cast to prevent overflow in %
Submitted by: "Alex G. Bulushev" <bag@demos.su>

Bandaid to make this compile.. These old mbuf types should probably be
ripped out.

Remove prog, unused variables.
Cosmetic in usage string.

Use the new sysctl(3) interface to mbuf statistics rather than
groveling about in kmem.

When the mbuf code was changed to use a private allocator instead of
the kernel malloc, netstat was never updated to reflect the fact that
there are once again allocated-but-free mbufs, just as there are
clusters, and so the information presented about how much memory was
allocated to the network was bogus. Fixed.

Code cleanup: remove unused variables, use correct *printf format
specifiers (some unsigned values were printed as signed, some longs
were printed as ints), and place parentheses around assignments in
if statements.

Changed "mapped pages in use" to "mbuf clusters in use".

Don't print protocol-cloned routes by default (enabled by -a flag).
Print non-zero expiration timers.
Use err(3) for error reporting.

This commit was generated by cvs2svn to compensate for changes in r1590,
which included commits to RCS files with non-trunk default branches.

BSD 4.4 Lite Usr.bin Sources