Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Merge r230119, r238498 from head:

------------------------------------------------------------------------
r230119 | csjp | 2012-01-15 02:51:34 +0400 (вс, 15 янв 2012) | 9 lines

Revert to the old behavior of allocating table/table entries using
M_NOWAIT. Currently, the code allows for sleeping in the ioctl path
to guarantee allocation. However code also handles ENOMEM gracefully, so
propagate this error back to user-space, rather than sleeping while
holding the global pf mutex.

Reviewed by: glebius
Discussed with: bz

------------------------------------------------------------------------
r238498 | glebius | 2012-07-15 23:10:00 +0400 (вс, 15 июл 2012) | 2 lines

Use M_NOWAIT while holding the pf giant lock.

Approved by: re (kib)

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

Update packet filter (pf) code to OpenBSD 4.5.

You need to update userland (world and ports) tools
to be in sync with the kernel.

Submitted by: mlaier
Submitted by: eri

Fix argument ordering to memcpy as well as the size of the copy in the
(theoretical) case that pfi_buffer_cnt should be greater than ~_max.

Submitted by: pjd
Reviewed by: {krw,sthen,markus}@openbsd.org
MFC after: 3 days

Merge the remainder of kern_vimage.c and vimage.h into vnet.c and
vnet.h, we now use jails (rather than vimages) as the abstraction
for virtualization management, and what remained was specific to
virtual network stacks. Minor cleanups are done in the process,
and comments updated to reflect these changes.

Reviewed by: bz
Approved by: re (vimage blanket)

Build on Jeff Roberson's linker-set based dynamic per-CPU allocator
(DPCPU), as suggested by Peter Wemm, and implement a new per-virtual
network stack memory allocator. Modify vnet to use the allocator
instead of monolithic global container structures (vinet, ...). This
change solves many binary compatibility problems associated with
VIMAGE, and restores ELF symbols for virtualized global variables.

Each virtualized global variable exists as a "reference copy", and also
once per virtual network stack. Virtualized global variables are
tagged at compile-time, placing the in a special linker set, which is
loaded into a contiguous region of kernel memory. Virtualized global
variables in the base kernel are linked as normal, but those in modules
are copied and relocated to a reserved portion of the kernel's vnet
region with the help of a the kernel linker.

Virtualized global variables exist in per-vnet memory set up when the
network stack instance is created, and are initialized statically from
the reference copy. Run-time access occurs via an accessor macro, which
converts from the current vnet and requested symbol to a per-vnet
address. When "options VIMAGE" is not compiled into the kernel, normal
global ELF symbols will be used instead and indirection is avoided.

This change restores static initialization for network stack global
variables, restores support for non-global symbols and types, eliminates
the need for many subsystem constructors, eliminates large per-subsystem
structures that caused many binary compatibility issues both for
monitoring applications (netstat) and kernel modules, removes the
per-function INIT_VNET_*() macros throughout the stack, eliminates the
need for vnet_symmap ksym(2) munging, and eliminates duplicate
definitions of virtualized globals under VIMAGE_GLOBALS.

Bump __FreeBSD_version and update UPDATING.

Portions submitted by: bz
Reviewed by: bz, zec
Discussed with: gnn, jamie, jeff, jhb, julian, sam
Suggested by: peter
Approved by: re (kensmith)

After cleaning up rt_tables from vnet.h and cleaning up opt_route.h
a lot of files no longer need route.h either. Garbage collect them.
While here remove now unneeded vnet.h #includes as well.

After r193232 rt_tables in vnet.h are no longer indirectly dependent on
the ROUTETABLES kernel option thus there is no need to include opt_route.h
anymore in all consumers of vnet.h and no longer depend on it for module
builds.

Remove the hidden include in flowtable.h as well and leave the two
explicit #includes in ip_input.c and ip_output.c.

For all files including net/vnet.h directly include opt_route.h and
net/route.h.

Remove the hidden include of opt_route.h and net/route.h from net/vnet.h.

We need to make sure that both opt_route.h and net/route.h are included
before net/vnet.h because of the way MRT figures out the number of FIBs
from the kernel option. If we do not, we end up with the default number
of 1 when including net/vnet.h and array sizes are wrong.

This does not change the list of files which depend on opt_route.h
but we can identify them now more easily.

Second round of putting global variables, which were virtualized
but formerly missed under VIMAGE_GLOBAL.

Put the extern declarations of the virtualized globals
under VIMAGE_GLOBAL as the globals themsevles are already.
This will help by the time when we are going to remove the globals
entirely.

Sponsored by: The FreeBSD Foundation

Rather than using hidden includes (with cicular dependencies),
directly include only the header files needed. This reduces the
unneeded spamming of various headers into lots of files.

For now, this leaves us with very few modules including vnet.h
and thus needing to depend on opt_route.h.

Reviewed by: brooks, gnn, des, zec, imp
Sponsored by: The FreeBSD Foundation

Step 1.5 of importing the network stack virtualization infrastructure
from the vimage project, as per plan established at devsummit 08/08:
http://wiki.freebsd.org/Image/Notes200808DevSummit

Introduce INIT_VNET_*() initializer macros, VNET_FOREACH() iterator
macros, and CURVNET_SET() context setting macros, all currently
resolving to NOPs.

Prepare for virtualization of selected SYSCTL objects by introducing a
family of SYSCTL_V_*() macros, currently resolving to their global
counterparts, i.e. SYSCTL_V_INT() == SYSCTL_INT().

Move selected #defines from sys/sys/vimage.h to newly introduced header
files specific to virtualized subsystems (sys/net/vnet.h,
sys/netinet/vinet.h etc.).

All the changes are verified to have zero functional impact at this
point in time by doing MD5 comparision between pre- and post-change
object files(*).

(*) netipsec/keysock.c did not validate depending on compile time options.

Implemented by: julian, bz, brooks, zec
Reviewed by: julian, bz, brooks, kris, rwatson, ...
Approved by: julian (mentor)
Obtained from: //depot/projects/vimage-commit2/...
X-MFC after: never
Sponsored by: NLnet Foundation, The FreeBSD Foundation

Commit step 1 of the vimage project, (network stack)
virtualization work done by Marko Zec (zec@).

This is the first in a series of commits over the course
of the next few weeks.

Mark all uses of global variables to be virtualized
with a V_ prefix.
Use macros to map them back to their global names for
now, so this is a NOP change only.

We hope to have caught at least 85-90% of what is needed
so we do not invalidate a lot of outstanding patches again.

Obtained from: //depot/projects/vimage-commit2/...
Reviewed by: brooks, des, ed, mav, julian,
jamie, kris, rwatson, zec, ...
(various people I forgot, different versions)
md5 (with a bit of help)
Sponsored by: NLnet Foundation, The FreeBSD Foundation
X-MFC after: never
V_Commit_Message_Reviewed_By: more people than the patch

Make ALTQ cope with disappearing interfaces (particularly common with mpd
and netgraph in gernal). This also allows to add queues for an interface
that is not yet existing (you have to provide the bandwidth for the
interface, however).

PR: kern/106400, kern/117827
MFC after: 2 weeks

Bring back pf_if.c revs 1.8 and 1.6 also lost during last import:
- Use correct time for tzero when compiled in
- Don't use bogus interface addresses on ptp-interfaces with :0

MFC after: 3 days

Cleanup pf interface mangement - esp. remove EVENTHANDLER before unloading
the coresponding code. This was lost during 4.1 import.

Reported by: ru
MFC after: 3 days

Properly drop the pf mutex around all copyout (consistency still protected
by the sx) and avoid a WITNESS panic. Overlooked during last import.

Reported and tested by: Max N. Boyarov
MFC after: 3 days

Commit resolved import of OpenBSD 4.1 pf from perforce.

Approved by: re (kensmith)

Resolve conflicts created during the import of pf 3.7 Some features are
missing and will be implemented in a second step. This is functional as is.

Tested by: freebsd-pf, pfsense.org
Obtained from: OpenBSD
X-MFC after: never (breaks API/ABI)

Fix a terrible braino in pfi_maybe_destroy() and unbreak "$pfctl -Fall" with
renamed interfaces.

PR: kern/77645
Reported by: Harald Schmalzbauer <harryNOschmalzbauerSPAMde>
Reviewed by: yongari
MFC after: 3 days

Fix inteface clear time. pf printed "Thu Jan 1 09:00:01 1970"
in "pfctl -vvsI" output when pf was statically linked to kernel.

Discussed with: mlaier

Improved interface handling. This should fix the use of renamed interfaces
(ifconfig xl0 name foo) as well as some special interfaces such as the 6to4
tunnel.

Reported by: Ed Schouten <ed (at) il ! fontys , nl>
Tested by: freebsd-pf
PR: kern/72444
MFC after: 3 weeks

Reactivate skipping over bogus IPCP addresses on ppp interfaces. Be more
careful with the skip condition this time. Addresses are only not taken into
account if:
- The interface is POINTTOPOINT
- There is no route installed for the address
- The user specified noalias (:0)
and - We are looking at an IPv4 address.

This should be enough paranoia to not cause any false positives.

PR: misc/69954
Discussed with: yongari
MFC after: 4 days

Comment out rev. 1.4 after problems w/ IPv6, a better solution must be
found.

Loopback fix from Henning Brauer:
skip over interface addresses without IFA_ROUTE, fixes some issue
with pppd

PR: misc/69954

Fix the following LOR on pf module unload:
1st ifnet (ifnet) @/usr/src/sys/contrib/pf/net/pf_if.c:191
2nd pf task mtx (pf task mtx) @/usr/src/sys/contrib/pf/net/pf_if.c:197

Reported by: Pyun YongHyeon (a long time ago)

Commit pf version 3.5 and link additional files to the kernel build.

Version 3.5 brings:
- Atomic commits of ruleset changes (reduce the chance of ending up in an
inconsistent state).
- A 30% reduction in the size of state table entries.
- Source-tracking (limit number of clients and states per client).
- Sticky-address (the flexibility of round-robin with the benefits of
source-hash).
- Significant improvements to interface handling.
- and many more ...

This commit was generated by cvs2svn to compensate for changes in r130610,
which included commits to RCS files with non-trunk default branches.

Import pf from OpenBSD 3.5 (OPENBSD_3_5_BASE)