- Copy stable/10 (r259064) to releng/10.0 as part of the
10.0-RELEASE cycle.
- Update __FreeBSD_version [1]
- Set branch name to -RC1

[1] 10.0-CURRENT __FreeBSD_version value ended at '55', so
start releng/10.0 at '100' so the branch is started with
a value ending in zero.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

ng_ether_ifnet_arrival_event: check interface type before using IFP2NG

The check is copied from vnet_ng_ether_init.
Not sure if it covers all the types that we want to support with
ng_ether.

Reported by: markj
Discussed with: zec
MFC after: 10 days
X-MFC with: r246245

ng_ether: track interface renaming

Also sanitize interface names that can potentially contain characters
that are prohibited in netgraph names.

PR: kern/154850 (sanitizing of names)
Discussed with: eri, melifaro
Submitted by: Nikolay Denev <ndenev@gmail.com> (sanitizing code)
Reviewed by: eri, glebius
MFC after: 17 days

Mechanically remove the last stray remains of spl* calls from net*/*.
They have been Noop's for a long time now.

Add version so others can depend on this module

remove RESTARTABLE_PANICS option

This is done per request/suggestion from John Baldwin
who introduced the option. Trying to resume normal
system operation after a panic is very unpredictable
and dangerous. It will become even more dangerous
when we allow a thread in panic(9) to penetrate all
lock contexts.
I understand that the only purpose of this option was
for testing scenarios potentially resulting in panic.

Suggested by: jhb
Reviewed by: attilio, jhb
X-MFC-After: never
Approved by: re (kib)

Clear pending ifnet events, in an attempt at preventing
ng_ether_link_state() from being dispatched after we have
cleared our IFP2NG(ifp).

MFC after: 3 days

Be consistent with r160968: keep autoSrcAddr flag untouched when
node receives NGM_SHUTDOWN.

Submitted by: pluknet

Declare a new EVENTHANDLER called iflladdr_event which signals that the L2
address on an interface has changed. This lets stacked interfaces such as
vlan(4) detect that their lower interface has changed and adjust things in
order to keep working. Previously this situation broke at least vlan(4) and
lagg(4) configurations.

The EVENTHANDLER_INVOKE call was not placed within if_setlladdr() due to the
risk of a loop.

PR: kern/142927
Submitted by: Nikolay Denev

Send link state change control messages to "orphans" hook as well.

MFC after: 1 week

Merge the remainder of kern_vimage.c and vimage.h into vnet.c and
vnet.h, we now use jails (rather than vimages) as the abstraction
for virtualization management, and what remained was specific to
virtual network stacks. Minor cleanups are done in the process,
and comments updated to reflect these changes.

Reviewed by: bz
Approved by: re (vimage blanket)

Introduce and use a sysinit-based initialization scheme for virtual
network stacks, VNET_SYSINIT:

- Add VNET_SYSINIT and VNET_SYSUNINIT macros to declare events that will
occur each time a network stack is instantiated and destroyed. In the
!VIMAGE case, these are simply mapped into regular SYSINIT/SYSUNINIT.
For the VIMAGE case, we instead use SYSINIT's to track their order and
properties on registration, using them for each vnet when created/
destroyed, or immediately on module load for already-started vnets.
- Remove vnet_modinfo mechanism that existed to serve this purpose
previously, as well as its dependency scheme: we now just use the
SYSINIT ordering scheme.
- Implement VNET_DOMAIN_SET() to allow protocol domains to declare that
they want init functions to be called for each virtual network stack
rather than just once at boot, compiling down to DOMAIN_SET() in the
non-VIMAGE case.
- Walk all virtualized kernel subsystems and make use of these instead
of modinfo or DOMAIN_SET() for init/uninit events. In some cases,
convert modular components from using modevent to using sysinit (where
appropriate). In some cases, do minor rejuggling of SYSINIT ordering
to make room for or better manage events.

Portions submitted by: jhb (VNET_SYSINIT), bz (cleanup)
Discussed with: jhb, bz, julian, zec
Reviewed by: bz
Approved by: re (VIMAGE blanket)

Build on Jeff Roberson's linker-set based dynamic per-CPU allocator
(DPCPU), as suggested by Peter Wemm, and implement a new per-virtual
network stack memory allocator. Modify vnet to use the allocator
instead of monolithic global container structures (vinet, ...). This
change solves many binary compatibility problems associated with
VIMAGE, and restores ELF symbols for virtualized global variables.

Each virtualized global variable exists as a "reference copy", and also
once per virtual network stack. Virtualized global variables are
tagged at compile-time, placing the in a special linker set, which is
loaded into a contiguous region of kernel memory. Virtualized global
variables in the base kernel are linked as normal, but those in modules
are copied and relocated to a reserved portion of the kernel's vnet
region with the help of a the kernel linker.

Virtualized global variables exist in per-vnet memory set up when the
network stack instance is created, and are initialized statically from
the reference copy. Run-time access occurs via an accessor macro, which
converts from the current vnet and requested symbol to a per-vnet
address. When "options VIMAGE" is not compiled into the kernel, normal
global ELF symbols will be used instead and indirection is avoided.

This change restores static initialization for network stack global
variables, restores support for non-global symbols and types, eliminates
the need for many subsystem constructors, eliminates large per-subsystem
structures that caused many binary compatibility issues both for
monitoring applications (netstat) and kernel modules, removes the
per-function INIT_VNET_*() macros throughout the stack, eliminates the
need for vnet_symmap ksym(2) munging, and eliminates duplicate
definitions of virtualized globals under VIMAGE_GLOBALS.

Bump __FreeBSD_version and update UPDATING.

Portions submitted by: bz
Reviewed by: bz, zec
Discussed with: gnn, jamie, jeff, jhb, julian, sam
Suggested by: peter
Approved by: re (kensmith)

Use if_maddr_rlock()/if_maddr_runlock() rather than IF_ADDR_LOCK()/
IF_ADDR_UNLOCK() across network device drivers when accessing the
per-interface multicast address list, if_multiaddrs. This will
allow us to change the locking strategy without affecting our driver
programming interface or binary interface.

For two wireless drivers, remove unnecessary locking, since they
don't actually access the multicast address list.

Approved by: re (kib)
MFC after: 6 weeks

After cleaning up rt_tables from vnet.h and cleaning up opt_route.h
a lot of files no longer need route.h either. Garbage collect them.
While here remove now unneeded vnet.h #includes as well.

Mark ng_ether node hooks as HI_STACK. It is usually the last point when
netgraph may unroll the call stack, and I have found that in some cases 2K
guarantied there for i386 may be not enough for NIC driver and BPF.

Introduce a mechanism for detecting calls from outbound path of the
network stack when reentering the inbound path from netgraph, and
force queueing of mbufs at the outbound netgraph node.

The mechanism relies on two components. First, in netgraph nodes
where outbound path of the network stack calls into netgraph, the
current thread has to be appropriately marked using the new
NG_OUTBOUND_THREAD_REF() macro before proceeding to call further
into the netgraph topology, and unmarked using the
NG_OUTBOUND_THREAD_UNREF() macro before returning to the caller.
Second, netgraph nodes which can potentially reenter the network
stack in the inbound path have to mark their inbound hooks using
NG_HOOK_SET_TO_INBOUND() macro. The netgraph framework will then
detect when there is a danger of a call graph looping back from
outbound to inbound path via netgraph, and defer handing off the
mbufs to the "inbound" node to a worker thread with a clean stack.

In this first pass only the most obvious netgraph nodes have been
updated to ensure no outbound to inbound calls can occur. Nodes
such as ng_ipfw, ng_gif etc. should be further examined whether a
potential for outbound to inbound call looping exists.

This commit changes the layout of struct thread, but due to
__FreeBSD_version number shortage a version bump has been omitted
at this time, nevertheless kernel and modules have to be rebuilt.

Reviewed by: julian, rwatson, bz
Approved by: julian (mentor)

After r193232 rt_tables in vnet.h are no longer indirectly dependent on
the ROUTETABLES kernel option thus there is no need to include opt_route.h
anymore in all consumers of vnet.h and no longer depend on it for module
builds.

Remove the hidden include in flowtable.h as well and leave the two
explicit #includes in ip_input.c and ip_output.c.

In preparation to make options VIMAGE operational, where needed,
initialize / release netgraph related state in iattach() / idetach()
functions called via the vnet module registration / initialization
framework, instead of initialization / cleanups being done in
mod_event handlers.

While here, introduce a crude hack aimed at preventing ng_ether to
autoattach to ng_eiface ifnets, which are also netgraph nodes already.

Reviewed by: bz
Approved by: julian (mentor)

For all files including net/vnet.h directly include opt_route.h and
net/route.h.

Remove the hidden include of opt_route.h and net/route.h from net/vnet.h.

We need to make sure that both opt_route.h and net/route.h are included
before net/vnet.h because of the way MRT figures out the number of FIBs
from the kernel option. If we do not, we end up with the default number
of 1 when including net/vnet.h and array sizes are wrong.

This does not change the list of files which depend on opt_route.h
but we can identify them now more easily.

shave about 7% off the overhead of ng_ether by using per-hook
receive data methods.

Rather than using hidden includes (with cicular dependencies),
directly include only the header files needed. This reduces the
unneeded spamming of various headers into lots of files.

For now, this leaves us with very few modules including vnet.h
and thus needing to depend on opt_route.h.

Reviewed by: brooks, gnn, des, zec, imp
Sponsored by: The FreeBSD Foundation

Retire the MALLOC and FREE macros. They are an abomination unto style(9).

MFC after: 3 months

Commit step 1 of the vimage project, (network stack)
virtualization work done by Marko Zec (zec@).

This is the first in a series of commits over the course
of the next few weeks.

Mark all uses of global variables to be virtualized
with a V_ prefix.
Use macros to map them back to their global names for
now, so this is a NOP change only.

We hope to have caught at least 85-90% of what is needed
so we do not invalidate a lot of outstanding patches again.

Obtained from: //depot/projects/vimage-commit2/...
Reviewed by: brooks, des, ed, mav, julian,
jamie, kris, rwatson, zec, ...
(various people I forgot, different versions)
md5 (with a bit of help)
Sponsored by: NLnet Foundation, The FreeBSD Foundation
X-MFC after: never
V_Commit_Message_Reviewed_By: more people than the patch

Implement reference counting for ifmultiaddr, in_multi, and in6_multi
structures. Detect when ifnet instances are detached from the network
stack and perform appropriate cleanup to prevent memory leaks.

This has been implemented in such a way as to be backwards ABI compatible.
Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti()
is unable to detect interface removal by design, as it performs searches
on structures which are removed with the interface.

With this architectural change, the panics FreeBSD users have experienced
with carp and pfsync should be resolved.

Obtained from: p4 branch bms_netdev
Reviewed by: andre
Sponsored by: Garance A Drosehn
Idea from: NetBSD
MFC after: 1 month

Turn off by default "feature" that overwrites MAC address
on output frames.

Many people were confused with not working CARP, ng_bridge(4)
and other subsystems, because ng_ether(4) overwritten source
MAC address.

Remove the dependency of bridgestp.h on if_bridgevar.h by moving a couple of
private structures to if_bridge.c.

Fixing compilation bustage: net/if_bridgevar.h depends on net/bridgestp.h.

Do not force queueing on peer hooks. This was important only for
5.0-CURRENT. And it looks like this didn't work before Julian's
revamp of netgraph queue code.

Reviewed by: julian

- Store pointer to the link-level address right in "struct ifnet"
rather than in ifindex_table[]; all (except one) accesses are
through ifp anyway. IF_LLADDR() works faster, and all (except
one) ifaddr_byindex() users were converted to use ifp->if_addr.

- Stop storing a (pointer to) Ethernet address in "struct arpcom",
and drop the IFP2ENADDR() macro; all users have been converted
to use IF_LLADDR() instead.

Make IFP2NG() usable as an lvalue.

Restore part of the code dropped in rev. 1.25 that makes sure ether_demux()
will receive an mbuf chain with Ethernet header in the data portion of the
first mbuf.

Further clean up the bridge hooks in if_ethersubr.c and ng_ether.c

- move the function pointer definitions to if_bridgevar.h
- move most of the logic to the new BRIDGE_INPUT and BRIDGE_OUTPUT macros
- remove unneeded functions from if_bridgevar.h and sort a little.

Clean up the if_bridge hooks a bit in if_ethersubr.c and ng_ether.c, move
the broadcast/multicast test to bridge_input().

Requested by: glebius

A deja vu of:

http://lists.freebsd.org/pipermail/cvs-src/2004-October/033496.html

The same problem applies to if_bridge(4), too.

- Copy-and-paste the if_bridge(4) related block from
if_ethersubr.c to ng_ether.c
- Add XXXs, so that copy-and-paste would be noticed by
any future editors of this code.
- Also add XXXs near if_bridge(4) declarations.

Silence from: thompsa

Remove bridge(4) from the tree. if_bridge(4) is a full functional
replacement and has additional features which make it superior.

Discussed on: -arch
Reviewed by: thompsa
X-MFC-after: never (RELENG_6 as transition period)

Propagate rename of IFF_OACTIVE and IFF_RUNNING to IFF_DRV_OACTIVE and
IFF_DRV_RUNNING, as well as the move from ifnet.if_flags to
ifnet.if_drv_flags. Device drivers are now responsible for
synchronizing access to these flags, as they are in if_drv_flags. This
helps prevent races between the network stack and device driver in
maintaining the interface flags field.

Many __FreeBSD__ and __FreeBSD_version checks maintained and continued;
some less so.

Reviewed by: pjd, bz
MFC after: 7 days

Stop embedding struct ifnet at the top of driver softcs. Instead the
struct ifnet or the layer 2 common structure it was embedded in have
been replaced with a struct ifnet pointer to be filled by a call to the
new function, if_alloc(). The layer 2 common structure is also allocated
via if_alloc() based on the interface type. It is hung off the new
struct ifnet member, if_l2com.

This change removes the size of these structures from the kernel ABI and
will allow us to better manage them as interfaces come and go.

Other changes of note:
- Struct arpcom is no longer referenced in normal interface code.
Instead the Ethernet address is accessed via the IFP2ENADDR() macro.
To enforce this ac_enaddr has been renamed to _ac_enaddr.
- The second argument to ether_ifattach is now always the mac address
from driver private storage rather than sometimes being ac_enaddr.

Reviewed by: sobomax, sam

Add new netgraph control message NGM_ETHER_DETACH, which actually
removes netgraph node and unwraps Ethernet interface.

This gives us ability to unload ng_ether.ko, when all interfaces
are detached, making ng_ether(4) developers happy.

Reviewed by: ru

- bzero sockaddr_dl
- use constant instead of number

Suggested by: ru

Add two new netgraph messages NGM_ETHER_ADD_MULTI and NGM_ETHER_DEL_MULTI,
to join and leave Ethernet multicast membership, respectively. Messages
take MAC address as argument.

Sponsored by: Rinet ISP

Removed redundant MODULE_VERSION().

Fixed an off-by-one error when dealing with interface name
(if_xname is IFNAMSIZ-sized and NUL-terminated).

This change adds reliability for Ethernet trunks built with ng_one2many:

- Introduce another ng_ether(4) callback ng_ether_link_state_p, which
is called from if_link_state_change(), every time link is changed.
- In ng_ether_link_state() send netgraph control message notifying
of link state change to a node connected to "lower" hook.

Reviewed by: sam
MFC after: 2 weeks

/* -> /*- for license, minor formatting changes

Remove a check that never returns true, because in this case we have panic
before the check.

Fix packet flow when both ng_ether(4) and bridge(4) are in use:

- push all bridge logic from if_ethersubr.c into bridge.c
make bridge_in() return mbuf pointer (or NULL).
- call only bridge_in() from ether_input(), after ng_ether_input()
was optinally called.
- call bridge_in() from ng_ether_rcv_upper().

Long description: http://lists.freebsd.org/mailman/htdig/freebsd-net/2004-May/003881.html
Reported by: Jian-Wei Wang <jwwang at FreeBSD.csie.NCTU.edu.tw>
Tested by: myself, Sergey Lyubka
Reviewed by: sam
Approved by: julian (mentor)
MFC after: 2 months

Return 0, not NULL, from a function declared as returning int.

Avoid casts as lvalues.

Slight cosmetic changes.
Also introduce a macro to be called by persistent nodes to signal their
persistence during shutdown to hide this mechanism from the node author.

Make node flags have a consistent style in naming.

Document the change.

Having moved metadata usage to mbuf tags, remove code that supports
the old way of doing it.

Submitted by: Gleb Smirnoff <glebius@cell.sick.ru>

Switch to using C99 sparse initialisers for the type methods array.
Should make no binary difference.

Submitted by: Gleb Smirnoff <glebius@cell.sick.ru>
Reviewed by: Harti Brandt <harti@freebsd.org>
MFC after: 1 week

Allow ng_ether "lower" and "orphans" hooks to be connected at the same time.

Reviewed by: julian
PR: kern/63317

+ rename and document an unused field in struct arpcom (field is still
there so there are no ABI changes);
+ replace 5 redefinitions of the IPF2AC macro with one in if_arp.h

Eventually (but before freezing the ABI) we need to get rid of
struct arpcom (initially with the help of some smart #defines
to avoid having to touch each and every driver, see below).

Apart from the struct ifnet, struct arpcom now only stores a copy
of the MAC address (ac_enaddr, but we already have another copy in
the struct ifnet -- if_addrhead), and a netgraph-specific field
which is _always_ accessed through the ifp, so it might well go
into the struct ifnet too (where, besides, there is already an entry
for AF_NETGRAPH data...)

Too bad ac_enaddr is widely referenced by all drivers. But
this can be fixed as follows:

#define ac_enaddr ac_if.the_original_ac_enaddr_in_struct_ifnet

(note that the right hand side would likely be a pointer rather than
the base address of an array.)

Backout previous change, it breaks build and it is not needed
layering violation. As pointed out, there is much better way to do this.
Sorry guys, I need to find a better way to force reviews.

Requested by: harti, julian, scottl (mentor)
Pointy hat to: pjd

Export private structure owned by ng_ether(4) module outside.
It'll is required by new failure detection algorithm for ng_one2many(4).

No response from: julian, archie
Silent on: net@
Approved by: scottl (mentor)

Also, don't crash in the netgraph disconnect node if the interface is
detached from the other direction.

Don't try to ifpromisc() on an interface that's deleted already.

Made the Ethernet address parse type standard.

OK'ed by: archie

Replace the if_name and if_unit members of struct ifnet with new members
if_xname, if_dname, and if_dunit. if_xname is the name of the interface
and if_dname/unit are the driver name and instance.

This change paves the way for interface renaming and enhanced pseudo
device creation and configuration symantics.

Approved By: re (in principle)
Reviewed By: njl, imp
Tested On: i386, amd64, sparc64
Obtained From: NetBSD (if_xname)

SMP locking for ifnet list.

o track changes to ethernet input packet handling
o track changes to bpf
o track changes to make ng hooks more private

Reviewed by: many
Approved by: re

Fix bug where an mbuf was being written to without checking M_WRITABLE().
Eliminate some of the unnecessary complexity of ng_ether_glueback_header().
Simplify two functions a bit by doing the NG_FREE_META(meta) earlier.

Reviewed by: julian, brian
MFC after: 1 week

Don't send packets out an interface unless it is IFF_UP|IFF_RUNNING.
This fixes panics with certain Ethernet drivers when doing bridging,
PPPoE, etc. before the driver has been fully brought up.

MFC after: 1 week

The hardware checksum code makes the assumption that a packet routed out
a particular Ethernet interface will actually be delivered by (only) that
device driver. This is not necessarily true when ng_ether(4) is used.

To word around this, while a ng_ether(4)'s "upper" hook is connected,
turn off all hardware checksum, fragmentation, etc., features for that
interface.

PR: kern/31586
MFC after: 1 week

Update to C99, s/__FUNCTION__/__func__/,
also don't use ANSI string concatenation.

KSE Milestone 2
Note ALL MODULES MUST BE RECOMPILED
make the kernel aware that there are smaller units of scheduling than the
process. (but only allow one thread per process at this time).
This is functionally equivalent to teh previousl -current except
that there is a thread associated with each process.

Sorry john! (your next MFC will be a doosie!)

Reviewed by: peter@freebsd.org, dillon@freebsd.org

X-MFC after: ha ha ha ha

Fix kernel crash when VLANs are combined with ng_ether(4), by attaching
interfaces of type IFT_L2VLAN as well as IFT_ETHER during module load.

Submitted by: yar

Implement direct support for semipersistant nodes.
(e.g. ethernet nodes are persistent until you rip out the hardware)
Use this support in the ethernet and sample nodes.
Add some more abstraction on the 'item's so that node and
hook reference counting can be checked easier.
Slight man page correction.
Make pppoe type dependent on ethernet type.
Clean up node shutdown a little.
Move a mutex from MTX_SPIN to MTX_DEF (oops)
Fix small ref-counting bug.
remove warning on one2many type.

Add a generic "queued function callin" mechanism
Use it to implement (hopefully) SMP safe node/hook addition
and removal.
Fix some debug stuff.

Part 2 of the netgraph rewrite.
This is mostly cosmetic changes, (though I caught a bug or two while
makeing them)
Reviewed by: archie@freebsd.org

Rewrite of netgraph to start getting ready for SMP.
This version is functional and is aproaching solid..
notice I said APROACHING. There are many node types I cannot test
I have tested: echo hole ppp socket vjc iface tee bpf async tty
The rest compile and "Look" right. More changes to follow.
DEBUGGING is enabled in this code to help if people have problems.

Divorce the kernel binary ABI version number from the message
format version number. (userland programs should not need to be
recompiled when the netgraph kernel internal ABI is changed.

Also fix modules that don;t handle the fact that a caller may not supply
a return message pointer. (benign at the moment because the calling code
checks, but that will change)

Reviewed by: Archie@freebsd.org
This clears out my outstanding netgraph changes.
There is a netgraph change of design in the offing and this is to some
extent a superset of soem of the new functionality and some of the old
functionality that may be removed.

This code works as before, but allows some new features that I want to
work with and evaluate. It is the basis for a version of netgraph
with integral locking for SMP use.

This is running on my test machine with no new problems :-)

Fix a bug where if the interface was in promiscuous mode when the
last hook was disconnected, the interface would not get reset to
non-promiscuous mode.

Reported by: jdp

Add the use of M_ZERO to netgraph.

Submitted by: josh@zipperup.org
Submitted by: Robert Drehmel <robd@gmx.net>
Submitted by: archie
Approved by: archie

When sending a packet back to a network interface to simulate an arrived
packet, make sure that the packet has the interface marked in the first mbuf,
the same a truely arrived packets would have.

Fix wrong offset bug in ng_enaddr_unparse().

Add three more control messages to complement their opposites:
NGM_ETHER_SET_ENADDR, NGM_ETHER_GET_PROMISC, and NGM_ETHER_GET_AUTOSRC.
Alter parsing algorithm so the EN address really looks like one.

Take advantage of the new unsigned and hex integer types.

Add three new control messages to the ng_ether(4) netgraph node type:

NGM_ETHER_GET_ENADDR: Get the device's Ethernet address
NGM_ETHER_SET_PROMISC: Enable/disable promiscuous mode
NGM_ETHER_SET_AUTOSRC: Enable/disable packet source address override

Remove node's name reference when the interface is detached;
otherwise, the ng_ether.ko KLD will never be unloadable after
all Ethernet interfaces are detached, as it should be, because
of the lingering extra reference.

Submitted by: "Yevmenkin, Maksim N, CSCIO" <myevmenkin@att.com>

Set NG_INVALID flag when destroying node.

Fix race condition caused by using NG_SEND_DATAQ() where we meant
to use ng_queue_data().

Reported by: Udo Erdelhoff <ue@nathan.ruhr.de>

Don't forget to set our MAC address into packets we wre sending out via
netgraph. Eventually we may need to have a separate hook for packets
that already have a source AMC address but for now just drop it in.
Should fix PPPoE.

Make the ng_ether(4) node type dynamically loadable like the rest.
This means 'options NETGRAPH' is no longer necessary in order to get
netgraph-enabled Ethernet interfaces. This supports loading/unloading
the ng_ether.ko and attaching/detaching the Ethernet interface in any
order.

Add two new hooks 'upper' and 'lower' to allow access to the protocol
demux engine and the raw device, respectively. This enables bridging
to be defined as a netgraph node, if so desired.

Reviewed by: freebsd-net@freebsd.org