Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r244090, r250945

Changes to resolve races in bpfread() and catchpacket() that, at worst,
cause kernel panics.

Add a flag to the bpf descriptor to indicate whether the hold buffer
is in use. In bpfread(), set the "hold buffer in use" flag before
dropping the descriptor lock during the call to bpf_uiomove().
Everywhere else the hold buffer is used or changed, wait while
the hold buffer is in use by bpfread(). Add a KASSERT in bpfread()
after re-acquiring the descriptor lock to assist uncovering any
additional hold buffer races.

While waiting for the bpf hold buffer to become idle, check
the return value from mtx_sleep() and exit bpfread() on
errors such as EINTR.

Merge
* r233937 - Improve BPF locking model
* r233938 - Improve performace for writer-only BPF users
* r233946 - Fix build
* r235744 - Fix (new) panic on attaching to non-existent interface
* r235745 - Fix old panic when BPF consumer attaches to destroying interface
* r235746 - Call bpf_jitter() before acquiring BPF global lock
* r235747 - Make most BPF ioctls() SMP-safe.
* r236231 - Fix BPF_JITTER code broken by r235746.
* r236251 - Fix shim for BIOCSETF to drop all packets buffered on the descriptor.
* r236261 - Save the previous filter right before we set new one.
* r236262 - Fix style(9) nits, reduce unnecessary type castings.
* r236559 - Fix panic introduced by r235745
* r236806 - Fix typo introduced in r236559.

r233937
- Improve BPF locking model.

Interface locks and descriptor locks are converted from mutex(9) to rwlock(9).
This greately improves performance: in most common case we need to acquire 1
reader lock instead of 2 mutexes.

- Remove filter(descriptor) (reader) lock in bpf_mtap[2]
This was suggested by glebius@. We protect filter by requesting interface
writer lock on filter change.

- Cover struct bpf_if under BPF_INTERNAL define. This permits including bpf.h
without including rwlock stuff. However, this is is temporary solution,
struct bpf_if should be made opaque for any external caller.

r233938
- Improve performace for writer-only BPF users.

Linux and Solaris (at least OpenSolaris) has PF_PACKET socket families to send
raw ethernet frames. The only FreeBSD interface that can be used to send raw
frames is BPF. As a result, many programs like cdpd, lldpd, various dhcp stuff
uses BPF only to send data. This leads us to the situation when software like
cdpd, being run on high-traffic-volume interface significantly reduces overall
performance since we have to acquire additional locks for every packet.

Here we add sysctl that changes BPF behavior in the following way:
If program came and opens BPF socket without explicitly specifyin read filter
we assume it to be write-only and add it to special writer-only per-interface
list. This makes bpf_peers_present() return 0, so no additional overhead is
introduced. After filter is supplied, descriptor is added to original
per-interface list permitting packets to be captured.

Unfortunately, pcap_open_live() sets catch-all filter itself for the purpose
of setting snap length.

Fortunately, most programs explicitly sets (event catch-all) filter after
that. tcpdump(1) is a good example.

So a bit hackis approach is taken: we upgrade description only after second
BIOCSETF is received.

Sysctl is named net.bpf.optimize_writers and is turned off by default.

- While here, document all sysctl variables in bpf.4

r233946
Fix build broken by r233938.

r235744
Fix panic on attaching to non-existent interface
(introduced by r233937, pointed by hrs@)
Fix panic on tcpdump being attached to interface being removed
(introduced by r233937, pointed by hrs@ and adrian@)
Protect most of bpf_setf() by BPF global lock

Add several forgotten assertions (thanks to adrian@)

Document current locking model inside bpf.c
Document EVENTHANDLER(9) usage inside BPF.

r235745
Fix old panic when BPF consumer attaches to destroying interface.
'flags' field is added to the end of bpf_if structure. Currently the only
flag is BPFIF_FLAG_DYING which is set on bpf detach and checked by bpf_attachd()
Problem can be easily triggered on SMP stable/[89] by the following command
(sort of):
'while true; do ifconfig vlan222 create vlan 222 vlandev em0 up ; \
tcpdump -pi vlan222 & ; ifconfig vlan222 destroy ; done'

Fix possible use-after-free when BPF detaches itself from interface, freeing
bpf_bif memory, while interface is still UP and there can be routes via this
interface. Freeing is now delayed till ifnet_departure_event is received via
eventhandler(9) api.

Convert bpfd rwlock back to mutex due lack of performance gain
(currently checking if packet matches filter is done without holding bpfd
lock and we have to acquire write lock if packet matches)

r235746
Call bpf_jitter() before acquiring BPF global lock due to malloc() being
used inside bpf_jitter.

Eliminate bpf_buffer_alloc() and allocate BPF buffers on descriptor creation
and BIOCSBLEN ioctl. This permits us not to allocate buffers inside
bpf_attachd() which is protected by global lock.

r235747
Make most BPF ioctls() SMP-safe.

r236559
Fix panic introduced by r235745. Panic occurs after first packet traverse
renamed interface.
Add several comments on locking

r236231
Fix BPF_JITTER code broken by r235746.

r236251
Fix 32-bit shim for BIOCSETF to drop all packets buffered on the descriptor
and reset statistics as it should.

r236261
- Save the previous filter right before we set new one.
- Reduce duplicate code and make it little easier to read.

r236262
Fix style(9) nits, reduce unnecessary type castings, etc., for bpf_setf().

r236806
Fix typo introduced in r236559.

MFC r241231:

MFV: libpcap 1.3.0.

MFC r235425:

Sync DLTs with the latest pcap version.

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

Sync DLTs with the latest pcap version.

Implement flexible BPF timestamping framework.

- Allow setting format, resolution and accuracy of BPF time stamps per
listener. Previously, we were only able to use microtime(9). Now we can
set various resolutions and accuracies with ioctl(2) BIOCSTSTAMP command.
Similarly, we can get the current resolution and accuracy with BIOCGTSTAMP
command. Document all supported options in bpf(4) and their uses.

- Introduce new time stamp 'struct bpf_ts' and header 'struct bpf_xhdr'.
The new time stamp has both 64-bit second and fractional parts. bpf_xhdr
has this time stamp instead of 'struct timeval' for bh_tstamp. The new
structures let us use bh_tstamp of same size on both 32-bit and 64-bit
platforms without adding additional shims for 32-bit binaries. On 64-bit
platforms, size of BPF header does not change compared to bpf_hdr as its
members are already all 64-bit long. On 32-bit platforms, the size may
increase by 8 bytes. For backward compatibility, struct bpf_hdr with
struct timeval is still the default header unless new time stamp format is
explicitly requested. However, the behaviour may change in the future and
all relevant code is wrapped around "#ifdef BURN_BRIDGES" for now.

- Add experimental support for tagging mbufs with time stamps from a lower
layer, e.g., device driver. Currently, mbuf_tags(9) is used to tag mbufs.
The time stamps must be uptime in 'struct bintime' format as binuptime(9)
and getbinuptime(9) do.

Reviewed by: net@

Sync DLTs with latest libpcap version.

Revert the previous commit to fix buildworld for now.

We have constified 'struct bpf_insn *' for bpf_filter(9) and bpf_validate(9)
since r1.19 but they conflict with pcap.h from libpcap.

Make sys/net/bpf_filter.c build cleanly on user land.

Add a new ioctl for changing the read filter (BIOCSETFNR). This is
just like BIOCSETF but it doesn't drop all the packets buffered on
the discriptor and reset the statistics.

Also, when setting the write filter, don't drop packets waiting to
be read or reset the statistics.

PR: 118486
Submitted by: Matthew Luckie <mluckie@cs.waikato.ac.nz>
MFC after: 1 month

Introduce support for zero-copy BPF buffering, which reduces the
overhead of packet capture by allowing a user process to directly "loan"
buffer memory to the kernel rather than using read(2) to explicitly copy
data from kernel address space.

The user process will issue new BPF ioctls to set the shared memory
buffer mode and provide pointers to buffers and their size. The kernel
then wires and maps the pages into kernel address space using sf_buf(9),
which on supporting architectures will use the direct map region. The
current "buffered" access mode remains the default, and support for
zero-copy buffers must, for the time being, be explicitly enabled using
a sysctl for the kernel to accept requests to use it.

The kernel and user process synchronize use of the buffers with atomic
operations, avoiding the need for system calls under load; the user
process may use select()/poll()/kqueue() to manage blocking while
waiting for network data if the user process is able to consume data
faster than the kernel generates it. Patchs to libpcap are available
to allow libpcap applications to transparently take advantage of this
support. Detailed information on the new API may be found in bpf(4),
including specific atomic operations and memory barriers required to
synchronize buffer use safely.

These changes modify the base BPF implementation to (roughly) abstrac
the current buffer model, allowing the new shared memory model to be
added, and add new monitoring statistics for netstat to print. The
implementation, with the exception of some monitoring hanges that break
the netstat monitoring ABI for BPF, will be MFC'd.

Zerocopy bpf buffers are still considered experimental are disabled
by default. To experiment with this new facility, adjust the
net.bpf.zerocopy_enable sysctl variable to 1.

Changes to libpcap will be made available as a patch for the time being,
and further refinements to the implementation are expected.

Sponsored by: Seccuris Inc.
In collaboration with: rwatson
Tested by: pwood, gallatin
MFC after: 4 months [1]

[1] Certain portions will probably not be MFCed, specifically things
that can break the monitoring ABI.

Remove trailing whitespace from lines in BPF.

MFC after: 3 days

Additions from libpcap 0.9.8 unbreak the build.

Pointy hat to: mlaier
X-MFC after: RELENG_7 buildworld

Add three new ioctl(2) commands for bpf(4).

- BIOCGDIRECTION and BIOCSDIRECTION get or set the setting determining
whether incoming, outgoing, or all packets on the interface should be
returned by BPF. Set to BPF_D_IN to see only incoming packets on the
interface. Set to BPF_D_INOUT to see packets originating locally and
remotely on the interface. Set to BPF_D_OUT to see only outgoing
packets on the interface. This setting is initialized to BPF_D_INOUT
by default. BIOCGSEESENT and BIOCSSEESENT are obsoleted by these but
kept for backward compatibility.

- BIOCFEEDBACK sets packet feedback mode. This allows injected packets
to be fed back as input to the interface when output via the interface is
successful. When BPF_D_INOUT direction is set, injected outgoing packet
is not returned by BPF to avoid duplication. This flag is initialized to
zero by default.

Note that libpcap has been modified to support BPF_D_OUT direction for
pcap_setdirection(3) and PCAP_D_OUT direction is functional now.

Reviewed by: rwatson

more juniper dlt's

MFC after: 1 month

If bpf(4) has not been compiled into the kernel, initialize the bpf interface
pointer to a zeroed, statically allocated bpf_if structure. This way the
LIST_EMPTY() macro will always return true. This allows us to remove the
additional unconditional memory reference for each packet in the fast path.

Discussed with: sam

Back out previous two commits, this caused some problems in the namespace
resulting in some build failures. Instead, to fix the problem of bpf not
being present, check the pointer before dereferencing it.

This is a temporary bandaid until we can decide on how we want to handle
the bpf code not being present. This will be fixed shortly.

Temporarily include files so that our macro checks do something useful.

Make sure we don't try to dereference the the if_bpf pointer when bpf has
not been compiled into the the kernel.

Submitted by: benno

Fix the following bpf(4) race condition which can result in a panic:

(1) bpf peer attaches to interface netif0
(2) Packet is received by netif0
(3) ifp->if_bpf pointer is checked and handed off to bpf
(4) bpf peer detaches from netif0 resulting in ifp->if_bpf being
initialized to NULL.
(5) ifp->if_bpf is dereferenced by bpf machinery
(6) Kaboom

This race condition likely explains the various different kernel panics
reported around sending SIGINT to tcpdump or dhclient processes. But really
this race can result in kernel panics anywhere you have frequent bpf attach
and detach operations with high packet per second load.

Summary of changes:

- Remove the bpf interface's "driverp" member
- When we attach bpf interfaces, we now set the ifp->if_bpf member to the
bpf interface structure. Once this is done, ifp->if_bpf should never be
NULL. [1]
- Introduce bpf_peers_present function, an inline operation which will do
a lockless read bpf peer list associated with the interface. It should
be noted that the bpf code will pickup the bpf_interface lock before adding
or removing bpf peers. This should serialize the access to the bpf descriptor
list, removing the race.
- Expose the bpf_if structure in bpf.h so that the bpf_peers_present function
can use it. This also removes the struct bpf_if; hack that was there.
- Adjust all consumers of the raw if_bpf structure to use bpf_peers_present

Now what happens is:

(1) Packet is received by netif0
(2) Check to see if bpf descriptor list is empty
(3) Pickup the bpf interface lock
(4) Hand packet off to process

From the attach/detach side:

(1) Pickup the bpf interface lock
(2) Add/remove from bpf descriptor list

Now that we are storing the bpf interface structure with the ifnet, there is
is no need to walk the bpf interface list to locate the correct bpf interface.
We now simply look up the interface, and initialize the pointer. This has a
nice side effect of changing a bpf interface attach operation from O(N) (where
N is the number of bpf interfaces), to O(1).

[1] From now on, we can no longer check ifp->if_bpf to tell us whether or
not we have any bpf peers that might be interested in receiving packets.

In collaboration with: sam@
MFC after: 1 month

Introduce two new ioctl(2) commands, BIOCLOCK and BIOCSETWF. These commands
enhance the security of bpf(4) by further relinquishing the privilege of
the bpf(4) consumer (assuming the ioctl commands are being implemented).

Once BIOCLOCK is executed, the device becomes locked which prevents the
execution of ioctl(2) commands which can change the underly parameters of the
bpf(4) device. An example might be the setting of bpf(4) filter programs or
attaching to different network interfaces.

BIOCSETWF can be used to set write filters for outgoing packets. Currently if
a bpf(4) consumer is compromised, the bpf(4) descriptor can essentially be used
as a raw socket, regardless of consumer's UID. Write filters give users the
ability to constrain which packets can be sent through the bpf(4) descriptor.

These features are currently implemented by a couple programs which came from
OpenBSD, such as the new dhclient and pflogd.

-Modify bpf_setf(9) to accept a "cmd" parameter. This will be used to specify
whether a read or write filter is to be set.
-Add a bpf(4) filter program as a parameter to bpf_movein(9) as we will run the
filter program on the mbuf data once we move the packet in from user-space.
-Rather than execute two uiomove operations, (one for the link header and the
other for the packet data), execute one and manually copy the linker header
into the sockaddr structure via bcopy.
-Restructure bpf_setf to compensate for write filters, as well as read.
-Adjust bpf(4) stats structures to include a bd_locked member.

It should be noted that the FreeBSD and OpenBSD implementations differ a bit in
the sense that we unconditionally enforce the lock, where OpenBSD enforces it
only if the calling credential is not root.

Idea from: OpenBSD
Reviewed by: mlaier

additions from libpcap 0.9.1 release

Approved by: re (scottl)

integrate changes from libpcap-0.9.1-096

Reviewed by: bms

/* -> /*- for license, minor formatting changes

Make the comment for DLT_NULL slightly more accurate.

PR: 62272
Submitted by: Radim Kolar <hsn@netmag.cz>
MFC after: 1 week

Remove advertising clause from University of California Regent's
license, per letter dated July 22, 1999 and email from Peter Wemm,
Alan Cox and Robert Watson.

Approved by: core, peter, alc, rwatson

Add more DLT types required by libpcap 0.8.3.
Maintain numeric sort order.

Update system bpf headers for libpcap 0.8.3.
Maintain listing of DLT link types in numeric order.

Tweak existing header and other build infrastructure to be able to build
pf/pflog/pfsync as modules. Do not list them in NOTES or modules/Makefile
(i.e. do not connect it to any (automatic) builds - yet).

Approved by: bms(mentor)

o eliminate widespread on-stack mbuf use for bpf by introducing
a new bpf_mtap2 routine that does the right thing for an mbuf
and a variable-length chunk of data that should be prepended.
o while we're sweeping the drivers, use u_int32_t uniformly when
when prepending the address family (several places were assuming
sizeof(int) was 4)
o return M_ASSERTVALID to BPF_MTAP* now that all stack-allocated
mbufs have been eliminated; this may better be moved to the bpf
routines

Reviewed by: arch@ and several others

Remove the call to M_ASSERTVALID from BPF_MTAP; some mbufs passed to
mpf are allocated on the stack, which causes this check to falsely trigger.

A new check which takes on-stack mbufs into account will be reintroduced
after 5.2 is out the door.

Approved by: re (watson)
Requested by: many

Add a new macro M_ASSERTVALID which ensures that the mbuf in question
is non-free. (More checks can/should be added in the future.)

Use M_ASSERTVALID in BPF_MTAP so that we catch when freed mbufs are
passed in, even if no bpf listeners are active.

Inspired by a bug in if_dc caught by Kenjiro Cho.

o add BIOCGDLTLIST and BIOCSDLT ioctls to get the data link type list
and set the link type for use by libpcap and tcpdump
o move mtx unlock in bpfdetach up; it doesn't need to be held so long
o change printf in bpf_detach to distinguish it from the same one in bpfsetdlt

Note there are locking issues here related to ioctl processing; they
have not been addressed here.

Submitted by: Guy Harris <guy@alum.mit.edu>
Obtained from: NetBSD (w/ locking modifications)

o add support for multiple link types per interface (e.g. 802.11 and Ethernet)
o introduce BPF_TAP and BPF_MTAP macros to hide implementation details and
ease code portability
o use m_getcl where appropriate

Reviewed by: many
Approved by: re
Obtained from: NetBSD (multiple link type support)

Update for libpcap 0.7.1

Originally-committed-to-wrong-repository by: fenner

Remove __P.

Update our bpf.h with tcpdump.org's new DLT_ types.
Use our bpf.h instead of tcpdump.org's to build libpcap.

The advent of if_detach, allowing interface removal at runtime, makes it
possible for a panic to occur if BPF is in use on the interface at the
time of the call to if_detach. This happens because BPF maintains pointers
to the struct ifnet describing the interface, which is freed by if_detach.

To correct this problem, a new call, bpfdetach, is introduced. bpfdetach
locates BPF descriptor references to the interface, and NULLs them. Other
BPF code is modified so that discovery of a NULL interface results in
ENXIO (already implemented for some calls). Processes blocked on a BPF
call will also be woken up so that they can receive ENXIO.

Interface drivers that invoke bpfattach and if_detach must be modified to
also call bpfattach(ifp) before calling if_detach(ifp). This is relevant
for buses that support hot removal, such as pccard and usb. Patches to
all effected devices will not be committed, only to if_wi.c, due to
testing limitations. To reproduce the crash, load up tcpdump on you
favorite pccard ethernet card, and then eject the card. As some pccard
drivers do not invoke if_detach(ifp), this bug will not manifest itself
for those drivers.

Reviewed by: wes

Introduce a new bd_seesent flag to the BPF descriptor, indicating whether or
not the current BPF device should report locally generated packets or not.
This allows sniffing applications to see only packets that are not generated
locally, which can be useful for debugging bridging problems, or other
situations where MAC addresses are not sufficient to identify locally
sourced packets. Default to true for this flag, so as to provide existing
behavior by default.

Introduce two new ioctls, BIOCGSEESENT and BIOCSSEESENT, which may be used
to manipulate this flag from userland, given appropriate privilege.

Modify bpf.4 to document these two new ioctl arguments.

Reviewed by: asmodai

|The hard limit for the BPF buffer size is 32KB, which appears too low
|for high speed networks (even at 100Mbit/s this corresponds to 1/300th
|of a second). The default buffer size is 4KB, but libpcap and ipfilter
|both override this (using the BIOCSBLEN ioctl) and allocate 32KB.
|
|The following patch adds an sysctl for bpf_maxbufsize, similar to the
|one for bpf_bufsize that you added back in December 1995. I choose to
|make the default for this limit 512KB (the value suggested by NFR).

Submitted by: se
Reviewed by: phk

Change #ifdef KERNEL to #ifdef _KERNEL in the public headers. "KERNEL"
is an application space macro and the applications are supposed to be free
to use it as they please (but cannot). This is consistant with the other
BSD's who made this change quite some time ago. More commits to come.

Add 'const' to the bpf_filter() and bpf_validate() prototypes.
Remove a stale comment from bpf_validate().

Implement pseudo_AF_HDRCMPLT, which controls the state of the 'header
completion' flag. If set, the interface output routine will assume that
the packet already has a valid link-level source address. This defaults
to off (the address is overwritten)

PR: kern/10680
Submitted by: "Christopher N . Harrell" <cnh@mindspring.net>
Obtained from: NetBSD

$Id$ -> $FreeBSD$

Change BPF_ALIGNMENT to long, necessary for correct alignment on Alpha.

Add DLT_{SLIP,PPP}_BSDOS from libpcap 0.4

Implement DLT_RAW from libpcap

Don't attempt to optimize the space allocated for bpf headers if
sizeof(struct bpf_hdr) > 20. 20 is normal on 32-bit systems with
32-bit alignment, but we still assume that the last 2 bytes of the
struct are unnecessary padding on such systems. On systems with
64-bit longs, struct timeval is bloated to 16 bytes, so bpf headers
certainly don't fit in 18 bytes.

Back out part 1 of the MCFH that changed $Id$ to $FreeBSD$. We are not
ready for it yet.

Make the long-awaited change from $Id$ to $FreeBSD$

This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.

Update to match definitions in LBL June 96 release

Clean up Ethernet drivers:
- fill in and use ifp->if_softc
- use if_bpf rather than private cookie variables
- change bpf interface to take advantage of this
- call ether_ifattach() directly from Ethernet drivers
- delete kludge in if_attach() that did this indirectly

Fix a bunch of spelling errors in the comment fields of
a bunch of system include files.

Moved prototypes for devswitch functions from conf.c and driver sources
to <machine/conf.h>. conf.h was mechanically generated by
`grep ^d_ conf.c >conf.h'. This accounts for part of its ugliness. The
prototypes should be moved back to the driver sources when the functions
are staticalized.

Fix benign type mismatches in devsw functions. 82 out of 299 devsw
functions were wrong.

Give the BPF the ability to generate signals when a packet is available.

Reviewed by: pst & wollman
Submitted by: grossman@cygnus.com

Remove trailing whitespace.

Make idempotent.

Submitted by: Paul

Added $Id$

This commit was generated by cvs2svn to compensate for changes in r1541,
which included commits to RCS files with non-trunk default branches.

BSD 4.4 Lite Kernel Sources