remove unneeded includes
ok miod@ mpi@

Fix lock order reversal in nfs_inactive()

Make the silly file removal happen after the vnode has been unlocked.
This avoids a file-directory reversal in the vnode locking order.

OK jca@

revert vnode: remove VLOCKSWORK and check locking when vop_islocked != nullop
(both kernel and userland bits)

GENERIC + VFSLCKDEBUG is broken with it.

vnode: remove VLOCKSWORK and check locking when vop_islocked != nullop

This flag is currently used to mark or unmark a vnode to actively
check vnode locking semantic (when compiled with VFSLCKDEBUG).

Currently, VLOCKSWORK flag isn't properly set for several FS
implementation which have full locking support. This commit enable
proper checking for them too (cd9660, udf, fuse, msdosfs, tmpfs).

Instead of using a particular flag, it directly check if
v_op->vop_islocked is nullop or not to activate or not the vnode
locking checks.

ok mpi@

vnode: do not manipulate vnode lock directly

use VOP_LOCK / VOP_UNLOCK wrappers.

VOP_LOCK() is prefered over vn_lock() here in order to keep equivalent code.

ok mpi@ visa@ (as part of larger diff)

struct vops is not modified during runtime so use const which moves each
into read-only data segment.
OK deraadt@ tedu@

Drop unnecessary `p' parameter from vget(9).

OK mpi@

Implement proper locking for NFS nodes.

Tested in bulks by many. ok visa@, beck@

Clean up the parameters of VOP_LOCK() and VOP_UNLOCK(). It is always
curproc that does the locking or unlocking, so the proc parameter
is pointless and can be dropped.

OK mpi@, deraadt@

Change the representation of an NFS mount point by caching the root
nodes.

nfs_root() now returns a "locked" vnode, so vput(9) must be called to
release it. Note that this has currently no effect as nfs_lock/unlock
are still stubs.

This will prevent some lock odering problems with upcoming NFSnode
locking.

Tested by landry@, sthen@, visa@, naddy@ and myself.

From NetBSD with some tweaks, ok visa@

Check for possible race after sleeping instead of using a rwlock to
protect insertions in `nm_ntree'.

This will prevent a future lock ordering problem with NFSnode's lock.

ok tedu@, visa@

replace the use of RB macros with the RBT functions.

Remove the unused flags argument from VOP_UNLOCK().

torture tested on amd64, i386 and macppc
ok beck mpi stefan
"the change looks right" deraadt

sync a function's comment with its signature

ok tedu@, with a slight tweak

Remove some includes include-what-you-use claims don't
have any direct symbols used. Tested for indirect use by compiling
amd64/i386/sparc64 kernels.

ok tedu@ deraadt@

to be safe, revert locking change until further tested.

optimize locking a wee bit by holding it for critical parts only

primary change: move uvm_vnode out of vnode, keeping only a pointer.
objective: vnode.h doesn't include uvm_extern.h anymore.
followup changes: include uvm_extern.h or lock.h where necessary.
ok and help from deraadt

add sizes for free(ptr, sizeof(*ptr)). use sizeof(*ptr) for malloc sizes.

remove uneeded proc.h includes
ok mpi@ kspillner@

add a size argument to free. will be used soon, but for now default to 0.
after discussions with beck deraadt kettenis.

Bring back the "End the VOP experiment." diff, naddy's issues where
unrelated, and his alpha is much happier now.

OK deraadt@

Backout the VOP diff until the issues naddy was seeing on alpha (gcc3)
have been resolved.

End the VOP experiment. Instead of the ridicolusly complicated operation
vector setup that has questionable features (that have, as far as I can
tell never been used in practice, atleast not in OpenBSD), remove all
the gunk and favor a simple struct full of function pointers that get
set directly by each of the filesystems.

Removes gobs of ugly code and makes things simpler by a magnitude.

The only downside of this is that we loose the vnoperate feature so
the spec/fifo operations of the filesystems need to be kept in sync
with specfs and fifofs, this is no big deal as the API it self is pretty
static.

Many thanks to armani@ who pulled an earlier version of this diff to
current after c2k10 and Gabriel Kihlman on tech@ for testing.

Liked by many. "come on, find your balls" deraadt@.

No "\n" needed at the end of panic() strings.

Bogus chunks pointed out by matthew@ and miod@. No cookies for
marco@ and jasper@.

ok deraadt@ miod@ matthew@ jasper@ macro@

This fixes a case where we could panic on a null deref with a bad vnode
in nfs_inactive, on a reboot.

The core of the problem was in nfs_nget, when we lose the race to put a new
nfsnode in the tree, we have previously allocated a vnode, which getnewvnode
has done an insmntque into the nfs mp's mntlist. The problem being we then
try again with a new vnode, abandoning this one on the mntlist, leaving
junk there for us to die on when we unmount.

This introduces VLARVAL - so we can indicate in a vnode that the higher
level stuff hiding in v_data is incompletely set up. This flag is then
used by nfs to deal with a halfway set up vnode and release it correctly.

analysis and bogus fix by art@, correct fix by me after serveral failed
attempts and much painful testing by krw@, good suggestions by tedu and miod

ok krw@ oga@ thib@ blambert@ art@

back out previous fix, apparently neither art or i can fix anything right on the first try

vgone doesn't work in other cases of this. I must fix this slightly differntly

This fixes a case where we could panic on a null deref with a bad vnode
in nfs_inactive, on a reboot.

The core of the problem was in nfs_nget, when we lose the race to put a new
nfsnode in the tree, we have previously allocated a vnode, which getnewvnode
has done an insmntque into the nfs mp's mntlist. The problem being we then
try again with a new vnode, abandoning this one on the mntlist, leaving
junk there for us to die on when we unmount.

analysis and bogus fix by art@, correct fix by me. much painful testing by
krw@

ok oga@, art@

Use the nfs_hashlock to protect the nfs_nodetree hanging of the mount.

What can happen is that a recycling of a vnode could pull one from out
under us (since NFS has issues with ref counts...).

Dance around getnewvnode() since we can end up recycling vnodes that
where formerly owned by NFS, causing recursive locking.

We where lucky with the old hashtables has the race was rare but now
with more aggresive recycling we loose, just as theo found out on vax.

help from oga, beck and blambert (beck mostly screamed though).

ok oga@, beck@, blambert@

tiny knf and use curproc directly instead of a setting up a local proc
pointer since its just used in one place.

ok blambert@

Use an RB tree instead of a hashtable for fh/node lookups.

Idea from NetBSD.

OK blambert@

(struct foo *)0 -> NULL, every where I could find it.

OK blambert@

Replace the TRUE/FALSE defines with 1/0 respectively. This doesn't
hurt readability and it was just plain annoying seeing them defined
in every other .c file.

OK blambert@

small indent fix

o nfs_vinvalbuf() is always called with the intrflag as 1, and then
checks if the mount is actually interrutable, and if not sets it 0.
remove this argument from nfs_vinvalbuf and just do the checking inside
the function.
o give nfs_vinvalbuf() a makeover so it looks nice. (spacing, casts, &c);
o Actually pass PCATCH too tsleep() if the mount it interrutable.

ok art@, blambert@

Remove code for variable-sized allocations of NFS filehandles by malloc(),
as it was never used because there were 64-bit buffers already allocated
for filehandles in nfsnode structs.

ok thib@

A bunch of pool_get() + bzero() -> pool_get(..., .. | PR_ZERO)
conversions that should shave a few bytes off the kernel.

ok henning, krw, jsing, oga, miod, and thib (``even though i usually prefer
FOO|BAR''; thanks for looking.

Canonical for() -> queue.h FOREACH macro conversions.
Also, it is historical practice to #include <sys/queue.h>
when using queue.h macros.

ok thib@ krw@

special thanks to krw@ for reminders vice violence

o Avoid putting duplicate entries into the name cache,
add a function to handle cache_enter()'s for us since
we need to update the appropriate timestamps so we don't
miss on lookup;
o Do not purge the dvp in nfs_rmdir(), since there's no need;
o cache the new entry created in nfs_mkdir();
o Do caching of the access modes, this drastically reduces the
amount of over the wire access RPCs we do in the NFSv3 case.

This diff was written by Pedro Martelletto, sometime in 2004.
A lot of people have been running with it at one time or another,
this includes at least markus and matthieu.

OK deraadt@, blambert@

Garbage collect nfsdmap and leftover code. That structure
was used as a part of the "old-style directory caching"
that was removed in 2001;

ok beck@, blambert@

MALLOC/FREE -> malloc/free + M_ZERO.
Uneeded includes and casts...

ok krw@

pedro ok'd this ~3500 line diff which removes the vop argument
"ap = v" comments in under 8 seconds, so it must be ok. and it compiles
too.

Add a name argument to the RWLOCK_INITIALIZER macro.
Pick reasonble names for the locks involved..

ok tedu@, art@

lockmgr -> rwlock for the nfs_hashlock.

ok tedu@

add a name to rwlock so that we can tell where procs are getting stuck
without breaking into ddb. doubles the size of rwlock [1], but moving
forward this really helps. ok/tested pedro fgsch millert krw
[1 - next person to add a field to this struct gets whipped with a wet noodle]

Put vprint() under DIAGNOSTIC, as to save space in generated ramdisks.
Inspiration from miod@, okay deraadt@. Tested on i386, macppc and amd64.

Use NULL where NULL is meant

Remove unnecessary lockmgr() archaism that was costing too much in terms
of panics and bugfixes. Access curproc directly, do not expect a process
pointer as an argument. Should fix many "process context required" bugs.
Incentive and okay millert@, okay marc@. Various testing, thanks.

Use list and queue macros where applicable to make the code easier to read;
no change in compiler assembly output.

NFS commit coalescion: instead of sending a commit for each block, coalesce
these into larger ranges wherever possible.

this should speed up NFS writes quite a bit.

ok art@ millert@ pedro@ tedu@

Remove the advertising clause in the UCB license which Berkeley
rescinded 22 July 1999. Proofed by myself and Theo.

use hash.h for nfs_hash as well as namei's hash
ok art@ costa@

First round of __P removal in sys

Sigh. The diff is out for testing for three months without a single problem.
Two days after commit people report serious lockups all over the place.
Back out nfs locking changes.

More locking in the NFS code

Pool deals fairly well with physical memory shortage, but it doesn't deal
well (not at all) with shortages of the vm_map where the pages are mapped
(usually kmem_map).

Try to deal with it:
- group all information the backend allocator for a pool in a separate
struct. The pool will only have a pointer to that struct.
- change the pool_init API to reflect that.
- link all pools allocating from the same allocator on a linked list.
- Since an allocator is responsible to wait for physical memory it will
only fail (waitok) when it runs out of its backing vm_map, carefully
drain pools using the same allocator so that va space is freed.
(see comments in code for caveats and details).
- change pool_reclaim to return if it actually succeeded to free some
memory, use that information to make draining easier and more efficient.
- get rid of PR_URGENT, noone uses it.

use queue.h macro's
remove register

UBC was a disaster. It worked very good when it worked, but on some
machines or some configurations or in some phase of the moon (we actually
don't know when or why) files disappeared. Since we've not been able to
track down the problem in two weeks intense debugging and we need -current
to be stable, back out everything to a state it had before UBC.

We apologise for the inconvenience.

branches: 1.18.2;
Unlock and drop vnode if VOP_GETATTR fails.

Merge in the unified buffer cache code as found in NetBSD 2001/03/10. The
code is written mostly by Chuck Silvers <chuq@chuq.com>/<chs@netbsd.org>.

Tested for the past few weeks by many developers, should be in a pretty stable
state, but will require optimizations and additional cleanups.

Remove creds from struct buf, move the creds that nfs need into the nfs node.
While in the area, convert nfs node allocation from malloc to pool and do
some cleanups.
Based on the UBC changes in NetBSD. niklas@ ok.

Remove NQNFS

Change handling of NFS root vnode. Moves recognition of NFS root vnode into
nfs_nget.

Root vnode no longer pinned in inode cache. Also, forceable unmounts of an
nfs file system now work even if there are extra references to the NFS root.

branches: 1.13.4;
zap the newhashinit hack.
Add an extra flag to hashinit telling if it should wait in malloc.
update all calls to hashinit.

Fix malloc race in nfs_node.c

Rename vop_revoke, vn_bwrite, vop_noislocked, vop_nolock, vop_nounlock
to be vop_generic_revoke, vop_generic_bwrite, vop_generic_islocked,
vop_generic_lock and vop_generic_unlock.

Create vop_generic_abortop and propogate change to all file systems.

Fix PR/371.

Get rid of locking in NULLFS (should be mostly unnecessary now except for
forced unmounts).

Updates for VFS Lite 2 + soft update.

back out vfs lite2 till after 2.2

VFS Lite2 Changes

Don't set sillyrename field to 0 for directories, as it's in a union with
the head of the cookie list. Fixes PR 3215, fix supplied by Hiroshi Tezuka
<tezuka@trc.rwcp.or.jp>. Should also fix M_NFSDIROFF memory leak; fvdl

partial sync with netbsd 960418, more to come

Minor cleanups. Checked against Lite2.
(NetBSD's was really just a Lite2's, but w/ 64bit support)

From NetBSD: NFSv3 import (tomorrow's Net's kernel)
Open's patches kept in. i'll possibly take a look at Lite2 soon,
is there smth usefull ?..

From NetBSD: merge with 960217 (still NFSv2)

from cgd; handle 64-bit pointers and longs

branches: 1.1.1;
Initial revision

Fix lock order reversal in nfs_inactive()

Make the silly file removal happen after the vnode has been unlocked.
This avoids a file-directory reversal in the vnode locking order.

OK jca@

revert vnode: remove VLOCKSWORK and check locking when vop_islocked != nullop
(both kernel and userland bits)

GENERIC + VFSLCKDEBUG is broken with it.

vnode: remove VLOCKSWORK and check locking when vop_islocked != nullop

This flag is currently used to mark or unmark a vnode to actively
check vnode locking semantic (when compiled with VFSLCKDEBUG).

Currently, VLOCKSWORK flag isn't properly set for several FS
implementation which have full locking support. This commit enable
proper checking for them too (cd9660, udf, fuse, msdosfs, tmpfs).

Instead of using a particular flag, it directly check if
v_op->vop_islocked is nullop or not to activate or not the vnode
locking checks.

ok mpi@

vnode: do not manipulate vnode lock directly

use VOP_LOCK / VOP_UNLOCK wrappers.

VOP_LOCK() is prefered over vn_lock() here in order to keep equivalent code.

ok mpi@ visa@ (as part of larger diff)

struct vops is not modified during runtime so use const which moves each
into read-only data segment.
OK deraadt@ tedu@

Drop unnecessary `p' parameter from vget(9).

OK mpi@

Implement proper locking for NFS nodes.

Tested in bulks by many. ok visa@, beck@

Clean up the parameters of VOP_LOCK() and VOP_UNLOCK(). It is always
curproc that does the locking or unlocking, so the proc parameter
is pointless and can be dropped.

OK mpi@, deraadt@

Change the representation of an NFS mount point by caching the root
nodes.

nfs_root() now returns a "locked" vnode, so vput(9) must be called to
release it. Note that this has currently no effect as nfs_lock/unlock
are still stubs.

This will prevent some lock odering problems with upcoming NFSnode
locking.

Tested by landry@, sthen@, visa@, naddy@ and myself.

From NetBSD with some tweaks, ok visa@

Check for possible race after sleeping instead of using a rwlock to
protect insertions in `nm_ntree'.

This will prevent a future lock ordering problem with NFSnode's lock.

ok tedu@, visa@

replace the use of RB macros with the RBT functions.

Remove the unused flags argument from VOP_UNLOCK().

torture tested on amd64, i386 and macppc
ok beck mpi stefan
"the change looks right" deraadt

sync a function's comment with its signature

ok tedu@, with a slight tweak

Remove some includes include-what-you-use claims don't
have any direct symbols used. Tested for indirect use by compiling
amd64/i386/sparc64 kernels.

ok tedu@ deraadt@

to be safe, revert locking change until further tested.

optimize locking a wee bit by holding it for critical parts only

primary change: move uvm_vnode out of vnode, keeping only a pointer.
objective: vnode.h doesn't include uvm_extern.h anymore.
followup changes: include uvm_extern.h or lock.h where necessary.
ok and help from deraadt

add sizes for free(ptr, sizeof(*ptr)). use sizeof(*ptr) for malloc sizes.

remove uneeded proc.h includes
ok mpi@ kspillner@

add a size argument to free. will be used soon, but for now default to 0.
after discussions with beck deraadt kettenis.

Bring back the "End the VOP experiment." diff, naddy's issues where
unrelated, and his alpha is much happier now.

OK deraadt@

Backout the VOP diff until the issues naddy was seeing on alpha (gcc3)
have been resolved.

End the VOP experiment. Instead of the ridicolusly complicated operation
vector setup that has questionable features (that have, as far as I can
tell never been used in practice, atleast not in OpenBSD), remove all
the gunk and favor a simple struct full of function pointers that get
set directly by each of the filesystems.

Removes gobs of ugly code and makes things simpler by a magnitude.

The only downside of this is that we loose the vnoperate feature so
the spec/fifo operations of the filesystems need to be kept in sync
with specfs and fifofs, this is no big deal as the API it self is pretty
static.

Many thanks to armani@ who pulled an earlier version of this diff to
current after c2k10 and Gabriel Kihlman on tech@ for testing.

Liked by many. "come on, find your balls" deraadt@.

No "\n" needed at the end of panic() strings.

Bogus chunks pointed out by matthew@ and miod@. No cookies for
marco@ and jasper@.

ok deraadt@ miod@ matthew@ jasper@ macro@

This fixes a case where we could panic on a null deref with a bad vnode
in nfs_inactive, on a reboot.

The core of the problem was in nfs_nget, when we lose the race to put a new
nfsnode in the tree, we have previously allocated a vnode, which getnewvnode
has done an insmntque into the nfs mp's mntlist. The problem being we then
try again with a new vnode, abandoning this one on the mntlist, leaving
junk there for us to die on when we unmount.

This introduces VLARVAL - so we can indicate in a vnode that the higher
level stuff hiding in v_data is incompletely set up. This flag is then
used by nfs to deal with a halfway set up vnode and release it correctly.

analysis and bogus fix by art@, correct fix by me after serveral failed
attempts and much painful testing by krw@, good suggestions by tedu and miod

ok krw@ oga@ thib@ blambert@ art@

back out previous fix, apparently neither art or i can fix anything right on the first try

vgone doesn't work in other cases of this. I must fix this slightly differntly

This fixes a case where we could panic on a null deref with a bad vnode
in nfs_inactive, on a reboot.

The core of the problem was in nfs_nget, when we lose the race to put a new
nfsnode in the tree, we have previously allocated a vnode, which getnewvnode
has done an insmntque into the nfs mp's mntlist. The problem being we then
try again with a new vnode, abandoning this one on the mntlist, leaving
junk there for us to die on when we unmount.

analysis and bogus fix by art@, correct fix by me. much painful testing by
krw@

ok oga@, art@

Use the nfs_hashlock to protect the nfs_nodetree hanging of the mount.

What can happen is that a recycling of a vnode could pull one from out
under us (since NFS has issues with ref counts...).

Dance around getnewvnode() since we can end up recycling vnodes that
where formerly owned by NFS, causing recursive locking.

We where lucky with the old hashtables has the race was rare but now
with more aggresive recycling we loose, just as theo found out on vax.

help from oga, beck and blambert (beck mostly screamed though).

ok oga@, beck@, blambert@

tiny knf and use curproc directly instead of a setting up a local proc
pointer since its just used in one place.

ok blambert@

Use an RB tree instead of a hashtable for fh/node lookups.

Idea from NetBSD.

OK blambert@

(struct foo *)0 -> NULL, every where I could find it.

OK blambert@

Replace the TRUE/FALSE defines with 1/0 respectively. This doesn't
hurt readability and it was just plain annoying seeing them defined
in every other .c file.

OK blambert@

small indent fix

o nfs_vinvalbuf() is always called with the intrflag as 1, and then
checks if the mount is actually interrutable, and if not sets it 0.
remove this argument from nfs_vinvalbuf and just do the checking inside
the function.
o give nfs_vinvalbuf() a makeover so it looks nice. (spacing, casts, &c);
o Actually pass PCATCH too tsleep() if the mount it interrutable.

ok art@, blambert@

Remove code for variable-sized allocations of NFS filehandles by malloc(),
as it was never used because there were 64-bit buffers already allocated
for filehandles in nfsnode structs.

ok thib@

A bunch of pool_get() + bzero() -> pool_get(..., .. | PR_ZERO)
conversions that should shave a few bytes off the kernel.

ok henning, krw, jsing, oga, miod, and thib (``even though i usually prefer
FOO|BAR''; thanks for looking.

Canonical for() -> queue.h FOREACH macro conversions.
Also, it is historical practice to #include <sys/queue.h>
when using queue.h macros.

ok thib@ krw@

special thanks to krw@ for reminders vice violence

o Avoid putting duplicate entries into the name cache,
add a function to handle cache_enter()'s for us since
we need to update the appropriate timestamps so we don't
miss on lookup;
o Do not purge the dvp in nfs_rmdir(), since there's no need;
o cache the new entry created in nfs_mkdir();
o Do caching of the access modes, this drastically reduces the
amount of over the wire access RPCs we do in the NFSv3 case.

This diff was written by Pedro Martelletto, sometime in 2004.
A lot of people have been running with it at one time or another,
this includes at least markus and matthieu.

OK deraadt@, blambert@

Garbage collect nfsdmap and leftover code. That structure
was used as a part of the "old-style directory caching"
that was removed in 2001;

ok beck@, blambert@

MALLOC/FREE -> malloc/free + M_ZERO.
Uneeded includes and casts...

ok krw@

pedro ok'd this ~3500 line diff which removes the vop argument
"ap = v" comments in under 8 seconds, so it must be ok. and it compiles
too.

Add a name argument to the RWLOCK_INITIALIZER macro.
Pick reasonble names for the locks involved..

ok tedu@, art@

lockmgr -> rwlock for the nfs_hashlock.

ok tedu@

add a name to rwlock so that we can tell where procs are getting stuck
without breaking into ddb. doubles the size of rwlock [1], but moving
forward this really helps. ok/tested pedro fgsch millert krw
[1 - next person to add a field to this struct gets whipped with a wet noodle]

Put vprint() under DIAGNOSTIC, as to save space in generated ramdisks.
Inspiration from miod@, okay deraadt@. Tested on i386, macppc and amd64.

Use NULL where NULL is meant

Remove unnecessary lockmgr() archaism that was costing too much in terms
of panics and bugfixes. Access curproc directly, do not expect a process
pointer as an argument. Should fix many "process context required" bugs.
Incentive and okay millert@, okay marc@. Various testing, thanks.

Use list and queue macros where applicable to make the code easier to read;
no change in compiler assembly output.

NFS commit coalescion: instead of sending a commit for each block, coalesce
these into larger ranges wherever possible.

this should speed up NFS writes quite a bit.

ok art@ millert@ pedro@ tedu@

Remove the advertising clause in the UCB license which Berkeley
rescinded 22 July 1999. Proofed by myself and Theo.

use hash.h for nfs_hash as well as namei's hash
ok art@ costa@

First round of __P removal in sys

Sigh. The diff is out for testing for three months without a single problem.
Two days after commit people report serious lockups all over the place.
Back out nfs locking changes.

More locking in the NFS code

Pool deals fairly well with physical memory shortage, but it doesn't deal
well (not at all) with shortages of the vm_map where the pages are mapped
(usually kmem_map).

Try to deal with it:
- group all information the backend allocator for a pool in a separate
struct. The pool will only have a pointer to that struct.
- change the pool_init API to reflect that.
- link all pools allocating from the same allocator on a linked list.
- Since an allocator is responsible to wait for physical memory it will
only fail (waitok) when it runs out of its backing vm_map, carefully
drain pools using the same allocator so that va space is freed.
(see comments in code for caveats and details).
- change pool_reclaim to return if it actually succeeded to free some
memory, use that information to make draining easier and more efficient.
- get rid of PR_URGENT, noone uses it.

use queue.h macro's
remove register

UBC was a disaster. It worked very good when it worked, but on some
machines or some configurations or in some phase of the moon (we actually
don't know when or why) files disappeared. Since we've not been able to
track down the problem in two weeks intense debugging and we need -current
to be stable, back out everything to a state it had before UBC.

We apologise for the inconvenience.

branches: 1.18.2;
Unlock and drop vnode if VOP_GETATTR fails.

Merge in the unified buffer cache code as found in NetBSD 2001/03/10. The
code is written mostly by Chuck Silvers <chuq@chuq.com>/<chs@netbsd.org>.

Tested for the past few weeks by many developers, should be in a pretty stable
state, but will require optimizations and additional cleanups.

Remove creds from struct buf, move the creds that nfs need into the nfs node.
While in the area, convert nfs node allocation from malloc to pool and do
some cleanups.
Based on the UBC changes in NetBSD. niklas@ ok.

Remove NQNFS

Change handling of NFS root vnode. Moves recognition of NFS root vnode into
nfs_nget.

Root vnode no longer pinned in inode cache. Also, forceable unmounts of an
nfs file system now work even if there are extra references to the NFS root.

branches: 1.13.4;
zap the newhashinit hack.
Add an extra flag to hashinit telling if it should wait in malloc.
update all calls to hashinit.

Fix malloc race in nfs_node.c

Rename vop_revoke, vn_bwrite, vop_noislocked, vop_nolock, vop_nounlock
to be vop_generic_revoke, vop_generic_bwrite, vop_generic_islocked,
vop_generic_lock and vop_generic_unlock.

Create vop_generic_abortop and propogate change to all file systems.

Fix PR/371.

Get rid of locking in NULLFS (should be mostly unnecessary now except for
forced unmounts).

Updates for VFS Lite 2 + soft update.

back out vfs lite2 till after 2.2

VFS Lite2 Changes

Don't set sillyrename field to 0 for directories, as it's in a union with
the head of the cookie list. Fixes PR 3215, fix supplied by Hiroshi Tezuka
<tezuka@trc.rwcp.or.jp>. Should also fix M_NFSDIROFF memory leak; fvdl

partial sync with netbsd 960418, more to come

Minor cleanups. Checked against Lite2.
(NetBSD's was really just a Lite2's, but w/ 64bit support)

From NetBSD: NFSv3 import (tomorrow's Net's kernel)
Open's patches kept in. i'll possibly take a look at Lite2 soon,
is there smth usefull ?..

From NetBSD: merge with 960217 (still NFSv2)

from cgd; handle 64-bit pointers and longs

branches: 1.1.1;
Initial revision

revert vnode: remove VLOCKSWORK and check locking when vop_islocked != nullop
(both kernel and userland bits)

GENERIC + VFSLCKDEBUG is broken with it.

vnode: remove VLOCKSWORK and check locking when vop_islocked != nullop

This flag is currently used to mark or unmark a vnode to actively
check vnode locking semantic (when compiled with VFSLCKDEBUG).

Currently, VLOCKSWORK flag isn't properly set for several FS
implementation which have full locking support. This commit enable
proper checking for them too (cd9660, udf, fuse, msdosfs, tmpfs).

Instead of using a particular flag, it directly check if
v_op->vop_islocked is nullop or not to activate or not the vnode
locking checks.

ok mpi@

vnode: do not manipulate vnode lock directly

use VOP_LOCK / VOP_UNLOCK wrappers.

VOP_LOCK() is prefered over vn_lock() here in order to keep equivalent code.

ok mpi@ visa@ (as part of larger diff)

struct vops is not modified during runtime so use const which moves each
into read-only data segment.
OK deraadt@ tedu@

Drop unnecessary `p' parameter from vget(9).

OK mpi@

Implement proper locking for NFS nodes.

Tested in bulks by many. ok visa@, beck@

Clean up the parameters of VOP_LOCK() and VOP_UNLOCK(). It is always
curproc that does the locking or unlocking, so the proc parameter
is pointless and can be dropped.

OK mpi@, deraadt@

Change the representation of an NFS mount point by caching the root
nodes.

nfs_root() now returns a "locked" vnode, so vput(9) must be called to
release it. Note that this has currently no effect as nfs_lock/unlock
are still stubs.

This will prevent some lock odering problems with upcoming NFSnode
locking.

Tested by landry@, sthen@, visa@, naddy@ and myself.

From NetBSD with some tweaks, ok visa@

Check for possible race after sleeping instead of using a rwlock to
protect insertions in `nm_ntree'.

This will prevent a future lock ordering problem with NFSnode's lock.

ok tedu@, visa@

replace the use of RB macros with the RBT functions.

Remove the unused flags argument from VOP_UNLOCK().

torture tested on amd64, i386 and macppc
ok beck mpi stefan
"the change looks right" deraadt

sync a function's comment with its signature

ok tedu@, with a slight tweak

Remove some includes include-what-you-use claims don't
have any direct symbols used. Tested for indirect use by compiling
amd64/i386/sparc64 kernels.

ok tedu@ deraadt@

to be safe, revert locking change until further tested.

optimize locking a wee bit by holding it for critical parts only

primary change: move uvm_vnode out of vnode, keeping only a pointer.
objective: vnode.h doesn't include uvm_extern.h anymore.
followup changes: include uvm_extern.h or lock.h where necessary.
ok and help from deraadt

add sizes for free(ptr, sizeof(*ptr)). use sizeof(*ptr) for malloc sizes.

remove uneeded proc.h includes
ok mpi@ kspillner@

add a size argument to free. will be used soon, but for now default to 0.
after discussions with beck deraadt kettenis.

Bring back the "End the VOP experiment." diff, naddy's issues where
unrelated, and his alpha is much happier now.

OK deraadt@

Backout the VOP diff until the issues naddy was seeing on alpha (gcc3)
have been resolved.

End the VOP experiment. Instead of the ridicolusly complicated operation
vector setup that has questionable features (that have, as far as I can
tell never been used in practice, atleast not in OpenBSD), remove all
the gunk and favor a simple struct full of function pointers that get
set directly by each of the filesystems.

Removes gobs of ugly code and makes things simpler by a magnitude.

The only downside of this is that we loose the vnoperate feature so
the spec/fifo operations of the filesystems need to be kept in sync
with specfs and fifofs, this is no big deal as the API it self is pretty
static.

Many thanks to armani@ who pulled an earlier version of this diff to
current after c2k10 and Gabriel Kihlman on tech@ for testing.

Liked by many. "come on, find your balls" deraadt@.

No "\n" needed at the end of panic() strings.

Bogus chunks pointed out by matthew@ and miod@. No cookies for
marco@ and jasper@.

ok deraadt@ miod@ matthew@ jasper@ macro@

This fixes a case where we could panic on a null deref with a bad vnode
in nfs_inactive, on a reboot.

The core of the problem was in nfs_nget, when we lose the race to put a new
nfsnode in the tree, we have previously allocated a vnode, which getnewvnode
has done an insmntque into the nfs mp's mntlist. The problem being we then
try again with a new vnode, abandoning this one on the mntlist, leaving
junk there for us to die on when we unmount.

This introduces VLARVAL - so we can indicate in a vnode that the higher
level stuff hiding in v_data is incompletely set up. This flag is then
used by nfs to deal with a halfway set up vnode and release it correctly.

analysis and bogus fix by art@, correct fix by me after serveral failed
attempts and much painful testing by krw@, good suggestions by tedu and miod

ok krw@ oga@ thib@ blambert@ art@

back out previous fix, apparently neither art or i can fix anything right on the first try

vgone doesn't work in other cases of this. I must fix this slightly differntly

This fixes a case where we could panic on a null deref with a bad vnode
in nfs_inactive, on a reboot.

The core of the problem was in nfs_nget, when we lose the race to put a new
nfsnode in the tree, we have previously allocated a vnode, which getnewvnode
has done an insmntque into the nfs mp's mntlist. The problem being we then
try again with a new vnode, abandoning this one on the mntlist, leaving
junk there for us to die on when we unmount.

analysis and bogus fix by art@, correct fix by me. much painful testing by
krw@

ok oga@, art@

Use the nfs_hashlock to protect the nfs_nodetree hanging of the mount.

What can happen is that a recycling of a vnode could pull one from out
under us (since NFS has issues with ref counts...).

Dance around getnewvnode() since we can end up recycling vnodes that
where formerly owned by NFS, causing recursive locking.

We where lucky with the old hashtables has the race was rare but now
with more aggresive recycling we loose, just as theo found out on vax.

help from oga, beck and blambert (beck mostly screamed though).

ok oga@, beck@, blambert@

tiny knf and use curproc directly instead of a setting up a local proc
pointer since its just used in one place.

ok blambert@

Use an RB tree instead of a hashtable for fh/node lookups.

Idea from NetBSD.

OK blambert@

(struct foo *)0 -> NULL, every where I could find it.

OK blambert@

Replace the TRUE/FALSE defines with 1/0 respectively. This doesn't
hurt readability and it was just plain annoying seeing them defined
in every other .c file.

OK blambert@

small indent fix

o nfs_vinvalbuf() is always called with the intrflag as 1, and then
checks if the mount is actually interrutable, and if not sets it 0.
remove this argument from nfs_vinvalbuf and just do the checking inside
the function.
o give nfs_vinvalbuf() a makeover so it looks nice. (spacing, casts, &c);
o Actually pass PCATCH too tsleep() if the mount it interrutable.

ok art@, blambert@

Remove code for variable-sized allocations of NFS filehandles by malloc(),
as it was never used because there were 64-bit buffers already allocated
for filehandles in nfsnode structs.

ok thib@

A bunch of pool_get() + bzero() -> pool_get(..., .. | PR_ZERO)
conversions that should shave a few bytes off the kernel.

ok henning, krw, jsing, oga, miod, and thib (``even though i usually prefer
FOO|BAR''; thanks for looking.

Canonical for() -> queue.h FOREACH macro conversions.
Also, it is historical practice to #include <sys/queue.h>
when using queue.h macros.

ok thib@ krw@

special thanks to krw@ for reminders vice violence

o Avoid putting duplicate entries into the name cache,
add a function to handle cache_enter()'s for us since
we need to update the appropriate timestamps so we don't
miss on lookup;
o Do not purge the dvp in nfs_rmdir(), since there's no need;
o cache the new entry created in nfs_mkdir();
o Do caching of the access modes, this drastically reduces the
amount of over the wire access RPCs we do in the NFSv3 case.

This diff was written by Pedro Martelletto, sometime in 2004.
A lot of people have been running with it at one time or another,
this includes at least markus and matthieu.

OK deraadt@, blambert@

Garbage collect nfsdmap and leftover code. That structure
was used as a part of the "old-style directory caching"
that was removed in 2001;

ok beck@, blambert@

MALLOC/FREE -> malloc/free + M_ZERO.
Uneeded includes and casts...

ok krw@

pedro ok'd this ~3500 line diff which removes the vop argument
"ap = v" comments in under 8 seconds, so it must be ok. and it compiles
too.

Add a name argument to the RWLOCK_INITIALIZER macro.
Pick reasonble names for the locks involved..

ok tedu@, art@

lockmgr -> rwlock for the nfs_hashlock.

ok tedu@

add a name to rwlock so that we can tell where procs are getting stuck
without breaking into ddb. doubles the size of rwlock [1], but moving
forward this really helps. ok/tested pedro fgsch millert krw
[1 - next person to add a field to this struct gets whipped with a wet noodle]

Put vprint() under DIAGNOSTIC, as to save space in generated ramdisks.
Inspiration from miod@, okay deraadt@. Tested on i386, macppc and amd64.

Use NULL where NULL is meant

Remove unnecessary lockmgr() archaism that was costing too much in terms
of panics and bugfixes. Access curproc directly, do not expect a process
pointer as an argument. Should fix many "process context required" bugs.
Incentive and okay millert@, okay marc@. Various testing, thanks.

Use list and queue macros where applicable to make the code easier to read;
no change in compiler assembly output.

NFS commit coalescion: instead of sending a commit for each block, coalesce
these into larger ranges wherever possible.

this should speed up NFS writes quite a bit.

ok art@ millert@ pedro@ tedu@

Remove the advertising clause in the UCB license which Berkeley
rescinded 22 July 1999. Proofed by myself and Theo.

use hash.h for nfs_hash as well as namei's hash
ok art@ costa@

First round of __P removal in sys

Sigh. The diff is out for testing for three months without a single problem.
Two days after commit people report serious lockups all over the place.
Back out nfs locking changes.

More locking in the NFS code

Pool deals fairly well with physical memory shortage, but it doesn't deal
well (not at all) with shortages of the vm_map where the pages are mapped
(usually kmem_map).

Try to deal with it:
- group all information the backend allocator for a pool in a separate
struct. The pool will only have a pointer to that struct.
- change the pool_init API to reflect that.
- link all pools allocating from the same allocator on a linked list.
- Since an allocator is responsible to wait for physical memory it will
only fail (waitok) when it runs out of its backing vm_map, carefully
drain pools using the same allocator so that va space is freed.
(see comments in code for caveats and details).
- change pool_reclaim to return if it actually succeeded to free some
memory, use that information to make draining easier and more efficient.
- get rid of PR_URGENT, noone uses it.

use queue.h macro's
remove register

UBC was a disaster. It worked very good when it worked, but on some
machines or some configurations or in some phase of the moon (we actually
don't know when or why) files disappeared. Since we've not been able to
track down the problem in two weeks intense debugging and we need -current
to be stable, back out everything to a state it had before UBC.

We apologise for the inconvenience.

branches: 1.18.2;
Unlock and drop vnode if VOP_GETATTR fails.

Merge in the unified buffer cache code as found in NetBSD 2001/03/10. The
code is written mostly by Chuck Silvers <chuq@chuq.com>/<chs@netbsd.org>.

Tested for the past few weeks by many developers, should be in a pretty stable
state, but will require optimizations and additional cleanups.

Remove creds from struct buf, move the creds that nfs need into the nfs node.
While in the area, convert nfs node allocation from malloc to pool and do
some cleanups.
Based on the UBC changes in NetBSD. niklas@ ok.

Remove NQNFS

Change handling of NFS root vnode. Moves recognition of NFS root vnode into
nfs_nget.

Root vnode no longer pinned in inode cache. Also, forceable unmounts of an
nfs file system now work even if there are extra references to the NFS root.

branches: 1.13.4;
zap the newhashinit hack.
Add an extra flag to hashinit telling if it should wait in malloc.
update all calls to hashinit.

Fix malloc race in nfs_node.c

Rename vop_revoke, vn_bwrite, vop_noislocked, vop_nolock, vop_nounlock
to be vop_generic_revoke, vop_generic_bwrite, vop_generic_islocked,
vop_generic_lock and vop_generic_unlock.

Create vop_generic_abortop and propogate change to all file systems.

Fix PR/371.

Get rid of locking in NULLFS (should be mostly unnecessary now except for
forced unmounts).

Updates for VFS Lite 2 + soft update.

back out vfs lite2 till after 2.2

VFS Lite2 Changes

Don't set sillyrename field to 0 for directories, as it's in a union with
the head of the cookie list. Fixes PR 3215, fix supplied by Hiroshi Tezuka
<tezuka@trc.rwcp.or.jp>. Should also fix M_NFSDIROFF memory leak; fvdl

partial sync with netbsd 960418, more to come

Minor cleanups. Checked against Lite2.
(NetBSD's was really just a Lite2's, but w/ 64bit support)

From NetBSD: NFSv3 import (tomorrow's Net's kernel)
Open's patches kept in. i'll possibly take a look at Lite2 soon,
is there smth usefull ?..

From NetBSD: merge with 960217 (still NFSv2)

from cgd; handle 64-bit pointers and longs

branches: 1.1.1;
Initial revision

vnode: remove VLOCKSWORK and check locking when vop_islocked != nullop

This flag is currently used to mark or unmark a vnode to actively
check vnode locking semantic (when compiled with VFSLCKDEBUG).

Currently, VLOCKSWORK flag isn't properly set for several FS
implementation which have full locking support. This commit enable
proper checking for them too (cd9660, udf, fuse, msdosfs, tmpfs).

Instead of using a particular flag, it directly check if
v_op->vop_islocked is nullop or not to activate or not the vnode
locking checks.

ok mpi@

vnode: do not manipulate vnode lock directly

use VOP_LOCK / VOP_UNLOCK wrappers.

VOP_LOCK() is prefered over vn_lock() here in order to keep equivalent code.

ok mpi@ visa@ (as part of larger diff)

struct vops is not modified during runtime so use const which moves each
into read-only data segment.
OK deraadt@ tedu@

Drop unnecessary `p' parameter from vget(9).

OK mpi@

Implement proper locking for NFS nodes.

Tested in bulks by many. ok visa@, beck@

Clean up the parameters of VOP_LOCK() and VOP_UNLOCK(). It is always
curproc that does the locking or unlocking, so the proc parameter
is pointless and can be dropped.

OK mpi@, deraadt@

Change the representation of an NFS mount point by caching the root
nodes.

nfs_root() now returns a "locked" vnode, so vput(9) must be called to
release it. Note that this has currently no effect as nfs_lock/unlock
are still stubs.

This will prevent some lock odering problems with upcoming NFSnode
locking.

Tested by landry@, sthen@, visa@, naddy@ and myself.

From NetBSD with some tweaks, ok visa@

Check for possible race after sleeping instead of using a rwlock to
protect insertions in `nm_ntree'.

This will prevent a future lock ordering problem with NFSnode's lock.

ok tedu@, visa@

replace the use of RB macros with the RBT functions.

Remove the unused flags argument from VOP_UNLOCK().

torture tested on amd64, i386 and macppc
ok beck mpi stefan
"the change looks right" deraadt

sync a function's comment with its signature

ok tedu@, with a slight tweak

Remove some includes include-what-you-use claims don't
have any direct symbols used. Tested for indirect use by compiling
amd64/i386/sparc64 kernels.

ok tedu@ deraadt@

to be safe, revert locking change until further tested.

optimize locking a wee bit by holding it for critical parts only

primary change: move uvm_vnode out of vnode, keeping only a pointer.
objective: vnode.h doesn't include uvm_extern.h anymore.
followup changes: include uvm_extern.h or lock.h where necessary.
ok and help from deraadt

add sizes for free(ptr, sizeof(*ptr)). use sizeof(*ptr) for malloc sizes.

remove uneeded proc.h includes
ok mpi@ kspillner@

add a size argument to free. will be used soon, but for now default to 0.
after discussions with beck deraadt kettenis.

Bring back the "End the VOP experiment." diff, naddy's issues where
unrelated, and his alpha is much happier now.

OK deraadt@

Backout the VOP diff until the issues naddy was seeing on alpha (gcc3)
have been resolved.

End the VOP experiment. Instead of the ridicolusly complicated operation
vector setup that has questionable features (that have, as far as I can
tell never been used in practice, atleast not in OpenBSD), remove all
the gunk and favor a simple struct full of function pointers that get
set directly by each of the filesystems.

Removes gobs of ugly code and makes things simpler by a magnitude.

The only downside of this is that we loose the vnoperate feature so
the spec/fifo operations of the filesystems need to be kept in sync
with specfs and fifofs, this is no big deal as the API it self is pretty
static.

Many thanks to armani@ who pulled an earlier version of this diff to
current after c2k10 and Gabriel Kihlman on tech@ for testing.

Liked by many. "come on, find your balls" deraadt@.

No "\n" needed at the end of panic() strings.

Bogus chunks pointed out by matthew@ and miod@. No cookies for
marco@ and jasper@.

ok deraadt@ miod@ matthew@ jasper@ macro@

This fixes a case where we could panic on a null deref with a bad vnode
in nfs_inactive, on a reboot.

The core of the problem was in nfs_nget, when we lose the race to put a new
nfsnode in the tree, we have previously allocated a vnode, which getnewvnode
has done an insmntque into the nfs mp's mntlist. The problem being we then
try again with a new vnode, abandoning this one on the mntlist, leaving
junk there for us to die on when we unmount.

This introduces VLARVAL - so we can indicate in a vnode that the higher
level stuff hiding in v_data is incompletely set up. This flag is then
used by nfs to deal with a halfway set up vnode and release it correctly.

analysis and bogus fix by art@, correct fix by me after serveral failed
attempts and much painful testing by krw@, good suggestions by tedu and miod

ok krw@ oga@ thib@ blambert@ art@

back out previous fix, apparently neither art or i can fix anything right on the first try

vgone doesn't work in other cases of this. I must fix this slightly differntly

This fixes a case where we could panic on a null deref with a bad vnode
in nfs_inactive, on a reboot.

The core of the problem was in nfs_nget, when we lose the race to put a new
nfsnode in the tree, we have previously allocated a vnode, which getnewvnode
has done an insmntque into the nfs mp's mntlist. The problem being we then
try again with a new vnode, abandoning this one on the mntlist, leaving
junk there for us to die on when we unmount.

analysis and bogus fix by art@, correct fix by me. much painful testing by
krw@

ok oga@, art@

Use the nfs_hashlock to protect the nfs_nodetree hanging of the mount.

What can happen is that a recycling of a vnode could pull one from out
under us (since NFS has issues with ref counts...).

Dance around getnewvnode() since we can end up recycling vnodes that
where formerly owned by NFS, causing recursive locking.

We where lucky with the old hashtables has the race was rare but now
with more aggresive recycling we loose, just as theo found out on vax.

help from oga, beck and blambert (beck mostly screamed though).

ok oga@, beck@, blambert@

tiny knf and use curproc directly instead of a setting up a local proc
pointer since its just used in one place.

ok blambert@

Use an RB tree instead of a hashtable for fh/node lookups.

Idea from NetBSD.

OK blambert@

(struct foo *)0 -> NULL, every where I could find it.

OK blambert@

Replace the TRUE/FALSE defines with 1/0 respectively. This doesn't
hurt readability and it was just plain annoying seeing them defined
in every other .c file.

OK blambert@

small indent fix

o nfs_vinvalbuf() is always called with the intrflag as 1, and then
checks if the mount is actually interrutable, and if not sets it 0.
remove this argument from nfs_vinvalbuf and just do the checking inside
the function.
o give nfs_vinvalbuf() a makeover so it looks nice. (spacing, casts, &c);
o Actually pass PCATCH too tsleep() if the mount it interrutable.

ok art@, blambert@

Remove code for variable-sized allocations of NFS filehandles by malloc(),
as it was never used because there were 64-bit buffers already allocated
for filehandles in nfsnode structs.

ok thib@

A bunch of pool_get() + bzero() -> pool_get(..., .. | PR_ZERO)
conversions that should shave a few bytes off the kernel.

ok henning, krw, jsing, oga, miod, and thib (``even though i usually prefer
FOO|BAR''; thanks for looking.

Canonical for() -> queue.h FOREACH macro conversions.
Also, it is historical practice to #include <sys/queue.h>
when using queue.h macros.

ok thib@ krw@

special thanks to krw@ for reminders vice violence

o Avoid putting duplicate entries into the name cache,
add a function to handle cache_enter()'s for us since
we need to update the appropriate timestamps so we don't
miss on lookup;
o Do not purge the dvp in nfs_rmdir(), since there's no need;
o cache the new entry created in nfs_mkdir();
o Do caching of the access modes, this drastically reduces the
amount of over the wire access RPCs we do in the NFSv3 case.

This diff was written by Pedro Martelletto, sometime in 2004.
A lot of people have been running with it at one time or another,
this includes at least markus and matthieu.

OK deraadt@, blambert@

Garbage collect nfsdmap and leftover code. That structure
was used as a part of the "old-style directory caching"
that was removed in 2001;

ok beck@, blambert@

MALLOC/FREE -> malloc/free + M_ZERO.
Uneeded includes and casts...

ok krw@

pedro ok'd this ~3500 line diff which removes the vop argument
"ap = v" comments in under 8 seconds, so it must be ok. and it compiles
too.

Add a name argument to the RWLOCK_INITIALIZER macro.
Pick reasonble names for the locks involved..

ok tedu@, art@

lockmgr -> rwlock for the nfs_hashlock.

ok tedu@

add a name to rwlock so that we can tell where procs are getting stuck
without breaking into ddb. doubles the size of rwlock [1], but moving
forward this really helps. ok/tested pedro fgsch millert krw
[1 - next person to add a field to this struct gets whipped with a wet noodle]

Put vprint() under DIAGNOSTIC, as to save space in generated ramdisks.
Inspiration from miod@, okay deraadt@. Tested on i386, macppc and amd64.

Use NULL where NULL is meant

Remove unnecessary lockmgr() archaism that was costing too much in terms
of panics and bugfixes. Access curproc directly, do not expect a process
pointer as an argument. Should fix many "process context required" bugs.
Incentive and okay millert@, okay marc@. Various testing, thanks.

Use list and queue macros where applicable to make the code easier to read;
no change in compiler assembly output.

NFS commit coalescion: instead of sending a commit for each block, coalesce
these into larger ranges wherever possible.

this should speed up NFS writes quite a bit.

ok art@ millert@ pedro@ tedu@

Remove the advertising clause in the UCB license which Berkeley
rescinded 22 July 1999. Proofed by myself and Theo.

use hash.h for nfs_hash as well as namei's hash
ok art@ costa@

First round of __P removal in sys

Sigh. The diff is out for testing for three months without a single problem.
Two days after commit people report serious lockups all over the place.
Back out nfs locking changes.

More locking in the NFS code

Pool deals fairly well with physical memory shortage, but it doesn't deal
well (not at all) with shortages of the vm_map where the pages are mapped
(usually kmem_map).

Try to deal with it:
- group all information the backend allocator for a pool in a separate
struct. The pool will only have a pointer to that struct.
- change the pool_init API to reflect that.
- link all pools allocating from the same allocator on a linked list.
- Since an allocator is responsible to wait for physical memory it will
only fail (waitok) when it runs out of its backing vm_map, carefully
drain pools using the same allocator so that va space is freed.
(see comments in code for caveats and details).
- change pool_reclaim to return if it actually succeeded to free some
memory, use that information to make draining easier and more efficient.
- get rid of PR_URGENT, noone uses it.

use queue.h macro's
remove register

UBC was a disaster. It worked very good when it worked, but on some
machines or some configurations or in some phase of the moon (we actually
don't know when or why) files disappeared. Since we've not been able to
track down the problem in two weeks intense debugging and we need -current
to be stable, back out everything to a state it had before UBC.

We apologise for the inconvenience.

branches: 1.18.2;
Unlock and drop vnode if VOP_GETATTR fails.

Merge in the unified buffer cache code as found in NetBSD 2001/03/10. The
code is written mostly by Chuck Silvers <chuq@chuq.com>/<chs@netbsd.org>.

Tested for the past few weeks by many developers, should be in a pretty stable
state, but will require optimizations and additional cleanups.

Remove creds from struct buf, move the creds that nfs need into the nfs node.
While in the area, convert nfs node allocation from malloc to pool and do
some cleanups.
Based on the UBC changes in NetBSD. niklas@ ok.

Remove NQNFS

Change handling of NFS root vnode. Moves recognition of NFS root vnode into
nfs_nget.

Root vnode no longer pinned in inode cache. Also, forceable unmounts of an
nfs file system now work even if there are extra references to the NFS root.

branches: 1.13.4;
zap the newhashinit hack.
Add an extra flag to hashinit telling if it should wait in malloc.
update all calls to hashinit.

Fix malloc race in nfs_node.c

Rename vop_revoke, vn_bwrite, vop_noislocked, vop_nolock, vop_nounlock
to be vop_generic_revoke, vop_generic_bwrite, vop_generic_islocked,
vop_generic_lock and vop_generic_unlock.

Create vop_generic_abortop and propogate change to all file systems.

Fix PR/371.

Get rid of locking in NULLFS (should be mostly unnecessary now except for
forced unmounts).

Updates for VFS Lite 2 + soft update.

back out vfs lite2 till after 2.2

VFS Lite2 Changes

Don't set sillyrename field to 0 for directories, as it's in a union with
the head of the cookie list. Fixes PR 3215, fix supplied by Hiroshi Tezuka
<tezuka@trc.rwcp.or.jp>. Should also fix M_NFSDIROFF memory leak; fvdl

partial sync with netbsd 960418, more to come

Minor cleanups. Checked against Lite2.
(NetBSD's was really just a Lite2's, but w/ 64bit support)

From NetBSD: NFSv3 import (tomorrow's Net's kernel)
Open's patches kept in. i'll possibly take a look at Lite2 soon,
is there smth usefull ?..

From NetBSD: merge with 960217 (still NFSv2)

from cgd; handle 64-bit pointers and longs

branches: 1.1.1;
Initial revision

struct vops is not modified during runtime so use const which moves each
into read-only data segment.
OK deraadt@ tedu@

Drop unnecessary `p' parameter from vget(9).

OK mpi@

Implement proper locking for NFS nodes.

Tested in bulks by many. ok visa@, beck@

Clean up the parameters of VOP_LOCK() and VOP_UNLOCK(). It is always
curproc that does the locking or unlocking, so the proc parameter
is pointless and can be dropped.

OK mpi@, deraadt@

Change the representation of an NFS mount point by caching the root
nodes.

nfs_root() now returns a "locked" vnode, so vput(9) must be called to
release it. Note that this has currently no effect as nfs_lock/unlock
are still stubs.

This will prevent some lock odering problems with upcoming NFSnode
locking.

Tested by landry@, sthen@, visa@, naddy@ and myself.

From NetBSD with some tweaks, ok visa@

Check for possible race after sleeping instead of using a rwlock to
protect insertions in `nm_ntree'.

This will prevent a future lock ordering problem with NFSnode's lock.

ok tedu@, visa@

replace the use of RB macros with the RBT functions.

Remove the unused flags argument from VOP_UNLOCK().

torture tested on amd64, i386 and macppc
ok beck mpi stefan
"the change looks right" deraadt

sync a function's comment with its signature

ok tedu@, with a slight tweak

Remove some includes include-what-you-use claims don't
have any direct symbols used. Tested for indirect use by compiling
amd64/i386/sparc64 kernels.

ok tedu@ deraadt@

to be safe, revert locking change until further tested.

optimize locking a wee bit by holding it for critical parts only

primary change: move uvm_vnode out of vnode, keeping only a pointer.
objective: vnode.h doesn't include uvm_extern.h anymore.
followup changes: include uvm_extern.h or lock.h where necessary.
ok and help from deraadt

add sizes for free(ptr, sizeof(*ptr)). use sizeof(*ptr) for malloc sizes.

remove uneeded proc.h includes
ok mpi@ kspillner@

add a size argument to free. will be used soon, but for now default to 0.
after discussions with beck deraadt kettenis.

Bring back the "End the VOP experiment." diff, naddy's issues where
unrelated, and his alpha is much happier now.

OK deraadt@

Backout the VOP diff until the issues naddy was seeing on alpha (gcc3)
have been resolved.

End the VOP experiment. Instead of the ridicolusly complicated operation
vector setup that has questionable features (that have, as far as I can
tell never been used in practice, atleast not in OpenBSD), remove all
the gunk and favor a simple struct full of function pointers that get
set directly by each of the filesystems.

Removes gobs of ugly code and makes things simpler by a magnitude.

The only downside of this is that we loose the vnoperate feature so
the spec/fifo operations of the filesystems need to be kept in sync
with specfs and fifofs, this is no big deal as the API it self is pretty
static.

Many thanks to armani@ who pulled an earlier version of this diff to
current after c2k10 and Gabriel Kihlman on tech@ for testing.

Liked by many. "come on, find your balls" deraadt@.

No "\n" needed at the end of panic() strings.

Bogus chunks pointed out by matthew@ and miod@. No cookies for
marco@ and jasper@.

ok deraadt@ miod@ matthew@ jasper@ macro@

This fixes a case where we could panic on a null deref with a bad vnode
in nfs_inactive, on a reboot.

The core of the problem was in nfs_nget, when we lose the race to put a new
nfsnode in the tree, we have previously allocated a vnode, which getnewvnode
has done an insmntque into the nfs mp's mntlist. The problem being we then
try again with a new vnode, abandoning this one on the mntlist, leaving
junk there for us to die on when we unmount.

This introduces VLARVAL - so we can indicate in a vnode that the higher
level stuff hiding in v_data is incompletely set up. This flag is then
used by nfs to deal with a halfway set up vnode and release it correctly.

analysis and bogus fix by art@, correct fix by me after serveral failed
attempts and much painful testing by krw@, good suggestions by tedu and miod

ok krw@ oga@ thib@ blambert@ art@

back out previous fix, apparently neither art or i can fix anything right on the first try

vgone doesn't work in other cases of this. I must fix this slightly differntly

This fixes a case where we could panic on a null deref with a bad vnode
in nfs_inactive, on a reboot.

The core of the problem was in nfs_nget, when we lose the race to put a new
nfsnode in the tree, we have previously allocated a vnode, which getnewvnode
has done an insmntque into the nfs mp's mntlist. The problem being we then
try again with a new vnode, abandoning this one on the mntlist, leaving
junk there for us to die on when we unmount.

analysis and bogus fix by art@, correct fix by me. much painful testing by
krw@

ok oga@, art@

Use the nfs_hashlock to protect the nfs_nodetree hanging of the mount.

What can happen is that a recycling of a vnode could pull one from out
under us (since NFS has issues with ref counts...).

Dance around getnewvnode() since we can end up recycling vnodes that
where formerly owned by NFS, causing recursive locking.

We where lucky with the old hashtables has the race was rare but now
with more aggresive recycling we loose, just as theo found out on vax.

help from oga, beck and blambert (beck mostly screamed though).

ok oga@, beck@, blambert@

tiny knf and use curproc directly instead of a setting up a local proc
pointer since its just used in one place.

ok blambert@

Use an RB tree instead of a hashtable for fh/node lookups.

Idea from NetBSD.

OK blambert@

(struct foo *)0 -> NULL, every where I could find it.

OK blambert@

Replace the TRUE/FALSE defines with 1/0 respectively. This doesn't
hurt readability and it was just plain annoying seeing them defined
in every other .c file.

OK blambert@

small indent fix

o nfs_vinvalbuf() is always called with the intrflag as 1, and then
checks if the mount is actually interrutable, and if not sets it 0.
remove this argument from nfs_vinvalbuf and just do the checking inside
the function.
o give nfs_vinvalbuf() a makeover so it looks nice. (spacing, casts, &c);
o Actually pass PCATCH too tsleep() if the mount it interrutable.

ok art@, blambert@

Remove code for variable-sized allocations of NFS filehandles by malloc(),
as it was never used because there were 64-bit buffers already allocated
for filehandles in nfsnode structs.

ok thib@

A bunch of pool_get() + bzero() -> pool_get(..., .. | PR_ZERO)
conversions that should shave a few bytes off the kernel.

ok henning, krw, jsing, oga, miod, and thib (``even though i usually prefer
FOO|BAR''; thanks for looking.

Canonical for() -> queue.h FOREACH macro conversions.
Also, it is historical practice to #include <sys/queue.h>
when using queue.h macros.

ok thib@ krw@

special thanks to krw@ for reminders vice violence

o Avoid putting duplicate entries into the name cache,
add a function to handle cache_enter()'s for us since
we need to update the appropriate timestamps so we don't
miss on lookup;
o Do not purge the dvp in nfs_rmdir(), since there's no need;
o cache the new entry created in nfs_mkdir();
o Do caching of the access modes, this drastically reduces the
amount of over the wire access RPCs we do in the NFSv3 case.

This diff was written by Pedro Martelletto, sometime in 2004.
A lot of people have been running with it at one time or another,
this includes at least markus and matthieu.

OK deraadt@, blambert@

Garbage collect nfsdmap and leftover code. That structure
was used as a part of the "old-style directory caching"
that was removed in 2001;

ok beck@, blambert@

MALLOC/FREE -> malloc/free + M_ZERO.
Uneeded includes and casts...

ok krw@

pedro ok'd this ~3500 line diff which removes the vop argument
"ap = v" comments in under 8 seconds, so it must be ok. and it compiles
too.

Add a name argument to the RWLOCK_INITIALIZER macro.
Pick reasonble names for the locks involved..

ok tedu@, art@

lockmgr -> rwlock for the nfs_hashlock.

ok tedu@

add a name to rwlock so that we can tell where procs are getting stuck
without breaking into ddb. doubles the size of rwlock [1], but moving
forward this really helps. ok/tested pedro fgsch millert krw
[1 - next person to add a field to this struct gets whipped with a wet noodle]

Put vprint() under DIAGNOSTIC, as to save space in generated ramdisks.
Inspiration from miod@, okay deraadt@. Tested on i386, macppc and amd64.

Use NULL where NULL is meant

Remove unnecessary lockmgr() archaism that was costing too much in terms
of panics and bugfixes. Access curproc directly, do not expect a process
pointer as an argument. Should fix many "process context required" bugs.
Incentive and okay millert@, okay marc@. Various testing, thanks.

Use list and queue macros where applicable to make the code easier to read;
no change in compiler assembly output.

NFS commit coalescion: instead of sending a commit for each block, coalesce
these into larger ranges wherever possible.

this should speed up NFS writes quite a bit.

ok art@ millert@ pedro@ tedu@

Remove the advertising clause in the UCB license which Berkeley
rescinded 22 July 1999. Proofed by myself and Theo.

use hash.h for nfs_hash as well as namei's hash
ok art@ costa@

First round of __P removal in sys

Sigh. The diff is out for testing for three months without a single problem.
Two days after commit people report serious lockups all over the place.
Back out nfs locking changes.

More locking in the NFS code

Pool deals fairly well with physical memory shortage, but it doesn't deal
well (not at all) with shortages of the vm_map where the pages are mapped
(usually kmem_map).

Try to deal with it:
- group all information the backend allocator for a pool in a separate
struct. The pool will only have a pointer to that struct.
- change the pool_init API to reflect that.
- link all pools allocating from the same allocator on a linked list.
- Since an allocator is responsible to wait for physical memory it will
only fail (waitok) when it runs out of its backing vm_map, carefully
drain pools using the same allocator so that va space is freed.
(see comments in code for caveats and details).
- change pool_reclaim to return if it actually succeeded to free some
memory, use that information to make draining easier and more efficient.
- get rid of PR_URGENT, noone uses it.

use queue.h macro's
remove register

UBC was a disaster. It worked very good when it worked, but on some
machines or some configurations or in some phase of the moon (we actually
don't know when or why) files disappeared. Since we've not been able to
track down the problem in two weeks intense debugging and we need -current
to be stable, back out everything to a state it had before UBC.

We apologise for the inconvenience.

branches: 1.18.2;
Unlock and drop vnode if VOP_GETATTR fails.

Merge in the unified buffer cache code as found in NetBSD 2001/03/10. The
code is written mostly by Chuck Silvers <chuq@chuq.com>/<chs@netbsd.org>.

Tested for the past few weeks by many developers, should be in a pretty stable
state, but will require optimizations and additional cleanups.

Remove creds from struct buf, move the creds that nfs need into the nfs node.
While in the area, convert nfs node allocation from malloc to pool and do
some cleanups.
Based on the UBC changes in NetBSD. niklas@ ok.

Remove NQNFS

Change handling of NFS root vnode. Moves recognition of NFS root vnode into
nfs_nget.

Root vnode no longer pinned in inode cache. Also, forceable unmounts of an
nfs file system now work even if there are extra references to the NFS root.

branches: 1.13.4;
zap the newhashinit hack.
Add an extra flag to hashinit telling if it should wait in malloc.
update all calls to hashinit.

Fix malloc race in nfs_node.c

Rename vop_revoke, vn_bwrite, vop_noislocked, vop_nolock, vop_nounlock
to be vop_generic_revoke, vop_generic_bwrite, vop_generic_islocked,
vop_generic_lock and vop_generic_unlock.

Create vop_generic_abortop and propogate change to all file systems.

Fix PR/371.

Get rid of locking in NULLFS (should be mostly unnecessary now except for
forced unmounts).

Updates for VFS Lite 2 + soft update.

back out vfs lite2 till after 2.2

VFS Lite2 Changes

Don't set sillyrename field to 0 for directories, as it's in a union with
the head of the cookie list. Fixes PR 3215, fix supplied by Hiroshi Tezuka
<tezuka@trc.rwcp.or.jp>. Should also fix M_NFSDIROFF memory leak; fvdl

partial sync with netbsd 960418, more to come

Minor cleanups. Checked against Lite2.
(NetBSD's was really just a Lite2's, but w/ 64bit support)

From NetBSD: NFSv3 import (tomorrow's Net's kernel)
Open's patches kept in. i'll possibly take a look at Lite2 soon,
is there smth usefull ?..

From NetBSD: merge with 960217 (still NFSv2)

from cgd; handle 64-bit pointers and longs

branches: 1.1.1;
Initial revision

Drop unnecessary `p' parameter from vget(9).

OK mpi@

Implement proper locking for NFS nodes.

Tested in bulks by many. ok visa@, beck@

Clean up the parameters of VOP_LOCK() and VOP_UNLOCK(). It is always
curproc that does the locking or unlocking, so the proc parameter
is pointless and can be dropped.

OK mpi@, deraadt@

Change the representation of an NFS mount point by caching the root
nodes.

nfs_root() now returns a "locked" vnode, so vput(9) must be called to
release it. Note that this has currently no effect as nfs_lock/unlock
are still stubs.

This will prevent some lock odering problems with upcoming NFSnode
locking.

Tested by landry@, sthen@, visa@, naddy@ and myself.

From NetBSD with some tweaks, ok visa@

Check for possible race after sleeping instead of using a rwlock to
protect insertions in `nm_ntree'.

This will prevent a future lock ordering problem with NFSnode's lock.

ok tedu@, visa@

replace the use of RB macros with the RBT functions.

Remove the unused flags argument from VOP_UNLOCK().

torture tested on amd64, i386 and macppc
ok beck mpi stefan
"the change looks right" deraadt

sync a function's comment with its signature

ok tedu@, with a slight tweak

Remove some includes include-what-you-use claims don't
have any direct symbols used. Tested for indirect use by compiling
amd64/i386/sparc64 kernels.

ok tedu@ deraadt@

to be safe, revert locking change until further tested.

optimize locking a wee bit by holding it for critical parts only

primary change: move uvm_vnode out of vnode, keeping only a pointer.
objective: vnode.h doesn't include uvm_extern.h anymore.
followup changes: include uvm_extern.h or lock.h where necessary.
ok and help from deraadt

add sizes for free(ptr, sizeof(*ptr)). use sizeof(*ptr) for malloc sizes.

remove uneeded proc.h includes
ok mpi@ kspillner@

add a size argument to free. will be used soon, but for now default to 0.
after discussions with beck deraadt kettenis.

Bring back the "End the VOP experiment." diff, naddy's issues where
unrelated, and his alpha is much happier now.

OK deraadt@

Backout the VOP diff until the issues naddy was seeing on alpha (gcc3)
have been resolved.

End the VOP experiment. Instead of the ridicolusly complicated operation
vector setup that has questionable features (that have, as far as I can
tell never been used in practice, atleast not in OpenBSD), remove all
the gunk and favor a simple struct full of function pointers that get
set directly by each of the filesystems.

Removes gobs of ugly code and makes things simpler by a magnitude.

The only downside of this is that we loose the vnoperate feature so
the spec/fifo operations of the filesystems need to be kept in sync
with specfs and fifofs, this is no big deal as the API it self is pretty
static.

Many thanks to armani@ who pulled an earlier version of this diff to
current after c2k10 and Gabriel Kihlman on tech@ for testing.

Liked by many. "come on, find your balls" deraadt@.

No "\n" needed at the end of panic() strings.

Bogus chunks pointed out by matthew@ and miod@. No cookies for
marco@ and jasper@.

ok deraadt@ miod@ matthew@ jasper@ macro@

This fixes a case where we could panic on a null deref with a bad vnode
in nfs_inactive, on a reboot.

The core of the problem was in nfs_nget, when we lose the race to put a new
nfsnode in the tree, we have previously allocated a vnode, which getnewvnode
has done an insmntque into the nfs mp's mntlist. The problem being we then
try again with a new vnode, abandoning this one on the mntlist, leaving
junk there for us to die on when we unmount.

This introduces VLARVAL - so we can indicate in a vnode that the higher
level stuff hiding in v_data is incompletely set up. This flag is then
used by nfs to deal with a halfway set up vnode and release it correctly.

analysis and bogus fix by art@, correct fix by me after serveral failed
attempts and much painful testing by krw@, good suggestions by tedu and miod

ok krw@ oga@ thib@ blambert@ art@

back out previous fix, apparently neither art or i can fix anything right on the first try

vgone doesn't work in other cases of this. I must fix this slightly differntly

This fixes a case where we could panic on a null deref with a bad vnode
in nfs_inactive, on a reboot.

The core of the problem was in nfs_nget, when we lose the race to put a new
nfsnode in the tree, we have previously allocated a vnode, which getnewvnode
has done an insmntque into the nfs mp's mntlist. The problem being we then
try again with a new vnode, abandoning this one on the mntlist, leaving
junk there for us to die on when we unmount.

analysis and bogus fix by art@, correct fix by me. much painful testing by
krw@

ok oga@, art@

Use the nfs_hashlock to protect the nfs_nodetree hanging of the mount.

What can happen is that a recycling of a vnode could pull one from out
under us (since NFS has issues with ref counts...).

Dance around getnewvnode() since we can end up recycling vnodes that
where formerly owned by NFS, causing recursive locking.

We where lucky with the old hashtables has the race was rare but now
with more aggresive recycling we loose, just as theo found out on vax.

help from oga, beck and blambert (beck mostly screamed though).

ok oga@, beck@, blambert@

tiny knf and use curproc directly instead of a setting up a local proc
pointer since its just used in one place.

ok blambert@