347040 |
03-May-2019 |
rmacklem |
MFC: r346365 Fix the NFSv4.0 server so that it does not support NFSv4.1 attributes.
During inspection of a packet trace, I noticed that an NFSv4.0 mount reported that it supported attributes that are only defined for NFSv4.1. In practice, this bug appears to be benign, since NFSv4.0 clients will not use attributes that were added for NFSv4.1. However, this was not correct and this patch fixes the NFSv4.0 server so that it only supports attributes defined for NFSv4.0. It also adds a definition for NFSv4.1 attributes that can only be set, although it is only defined as 0 for now. This is anticipation of the addition of support for the NFSv4.1 mode+mask attribute soon. |
346466 |
21-Apr-2019 |
rmacklem |
MFC: r345992, r346087 Add INET6 support for the upcalls to the nfsuserd daemon.
The kernel code uses UDP to do upcalls to the nfsuserd(8) daemon to get updates to the username<->uid and groupname<->gid mappings. A change to AF_LOCAL last year had to be reverted, since it could result in vnode locking issues on the AF_LOCAL socket. This patch adds INET6 support and the required #ifdef INET and INET6 to the code. |
338308 |
24-Aug-2018 |
rmacklem |
MFC: r337438 Allow newnfs_request() to retry all callback RPCs with an NFSERR_DELAY reply.
The code in newnfs_request() retries RPCs that get a reply of NFSERR_DELAY, but exempts certain NFSv4 operations. However, for callback RPCs, there should not be any exemptions at this time. The code would have erroneously exempted the CBRECALL callback, since it has the same operation number as the CLOSE operation. This patch fixes this by checking for a callback RPC (indicated by clp != NULL) and not checking for exempt operations for callbacks. This would have only affected the NFSv4 server when delegations are enabled (they are not enabled by default) and the client replies to CBRECALL with NFSERR_DELAY. This may never actually happen. Spotted during code inspection. |
338132 |
21-Aug-2018 |
rmacklem |
MFC: r336839 Modify the NFSv4.1 server so that it allows ReclaimComplete as done by ESXi 6.7.
I believe that a ReclaimComplete with rca_one_fs == TRUE is only to be used after a file system has been transferred to a different file server. However, RFC5661 is somewhat vague w.r.t. this and the ESXi 6.7 client does both a ReclaimComplete with rca_one_fs == TRUE and one with ReclaimComplete with rca_one_fs == FALSE. Therefore, just ignore the rca_one_fs == TRUE operation and return NFS_OK without doing anything instead of replying NFS4ERR_NOTSUPP. This allows the ESXi 6.7 NFSv4.1 client to do a mount. After discussion on the NFSv4 IETF working group mailing list, doing this along with setting a flag to note that a ReclaimComplete with rca_one_fs TRUE was an appropriate way to handle this. The flag that indicates that a ReclaimComplete with rca_one_fs == TRUE was done may be used to disable replies of NFS4ERR_GRACE for non-reclaim state operations in a future commit.
This patch along with r332790, r334492 and r336357 allow ESXi 6.7 NFSv4.1 mounts work ok. ESX 6.5 NFSv4.1 mounts do not work well, due to what I believe are violations of RFC-5661 and should not be used. |
337058 |
01-Aug-2018 |
rmacklem |
MFC: r336357 Modify the reasons for not issuing a delegation in the NFSv4.1 server.
The ESXi NFSv4.1 client will generate warning messages when the reason for not issuing a delegation is two. Two refers to a resource limit and I do not see why it would be considered invalid. However it probably was not the best choice of reason for not issuing a delegation. This patch changes the reasons used to ones that the ESXi client doesn't complain about. This change does not affect the FreeBSD client and does not appear to affect behaviour of the Linux NFSv4.1 client. RFC5661 defines these "reasons" but does not give any guidance w.r.t. which ones are more appropriate to return to a client. |
336899 |
30-Jul-2018 |
rmacklem |
MFC: r334966 Add a couple of safety belt checks to the NFSv4.1 client related to sessions.
There were a couple of cases in newnfs_request() that it assumed that it was an NFSv4.1 mount with a session. This should always be the case when a Sequence operation is in the reply or the server replies NFSERR_BADSESSION. However, if a server was broken and sent an erroneous reply, these safety belt checks should avoid trouble. The one check required a small tweak to nfsmnt_mdssession() so that it returns NULL when there is no session instead of the offset of the field in the structure (0x8 for i386). This patch should have no effect on normal operation of the client. Found by inspection during pNFS server development. |
336846 |
28-Jul-2018 |
rmacklem |
MFC: r334492 Add the BindConnectiontoSession operation to the NFSv4.1 server.
Under some fairly unusual circumstances, the Linux NFSv4.1 client is doing a BindConnectiontoSession operation for TCP connections. It is also used by the ESXi6.5 NFSv4.1 client. This patch adds this operation to the NFSv4.1 server.
PR: 226493 |
336422 |
17-Jul-2018 |
rmacklem |
MFC: r333645 End grace for the NFSv4 server if all mounts do ReclaimComplete.
The NFSv4 protocol requires that the server only allow reclaim of state and not issue any new open/lock state for a grace period after booting. The NFSv4.0 protocol required this grace period to be greater than the lease duration (over 2minutes). For NFSv4.1, the client tells the server that it has done reclaiming state by doing a ReclaimComplete operation. If all NFSv4 clients are NFSv4.1, the grace period can end once all the clients have done ReclaimComplete, shortening the time period considerably. This patch does this. If there are any NFSv4.0 mounts, the grace period will still be over 2minutes. This change is only an optimization and does not affect correct operation. |
336179 |
10-Jul-2018 |
rmacklem |
MFC: r333508 Add support for the TestStateID operation to the NFSv4.1 server.
The Linux client now uses the TestStateID operation, so this patch adds support for it to the NFSv4.1 server. The FreeBSD client never uses this operation, so it should not be affected. |
325407 |
04-Nov-2017 |
rmacklem |
MFC: r324506 Fix forced dismount when a pNFS mount is hung on a DS.
When a "pnfs" NFSv4.1 mount is hung because of an unresponsive DS, a forced dismount wouldn't work, because the RPC socket for the DS was not being closed. This patch fixes this. This will only affect "pnfs" mounts where the pNFS server's DS is unresponsive (crashed or network partitioned or...). Found during testing of the pNFS server. |
321877 |
01-Aug-2017 |
trasz |
MFC r320359:
Add vfs.nfsd.nfsd_enable_uidtostring, which works just like vfs.nfsd.nfsd_enable_stringtouid, but in reverse - when set to 1, it forces the NFSv4 server to return numeric UIDs and GIDs instead of "user@domain" strings. This helps with clients that can't translate returned identifiers, eg when rerooting.
The same can be achieved by just never running nfsuserd(8), but the sysctl is useful to toggle the behaviour back and forth without rebooting.
MFC r320409:
Revert part of r320359, as suggested by rmacklem@. That case is only used for nfsuserd -manage-gids and shouldn't depend on sysctl.
MFC r321196:
Rename vfs.nfsd.enable_uidtostring to vfs.nfs.enable_uidtostring. It applies to both NFS client and NFS server, and is useful for both. This is different from vfs.nfsd.enable_stringtouid, which is specific to server side.
Sponsored by: DARPA, AFRL |
321031 |
15-Jul-2017 |
rmacklem |
MFC: r320345 Add support to the NFSv4.1/pNFS client for commits through the DS.
A NFSv4.1/pNFS server using File Layout can specify that Commit operations are to be done against the DS instead of MDS. Since no extant pNFS server did this, the code was untested and "#ifdef notyet". The FreeBSD pNFS server I am developing does specify that Commits be done through the DS, so the code has been enabled/tested. This patch should only affect the case of a pNFS server that specfies Commits through the DS.
Relnotes: yes |
320637 |
04-Jul-2017 |
rmacklem |
MFC: r320062, r320070, r320126 This is a partial merge of only the NFS changes and not the maxbcachebuf tunable.
The NFS client changes make the code handle different I/O sizes more correctly. However, with the limit at 64K, they are not actually necessary. This MFC is mainly being done so that subsequent MFCs to the NFS code will merge easily. |
320617 |
03-Jul-2017 |
rmacklem |
MFC: r319882 Define NFS_MAXXDR as the upper bound on XDR overhead in an NFS RPC.
This definition is a part of the maxiotune2 patch that will be committed soon. |
317984 |
08-May-2017 |
rmacklem |
MFC: r317345 Make the NFSv4 client to use a write open for reading if allowed by the server.
An NFSv4 server has the option of allowing a Read to be done using a Write Open. If this is not allowed, the server will return NFSERR_OPENMODE. This patch attempts the read with a write open and then disables this if the server replies NFSERR_OPENMODE. This change will avoid some uses of the special stateids. This will be useful for pNFS/DS Reads, since they cannot use special stateids. It will also be useful for any NFSv4 server that does not support reading via the special stateids. It has been tested against both types of NFSv4 server. |
317930 |
07-May-2017 |
rmacklem |
MFC: r317276 Don't set ND_NOMOREDATA for a failed Setattr operation (NFSv4).
The NFSv4 Setattr operation always has reply data even when it fails, so don't set the ND_NOMOREDATA for it. This would only affect unusual cases where Setattr fails and the RPC code wants to parse the rest of the compound. Detected during recent development related to the pNFS server. |
317927 |
07-May-2017 |
rmacklem |
MFC: r317275, r317344 Don't create a backchannel for a DS connection.
An NFSv4.1 client connection to a Data Server (DS) should not have a backchannel. This patch fixes the NFSv4.1/pNFS client to not do a backchannel for this case. Found during recent testing with the pNFS server under development. |
317917 |
07-May-2017 |
rmacklem |
MFC: r317269 Set default uid/gid to nobody/nogroup for NFSv4 mapping.
The default uid/gid for NFSv4 are set by the nfsuserd(8) daemon. However, they were 0 until the nfsuserd(8) was run. Since it is possible to use NFSv4 without running the nfsuserd(8) daemon, set them to nobody/nogroup initially. Without this patch, the values would be set by the nfsuserd(8) daemon and left changed even if the nfsuserd(8) daemon was killed. The default values of 0 meant that setting a group to "wheel" would fail even when done by root. It also adds a definition of GID_NOGROUP to sys/conf.h. |
317579 |
29-Apr-2017 |
rmacklem |
MFC: r316829 Remove unused "cred" argument to ncl_flush().
The "cred" argument of ncl_flush() is unused and it was confusing to have the code passing in NULL for this argument in some cases. This patch deletes this argument. There is no semantic change because of this patch. |
317525 |
27-Apr-2017 |
rmacklem |
MFC: r316792 Add an NFSv4.1 mount option for "use one openowner".
Some NFSv4.1 servers such as AmazonEFS can only support a small fixed number of open_owner4s. This patch adds a mount option called "oneopenown" that can be used for NFSv4.1 mounts to make the client do all Opens with the same open_owner4 string. This option can only be used with NFSv4.1 and may not work correctly when Delegations are is use.
Differential Revision: https://reviews.freebsd.org/D8988 |
317429 |
26-Apr-2017 |
rmacklem |
MFC: r316692 Set initial values for nfsstatfs in the NFSv4 client.
The AmazonEFS NFSv4.1 server does not support the FILES_FREE and FILES_TOTAL attributes. As such, an NFSv4.1 mount to the server would return garbage for these values. This patch initializes the fields of the nfsstatfs structure, so that "df" and friends will at least return consistent bogus values. |
317419 |
25-Apr-2017 |
rmacklem |
MFC: r316666 Fix the NFSv4.1 client for NFSERR_BADSESSION recovery via ReclaimComplete.
For the ReclaimComplete operation, the RPC layer should not loop on NFSERR_BADSESSION. If it does, the recovery thread (nfscl) can get stuck looping and will not do a recovery. This patch fixes it so it does not loop. This bug only affects NFSv4.1 and only when a server reboots. |
317404 |
25-Apr-2017 |
rmacklem |
MFC: r310491 Fix NFSv4.1 client recovery from NFS4ERR_BAD_SESSION errors.
For most NFSv4.1 servers, a NFS4ERR_BAD_SESSION error is a rare failure that indicates that the server has lost session/open/lock state. However, recent testing by cperciva@ against the AmazonEFS server found several problems with client recovery from this due to it generating this failure frequently. Briefly, the problems fixed are: - If all session slots were in use at the time of the failure, some processes would continue to loop waiting for a slot on the old session forever. - If an RPC that doesn't use open/lock state failed with NFS4ERR_BAD_SESSION, it would fail the RPC/syscall instead of initiating recovery and then looping to retry the RPC. - If a successful reply to an RPC for an old session wasn't processed until after a new session was created for a NFS4ERR_BAD_SESSION error, it would erroneously update the new session and corrupt it. - The use of the first element of the session list in the nfs mount structure (which is always the current metadata session) was slightly racey. With changes for the above problems it became more racey, so all uses of this head pointer was wrapped with a NFSLOCKMNT()/NFSUNLOCKMNT(). - Although the kernel malloc() usually allocates more bytes than requested and, as such, this wouldn't have caused problems, the allocation of a session structure was 1 byte smaller than it should have been. (Null termination byte for the string not included in byte count.)
There are probably still problems with a pNFS data server that fails with NFS4ERR_BAD_SESSION, but I have no server that does this to test against (the AmazonEFS server doesn't do pNFS), so I can't fix these yet.
Although this patch is fairly large, it should only affect the handling of NFS4ERR_BAD_SESSION error replies from an NFSv4.1 server. Thanks go to cperciva@ for the extension testing he did to help isolate/fix these problems. |
314667 |
04-Mar-2017 |
avg |
MFC r283291: don't use CALLOUT_MPSAFE with callout_init()
The main purpose of this MFC is to reduce conflicts for other merges. Parts of the original change have already "trickled down" via individual MFCs. |
310322 |
20-Dec-2016 |
cperciva |
MFC r308708: Reduce verbosity of warnings about truncating NFS fileids to 32-bit inode numbers. |
306809 |
07-Oct-2016 |
emaste |
MFC r299199: Add nid_namelen bounds check to nfssvc system call
This is only allowed by root and only used by the nfs daemon, which should not provide an incorrect value. However, it's still good practice to validate data provided by userland.
PR: 206626 |
306663 |
03-Oct-2016 |
rmacklem |
Revert r306659 since the userland changes won't merge and this would break the build. |
306659 |
03-Oct-2016 |
rmacklem |
MFC: r304026 Update the nfsstats structure to include the changes needed by the patch in D1626 plus changes so that it includes counts for NFSv4.1 (and the draft of NFSv4.2). Also, make all the counts uint64_t and add a vers field at the beginning, so that future revisions can easily be implemented. There is code in place to handle the old vesion of the nfsstats structure for backwards binary compatibility.
Subsequent commits will update nfsstat(8) to use the new fields. |
299222 |
07-May-2016 |
rmacklem |
MFC: r298495 Fix a LOR in the NFSv4.1 server.
The ordering of acquisition of the state and session mutexes was reversed in two cases executed when an NFSv4.1 client created/freed a session. Since clients will typically do this only when mounting and dismounting, the likelyhood of causing a deadlock was low but possible. This can only occur for NFSv4.1 mounts, since the others do not use sessions. This was detected while testing the pNFS server/client where the client crashed during dismounting. The patch also reorders the unlocks, although that isn't necessary for correct operation. |
299204 |
06-May-2016 |
rmacklem |
MFC: r297837 Bruce Evans reported that there was a performance regression between the old and new NFS clients. He did a good job of isolating the problem which was caused by the new NFS client not setting the post write mtime correctly. The new NFS client code was cloned from the old client, but was incorrect, because the mtime in the nfs vnode's cache wasn't yet updated. This patch fixes this problem. The patch also adds missing mutex locking. |
292895 |
30-Dec-2015 |
mckusick |
MFC of 291244, 291380, 291459, 291460, 291671, and 291743:
This MFC includes changes to better manage the vnode freelist and to streamline the allocation and freeing of vnodes.
Note that to maintain the KPI the VI_AGE flag is left defined in sys/vnode.h though its use is dropped as described in 291380. To maintain KBI the vfs.vlru_alloc_cache_src sysctl variable remains though it no longer has any effect as described in 291244.
MFC of 291244: Move the comment about resident pages preventing vnode from leaving active list, into the header comment for vdrop(), which is the function that decides whether to leave the vnode on the list. Note that dirty page write-out in vinactive() is asynchronous.
Discussed with: alc Sponsored by: The FreeBSD Foundation
MFC of 291380: Remove VI_AGE vnode iflag, it is unused.
Noted by: bde Sponsored by: The FreeBSD Foundation
MFC of 291459: For performance reasons, it is useful to have a single string used as the name of a filesystem when setting it as the first parameter to the getnewvnode() function. Most filesystems call getnewvnode from just one place so can use a literal string as the first parameter. However, NFS calls getnewvnode from two places, so we create a global constant string that can be used by the two instances. This change also collapses two instances of getnewvnode() in the UFS filesystem to a single call.
Reviewed by: kib Tested by: Peter Holm
MFC of 291460: As the kernel allocates and frees vnodes, it fully initializes them on every allocation and fully releases them on every free. These are not trivial costs: it starts by zeroing a large structure then initializes a mutex, a lock manager lock, an rw lock, four lists, and six pointers. And looking at vfs.vnodes_created, these operations are being done millions of times an hour on a busy machine.
As a performance optimization, this code update uses the uma_init and uma_fini routines to do these initializations and cleanups only as the vnodes enter and leave the vnode_zone. With this change the initializations are only done kern.maxvnodes times at system startup and then only rarely again. The frees are done only if the vnode_zone shrinks which never happens in practice. For those curious about the avoided work, look at the vnode_init() and vnode_fini() functions in kern/vfs_subr.c to see the code that has been removed from the main vnode allocation/free path.
Reviewed by: kib Tested by: Peter Holm
MFC of 291671: We need to zero out the union of pointers in a freed vnode structure.
Fix from: Mateusz Guzik Tested by: Jason Unovitch
MFC of 291743: We need to zero out the clustering variables in a freed vnode structure. For completeness add a VNASSERT that there are no threads waiting on a range lock (this was previously checked on every vnode free).
Reported by; Rick Macklem Fix from: Mateusz Guzik |
292307 |
16-Dec-2015 |
rmacklem |
MFC: r291638 Fix the memory leak that occurs when the nfscommon.ko module is unloaded. This leak was introduced by r291527 (r292223 in stable/10). Since the nfscommon.ko module is rarely unloaded, this leak would not have been much of an issue. |
292223 |
14-Dec-2015 |
rmacklem |
MFC: r291527 Add kernel support to the NFS server for the "-manage-gids" option that will be added to the nfsuserd daemon in a future commit. It modifies the cache used by NFSv4 for name<-->id translation (both username/uid and group/gid) to support this. When "-manage-gids" is set, the server looks up each uid for the RPC and uses the list of groups cached in the server instead of the list of groups provided in the RPC request. The cached group list is acquired for the cache by the nfsuserd daemon via getgrouplist(3). This avoids the 16 groups limit for the list in the RPC request. Since the cache is now used for every RPC when "-manage-gids" is enabled, the code also modifies the cache to use a separate mutex for each hash list instead of a single global mutex. |
291869 |
05-Dec-2015 |
rmacklem |
MFC: r291150 When the nfsd threads are terminated, the NFSv4 server state (opens, locks, etc) is retained, which I believe is correct behaviour. However, for NFSv4.1, the server also retained a reference to the xprt (RPC transport socket structure) for the backchannel. This caused svcpool_destroy() to not call SVC_DESTROY() for the xprt and allowed a socket upcall to occur after the mutexes in the svcpool were destroyed, causing a crash. This patch fixes the code so that the backchannel xprt structure is dereferenced just before svcpool_destroy() is called, so the code does do an SVC_DESTROY() on the xprt, which shuts down the socket upcall. |
284216 |
10-Jun-2015 |
rmacklem |
MFC: r283635 Make the size of the hash tables used by the NFSv4 server tunable. No appreciable change in performance was observed after increasing the sizes of these tables and then testing with a single client. However, there was an email that indicated high CPU overheads for a heavily loaded NFSv4 and it is hoped that increasing the sizes of the hash tables via these tunables might help. The tables remain the same size by default. |
282933 |
14-May-2015 |
rmacklem |
MFC: r281960 MAXBSIZE defines both the largest UFS block size and the largest size for a buffer in the buffer cache. This patch defines a new constant MAXBCACHEBUF, which is the largest size for a buffer in the buffer cache. Having a separate constant allows MAXBCACHEBUF to be set larger than MAXBSIZE on a per-architecture basis, so that NFS can do larger read/writes for these architectures. It modifies sys/param.h so that BKVASIZE can also be set on a per-architecture basis. A couple of cases where NFS used MAXBSIZE instead of NFS_MAXBSIZE is fixed as well. |
282271 |
30-Apr-2015 |
rmacklem |
MFC: r281628 mav@ has found that NFS servers exporting ZFS file systems can perform better when using a 128K read/write data size. This patch changes NFS_MAXDATA from 64K to 128K so that clients can use 128K for NFS mounts to allow this. The patch also renames NFS_MAXDATA to NFS_SRVMAXIO so that it is clear that it applies to the NFS server side only. It also avoids a name conflict with the NFS_MAXDATA defined in rpcsvc/nfs_prot.h, that is used for userland RPC. |
276327 |
28-Dec-2014 |
peterj |
MFH r275941: Adjust the test of a KASSERT to better match the intent.
This assertion was added in r246213 as a guard against corrupted mbufs arriving from drivers, the key distinguishing factor of said mbufs being that they had a negative length. Given we're in a while loop specifically designed to skip over zero-length mbufs, panicking on a zero-length mbuf seems incorrect.
Suggested by: rmacklem MFH go-ahead: benno Approved by: grog (co-mentor) |
269655 |
07-Aug-2014 |
kib |
MFC r269347: Do not generate 1000 unique lock names for nfsrc hash chain locks. Shorten the names of some nfs mutexes. |
269398 |
01-Aug-2014 |
rmacklem |
MFC: r268115 Merge the NFSv4.1 server code in projects/nfsv4.1-server over into head. The code is not believed to have any effect on the semantics of non-NFSv4.1 server behaviour. It is a rather large merge, but I am hoping that there will not be any regressions for the NFS server. |
265714 |
08-May-2014 |
rmacklem |
MFC: r265252 The new draft specification for NFSv4.0 specifies that a server should either accept owner and owner_group strings that are just the digits of the uid/gid or return NFS4ERR_BADOWNER. This patch adds a sysctl vfs.nfsd.enable_stringtouid, which can be set to enable the server w.r.t. accepting numeric string. It also ensures that NFS4ERR_BADOWNER is returned if numeric uid/gid strings are not enabled. This fixes the server for recent Linux nfs4 clients that use numeric uid/gid strings by default. |
265434 |
06-May-2014 |
rmacklem |
MFC: r264672 Modify the Lookup RPC for NFSv4 so that it acquires directory attributes. This allows the client to cache directory names when they are looked up, reducing the Lookup RPC count by about 40% for software builds. |
261055 |
22-Jan-2014 |
mav |
MFC r260229, r260258, r260367, r260390, r260459, r260648: Rework NFS Duplicate Request Cache cleanup logic.
- Introduce additional hash to group requests by hash of sockref. This allows to process TCP acknowledgements without looping though all the cache, and as result allows to do it every time. - Indroduce additional callbacks to notify application layer about sockets disconnection. Without this last few requests processed just before socket disconnection never processed their ACKs and stuck in cache for many hours. - Implement transport-specific method for tracking reply acknowledgements. New implementation does not cross multiple stack layers to get the data and does not have race conditions that previously made some requests stuck in cache. This could be done more efficiently at sockbuf layer, but that would broke some KBIs, while I don't know other consumers for it aside NFS. - Instead of traversing all DRC twice per request, run cleaning only once per request, and except in some conditions traverse only single hash slot at a time.
Together this limits NFS DRC growth only to situations of real connectivity problems. If network is working well, and so all replies are acknowledged, cache remains almost empty even after hours of heavy load. Without this change on the same test cache was growing to many thousand requests even with perfectly working local network.
As another result this reduces CPU time spent on the DRC handling during SPEC NFS benchmark from about 10% to 0.5%.
Sponsored by: iXsystems, Inc. |
260159 |
01-Jan-2014 |
rmacklem |
MFC: r259854 The NFSv4 server would call VOP_SETATTR() with a shared locked vnode when a Getattr for a file is done by a client other than the one that holds the file's delegation. This would only happen when delegations are enabled and the problem is fixed by this patch. |
260143 |
31-Dec-2013 |
rmacklem |
MFC: r259801 The NFSv4 client was passing both the p and cred arguments to nfsv4_fillattr() as NULLs for the Getattr callback. This caused nfsv4_fillattr() to not fill in the Change attribute for the reply. I believe this was a violation of the RFC, but had little effect on server behaviour. This patch passes a non-NULL p argument to fix this. |
259238 |
11-Dec-2013 |
rmacklem |
MFC: r257901 Fix an NFSv4.1 client specific case where a forced dismount would hang. The hang occurred in nfsv4_setsequence() when it couldn't find an available session slot and is fixed by checking for a forced dismount in progress and just returning for this case. |
259207 |
11-Dec-2013 |
rmacklem |
MFC: r257598 During code inspection, I spotted that there was a code path where CLNT_CONTROL() would be called on "client" after it was released via CLNT_RELEASE(). It was unlikely that this code path gets executed and I have not heard of any problem report caused by this bug. This patch fixes the code so that this cannot happen. |
256281 |
10-Oct-2013 |
gjb |
Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.
Approved by: re (implicit) Sponsored by: The FreeBSD Foundation
|
254925 |
26-Aug-2013 |
jhb |
Remove most of the remaining sysctl name list macros. They were only ever intended for use in sysctl(8) and it has not used them for many years.
Reviewed by: bde Tested by: exp-run by bdrewery
|
254337 |
14-Aug-2013 |
rmacklem |
Fix several performance related issues in the new NFS server's DRC for NFS over TCP. - Increase the size of the hash tables. - Create a separate mutex for each hash list of the TCP hash table. - Single thread the code that deletes stale cache entries. - Add a tunable called vfs.nfsd.tcphighwater, which can be increased to allow the cache to grow larger, avoiding the overhead of frequent scans to delete stale cache entries. (The default value will result in frequent scans to delete stale cache entries, analagous to what the pre-patched code does.) - Add a tunable called vfs.nfsd.cachetcp that can be used to disable DRC caching for NFS over TCP, since the old NFS server didn't DRC cache TCP. It also adjusts the size of nfsrc_floodlevel dynamically, so that it is always greater than vfs.nfsd.tcphighwater.
For UDP the algorithm remains the same as the pre-patched code, but the tunable vfs.nfsd.udphighwater can be used to allow the cache to grow larger and reduce the overhead caused by frequent scans for stale entries. UDP also uses a larger hash table size than the pre-patched code.
Reported by: wollman Tested by: wollman (earlier version of patch) Submitted by: ivoras (earlier patch) Reviewed by: jhb (earlier version of patch) MFC after: 1 month
|
253506 |
20-Jul-2013 |
rmacklem |
The NFSv4 server incorrectly assumed that the high order words of the attribute bitmap argument would be non-zero. This caused an interoperability problem for a recent patch to the Linux NFSv4 client. The Linux folks have changed their patch to avoid this, but this patch fixes the problem on the server.
Reported and tested by: Andre Heider (a.heider@gmail.com) MFC after: 3 days
|
253049 |
09-Jul-2013 |
rmacklem |
Add support for host-based (Kerberos 5 service principal) initiator credentials to the kernel rpc. Modify the NFSv4 client to add support for the gssname and allgssname mount options to use this capability. Requires the gssd daemon to be running with the "-h" option.
Reviewed by: jhb
|
249592 |
17-Apr-2013 |
ken |
Revamp the old NFS server's File Handle Affinity (FHA) code so that it will work with either the old or new server.
The FHA code keeps a cache of currently active file handles for NFSv2 and v3 requests, so that read and write requests for the same file are directed to the same group of threads (reads) or thread (writes). It does not currently work for NFSv4 requests. They are more complex, and will take more work to support.
This improves read-ahead performance, especially with ZFS, if the FHA tuning parameters are configured appropriately. Without the FHA code, concurrent reads that are part of a sequential read from a file will be directed to separate NFS threads. This has the effect of confusing the ZFS zfetch (prefetch) code and makes sequential reads significantly slower with clients like Linux that do a lot of prefetching.
The FHA code has also been updated to direct write requests to nearby file offsets to the same thread in the same way it batches reads, and the FHA code will now also send writes to multiple threads when needed.
This improves sequential write performance in ZFS, because writes to a file are now more ordered. Since NFS writes (generally less than 64K) are smaller than the typical ZFS record size (usually 128K), out of order NFS writes to the same block can trigger a read in ZFS. Sending them down the same thread increases the odds of their being in order.
In order for multiple write threads per file in the FHA code to be useful, writes in the NFS server have been changed to use a LK_SHARED vnode lock, and upgrade that to LK_EXCLUSIVE if the filesystem doesn't allow multiple writers to a file at once. ZFS is currently the only filesystem that allows multiple writers to a file, because it has internal file range locking. This change does not affect the NFSv4 code.
This improves random write performance to a single file in ZFS, since we can now have multiple writers inside ZFS at one time.
I have changed the default tuning parameters to a 22 bit (4MB) window size (from 256K) and unlimited commands per thread as a result of my benchmarking with ZFS.
The FHA code has been updated to allow configuring the tuning parameters from loader tunable variables in addition to sysctl variables. The read offset window calculation has been slightly modified as well. Instead of having separate bins, each file handle has a rolling window of bin_shift size. This minimizes glitches in throughput when shifting from one bin to another.
sys/conf/files: Add nfs_fha_new.c and nfs_fha_old.c. Compile nfs_fha.c when either the old or the new NFS server is built.
sys/fs/nfs/nfsport.h, sys/fs/nfs/nfs_commonport.c: Bring in changes from Rick Macklem to newnfs_realign that allow it to operate in blocking (M_WAITOK) or non-blocking (M_NOWAIT) mode.
sys/fs/nfs/nfs_commonsubs.c, sys/fs/nfs/nfs_var.h: Bring in a change from Rick Macklem to allow telling nfsm_dissect() whether or not to wait for mallocs.
sys/fs/nfs/nfsm_subs.h: Bring in changes from Rick Macklem to create a new nfsm_dissect_nonblock() inline function and NFSM_DISSECT_NONBLOCK() macro.
sys/fs/nfs/nfs_commonkrpc.c, sys/fs/nfsclient/nfs_clkrpc.c: Add the malloc wait flag to a newnfs_realign() call.
sys/fs/nfsserver/nfs_nfsdkrpc.c: Setup the new NFS server's RPC thread pool so that it will call the FHA code.
Add the malloc flag argument to newnfs_realign().
Unstaticize newnfs_nfsv3_procid[] so that we can use it in the FHA code.
sys/fs/nfsserver/nfs_nfsdsocket.c: In nfsrvd_dorpc(), add NFSPROC_WRITE to the list of RPC types that use the LK_SHARED lock type.
sys/fs/nfsserver/nfs_nfsdport.c: In nfsd_fhtovp(), if we're starting a write, check to see whether the underlying filesystem supports shared writes. If not, upgrade the lock type from LK_SHARED to LK_EXCLUSIVE.
sys/nfsserver/nfs_fha.c: Remove all code that is specific to the NFS server implementation. Anything that is server-specific is now accessed through a callback supplied by that server's FHA shim in the new softc.
There are now separate sysctls and tunables for the FHA implementations for the old and new NFS servers. The new NFS server has its tunables under vfs.nfsd.fha, the old NFS server's tunables are under vfs.nfsrv.fha as before.
In fha_extract_info(), use callouts for all server-specific code. Getting file handles and offsets is now done in the individual server's shim module.
In fha_hash_entry_choose_thread(), change the way we decide whether two reads are in proximity to each other. Previously, the calculation was a simple shift operation to see whether the offsets were in the same power of 2 bucket. The issue was that there would be a bucket (and therefore thread) transition, even if the reads were in close proximity. When there is a thread transition, reads wind up going somewhat out of order, and ZFS gets confused.
The new calculation simply tries to see whether the offsets are within 1 << bin_shift of each other. If they are, the reads will be sent to the same thread.
The effect of this change is that for sequential reads, if the client doesn't exceed the max_reqs_per_nfsd parameter and the bin_shift is set to a reasonable value (22, or 4MB works well in my tests), the reads in any sequential stream will largely be confined to a single thread.
Change fha_assign() so that it takes a softc argument. It is now called from the individual server's shim code, which will pass in the softc.
Change fhe_stats_sysctl() so that it takes a softc parameter. It is now called from the individual server's shim code. Add the current offset to the list of things printed out about each active thread.
Change the num_reads and num_writes counters in the fha_hash_entry structure to 32-bit values, and rename them num_rw and num_exclusive, respectively, to reflect their changed usage.
Add an enable sysctl and tunable that allows the user to disable the FHA code (when vfs.XXX.fha.enable = 0). This is useful for before/after performance comparisons.
nfs_fha.h: Move most structure definitions out of nfs_fha.c and into the header file, so that the individual server shims can see them.
Change the default bin_shift to 22 (4MB) instead of 18 (256K). Allow unlimited commands per thread.
sys/nfsserver/nfs_fha_old.c, sys/nfsserver/nfs_fha_old.h, sys/fs/nfsserver/nfs_fha_new.c, sys/fs/nfsserver/nfs_fha_new.h: Add shims for the old and new NFS servers to interface with the FHA code, and callbacks for the
The shims contain all of the code and definitions that are specific to the NFS servers.
They setup the server-specific callbacks and set the server name for the sysctl and loader tunable variables.
sys/nfsserver/nfs_srvkrpc.c: Configure the RPC code to call fhaold_assign() instead of fha_assign().
sys/modules/nfsd/Makefile: Add nfs_fha.c and nfs_fha_new.c.
sys/modules/nfsserver/Makefile: Add nfs_fha_old.c.
Reviewed by: rmacklem Sponsored by: Spectra Logic MFC after: 2 weeks
|
248255 |
13-Mar-2013 |
jhb |
Revert 195703 and 195821 as this special stop handling in NFS is now implemented via VFCF_SBDRY rather than passing PBDRY to individual sleep calls.
|
248188 |
12-Mar-2013 |
glebius |
Finish r243882: mechanically substitute flags from historic mbuf allocator with malloc(9) flags within sys.
Sponsored by: Nginx, Inc.
|
247602 |
02-Mar-2013 |
pjd |
Merge Capsicum overhaul:
- Capability is no longer separate descriptor type. Now every descriptor has set of its own capability rights.
- The cap_new(2) system call is left, but it is no longer documented and should not be used in new code.
- The new syscall cap_rights_limit(2) should be used instead of cap_new(2), which limits capability rights of the given descriptor without creating a new one.
- The cap_getrights(2) syscall is renamed to cap_rights_get(2).
- If CAP_IOCTL capability right is present we can further reduce allowed ioctls list with the new cap_ioctls_limit(2) syscall. List of allowed ioctls can be retrived with cap_ioctls_get(2) syscall.
- If CAP_FCNTL capability right is present we can further reduce fcntls that can be used with the new cap_fcntls_limit(2) syscall and retrive them with cap_fcntls_get(2).
- To support ioctl and fcntl white-listing the filedesc structure was heavly modified.
- The audit subsystem, kdump and procstat tools were updated to recognize new syscalls.
- Capability rights were revised and eventhough I tried hard to provide backward API and ABI compatibility there are some incompatible changes that are described in detail below:
CAP_CREATE old behaviour: - Allow for openat(2)+O_CREAT. - Allow for linkat(2). - Allow for symlinkat(2). CAP_CREATE new behaviour: - Allow for openat(2)+O_CREAT.
Added CAP_LINKAT: - Allow for linkat(2). ABI: Reuses CAP_RMDIR bit. - Allow to be target for renameat(2).
Added CAP_SYMLINKAT: - Allow for symlinkat(2).
Removed CAP_DELETE. Old behaviour: - Allow for unlinkat(2) when removing non-directory object. - Allow to be source for renameat(2).
Removed CAP_RMDIR. Old behaviour: - Allow for unlinkat(2) when removing directory.
Added CAP_RENAMEAT: - Required for source directory for the renameat(2) syscall.
Added CAP_UNLINKAT (effectively it replaces CAP_DELETE and CAP_RMDIR): - Allow for unlinkat(2) on any object. - Required if target of renameat(2) exists and will be removed by this call.
Removed CAP_MAPEXEC.
CAP_MMAP old behaviour: - Allow for mmap(2) with any combination of PROT_NONE, PROT_READ and PROT_WRITE. CAP_MMAP new behaviour: - Allow for mmap(2)+PROT_NONE.
Added CAP_MMAP_R: - Allow for mmap(PROT_READ). Added CAP_MMAP_W: - Allow for mmap(PROT_WRITE). Added CAP_MMAP_X: - Allow for mmap(PROT_EXEC). Added CAP_MMAP_RW: - Allow for mmap(PROT_READ | PROT_WRITE). Added CAP_MMAP_RX: - Allow for mmap(PROT_READ | PROT_EXEC). Added CAP_MMAP_WX: - Allow for mmap(PROT_WRITE | PROT_EXEC). Added CAP_MMAP_RWX: - Allow for mmap(PROT_READ | PROT_WRITE | PROT_EXEC).
Renamed CAP_MKDIR to CAP_MKDIRAT. Renamed CAP_MKFIFO to CAP_MKFIFOAT. Renamed CAP_MKNODE to CAP_MKNODEAT.
CAP_READ old behaviour: - Allow pread(2). - Disallow read(2), readv(2) (if there is no CAP_SEEK). CAP_READ new behaviour: - Allow read(2), readv(2). - Disallow pread(2) (CAP_SEEK was also required).
CAP_WRITE old behaviour: - Allow pwrite(2). - Disallow write(2), writev(2) (if there is no CAP_SEEK). CAP_WRITE new behaviour: - Allow write(2), writev(2). - Disallow pwrite(2) (CAP_SEEK was also required).
Added convinient defines:
#define CAP_PREAD (CAP_SEEK | CAP_READ) #define CAP_PWRITE (CAP_SEEK | CAP_WRITE) #define CAP_MMAP_R (CAP_MMAP | CAP_SEEK | CAP_READ) #define CAP_MMAP_W (CAP_MMAP | CAP_SEEK | CAP_WRITE) #define CAP_MMAP_X (CAP_MMAP | CAP_SEEK | 0x0000000000000008ULL) #define CAP_MMAP_RW (CAP_MMAP_R | CAP_MMAP_W) #define CAP_MMAP_RX (CAP_MMAP_R | CAP_MMAP_X) #define CAP_MMAP_WX (CAP_MMAP_W | CAP_MMAP_X) #define CAP_MMAP_RWX (CAP_MMAP_R | CAP_MMAP_W | CAP_MMAP_X) #define CAP_RECV CAP_READ #define CAP_SEND CAP_WRITE
#define CAP_SOCK_CLIENT \ (CAP_CONNECT | CAP_GETPEERNAME | CAP_GETSOCKNAME | CAP_GETSOCKOPT | \ CAP_PEELOFF | CAP_RECV | CAP_SEND | CAP_SETSOCKOPT | CAP_SHUTDOWN) #define CAP_SOCK_SERVER \ (CAP_ACCEPT | CAP_BIND | CAP_GETPEERNAME | CAP_GETSOCKNAME | \ CAP_GETSOCKOPT | CAP_LISTEN | CAP_PEELOFF | CAP_RECV | CAP_SEND | \ CAP_SETSOCKOPT | CAP_SHUTDOWN)
Added defines for backward API compatibility:
#define CAP_MAPEXEC CAP_MMAP_X #define CAP_DELETE CAP_UNLINKAT #define CAP_MKDIR CAP_MKDIRAT #define CAP_RMDIR CAP_UNLINKAT #define CAP_MKFIFO CAP_MKFIFOAT #define CAP_MKNOD CAP_MKNODAT #define CAP_SOCK_ALL (CAP_SOCK_CLIENT | CAP_SOCK_SERVER)
Sponsored by: The FreeBSD Foundation Reviewed by: Christoph Mallon <christoph.mallon@gmx.de> Many aspects discussed with: rwatson, benl, jonathan ABI compatibility discussed with: kib
|
247116 |
21-Feb-2013 |
jhb |
Further refine the handling of stop signals in the NFS client. The changes in r246417 were incomplete as they did not add explicit calls to sigdeferstop() around all the places that previously passed SBDRY to _sleep(). In addition, nfs_getcacheblk() could trigger a write RPC from getblk() resulting in sigdeferstop() recursing. Rather than manually deferring stop signals in specific places, change the VFS_*() and VOP_*() methods to defer stop signals for filesystems which request this behavior via a new VFCF_SBDRY flag. Note that this has to be a VFC flag rather than a MNTK flag so that it works properly with VFS_MOUNT() when the mount is not yet fully constructed. For now, only the NFS clients are set this new flag in VFS_SET().
A few other related changes: - Add an assertion to ensure that TDF_SBDRY doesn't leak to userland. - When a lookup request uses VOP_READLINK() to follow a symlink, mark the request as being on behalf of the thread performing the lookup (cnp_thread) rather than using a NULL thread pointer. This causes NFS to properly handle signals during this VOP on an interruptible mount.
PR: kern/176179 Reported by: Russell Cattelan (sigdeferstop() recursion) Reviewed by: kib MFC after: 1 month
|
246417 |
06-Feb-2013 |
jhb |
Rework the handling of stop signals in the NFS client. The changes in 195702, 195703, and 195821 prevented a thread from suspending while holding locks inside of NFS by forcing the thread to fail sleeps with EINTR or ERESTART but defer the thread suspension to the user boundary. However, this had the effect that stopping a process during an NFS request could abort the request and trigger EINTR errors that were visible to userland processes (previously the thread would have suspended and completed the request once it was resumed).
This change instead effectively masks stop signals while in the NFS client. It uses the existing TDF_SBDRY flag to effect this since SIGSTOP cannot be masked directly. Also, instead of setting PBDRY on individual sleeps, the NFS client now sets the TDF_SBDRY flag around each NFS request and stop signals are masked for all sleeps during that region (the previous change missed sleeps in lockmgr locks). The end result is that stop signals sent to threads performing an NFS request are completely ignored until after the NFS request has finished processing and the thread prepares to return to userland. This restores the behavior of stop signals being transparent to userland processes while still preventing threads from suspending while holding NFS locks.
Reviewed by: kib MFC after: 1 month
|
246213 |
01-Feb-2013 |
kib |
Assert that the mbuf in the chain has sane length. Proper place for this check is somewhere in the network code, but this assertion already proven to be useful in catching what seems to be driver bugs causing NFS scrambling random memory.
Discussed with: rmacklem MFC after: 1 week
|
245909 |
25-Jan-2013 |
jhb |
Further cleanups to use of timestamps in NFS: - Use NFSD_MONOSEC (which maps to time_uptime) instead of the seconds portion of wall-time stamps to manage timeouts on events. - Remove unused nd_starttime from the per-request structure in the new NFS server. - Use nanotime() for the modification time on a delegation to get as precise a time as possible. - Use time_second instead of extracting the second from a call to getmicrotime().
Submitted by: bde (3) Reviewed by: bde, rmacklem MFC after: 2 weeks
|
245508 |
16-Jan-2013 |
jhb |
Use the VA_UTIMES_NULL flag to detect when NULL was passed to utimes() instead of comparing the desired time against the current time as a heuristic.
Reviewed by: rmacklem MFC after: 1 week
|
245476 |
15-Jan-2013 |
jhb |
- More properly handle interrupted NFS requests on an interruptible mount by returning an error of EINTR rather than EACCES. - While here, bring back some (but not all) of the NFS RPC statistics lost when krpc was committed.
Reviewed by: rmacklem MFC after: 1 week
|
244042 |
08-Dec-2012 |
rmacklem |
Move the NFSv4.1 client patches over from projects/nfsv4.1-client to head. I don't think the NFS client behaviour will change unless the new "minorversion=1" mount option is used. It includes basic NFSv4.1 support plus support for pNFS using the Files Layout only. All problems detecting during an NFSv4.1 Bakeathon testing event in June 2012 have been resolved in this code and it has been tested against the NFSv4.1 server available to me. Although not reviewed, I believe that kib@ has looked at it.
|
243882 |
05-Dec-2012 |
glebius |
Mechanically substitute flags from historic mbuf allocator with malloc(9) flags within sys.
Exceptions:
- sys/contrib not touched - sys/mbuf.h edited manually
|
243782 |
02-Dec-2012 |
rmacklem |
Add an nfssvc() option to the kernel for the new NFS client which dumps out the actual options being used by an NFS mount. This will be used to implement a "-m" option for nfsstat(1).
Reviewed by: alfred MFC after: 2 weeks
|
241896 |
22-Oct-2012 |
kib |
Remove the support for using non-mpsafe filesystem modules.
In particular, do not lock Giant conditionally when calling into the filesystem module, remove the VFS_LOCK_GIANT() and related macros. Stop handling buffers belonging to non-mpsafe filesystems.
The VFS_VERSION is bumped to indicate the interface change which does not result in the interface signatures changes.
Conducted and reviewed by: attilio Tested by: pho
|
240720 |
20-Sep-2012 |
rmacklem |
Modify the NFSv4 client so that it can handle owner and owner_group strings that consist entirely of digits, interpreting them as the uid/gid number. This change was needed since new (>= 3.3) Linux servers reply with these strings by default. This change is mandated by the rfc3530bis draft. Reported on freebsd-stable@ under the Subject heading "Problem with Linux >= 3.3 as NFSv4 server" by Norbert Aschendorff on Aug. 20, 2012.
Tested by: norbert.aschendorff at yahoo.de Reviewed by: jhb MFC after: 2 weeks
|
240289 |
09-Sep-2012 |
rmacklem |
Add a simple printf() based debug facility to the new nfs client. Use it for a printf() that can be harmlessly generated for mmap()'d files. It will be used extensively for the NFSv4.1 client. Debugging printf()s are enabled by setting vfs.nfs.debuglevel to a non-zero value. The higher the value, the more debugging printf()s.
Reviewed by: jhb MFC after: 2 weeks
|
239039 |
04-Aug-2012 |
kib |
The header uma_int.h is internal uma header, unused by this source file. Do not include it needlessly.
Reviewed by: alc MFC after: 1 week
|
235568 |
17-May-2012 |
rmacklem |
A problem with the NFSv4 server was reported by Andrew Leonard to freebsd-fs@, where the setfacl of an NFSv4 acl would fail. This was caused by the VOP_ACLCHECK() call for ZFS replying EOPNOTSUPP. After discussion with rwatson@, it was determined that a call to VOP_ACLCHECK() before doing VOP_SETACL() is not required. This patch fixes the problem by deleting the VOP_ACLCHECK() call.
Tested by: Andrew Leonard (previous version) MFC after: 1 week
|
232821 |
11-Mar-2012 |
kib |
Remove fifo.h. The only used function declaration from the header is migrated to sys/vnode.h.
Submitted by: gianni
|
231805 |
16-Feb-2012 |
rmacklem |
Delete a couple of out of date comments that are no longer true in the new NFS client.
Requested by: bde MFC after: 1 week
|
230345 |
20-Jan-2012 |
rmacklem |
Martin Cracauer reported a problem to freebsd-current@ under the subject "Data corruption over NFS in -current". During investigation of this, I came across an ugly bogusity in the new NFS client where it replaced the cr_uid with the one used for the mount. This was done so that "system operations" like the NFSv4 Renew would be performed as the user that did the mount. However, if any other thread shares the credential with the one doing this operation, it could do an RPC (or just about anything else) as the wrong cr_uid. This patch fixes the above, by using the mount credentials instead of the one provided as an argument for this case. It appears to have fixed Martin's problem. This patch is needed for NFSv4 mounts and NFSv3 mounts against some non-FreeBSD servers that do not put post operation attributes in the NFSv3 Statfs RPC reply.
Tested by: Martin Cracauer (cracauer at cons.org) Reviewed by: jhb MFC after: 2 weeks
|
229956 |
11-Jan-2012 |
rmacklem |
jwd@ reported via email that the "CacheSize" field reported by "nfsstat -e -s" would go negative after using the "-z" option to zero out the stats. This patch fixes that by not zeroing out the srvcache_size field for "-z", since it is the size of the cache and not a counter.
MFC after: 2 weeks
|
229802 |
08-Jan-2012 |
rmacklem |
opt_inet6.h was missing from some files in the new NFS subsystem. The effect of this was, for clients mounted via inet6 addresses, that the DRC cache would never have a hit in the server. It also broke NFSv4 callbacks when an inet6 address was the only one available in the client. This patch fixes the above, plus deletes opt_inet6.h from a couple of files it is not needed for.
MFC after: 2 weeks
|
229272 |
02-Jan-2012 |
ed |
Use strchr() and strrchr().
It seems strchr() and strrchr() are used more often than index() and rindex(). Therefore, simply migrate all kernel code to use it.
For the XFS code, remove an empty line to make the code identical to the code in the Linux kernel.
|
228757 |
21-Dec-2011 |
rmacklem |
jwd@ reported a problem via email where the old NFS client would get a reply of EEXIST from an NFS server when a Mkdir RPC was retried, for an NFS over UDP mount. Upon investigation, it was found that the client was retransmitting the Mkdir RPC request over UDP, but with a different xid. As such, the retransmitted message would miss the Duplicate Request Cache in the server, causing it to reply EEXIST. The kernel client side UDP rpc code has two timers. The first one causes a retransmit using the same xid and socket and was set to a fixed value of 3seconds. (The default can be overridden via CLSET_RETRY_TIMEOUT.) The second one creates a new socket and xid and should be larger than the first. However, both NFS clients were setting the second timer to nm_timeo ("timeout=<value>" mount argument), which defaulted to 1second, so the first timer would never time out. This patch fixes both NFS clients so that they set the first timer using nm_timeo and makes the second timer larger than the first one.
Reported by: jwd Tested by: jwd Reviewed by: jhb MFC after: 2 weeks
|
228217 |
03-Dec-2011 |
rmacklem |
Post r223774, the NFSv4 client no longer has multiple instances of the same lock_owner4 string. As such, the handling of cleanup of lock_owners could be simplified. This simplification permitted the client to do a ReleaseLockOwner operation when the process that the lock_owner4 string represents, has exited. This permits the server to release any storage related to the lock_owner4 string before the associated open is closed. Without this change, it is possible to exhaust a server's storage when a long running process opens a file and then many child processes do locking on the file, because the open doesn't get closed. A similar patch was applied to the Linux NFSv4 client recently so that it wouldn't exhaust a server's storage.
Reviewed by: zack MFC after: 2 weeks
|
227760 |
20-Nov-2011 |
rmacklem |
Add two arguments to the nfsrpc_rellockown() function in the NFSv4 client. This does not change the client's behaviour, but prepares the code so that nfsrpc_rellockown() can be called elsewhere in a future commit.
MFC after: 2 weeks
|
227743 |
20-Nov-2011 |
rmacklem |
Post r223774 the NFSv4 client never uses the linked list with the head nfsc_defunctlockowner. This patch simply removes the code that loops through this always empty list, since the code no longer does anything useful. It should not have any effect on the client's behaviour.
MFC after: 2 weeks
|
224121 |
17-Jul-2011 |
zack |
Revert revision 224079 as Rick pointed out that I would be calling VOP_PATHCONF without the vnode lock held.
Implicitly approved by: zml (mentor)
|
224117 |
16-Jul-2011 |
rmacklem |
The new NFSv4 client handled NFSERR_GRACE as a fatal error for the remove and rename operations. Some NFSv4 servers will report NFSERR_GRACE for these operations. This patch changes the behaviour of the client so that it handles NFSERR_GRACE like NFSERR_DELAY for non-state related operations like remove and rename. It also exempts the delegreturn operation from handling within newnfs_request() for NFSERR_DELAY/NFSERR_GRACE so that it can handle NFSERR_GRACE in the same manner as before. This problem was resolved thanks to discussion with bfields at fieldses.org. The problem was identified at the recent NFSv4 ineroperability bakeathon.
MFC after: 2 weeks
|
224086 |
16-Jul-2011 |
zack |
Add DEXITCODE plumbing to NFS.
Isilon has the concept of an in-memory exit-code ring that saves the last exit code of a function and allows for stack tracing. This is very helpful when debugging tough issues.
This patch is essentially a no-op for BSD at this point, until we upstream the dexitcode logic itself. The patch adds DEXITCODE calls to every NFS function that returns an errno error code. A number of code paths were also reorganized to have single exit paths, to reduce code duplication.
Submitted by: David Kwan <dkwan@isilon.com> Reviewed by: rmacklem Approved by: zml (mentor) MFC after: 2 weeks
|
224082 |
16-Jul-2011 |
zack |
Simple find/replace of VOP_UNLOCK -> NFSVOPUNLOCK. This is done so that NFSVOPUNLOCK can be modified later to add enhanced logging and assertions.
Reviewed by: rmacklem Approved by: zml (mentor) MFC after: 2 weeks
|
224081 |
16-Jul-2011 |
zack |
Simple find/replace of vn_lock -> NFSVOPLOCK. This is done so that NFSVOPLOCK can be modified later to add enhanced logging and assertions.
Reviewed by: rmacklem Approved by: zml (mentor) MFC after: 2 weeks
|
224080 |
16-Jul-2011 |
zack |
Remove unnecessary thread pointer from VOPLOCK macros and current users.
Reviewed by: rmacklem Approved by: zml (mentor) MFC after: 2 weeks
|
224079 |
16-Jul-2011 |
zack |
Change loadattr and fillattr to ask the file system for the pathconf variable.
Small modification where VOP_PATHCONF was being called directly.
Reviewed by: rmacklem Approved by: zml (mentor) MFC after: 2 weeks
|
224078 |
16-Jul-2011 |
zack |
Move nfsvno_pathconf to be accessible to sys/fs/nfs; no functionality change.
Reviewed by: rmacklem Approved by: zml (mentor) MFC after: 2 weeks
|
224077 |
16-Jul-2011 |
zack |
Small acl patch to return the aclerror that comes back from nfsrv_dissectacl(). This fixes a problem where ATTRNOTSUPP was being returned instead of BADOWNER.
Reviewed by: rmacklem Approved by: zml (mentor) MFC after: 2 weeks
|
223747 |
03-Jul-2011 |
rmacklem |
Modify the new NFSv4 client so that it appends a file handle to the lock_owner4 string that goes on the wire. Also, add code to do a ReleaseLockOwner Op on the lock_owner4 string before a Close. Apparently not all NFSv4 servers handle multiple instances of the same lock_owner4 string, at least not in a compatible way. This patch avoids having multiple instances, except for one unusual case, which will be fixed by a future commit. Found at the recent NFSv4 interoperability Bakeathon.
Tested by: tdh at excfb.com MFC after: 2 weeks
|
223657 |
28-Jun-2011 |
rmacklem |
Fix the new NFSv4 client so that it doesn't fill the cached mode attribute in as 0 when doing writes. The change adds the Mode attribute plus the others except Owner and Owner_group to the list requested by the NFSv4 Write Operation. This fixed a problem where an executable file built by "cc" would get mode 0111 instead of 0755 for some NFSv4 servers. Found at the recent NFSv4 interoperability Bakeathon.
Tested by: tdh at excfb.com MFC after: 2 weeks
|
223441 |
22-Jun-2011 |
rmacklem |
Plug an mbuf leak in the new NFS client that occurred when a server replied NFS3ERR_JUKEBOX/NFS4ERR_DELAY to an rpc. This affected both NFSv3 and NFSv4. Found during testing at the recent NFSv4 interoperability Bakeathon.
MFC after: 2 weeks
|
223436 |
22-Jun-2011 |
rmacklem |
Fix the new NFSv4 client so that it uses the same uid as was used for doing a mount when performing system operations on AUTH_SYS mounts. This resolved an issue when mounting a Linux server. Found during testing at the recent NFSv4 interoperability Bakeathon.
MFC after: 2 weeks
|
223309 |
19-Jun-2011 |
rmacklem |
Fix the kgssapi so that it can be loaded as a module. Currently the NFS subsystems use five of the rpcsec_gss/kgssapi entry points, but since it was not obvious which others might be useful, all nineteen were included. Basically the nineteen entry points are set in a structure called rpc_gss_entries and inline functions defined in sys/rpc/rpcsec_gss.h check for the entry points being non-NULL and then call them. A default value is returned otherwise. Requested by rwatson.
Reviewed by: jhb MFC after: 2 weeks
|
223280 |
18-Jun-2011 |
rmacklem |
Add DTrace support to the new NFS client. This is essentially cloned from the old NFS client, plus additions for NFSv4. A review of this code is in progress, however it was felt by the reviewer that it could go in now, before code slush. Any changes required by the review can be committed as bug fixes later.
|
222719 |
05-Jun-2011 |
rmacklem |
The new NFSv4 client was erroneously using "p" instead of "p_leader" for the "id" for POSIX byte range locking. I think this would only have affected processes created by rfork(2) with the RFTHREAD flag specified. This patch fixes that by passing the "id" down through the various functions from nfs_advlock().
MFC after: 2 weeks
|
222389 |
27-May-2011 |
rmacklem |
Fix the new NFS client so that it handles NFSv4 state correctly during a forced dismount. This required that the exclusive and shared (refcnt) sleep lock functions check for MNTK_UMOUNTF before sleeping, so that they won't block while nfscl_umount() is getting rid of the state. As such, a "struct mount *" argument was added to the locking functions. I believe the only remaining case where a forced dismount can get hung in the kernel is when a thread is already attempting to do a TCP connect to a dead server when the krpc client structure called nr_client is NULL. This will only happen just after a "mount -u" with options that force a new TCP connection is done, so it shouldn't be a problem in practice.
MFC after: 2 weeks
|
222289 |
25-May-2011 |
rmacklem |
Fix the new NFS client so that it correctly sets the "must_commit" argument for a write RPC when it succeeds for the first one and fails for a subsequent RPC within the same call to the function. This makes it compatible with the old NFS client for this case.
MFC after: 2 weeks
|
221973 |
15-May-2011 |
rmacklem |
Change the sysctl naming for the old and new NFS clients to vfs.oldnfs.xxx and vfs.nfs.xxx respectively. This makes the default nfs client use vfs.nfs.xxx after r221124.
|
221523 |
06-May-2011 |
mav |
Increase NFS_TICKINTVL value from 10 to 500. Now that callout does useful things only once per second, so other 99 calls per second were useless and just don't allow idle system to sleep properly.
Reviewed by: rmacklem
|
221462 |
04-May-2011 |
rmacklem |
Add a comment noting that the NFS code assumes that the values of error numbers in sys/errno.h will be the same as the ones specified by the NFS RFCs and that the code needs to be fixed if error numbers are changed in sys/errno.h.
Suggested by: Peter Jeremy MFC after: 2 weeks
|
221439 |
04-May-2011 |
rmacklem |
Add kernel support for NFSSVC_ZEROCLTSTATS and NFSSVC_ZEROSRVSTATS so that they can be used by nfsstat(1) to implement the "-z" option for the new NFS subsystem.
MFC after: 2 weeks
|
221438 |
04-May-2011 |
rmacklem |
Revert r221306, since NFSSVC_ZEROSTATS zero'd both client and server stats, when separate modifiers for NFSSVC_GETSTATS for each of client and server stats is what it required by nfsstat(1).
|
221306 |
01-May-2011 |
rmacklem |
Add the kernel support needed to zero out the nfsstats structure for the new NFS subsystem. This will be used by nfsstats.c to implement the "-z" option.
MFC after: 2 weeks
|
221040 |
25-Apr-2011 |
rmacklem |
Modify the experimental (newnfs) NFS client so that it uses the same diskless NFS root code as the regular client, which was moved to sys/nfs by r221032. This fixes the newnfs client so that it can do an NFSv3 diskless root file system.
MFC after: 2 weeks
|
221014 |
25-Apr-2011 |
rmacklem |
Modify the experimental NFS client so that it uses the same "struct nfs_args" as the regular NFS client. This is needed so that the old mount(2) syscall will work and it makes sharing of the diskless NFS root code easier. Eary in the porting exercise I introduced a new revision of nfs_args, but didn't actually need it, thanks to nmount(2). I re-introduced the NFSMNT_KERB flag, since it does essentially the same thing and the old one would not have been used because it never worked. I also added a few new NFSMNT_xxx flags to sys/nfsclient/nfs_args.h that are used by the experimental NFS client.
MFC after: 2 weeks
|
220752 |
17-Apr-2011 |
rmacklem |
Get rid of the "nfscl: consider increasing kern.ipc.maxsockbuf" message that was generated when doing experimental NFS client mounts. I put that message in because the krpc would hang with the default size for mounts that used large rsize/wsize values. Since the bug that caused these hangs was fixed by r213756, I think the message is no longer needed.
MFC after: 2 weeks
|
220751 |
17-Apr-2011 |
rmacklem |
Fix up some of the sysctls for the experimental NFS client so that they use the same names as the regular client. Also add string descriptions for them.
MFC after: 2 weeks
|
220739 |
17-Apr-2011 |
rmacklem |
Change some defaults in the experimental NFS client to be the same as the regular NFS client for NFSv3. The main one is making use of a reserved port# the default. Also, set the retry limit for TCP the same and fix the code so that it doesn't disable readdirplus for NFSv4.
MFC after: 2 weeks
|
220732 |
16-Apr-2011 |
rmacklem |
Add a lktype flags argument to nfscl_nget() and ncl_nget() in the experimental NFS client so that its nfs_lookup() function can use cn_lkflags in a manner analagous to the regular NFS client.
MFC after: 2 weeks
|
220683 |
15-Apr-2011 |
rmacklem |
Change the experimental NFS client so that it creates nfsiod threads in the same manner as the regular NFS client after r214026 was committed. This resolves the lors fixed by r214026 and its predecessors for the regular client.
Reviewed by: jhb MFC after: 2 weeks
|
220648 |
14-Apr-2011 |
rmacklem |
Fix the experimental NFSv4 server so that it uses VOP_PATHCONF() to determine if a file system supports NFSv4 ACLs. Since VOP_PATHCONF() must be called with a locked vnode, the function is called before nfsvno_fillattr() and the result is passed in as an extra argument.
MFC after: 2 weeks
|
220645 |
14-Apr-2011 |
rmacklem |
Modify the experimental NFSv4 server so that it handles crossing of server mount points properly. The functions nfsvno_fillattr() and nfsv4_fillattr() were modified to take the extra arguments that are the mount point, a flag to indicate that it is a file system root and the mounted on fileno. The mount point argument needs to be busy when nfsvno_fillattr() is called, since the vp argument is not locked.
Reviewed by: kib MFC after: 2 weeks
|
220530 |
10-Apr-2011 |
rmacklem |
Add some cleanup code to the module unload operation for the experimental NFS server, so that it doesn't leak memory when unloaded. However, unloading the NFSv4 server is not recommended, since all NFSv4 state will be lost by the unload and clients will have to recover the state after a server reload/restart as if the server crashed/rebooted.
MFC after: 2 weeks
|
217922 |
27-Jan-2011 |
gnn |
Quick fix to a comment.
|
217535 |
18-Jan-2011 |
rmacklem |
Fix the experimental NFSv4 server so that it uses VOP_ACCESSX() to check for VREAD_ACL instead of VOP_ACCESS().
MFC after: 3 days
|
217432 |
14-Jan-2011 |
rmacklem |
Modify the experimental NFSv4 server so that it posts a SIGUSR2 signal to the master nfsd daemon whenever the stable restart file has been modified. This will allow the master nfsd daemon to maintain an up to date backup copy of the file. This is enabled via the nfssvc() syscall, so that older nfsd daemons will not be signaled.
Reviewed by: jhb MFC after: 1 week
|
217066 |
06-Jan-2011 |
rmacklem |
Delete the NFS_STARTWRITE() and NFS_ENDWRITE() macros that obscured vn_start_write() and vn_finished_write() for the old OpenBSD port, since most uses have been replaced by the correct calls.
MFC after: 12 days
|
217063 |
06-Jan-2011 |
rmacklem |
Since the VFS_LOCK_GIANT() code in the experimental NFS server is broken and the major file systems are now all mpsafe, modify the server so that it will only export mpsafe file systems. This was discussed on freebsd-fs@ and removes a fair bit of crufty code.
MFC after: 12 days
|
216875 |
01-Jan-2011 |
rmacklem |
Add support for shared vnode locks for the Read operation in the experimental NFSv4 server.
Reviewed by: kib MFC after: 2 weeks
|
216784 |
28-Dec-2010 |
rmacklem |
Delete the nfsvno_localconflict() function in the experimental NFS server since it is no longer used and is broken.
MFC after: 2 weeks
|
216700 |
25-Dec-2010 |
rmacklem |
Modify the experimental NFS server so that it uses LK_SHARED for RPC operations when it can. Since VFS_FHTOVP() currently always gets an exclusively locked vnode and is usually called at the beginning of each RPC, the RPCs for a given vnode will still be serialized. As such, passing a lock type argument to VFS_FHTOVP() would be preferable to doing the vn_lock() with LK_DOWNGRADE after the VFS_FHTOVP() call.
Reviewed by: kib MFC after: 2 weeks
|
216693 |
24-Dec-2010 |
rmacklem |
Add an argument to nfsvno_getattr() in the experimental NFS server, so that it can avoid calling VOP_ISLOCKED() when the vnode is known to be locked. This will allow LK_SHARED to be used for these cases, which happen to be all the cases that can use LK_SHARED. This does not fix any bug, but it reduces the number of calls to VOP_ISLOCKED() and prepares the code so that it can be switched to using LK_SHARED in a future patch.
Reviewed by: kib MFC after: 2 weeks
|
214406 |
26-Oct-2010 |
rmacklem |
Add a flag to the experimental NFSv4 client to indicate when delegations are being returned for reasons other than a Recall. Also, re-organize nfscl_recalldeleg() slightly, so that it leaves clearing NMODIFIED to the ncl_flush() call and invalidates the attribute cache after flushing. It is hoped that these changes might fix the problem others have seen when using the NFSv4 client with delegations enabled, since I can't reliably reproduce the problem. These changes only affect the client when doing NFSv4 mounts with delegations enabled.
MFC after: 10 days
|
214255 |
23-Oct-2010 |
rmacklem |
Modify the experimental NFSv4 server's file handle hash function to use the generic hash32_buf() function. Although adding the bytes seemed sufficient for UFS and ZFS, since most of the bytes are the same for file handles on the same volume, this might not be sufficient for other file systems. Use of a generic function also seems preferable to one specific to NFSv4.
Suggested by: gleb.kurtsou at gmail.com MFC after: 10 days
|
214224 |
22-Oct-2010 |
rmacklem |
Modify the file handle hash function in the experimental NFS server so that it will work better for non-UFS file systems. The new function simply sums the bytes of the fh_fid field of fhandle_t.
MFC after: 10 days
|
212439 |
10-Sep-2010 |
rmacklem |
Fix the NFSVNO_CMPFH() macro in the experimental NFS server so that it works correctly for ZFS file handles. It is possible to have two ZFS file handles that differ only in the bytes in the fid_reserved field of the generic "struct fid" and comparing the bytes in fid_data didn't catch this case. This patch changes the macro to compare all bytes of "struct fid".
Tested by: gull at gull.us MFC after: 2 weeks
|
211951 |
28-Aug-2010 |
rmacklem |
The timer routine in the experimental NFS server did not acquire the correct mutex when checking nfsv4root_lock. Although this could be fixed by adding mutex lock/unlock calls, zack.kirsch at isilon.com suggested a better fix that uses a non-blocking acquisition of a reference count on nfsv4root_lock. This fix allows the weird NFSLOCKSTATE(); NFSUNLOCKSTATE(); synchronization to be deleted. This patch applies this fix.
Tested by: zack.kirsch at isilon.com MFC after: 2 weeks
|
210786 |
03-Aug-2010 |
rmacklem |
Modify the return value for nfscl_mustflush() from boolean_t, which I mistakenly thought was correct w.r.t. style(9), back to int and add the checks for != 0. This is just a stylistic modification.
MFC after: 1 week
|
210268 |
19-Jul-2010 |
rmacklem |
For the experimental NFSv4 server's dumplocks operation, add the MPSAFE flag to cn_flags so that it doesn't panic. The panics weren't seen since nfsdumpstate(8) is broken for the "-l" case, so this was never done. I'll do a separate commit to fix nfsdumpstate(8).
Submitted by: zack.kirsch at isilon.com MFC after: 2 weeks
|
210201 |
18-Jul-2010 |
rmacklem |
Change the nfscl_mustflush() function in the experimental NFSv4 client to return a boolean_t in order to make it more compatible with style(9).
MFC after: 2 weeks
|
207785 |
08-May-2010 |
rmacklem |
Fix typos in macros.
PR: kern/146375 Submitted by: simon AT comsys.ntu-kpi.kiev.ua MFC after: 1 week
|
207764 |
08-May-2010 |
rmacklem |
Patch the experimental NFS client so that it works for NFSv2 by adding the necessary mapping from NFSv3 procedure numbers to NFSv2 procedure numbers when doing NFSv2 RPCs.
MFC after: 1 week
|
207170 |
24-Apr-2010 |
rmacklem |
An NFSv4 server will reply NFSERR_GRACE for non-recovery RPCs during the grace period after startup. This grace period must be at least the lease duration, which is typically 1-2 minutes. It seems prudent for the experimental NFS client to wait a few seconds before retrying such an RPC, so that the server isn't flooded with non-recovery RPCs during recovery. This patch adds an argument to nfs_catnap() to implement a 5 second delay for this case.
MFC after: 1 week
|
207082 |
22-Apr-2010 |
rmacklem |
When the experimental NFS client is handling an NFSv4 server reboot with delegations enabled, the recovery could fail if the renew thread is trying to return a delegation, since it will not do the recovery. This patch fixes the above by having nfscl_recalldeleg() fail with the I/O operations returning EIO, so that they will be attempted later. Most of the patch consists of adding an argument to various functions to indicate the delegation recall case where this needs to be done.
MFC after: 1 week
|
206818 |
18-Apr-2010 |
rmacklem |
Avoid extraneous recovery cycles in the experimental NFS client when an NFSv4 server reboots, by doing two things. 1 - Make the function that acquires a stateid for I/O operations block until recovery is complete, so that it doesn't acquire out of date stateids. 2 - Only allow a recovery once every 1/2 of a lease duration, since the NFSv4 server must provide a recovery grace period of at least a lease duration. This should avoid recoveries caused by an out of date stateid that was acquired for an I/O op. just before a recovery cycle started.
MFC after: 1 week
|
206688 |
15-Apr-2010 |
rmacklem |
The experimental NFS client was not filling in recovery credentials for opens done locally in the client when a delegation for the file was held. This could cause the client to crash in crsetgroups() when recovering from a server crash/reboot. This patch fills in the recovery credentials for this case, in order to avoid the client crash. Also, add KASSERT()s to the credential copy functions, to catch any other cases where the credentials aren't filled in correctly.
MFC after: 1 week
|
205941 |
30-Mar-2010 |
rmacklem |
This patch should fix handling of byte range locks locally on the server for the experimental nfs server. When enabled by setting vfs.newnfs.locallocks_enable to non-zero, the experimental nfs server will now acquire byte range locks on the file on behalf of NFSv4 clients, such that lock conflicts between the NFSv4 clients and processes running locally on the server, will be recognized and handled correctly.
MFC after: 2 weeks
|
205572 |
24-Mar-2010 |
rmacklem |
Fix the experimental NFS subsystem so that it uses the correct preprocessor macro name for not requiring strict data alignment.
Suggested by: marius MFC after: 2 weeks
|
200999 |
25-Dec-2009 |
rmacklem |
Modify the experimental server so that it uses VOP_ACCESSX(). This is necessary in order to enable NFSv4 ACL support. The argument to nfsvno_accchk() was changed to an accmode_t and the function nfsrv_aclaccess() was no longer needed and, therefore, deleted.
Reviewed by: trasz MFC after: 2 weeks
|
200069 |
03-Dec-2009 |
trasz |
Remove unneeded ifdefs.
Reviewed by: rmacklem
|
199616 |
20-Nov-2009 |
rmacklem |
Patch the experimental NFS server is a manner analagous to r197525, so that the creation verifier is handled correctly in va_atime for 64bit architectures. There were two problems. One was that the code incorrectly assumed that sizeof (struct timespec) == 8 and the other was that the tv_sec field needs to be assigned from a signed 32bit integer, so that sign extension occurs on 64bit architectures. This is required for correct operation when exporting ZFS volumes.
Reviewed by: pjd MFC after: 2 weeks
|
197650 |
30-Sep-2009 |
trasz |
Fix typo in the comment.
|
196019 |
01-Aug-2009 |
rwatson |
Merge the remainder of kern_vimage.c and vimage.h into vnet.c and vnet.h, we now use jails (rather than vimages) as the abstraction for virtualization management, and what remained was specific to virtual network stacks. Minor cleanups are done in the process, and comments updated to reflect these changes.
Reviewed by: bz Approved by: re (vimage blanket)
|
195821 |
22-Jul-2009 |
rmacklem |
Add changes to the experimental nfs client to use the PBDRY flag for msleep(9) when a vnode lock or similar may be held. The changes are just a clone of the changes applied to the regular nfs client by r195703.
Approved by: re (kensmith), kib (mentor)
|
195699 |
14-Jul-2009 |
rwatson |
Build on Jeff Roberson's linker-set based dynamic per-CPU allocator (DPCPU), as suggested by Peter Wemm, and implement a new per-virtual network stack memory allocator. Modify vnet to use the allocator instead of monolithic global container structures (vinet, ...). This change solves many binary compatibility problems associated with VIMAGE, and restores ELF symbols for virtualized global variables.
Each virtualized global variable exists as a "reference copy", and also once per virtual network stack. Virtualized global variables are tagged at compile-time, placing the in a special linker set, which is loaded into a contiguous region of kernel memory. Virtualized global variables in the base kernel are linked as normal, but those in modules are copied and relocated to a reserved portion of the kernel's vnet region with the help of a the kernel linker.
Virtualized global variables exist in per-vnet memory set up when the network stack instance is created, and are initialized statically from the reference copy. Run-time access occurs via an accessor macro, which converts from the current vnet and requested symbol to a per-vnet address. When "options VIMAGE" is not compiled into the kernel, normal global ELF symbols will be used instead and indirection is avoided.
This change restores static initialization for network stack global variables, restores support for non-global symbols and types, eliminates the need for many subsystem constructors, eliminates large per-subsystem structures that caused many binary compatibility issues both for monitoring applications (netstat) and kernel modules, removes the per-function INIT_VNET_*() macros throughout the stack, eliminates the need for vnet_symmap ksym(2) munging, and eliminates duplicate definitions of virtualized globals under VIMAGE_GLOBALS.
Bump __FreeBSD_version and update UPDATING.
Portions submitted by: bz Reviewed by: bz, zec Discussed with: gnn, jamie, jeff, jhb, julian, sam Suggested by: peter Approved by: re (kensmith)
|
195642 |
12-Jul-2009 |
rmacklem |
Add calls to the experimental nfs client for the case of an "intr" mount, so that signals that aren't supposed to terminate RPCs in progress are masked off during the RPC.
Approved by: re (kensmith), kib (mentor)
|
195510 |
09-Jul-2009 |
rmacklem |
Since the nfscl_getclose() function both decremented open counts and, optionally, created a separate list of NFSv4 opens to be closed, it was possible for the associated OpenOwner to be free'd before the Open was closed. The problem was that the Open was taken off the OpenOwner list before the Close RPC was done and OpenOwners can be free'd once the list is empty. This patch separates out the case of doing the Close RPC into a separate function called nfscl_doclose() and simplifies nfsrpc_doclose() so that it closes a single open instead of a list of them. This avoids removing the Open from the OpenOwner list before doing the Close RPC.
Approved by: re (kensmith), kib (mentor)
|
194541 |
20-Jun-2009 |
rmacklem |
Replace RPCAUTH_UNIXGIDS with NFS_MAXGRPS so that nfscbd.c will build.
Approved by: kib (mentor)
|
194523 |
20-Jun-2009 |
rmacklem |
Change the size of the nfsc_groups[] array in the experimental nfs client to RPCAUTH_UNIXGIDS + 1 (17), since that is what can go on the wire for AUTH_SYS authentication.
Reviewed by: brooks Approved by: kib (mentor)
|
194498 |
19-Jun-2009 |
brooks |
Rework the credential code to support larger values of NGROUPS and NGROUPS_MAX, eliminate ABI dependencies on them, and raise the to 1024 and 1023 respectively. (Previously they were equal, but under a close reading of POSIX, NGROUPS_MAX was defined to be too large by 1 since it is the number of supplemental groups, not total number of groups.)
The bulk of the change consists of converting the struct ucred member cr_groups from a static array to a pointer. Do the equivalent in kinfo_proc.
Introduce new interfaces crcopysafe() and crsetgroups() for duplicating a process credential before modifying it and for setting group lists respectively. Both interfaces take care for the details of allocating groups array. crsetgroups() takes care of truncating the group list to the current maximum (NGROUPS) if necessary. In the future, crsetgroups() may be responsible for insuring invariants such as sorting the supplemental groups to allow groupmember() to be implemented as a binary search.
Because we can not change struct xucred without breaking application ABIs, we leave it alone and introduce a new XU_NGROUPS value which is always 16 and is to be used or NGRPS as appropriate for things such as NFS which need to use no more than 16 groups. When feasible, truncate the group list rather than generating an error.
Minor changes: - Reduce the number of hand rolled versions of groupmember(). - Do not assign to both cr_gid and cr_groups[0]. - Modify ipfw to cache ucreds instead of part of their contents since they are immutable once referenced by more than one entity.
Submitted by: Isilon Systems (initial implementation) X-MFC after: never PR: bin/113398 kern/133867
|
194357 |
17-Jun-2009 |
bz |
Add the explicit include of vimage.h to another five .c files still missing it.
Remove the "hidden" kernel only include of vimage.h from ip_var.h added with the very first Vimage commit r181803 to avoid further kernel poisoning.
|
194292 |
16-Jun-2009 |
rmacklem |
Remove the "int *" typecast for the aresid argument to vn_rdwr() and change the type of the argument from size_t to int. This should avoid issues on 64bit architectures.
Suggested by: kib Approved by: kib (mentor)
|
193507 |
05-Jun-2009 |
rwatson |
Don't check MAC in the NFS server ACL set path, right now we aren't enforcing MAC for NFS clients.
|
193066 |
29-May-2009 |
jamie |
Place hostnames and similar information fully under the prison system. The system hostname is now stored in prison0, and the global variable "hostname" has been removed, as has the hostname_mtx mutex. Jails may have their own host information, or they may inherit it from the parent/system. The proper way to read the hostname is via getcredhostname(), which will copy either the hostname associated with the passed cred, or the system hostname if you pass NULL. The system hostname can still be accessed directly (and without locking) at prison0.pr_host, but that should be avoided where possible.
The "similar information" referred to is domainname, hostid, and hostuuid, which have also become prison parameters and had their associated global variables removed.
Approved by: bz (mentor)
|
192898 |
27-May-2009 |
rmacklem |
Add a function to the experimental nfs subsystem that tests to see if a local file system supports NFSv4 ACLs. This allows the NFSHASNFS4ACL() macro to be correctly implemented. The NFSv4 ACL support should now work when the server exports a ZFS volume.
Approved by: kib (mentor)
|
192861 |
26-May-2009 |
rmacklem |
Fix the experimental nfs subsystem so that it builds with the current NFSv4 ACLs, as defined in sys/acl.h. It still needs a way to test a mount point for NFSv4 ACL support before it will work. Until then, the NFSHASNFS4ACL() macro just always returns 0.
Approved by: kib (mentor)
|
192818 |
26-May-2009 |
trasz |
Adapt to the new ACL #define names.
Reviewed by: rmacklem@
|
192781 |
26-May-2009 |
rmacklem |
Fix the handling of NFSv4 Illegal Operation number to conform to RFC3530 (the operation number in the reply must be set to the value for OP_ILLEGAL). Also cleaned up some indentation.
Approved by: kib (mentor)
|
192705 |
25-May-2009 |
rmacklem |
Temporarily #undef NFS4_ACL_EXTATTR_NAME, so that the experimental nfs subsystem will build while the NFSv4 ACL support is going into the kernel.
Approved by: kib (mentor)
|
192695 |
24-May-2009 |
rmacklem |
Crib the realign function out of nfs_krpc.c and add a call to it for the client side reply. Hopefully this fixes the problem with using the new krpc for arm for the experimental nfs client.
Approved by: kib (mentor)
|
192675 |
24-May-2009 |
rmacklem |
Fix the experimental nfsv4 client so that it works for the case of a kerberized mount without a host based principal name. This will only work for mounts being done by a user other than root. Support for a host based principal name will not work until proposed changes to the rpcsec_gss part of the krpc are committed. It now builds for "options KGSSAPI".
Approved by: kib (mentor)
|
192616 |
23-May-2009 |
rmacklem |
Fix the rpc_gss_secfind() call in nfs_commonkrpc.c so that the code will build when "options KGSSAPI" is specified without requiring the proposed changes that add host based initiator principal support. It will not handle the case where the client uses a host based initiator principal until those changes are committed. The code that uses those changes is #ifdef'd notyet until the krpc rpcsec_changes are committed.
Approved by: kib (mentor)
|
192581 |
22-May-2009 |
rmacklem |
Fix the comment in sys/fs/nfs/nfs.h to correctly reflect the current use of the R_xxx flags. This changed when the NFS_LEGACYRPC code was removed from the subsystem.
Approved by: kib (mentor)
|
192578 |
22-May-2009 |
rwatson |
Remove the unmaintained University of Michigan NFSv4 client from 8.x prior to 8.0-RELEASE. Rick Macklem's new and more feature-rich NFSv234 client and server are replacing it.
Discussed with: rmacklem
|
192337 |
18-May-2009 |
rmacklem |
Change the experimental NFSv4 client so that it does not do the NFSv4 Close operations until ncl_inactive(). This is necessary so that the Open StateIDs are available for doing I/O on mmap'd files after VOP_CLOSE(). I also changed some indentation for the nfscl_getclose() function.
Approved by: kib (mentor)
|
192181 |
16-May-2009 |
rmacklem |
Fixed the Null callback RPCs so that they work with the new krpc. This required two changes: setting the program and version numbers before connect and fixing the handling of the Null Rpc case in newnfs_request().
Approved by: kib (mentor)
|
192152 |
15-May-2009 |
rmacklem |
Move the nfsstat structure and proc/op number definitions on the experimental nfs subsystem from sys/fs/nfs/nfs.h and sys/fs/nfs/nfsproto.h to sys/fs/nfs/nfsport.h and rename nfsstat to ext_nfsstat. This was done so that src/usr.bin/nfsstat.c could use it alongside the regular nfs include files and struct nfsstat.
Approved by: kib (mentor)
|
192121 |
14-May-2009 |
rmacklem |
Apply changes to the experimental nfs server so that it uses the security flavors as exported in FreeBSD-CURRENT. This allows it to use a slightly modified mountd.c instead of a different utility.
Approved by: kib (mentor)
|
192115 |
14-May-2009 |
rmacklem |
Change the file names in the comments in sys/fs/nfs/nfs_var.h so that they are the names used in FreeBSD-CURRENT. Also shuffled a few entries around, so that they under the correct comment.
Approved by: kib (mentor)
|
192000 |
11-May-2009 |
rmacklem |
Change the name of the nfs server addsock structure from nfsd_args to nfsd_addsock_args, so that it is consistent with the one in sys/nfsserver/nfs.h.
Approved by: kib (mentor)
|
191990 |
11-May-2009 |
attilio |
Remove the thread argument from the FSD (File-System Dependent) parts of the VFS. Now all the VFS_* functions and relating parts don't want the context as long as it always refers to curthread.
In some points, in particular when dealing with VOPs and functions living in the same namespace (eg. vflush) which still need to be converted, pass curthread explicitly in order to retain the old behaviour. Such loose ends will be fixed ASAP.
While here fix a bug: now, UFS_EXTATTR can be compiled alone without the UFS_EXTATTR_AUTOSTART option.
VFS KPI is heavilly changed by this commit so thirdy parts modules needs to be recompiled. Bump __FreeBSD_version in order to signal such situation.
|
191940 |
09-May-2009 |
kan |
Do not embed struct ucred into larger netcred parent structures.
Credential might need to hang around longer than its parent and be used outside of mnt_explock scope controlling netcred lifetime. Use separate reference-counted ucred allocated separately instead.
While there, extend mnt_explock coverage in vfs_stdexpcheck and clean-up some unused declarations in new NFS code.
Reported by: John Hickey PR: kern/133439 Reviewed by: dfr, kib
|
191783 |
04-May-2009 |
rmacklem |
Add the experimental nfs subtree to the kernel, that includes support for NFSv4 as well as NFSv2 and 3. It lives in 3 subdirs under sys/fs: nfs - functions that are common to the client and server nfsclient - a mutation of sys/nfsclient that call generic functions to do RPCs and handle state. As such, it retains the buffer cache handling characteristics and vnode semantics that are found in sys/nfsclient, for the most part. nfsserver - the server. It includes a DRC designed specifically for NFSv4, that is used instead of the generic DRC in sys/rpc. The build glue will be checked in later, so at this point, it consists of 3 new subdirs that should not affect kernel building.
Approved by: kib (mentor)
|