Copy stable/10@r272459 to releng/10.1 as part of
the 10.1-RELEASE process.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

libc: Make various internal file descriptors close-on-exec.

These are obtained via fopen().

More style.

MFC after: 3 days

Revert part of the r235740 which changed separate allocation of the
string buffer for each linelist l_line into one large string. Since
linelists parsed out during the previous passes store the pointers to
previously allocated l_lines, the reallocation caused undefined
behaviour on accessing the buffers, and quite deterministic fault on
freeing them (in mountd(8) startup).

This fixes reading of netgroup(5) file which contains more then one
netgroup.

Discussed with: ghelmer
MFC after: 3 days

Style(9) improvements: remove unnecessary parenthesis, improve order
of local variable declarations, remove bogus casts, and resolve long
lines.

Reviewed by: bde

Add checks for memory allocation failures in appropriate places, and
avoid creating bad entries in the grp list as a result of memory allocation
failures while building new entries.

PR: bin/83340
Reviewed by: delphij (prior version of patch)

Apply style(9) to return and switch/case statements.

Reviewed by: delphij (prior version of the patch)

Revert unintentional commit of changes to getnetgrent.c.

Add pidfile_fileno() to obtain the file descriptor for an open
pidfile.

Per Regents of the University of Calfornia letter, remove advertising
clause.

# If I've done so improperly on a file, please let me know.

fix innetgr() returning false positives and negatives when reverse netgroup
matching is used.

PR: 35506
MFC after: 3 days

remember to reset nextgrp in getnetgrent(), so that subsequent calls to getnetgrent() doesn't return bogus information.

MFC after: 3 days

Add knob NO_NIS (fka NO_YP_LIBC) and make world compileable when set.
If turned on no NIS support and related programs will be built.

Lost parts rediscovered by: Danny Braniss <danny at cs.huji.ac.il>
PR: bin/68303
No objections: des, gshapiro, nectar
Reviewed by: ru
Approved by: rwatson (mentor)
MFC after: 2 weeks

ANSIfy and constify; this now builds with GCC 3.4.

#include <ctype.h> for isspace()'s prototype (or a macro version).

Fix typos, mostly s/ an / a / where appropriate and a few s/an/and/
Add FreeBSD Id tag where missing.

#include <string.h> instead of <strings.h>

* Remove 'register'. (some functions had 7+ register functions...)
* Fix SCM ID's.

Add unsigned char cast to isspace arg

Replace previous commit with a check disallowing ptr from running
off the end of the list variable.

PR: 5345, 5610
Submitted by: nagao@cs.titech.ac.jp

Fix _listmatch() again so that it works with group lists containing only
one group. Thanks to Dirk Froemberg for supplying a patch for this. I will
be closing out the PR and moving this to the 2.2.5 branch later: my login
sessions to freefall from Columbia are ridiculously spotty today.

PR: 5610
Submitted by: Dirk Froemberg <ibex@physik.TU-Berlin.DE>

Close PR #4867: improve _listmatch() to avoid returning false positives.

PR: 4867

Improve the innetgr() NIS+ compat kludge. We should only fail over to the
'slow' lookup if we get a YPERR_MAP (no such map in server's domain) error
instead of failing over on any error. In the latter case, if the 'fast'
search fails legitimately (i.e. the user or host really isn't a member
of the specified netgroup) then we end up doing the 'slow' search and
failing all over again. The result is still correct, but cycles are
consumed for no good reason.

Also removed the #ifdef CHARITABLE since the compat kludge is no longer
optional.

NIS+ compatibility kludge. A long time ago, I set up innetgr() so
that if searching through the special netgroup.byhost or netgroup.byuser
maps didn't work, we would roll over to the 'slow' method of grovelling
though the netgroup map and working out the dependencies on the fly.
But I left this option hidden inside an #ifdef CHARITABLE since I
didn't think I'd ever need it.

Well, the Sun rpc.nisd NIS+ server in YP compat mode doesn't support
the .byhost and .byuser reverse maps, so the failover is necessary
in order to be compatible. *sigh*

This closes PR #3891, and should be merged into RELENG_2_2.

Merge from Lite2:
filesystem include updates, duplicate group suppression, cleanups,
filesystem whiteout support (unionfs), bidir popen().

Small yet significant tweaks/cleanups:

- getpwent:
o adjunctbuf should be NUL terminated after copying
o _pw_breakout_yp() needs to know the length of the buffer returned
from YP so it can properly NUL terminate its local buffer.

- getgrent:
o YP buffers should be YPMAXRECORD + 2 bytes long and NUL terminated.
(Previously they were hardcoded to 1024 bytes.)

- getnetgrent:
o YP data should be copied with snprintf(), not sprintf()

These are 2.2 candidates. I will wait a few days to make sure these don't
break anything and then, if there are no objections, move them to the 2.2
branch.

_key is a char array and we don't need to pass its address to _buildkey()
when buildkey is expecting a char *.

General -Wall warning cleanup, part I.
Submitted-By: Kent Vander Velden <graphix@iastate.edu>

- Fix _listmatch() to close PR #1207.
Fix submitted by: Alan Cox <alc@cs.rice.edu>

- Nuke yet another free(result) that isn't needed. (This one I found
without phkmalloc's help. :)

NIS cleanups and fixes, the next generation.

getnetgrent.c:

- Catch one bogon that snuck by: in _listmatch(), check for '\0'
rather than '\n'; strings returned from yp_match() are terminated
with a nul, not a newline.

getpwent.c:

- Rip out all of the +inclusion/-exclusion stuff from before and
replace it with something a little less grotty. The main problem
with the old mechanism was that it wasted many cycles processing
NIS entries even after it already knew they were to be exlcuded
(or not included, depending on your pointof view). The highlights
of these changes include:

o Uses an in-memory hash database table to keep track of all the
-@netgroup, -user, and -@group exclusions.

o Tries harder to duplicate the behavior normally obtained when using
NIS inclusions/exclusions on a flat /etc/passwd file (meaning things
come out in much the same order).

o Uses seperate methods for handling getpwent() and getpwnam()/getpwuid()
operations instead of trying to do everything with one general
function, which didn't work as well as I thought it would.

o Uses both getnetgrent() and innetgr() to try to save time where
possible.

o Use only one special token in the local password database
(_PW_KEYYPBYNUM) instead of seperate tokens to mark + and -
entries (and stop using the counter tokens too). If this new
token doesn't exist, the code will make due with the standard
_PW_KEYBYNUM token in order to support older databases that
won't have the new token in them.

All this is an attempt to make this stuff work better in environments
with large NIS passwd databases.

Fix a few NIS-related bogons:

- Clear the _yp_innetgr flag immediately after calling setnetgrent() from
innetgr(). We only need the flag set to temporarily alter setnetgrent()'s
behavior. Previously, it was being cleared too late.

- When in NIS-only mode, innetgr() was wasting time doing unecessary
extra processing after it had already found a match.

- Remember to free memory allocated by the NIS functions during innetgr()
searches.

Small tweak: don't try closing /etc/netgroup if we haven't opened it yet.

getpwent.c: turn the code that checks the override caches into a
seperate function to avoid duplication. Also fix getpwent() a
small bit to properly handle the case where the magic NIS '+'
entry appears before the end of the password file.

getgrent.c: be a little more SunOS-ish. Make it look like the NIS
group map is 'inserted' at the the point(s) where the magic NIS '+'
entry/entries appear.

getgrent: fix a file descriptor leak: remember to close the netgroup
file after we determine that we're using NIS-only innetgr() lookups.

Fix _listmatch() so that it doesn't fall off the end of the list string.

Just when you thought it was safe...

- getnetgrent.c: address some NIS compatibility problems. We really need
to use the netgroup.byuser and netgroup.byhost maps to speed up innetgr()
when using NIS. Also, change the NIS interaction in the following way:

If /etc/netgroup does not exist or is empty (or contains only the
NIS '+' token), we now use NIS exclusively. This lets us use the
'reverse netgroup' maps and is more or less the behavior of other
platforms.

If /etc/netgroup exists and contains local netgroup data (but no '+').
we use only lthe local stuff and ignore NIS.

If /etc/netgroup exists and contains both local data and the '+',
we use the local data nd the netgroup map as a single combined
database (which, unfortunately, can be slow when the netgroup
database is large). This is what we have been doing up until now.

Head off a potential NULL pointer dereference in the old innetgr()
matching code.

Also fix the way the NIS netgroup map is incorporated into things:
adding the '+' is supposed to make it seem as though the netgroup
database is 'inserted' wherever the '+' is placed. We didn't quite
do it that way before.

(The NetBSD people apparently use a real, honest-to-gosh, netgroup.db
database that works just like the password database. This is
actually a neat idea since netgroups is the sort of thing that
can really benefit from having multi-key search capability,
particularly since reverse lookups require more than a trivial
amount of processing. Should we do something like this too?)

- netgroup.5: document all this stuff.

- rcmd.c: some sleuthing with some test programs linked with my own
version of innetgr() has revealed that SunOS always passes the NIS
domain name to innetgr() in the 'domain' argument. We might as well
do the same (if YP is defined).

- ether_addr.c: also fix the NIS interaction so that placing the
'+' token in the /etc/ethers file makes it seem like the NIS
ethers data is 'inserted' at that point. (Chances are nobody will
notice the effect of this change, which is just te way I like it. :)

Fixes for PR #508 and #509 ('botched 'Bad netgroup' error message' and
'cycle in netgroup check too greedy').

PR #508 is apparently due to an inconsistency in the way the 4.4BSD
netgroup code deals with bad netgroups. When 4.4BSD code encounters
a badly formed netgroup entry (e.g. (somehost,-somedomain), which,
because of the missing comma between the '-' and 'somedomain,' has
only 2 fields instead of 3), it generates an error message and
then bails out without doing any more processing on the netgroup
containing the bad entry. Conversely, every other *NIX in the world
that usees netgroups just tries to parse the entry as best it can
and then silently continues on its way.

The result is that two bad things happen: 1) we ignore other valid entries
within the netgroup containing the bogus entry, which prevents
us from interoperating with other systems that don't behave this way,
and 2) by printing an error to stderr from inside libc, we hose certain
programs, in this case rlogind. In the problem report, Bill Fenner
noted that the 'B' from 'Bad' was missing, and that rlogind exited
immediately after generating the error. The missing 'B' is apparently
not caused by any problem in getnetgrent.c; more likely it's getting
swallowed up by rlogind somehow, and the error message itself causes
rlogind to become confused. I was able to duplicate this problem and
discovered that running a simple test program on my FreeBSD system
resulted in a properly formatted (if confusing) error, whereas triggering
the error by trying to rlogin to the machine yielded the missing 'B'
problem.

Anyway, the fixes for this are as follows:

- The error message has been reformatted so that it prints out more useful
information (e.g. Bad entry (somehost,-somedomain) in netgroup "foo").
We check for NULL entries so that we don't print '(null)' anymore too. :)

- Rearranged things in parse_netgrp() so that we make a best guess at
what bad entries are supposed to look like and then continue processing
instead of bailing out.

- Even though the error message has been cleaned up, it's wrapped inside
a #ifdef DEBUG. This way we match the behavior of other systems. Since we
now handle the error condition better anyway, this error message becomes
less important.

PR #507 is another case of inconsistency. The code that handles
duplicate/circular netgroup entries isn't really 'too greedy; -- it's
just too noisy. If you have a netgroup containing duplicate entries,
the code actually does the right thing, but it also generates an error
message. As with the 'Bad netgroup' message, spewing this out from
inside libc can also hose certain programs (like rlogind). Again, no
other system generates an error message in this case.

The only change here is to hide the error message inside an #ifdef DEBUG.
Like the other message, it's largely superfluous since the code handles
the condition correctly.

Note that PR #510 (+@netgroup host matching in /etc/hosts.equiv) is still
being investigated. I haven't been able to duplicate it myself, and I
strongly suspect it to be a configuration problem of some kind. However,
I'm leaving all three PRs open until I get 510 resolved just for the
sake of paranoia.

Remove trailing whitespace.

getpwent.c: fix problem with emacs dumping core when NIS is enabled. Also
add #includes for YP headers when compiling with -DYP to avoid some implicit
declarations.

getgrent.c & getnetgrent.c: add some #includes to avoid implicit declarations
of YP functions.

Make sanity checks saner: don't let setnetgrent() or innetgr() swallow
any bogus arguments.

Don't let setnetgrent() operate on a null or empty group name: it can
tickle a bug in ypserv and make a serious mess of things.

Use better/stronger/faster NIS lookup code: by using yp_match() instead of
the yp_first()/yp_next() combo, we let the database code in ypserv do some
of the work for us.

Whoops: expanding netgroups that reference multiple netgroups doesn't
work because parse_netgrp() doesn't recurse properly. Fixed by
changing

if (parse_netgrp(spos))
return(1);
to

if (parse_netgrp(spos))
continue;

inside parse_netgrp(). (Lucky for me I happen to have a fairly complex
'live' netgroup database to test this stuff with.)

Two major changes:

- Added support for reading netgroups from NIS/YP in addition to the
local /etc/netgroups file. (Note that SunOS and many other systems only
support reading netgroups via NIS, which is a bit odd.)

- Fix Evil Null Pointer Dereferences From Hell (tm) that caused
parse_netgrp() to SEGV when expanding netgroups that include
references to other netgroups. Funny how nobody else noticed this.

This is the first step in implimenting +@netgroup substitution in
getpwent.c and any other places that could use it and don't already
support it (which is probably everywhere).

This commit was generated by cvs2svn to compensate for changes in r1573,
which included commits to RCS files with non-trunk default branches.

BSD 4.4 Lite Lib Sources