- Copy stable/10 (r259064) to releng/10.0 as part of the
10.0-RELEASE cycle.
- Update __FreeBSD_version [1]
- Set branch name to -RC1

[1] 10.0-CURRENT __FreeBSD_version value ended at '55', so
start releng/10.0 at '100' so the branch is started with
a value ending in zero.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Fix compiler warnings.

Fix a wrong memcpy of struct sockaddr.

Spotted by: dt71@gmx.com

Fix socket calls on error post-r243965.

Submitted by: Garrett Cooper

Remove superfluous paragraph macro.

Consistently set RPCGEN_CPP when running rpcgen, so the C preprocessor
set via ${CPP} is used, instead of always using hardcoded /usr/bin/cpp.

MFC after: 1 week

Revert changes in r228790. It prevented the ypserv daemon from working with
with multiple socktypes.

Spelling fixes for usr.sbin/

- Fix style(9) bug I introduced in the last commit

Approved by: jhb

- Remove extraneous null ptr deref checks
- Fix memory leak

Submitted by: Slono Slono <slonoman2011@yandex.ru>
Approved by: jhb
MFC after: 1 week

Protect NIS server with madvise(2) since this daemon is required
for succesful authentication of users.

Add missing argument for -h in usage.

Spelling fixes.

Simply ignore unsupported protocols listed in /etc/netconfig and
abort only if no transport is available. This fixes
(INET && !INET6) and (!INET && INET6) case, for example.

Try a privileged port for binding whenever possible because the
ypbind daemon requires the ypserv daemon is running on a
privileged port.

Reported by: Andrzej Tobola

The last big commit: let usr.sbin/ use WARNS=6 by default.

(S)LIST_HEAD_INITIALIZER takes a (S)LIST_HEAD as an argument.
Fix some wrong usages.
Note: this does not affect generated binaries as this argument is not used.

PR: 137213
Submitted by: Eygene Ryabinkin (initial version)
MFC after: 1 month

To remove a server, one should use double backslash, and half of them
are eaten by shell. Fix this.

MFC after: 2 weeks

- Fix main() to use two separated sockets for the two transports
when "-P port" is specified. It invoked svc{tcp,udp}_create()
for only one of the two allocated sockets, and prevented the
TCP socket from binding to as the result.

- Use TI-RPC functions and handle sockets in a
transport-independent way. At this moment only AF_INET ("udp"
and "tcp") is supported because others need rewrites of ACL
handling and yp clients.

- Add '-h addr' to specify addresses to bind to.

- Convert _msgout() to use variable argument lists and remove
asprintf() for error strings.

- Remove register storage class specifier.

Discussed with: kuriyama
MFC after: 1 week

- In ypproc_all_2_svc(), yp_fork() is called only when !debug case.
So _exit() in the bottom of this function should be called with the
same case.

- Replace magic number with YPOLDVERS macro (which may be missed in
r14262).

Support shadow.byname and shadow.byuid maps, protecting them by
insisting on privileged port access.

Include /var/yp/Makefile.local if it exists and suggest using
it to override /var/yp/Makefile behaviour.

Approved by: re (kib)
MFC after: 3 weeks

Add -P <port> option to allow binding to a specific port.

PR: bin/109494
Submitted by: mtm
MFC after: 1 week

o Re-arrange parentheses and fix a conition logic for !do_dns case.

PR: bin/98625
MFC after: 1 month

o Add /etc/shells to a source files list.

o NIS clients ask for "shells" map not "shells.list".

PR: kern/86693
Submitted by: Victor Sudakov
MFC after: 1 month

Enable inadvertantly disabled "securenet" access controls in ypserv. [1]

Correct a bug in the handling of backslash characters in smbfs which can
allow an attacker to escape from a chroot(2). [2]

Security: FreeBSD-SA-06:15.ypserv [1]
Security: FreeBSD-SA-06:16.smbfs [2]

- Let make(1) track dependencies and regen netid map when needed.
- Add missing dependency for netid.byname.

Remove unused variables. int -> size_t for lenghts.

add targets for ipnodes.byname and ipnodes.byaddr which carries
IPv6 addresses. these targets are off by default, and these
are built when there is /var/yp/ipnodes.

fix typo in previos commit.

teach IPv6 to async DNS resolver in ypserv:
- query AAAA RR on ipnodes.byname query.
- query .ip6.arpa. on ipnodes.byaddr query.

Use socklen_t where appropriate.

Expand contractions.

Expand *n't contractions.

Sort sections.

Utilize FILES, SCRIPTS, and SYMLINKS. While here, fixed a bug in
the implementation of the following feature in revision 1.4:

- Install Makefile.yp as /var/yp/Makefile.dist and link it to
/var/yp/Makefile only if /var/yp/Makefile doesn't already exist.
Suggested by Peter Wemm.

The actual code was only symlinking when no /var/yp/Makefile.dist
existed, i.e., never.

Sort SEE ALSO references (in dictionary order, ignoring case).

Mechanically kill hard sentence breaks.

Eliminated two dozens of superfluous cats and two r-cats (sorry, Jordan).

Suggested by: Dan Nelson

Use __FBSDID over rcsid[]. Protect copyright[] where needed.

style.Makefile(5)

Add a target for the creation of a /etc/shells map and add it to
the maps which are built by default.

ypserv(8) is now started from rc.d

Approved by: markm (mentor)(implicit)

Fix typos, mostly s/ an / a / where appropriate and a few s/an/and/
Add FreeBSD Id tag where missing.

english(4) police.

Uniformly refer to a file system as "file system".

Approved by: re

The resolver library does a pretty acceptable set of domain
suffix attempts before failing. No need to try again by hand,
particularly when it fills your log with failures because
localhost.example.com..example.com fails to resolve. Also improve the
log message that helped find this error.

There is still (maybe) an uninitialised pointer problem here, but in a
month of testing I haven't triggered it.

mdoc(7) police: Removed redundant .Ns calls.

Drop support for COPY, -c has been the default mode of install(1)
for a long time now.

Approved by: bde

s/${INSTALL} -c/${INSTALL} ${COPY}/

The .Nm utility

Fix a boatload of format errors and constness violations.

Sponsored by: DARPA, NAI Labs

Spell void * as void * rather than caddr_t. This is complicated by the
fact that caddr_t is often misspelled as char *.

Sponsored by: DARPA, NAI Labs

Sort out the documentation WRT TCP_WRAPPERs, which is now enabled.

Fix an inconsistency between what FreeBSD NIS and AMD think the
AMD map name should be.

Enable TCP_WRAPPERs for the NIS server. The protection afforded is
not massive, but usable.

ANSIfy and remove some dead code.

Sponsored by: DARPA, NAI Labs

Apply the following mechanical transformations in preparation for
ansification and constification:

s{\s+__P\((\(.*?\))\)}{$1}g;
s{\(\s+}{\(}g;
s{\s+\)}{\)}g;
s{\s+,}{,}g;
s{(\s+)(for|if|switch|while)\(}{$1$2 \(}g;
s{return ([^\(].*?);}{return ($1);}g;
s{([\w\)])([!=+/\*-]?=)([\w\(+-])}{$1 $2 $3}g;
s{\s+$}{\n};g

Also add $FreeBSD$ where needed.

MFC after: 1 week

yp(4) -> yp(8).

PR: docs/30797

mdoc(7) police: join split punctuation to macro calls.

can not -> cannot

sprintf -> snprintf

Perform a major cleanup of the usr.sbin Makefiles.
These are not perfectly in agreement with each other style-wise, but they
are orders of orders of magnitude more consistent style-wise than before.

Remove whitespace at EOL.

mdoc(7) police: removed HISTORY info from the .Os call.

mdoc(7) police: remove extraneous .Pp before and/or after .Sh.

Fix the type of the NULL arg to execl()

Idea from: Theo de Raadt <deraadt@openbsd.org>

mdoc(7) police: sort SEE ALSO xrefs (sort -b -f +2 -3 +1 -2).

Mark some functions as __printflike()

MFC After: 1 week

The aliases file now lives in /etc/mail/.

PR: misc/26648
MFC after: 3 days

- Backout botched attempt to introduce MANSECT feature.
- MAN[1-9] -> MAN.

ypserv has intimate knowledge of the server transport handle which is
needed to make the asynchronous DNS lookup mechanism work. (It needs to
be able to get/set the transaction ID in the trasport handle so it can
deliver a delayed UDP response when a reply is received from a DNS
server.) With TI-TPC, the transport handle has changed slightly (what
used to be an int is now a size_t) so we need to account for this.

Bring in a hybrid of SunSoft's transport-independent RPC (TI-RPC) and
associated changes that had to happen to make this possible as well as
bugs fixed along the way.

Bring in required TLI library routines to support this.

Since we don't support TLI we've essentially copied what NetBSD
has done, adding a thin layer to emulate direct the TLI calls
into BSD socket calls.

This is mostly from Sun's tirpc release that was made in 1994,
however some fixes were backported from the 1999 release (supposedly
only made available after this porting effort was underway).

The submitter has agreed to continue on and bring us up to the
1999 release.

Several key features are introduced with this update:
Client calls are thread safe. (1999 code has server side thread
safe)
Updated, a more modern interface.

Many userland updates were done to bring the code up to par with
the recent RPC API.

There is an update to the pthreads library, a function
pthread_main_np() was added to emulate a function of Sun's threads
library.

While we're at it, bring in NetBSD's lockd, it's been far too
long of a wait.

New rpcbind(8) replaces portmap(8) (supporting communication over
an authenticated Unix-domain socket, and by default only allowing
set and unset requests over that channel). It's much more secure
than the old portmapper.

Umount(8), mountd(8), mount_nfs(8), nfsd(8) have also been upgraded
to support TI-RPC and to support IPV6.

Umount(8) is also fixed to unmount pathnames longer than 80 chars,
which are currently truncated by the Kernel statfs structure.

Submitted by: Martin Blapp <mb@imp.ch>
Manpage review: ru
Secure RPC implemented by: wpaul

Fix typo: seperate -> separate.

Seperate does not exist in the english language.

Convert from CIRCLEQ to TAILQ.

Prepare for mdoc(7)NG.

mdoc(7) police: removed history info from the .Os FreeBSD call.

mdoc(7) police: do not split author names in the AUTHORS section.

mdoc(7) police: use the new features of the Nm macro.

Use Fx macro wherever possible.

Add nsswitch support. By creating an /etc/nsswitch.conf file, you can
configure FreeBSD so that various databases such as passwd and group can be
looked up using flat files, NIS, or Hesiod.

= Hesiod has been added to libc (see hesiod(3)).

= A library routine for parsing nsswitch.conf and invoking callback
functions as specified has been added to libc (see nsdispatch(3)).

= The following C library functions have been modified to use nsdispatch:
. getgrent, getgrnam, getgrgid
. getpwent, getpwnam, getpwuid
. getusershell
. getaddrinfo
. gethostbyname, gethostbyname2, gethostbyaddr
. getnetbyname, getnetbyaddr
. getipnodebyname, getipnodebyaddr, getnodebyname, getnodebyaddr

= host.conf has been removed from src/etc. rc.network has been modified
to warn that host.conf is no longer used at boot time. In addition, if
there is a host.conf but no nsswitch.conf, the latter is created at boot
time from the former.

Obtained from: NetBSD

Don't call syslog() without a format string.

Further protection against comments in /etc/{passwd,group}

Submitted by: Andre Albsmeier <andre.albsmeier@mchp.siemens.de>
PR: 14269

Handle comments and empty lines when generating YP passwd and group files

PR: 14269
Submitted by: howard@ee.utah.edu

Fix typo.

PR: 18845
Submitted by: Kevin <kevin@ns1.stcp.net>

Back out the previous change to the queue(3) interface.
It was not discussed and should probably not happen.

Requested by: msmith and others

Change the way that the queue(3) structures are declared; don't assume that
the type argument to *_HEAD and *_ENTRY is a struct.

Suggested by: phk
Reviewed by: phk
Approved by: mdodd

Remove a bogus include.

Remove more single-space hard sentence breaks.

Remove single-space hard sentence breaks. These degrade the quality
of the typeset output, tend to make diffs harder to read and provide
bad examples for new-comers to mdoc.

Fix various man pages to stop abusing the .Bx macro to generate
the string "FreeBSD". Use the .Fx macro instead.

$Id$ -> $FreeBSD$

Modify yp_mkdb so that it only checks for bogus '+' and '-' characters
in source input if the -f flag is used, and modify Makefile.yp to only
use -f for the passwd, master.passwd and group maps. These should be
the only ones for which the + and - characters have special meaning
that make it important for us to avoid letting them into any of the map
databases. In some cases (namely the automounter maps) we have to allow
at least the - character through in order to create the map properly.

This closes PR #8699.

Various man page cleanup:

- Sort xrefs.
- FreeBSD.ORG -> FreeBSD.org
- Be consistent with section names as outlined in mdoc(7).
- Other misc mdoc cleanup.

PR: doc/13144
Submitted by: Alexey M. Zelkin <phantom@cris.net>

Fix typo (missing double quote)

Close PR #10971: flush the database cache after forking in ypproc_all_2_svc()
so that parent and child don't share DB handles.

Close PR #10970: exit at the end of ypproc_all_2_svc() instead of returning.
This really fixes the condition where a child creates children of its own.
I'm leaving the previous sanity tests in though, since they shouldn't hurt,
and will give an indication if this ever happens again.

Add some extra checks to make sure that a child ypserv process doesn't
try to fork() a child of its own, which could result in several children
ypservs running at once. I'm still not sure exactly what leads to this
condition, but these fixes should stop it from causing trouble. A new
function, yp_fork() checks to see if the current process is already a
child of the parent ypserv, and returns failure (and logs an error message)
rather than spawning another child.

Close PR #11122: check key length before calling strncmp()
in yp_next_record().

Fix reference to rpc.yppasswdd.

PR: docs/10171

Add a reference to the ypinit(8) command.

PR: docs/7679 (in spirit)

Add some tweaks to hopefully fix a problem I've started to notice recently.
Under certain conditions (possibly associated with heavy load), ypserv will
fork() child processes that don't exit like they're supposed to. I think
this is because of some suspect logic in the ypproc_all procedure. I updated
it to use what I hope is a more bulletproof approach.

Also tweaked yp_svc_run() a little so that the 'are we a child?' test happens
at every pass through the for(;;) loop, not just immediately after returning
from svc_getreqset2().

Sort cross references.

DNS lookups for partially qualified hosts didn't work right. I was trying
to be clever by avoiding the 'check all domains in the search list'
cycle in certain cases, but this would lose if handed a name like
"foo.ctr" which refers to an FQDN of "foo.ctr.columbia.edu". If
"columbia.edu" is in the search list in /etc/resolv.conf then the
DNS lookup code should resolve it, but it didn't.

A comment in /usr/src/usr.sbin/ypserv/Makefile.yp
(a.k.a. /var/yp/Makefile.dist) refers to an obsoleted usage of the
-m option of rpc.yppasswdd. It is currently taken over by the -t
option. -m is used for a different purpose now.

PR: 7279
Reviewed by: phk
Submitted by: Amakawa Shuhei <amakawa@nebula.sf.t.u-tokyo.ac.jp>

Allow blank lines in /var/yp/securenets.

If we get back too much data to fit in result, return NULL. This avoids
a buffer overflow, but might negatively impact those hosts who have
enough aliases to fill MAXHOSTNAMELEN * 2 characters in them.

Good candidate for merging back into -stable. Lightly tested by me, but
it came from OpenBSD a while ago.

Obtained from: OpenBSD

Protect errno in signal handlers, like in portmap.

Fixed double slashes in pathnames.

Removed bogus dependencies of generated .c files on generated headers.

.Sh AUTHOR -> .Sh AUTHORS. Use .An/.Aq.

Fixed `make -jN' for large N, as usual. Too usual for yp programs.

*sigh* I will not commit late at night. I will not commit late at night.
I will not commit late at night. I will not commit late at night.

I swear it's been Monday all week for me.

Apply proper fix for services target submitted by Andre Albsmeier
<andre.albsmeier@mchp.siemens.de>. Sorry for botching this that last
time, Andre. (Could have been worse: t least I didn't break the build.)

Arghhh.... another missing backslash. Yes, I feel stupid.

Doh! Replace backslash that got eaten by gremlins in the last commit.

Close PR #2206: fix the services.byname target so that it can handle
/etc/services entries with any protocol instead of just udp and tcp.
Rather thani having the awk script explicitly search for 'udp' or 'tcp'
in the second field using index(), use split() to break up the field
at the '/' character if it exists, which extracts the protocol from
the field no matter what it is.

PR: 2206

Fix a bug in the database handle caching. This has to do with the way the
underlying database code works. When dealing with first/next queries, you
have the notion of a database 'cursor,' which is essentially a file pointer
for the database. To select the first entry, you do a fetch with the
R_FIRST flag set, then you can use the R_NEXT flag to enumerate the other
entries in the database. Unfortunately, doing a direct fetch with no flag
does _not_ set the 'cursor,' so you can't do a direct fetch and then
enumerate the table from there.

The bug is that cached handles generated as the result of a YPPROC_MATCH
were being treated as though they were the same as handles generated by
a YPPROC_FIRST, which is not the case. The manifestation is that if you
do a 'ypmatch first-key-in-map map' followed by a yp_first()/yp_next()
pair, the yp_first() and yp_next() both return the first key in the
table, which makes the entry appear to be duplicated.

A couple smaller things since I'm here:

- yp_main.c and yp_error.c both have a global 'int debug' in them.
For some reason, our cc/ld doesn't flag this as a multiply defined
symbol even though it should. Removed the declaration from yp_main.c;
we want the one in yp_error.c.

- The Makefile wasn't installing ypinit in the right place.

Fix Makefile.yp so that it no longer creates a temporary file called
/tmp/ypmake, thereby fixing problems with successive map updates
possibly reading stale copies of this file left behind by a previous
failed run.

PR: 5571

Allow comments to begin with multiple '#' characters.

PR: 4452
Submitted by: Joao Carlos Mendes Luis <Jonny@mailhost.coppe.ufrj.br>

Some touchups courtesy of Philippe Charnier.

Add a ypinit script and man page (script lifted from OpenBSD and haqued
to work on FreeBSD, man page written by me.)

Also change Makefile.yp a little to be more tolerane in the face of
missing source files. Print a message if we can't find /var/yp/master.passwd
telling the user what to do to fix things.

Fix typo: 129.168.128.0 -> 192.168.128.0

PR: docs/4871
Submitted by: sec@42.org

Typos in man page. Cosmetics in error strings.

Don't initialize structures from other variables; gcc -ansi -pedantic
doesn't allow it. Use proper constants instead.

Fix for PR #4147: apparently BIND now returns a status of NO_RECOVERY
at the end of gethostanswer()/getanswer()/whatever where it used to
return TRY_AGAIN. This breaks the domain list traversal in ypserv's
async DNS lookup module: it would only retry using the domain(s) from
the 'domain' or 'search' lines in /etc/resolv.conf if __dns_getanswer()
returned TRY_AGAIN.

Changed the test so that either TRY_AGAIN or NO_RECOVERY will work.
This seemed to me the best solution in the event somebody tries to
compile this code on an older system with a different version of BIND.
(You shouldn't do that of course, but then there's a lot of things
in the world that you shouldn't do and people do them anyway.)

Fix a bug in the async DNS resolver that can crash ypserv. yp_prune_dnsq()
is not sane: if the TTL on a pending but unanswered query hits 0 and the
circular queue entry is removed and free()d, the for() loop may still try
to use the entry pointer (which now points at no longer valid memory).
usually, deleting only the last entry off the end of the queue worked, but
if more than one was deleted, the server would crash. I changed things a
bit so this shouldn't happen anymore.

Also arranged to call the prune routine a bit more often.

Fix a very stupid heap corruption bug: in ypproc_match_2_svc(), when
we decide to do a DNS lookup, we NUL terminate the key string provided
by the client before passing it into the DNS lookup module. This is
actually wrong. Assume the key is 'foo.com'. In this case, key.keydat_val
will be "foo.com" and key.keydat_len will be 7 (seven characters; the
string is not NUL-terminated so it is not 8 as you might expect).
The string "foo.com" is actually allocated by the XDR routines when the
RPC request is decoded; exactly 7 bytes are allocated. By adding a NUL,
the string becomes "foo.com\0", but the '\0' goes into an 8th byte which
was never allocated for this string and which could be anywhere. The result
is that while the initial request may succeed, we could trash other
dynamically allocated structures (like, oh, I dunno, the circular map
cache queue?) and SEGV later. This is in fact what happens.

The fix is to copy the string into a larger local buffer and NUL-terminate
that buffer instead.

Crash first reported by: Ricky Chan <ricky@come.net.uk>
Bug finally located with: Electric Fence 2.0.5

Back out one of my paranoia tests from the last commit here; yp_access()
already deals with it appropriately. (You know you've been working too
hard when you forget how your own code works.)

fix some typos, and some slight clean up...

Closes PR#3266

Add a sanity test to ypproc_xfr_2_svc(): check that the map and domain
in the transfer request actually exist. Technically ypxfr can do this too,
but why waste the cycles getting ypxfr off the ground for a transfer we
already know is going to fail.

Also apply stricter access control rules; ypproc_xfr_2_svc() is in a
different class than the normal map access procedures procedures.

compare return value from getopt against -1 rather than EOF, per the final
posix standard on the topic.

Revert $FreeBSD$ to $Id$

Two small tweaks:

- servers should be the first target listed in 'all:' in order for slave
servers to be updated correctly: yppush reads the ypservers map to figure
out where all the slaves are, so it needs to be loaded onto the master
ASAP.

- Fixed small bogon in publickey target which nobody has noticed since
we're not using the publickey.byname map yet.

Close PR# 2645: add curly braces where my fevered brain in collusion
with my frantic fingers forgot to put them.

Typo.

Fix bug that slipped through last big round of changes: sometimes
yp_next_record() is called without a key (from xdr_my_ypresp_all()),
in which case it returns the first key in the map. When doing this,
it also needs to update the key index in the map queue entry. Without
this, ypproc_all_2_svc() (and hence ypcat) don't work correctly.

Noticed by: Michael L. Hench <hench@watt.cae.uwm.edu>

Make the long-awaited change from $Id$ to $FreeBSD$

This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.

Ack. When I moved the getsockopt() calls, I forgot to move the lines
that initialize the getsockopt() args (type and len) too.

yp_server.c:
- Fail YPPROC_ALL requests when we hit the child process limit. This
is a little harsh, but it helps prevent the parent from blocking
and causing other requests to time out.

yp_dnslookup.c:
- Check for duplicate RPC transaction IDs that indicate duplicate
requests sent due to RPC retransmissions. We don't want to send
a second DNS request for the same data while an existing request
is in progress.

- Fix small formatting bogon in snprintf() in yp_async_lookup_addr().

Eek: the 'check for / in map names' test I added a while back detects
slashes in map names but doesn't return failure if it finds them. Add
missing return(1) to fix this.

Make the select() call work like the others in the system.. ie: don't
assume that the timeval will be preserved. As the man page says:
".. it is unwise to assume that the timeout value will be unmodified
by the select() call." This happens on Linux and on my system at least.

Fix harmless bug found by new pointer-to-function prototypes.

More async resolver refinements:

- yp_main.c: Always add the resolver socket to the set of fds
monitored by select(). It can happen that pending == 0 but we
still have some data in the socket buffer from an old query.
This way, the data will be flushed in a timely manner.

- yp_extern.h: remove proto for yp_dns_pending() since we don't need
it anynmore.

- yp_server.c: call yp_async_lookup_name()/yp_async_lookup_addr()
functions with the svc_req pointer as an arg instead of the xprt.
(The svc_req struct includes a pointer to the transport handle,
and it also has the service version number which the async DNS
code will need. (see below))

- yp_dnslookup.c:

o Nuke yp_dns_pending() since we don't need it anymore.
o In yp_run_dnsq(), swallow up and ignore replies if no requests
are pending or the ID doesn't match any of the IDs in the queue.
o In yp_send_dns_reply(), we assume that we will always be
replying to an NIS v2 client. While this will probably always
be the case, we do support the v1 'match' procedure, and it
has a different result struct than v2. For completeness,
support replying to both NIS v1 and v2 clients.
o Update the queue entry structure to include a member to
keep track of the NIS version number.
o Have yp_async_lookup_name/addr() extract the version number
from the svc_req structure and save it with the queue entry
for yp_send_dns_reply() to inspect later.
o Add some comments.

Fix some bugs:

- Don't dereference a NULL hostent pointer (if T_PTR lookup fails).

- Today I asked myself: "Self, you wrote this nifty async resolver
that does a great job handling delayed replies to clients using
the UDP transport, and the yplib code in libc always uses UDP
(except for yp_all()). But what if some dork makes a DNS lookup using
TCP?" Being the only dork on hand at the time, I tried it and was
enlightened. As I suspected, my transaction ID frobbing hacks cause
fireworks if called on a TCP transport handle (duh: the structures
are different). Fix: check the type of socket in xprt->xp_sock using
getsockopt() and don't use svcudp_get_xid() and svcudp_set_xid() for
anything except SOCK_DGRAM sockets. (Since accept() gives you a
new socket for each connection, the transaction ID munging isn't
needed for TCP anyway.)

Big round o changes:

- yp_dblookup.c: Create non-DB specific database access functions.
Using these allows access to the underlying database functions without
needing explicit knowledge of Berkeley DB. (These are used only
when DB_CACHE is #defined. Other programs that use the non-caching
functions (yp_mkdb, ypxfr, yppush, rpc.yppasswdd) shouldn't notice
the difference.)

- yp_dnslookup: Implement async DNS lookups. We send our own DNS
requests using UDP and put the request in a queue. When the response
arrives, we use the ID in the header to find the corresponsing queue
entry and then send the response to the client. We can go about our
business and handle other YP requests in the meantime. This way, we
can deal with time consuming DNS requests without blocking and without
forking.

- yp_server.c: Convert to using new non-DB-specific database access
functions. This simplifies the code a bit and removes the need for
this module to know anything about Berkeley DB. Also convert the
ypproc_match_2_svc() function to use the async DNS lookup routines.

- yp_main.c: tweak yp_svc_run() to add the resolver socket to the
set of descriptors monitored in the select() loop. Also add a
timeout to select(); we may get stale DNS requests stuck in the
queue which we want to invalidate after a while. If the timeout
hits, we decrement the ttl on all pending DNS requests and nuke
those requests that aren't handled before ttl hits zero.

- yp_extern.h: Add prototypes for new stuff.

- yp_svc_udp.c (new file): The async resolver code needs to be able
to rummage around inside the RPC UDP transport handle in order to
work correcty. There's basically one transport handle, and each time
a request comes in, the transaction ID in the handle is changed.
This means that if we queue a DNS request, then we handle some other
unrelated requests, we will be unable to send the DNS response because
the transaction ID and remote address of the client that made the DNS
request will have been lost. What we need to do is save the client
address and transaction ID in the queue entry for the DNS request,
then put the transaction ID and address back in the transport handle
when we're ready to reply. (And then we have to undo the change so
as not to confuse any other part of the server.) The trouble is that
the transaction ID is hidden in an opaque part of the transport handle,
and only the code in the svc_udp module in the RPC library knows how
to handle it. This file contains a couple of functions that let us
read and set the transaction ID in spite of this. This is really a
dirty trick and I should be taken out and shot for even thinking about
it, but there's no other way to get this stuff to work.

- Makefile: add yp_svc_udp.c to SRCS.

Back out the non-forking YPPROC_ALL stuff. Whatever drugs I was doing
when I came up with this idea weren't strong enough to help me see it
through. If this was a self-contained application and I had complete
control over what data got sent through what socket and when, I might
be able to get everything to work right without blocking, but instead
I have RPC/XDR in between me and the socket layer, and they have their
own ideas about what to do.

Maybe one day I'll go totally mad and figure out the right way to do
this; in the meantime this mess goes on the back burner.

This commit changes the YPPROC_ALL procecdure so that it handles requests
_without_ using fork().

The problem with YPPROC_ALL is that it transmits an entire map through
a TCP pipe as the result of a single RPC call. First of all, this requires
certain hackery in the XDR filter. Second, if the map being sent is
large, the server can end up spending lots of time in the XDR filter
sending to just the one client, while requests for other clients will
go unanswered.

My original solution for this was to fork() the request into a child
process which terminates after the map has been transmitted (or the
transfer is interrupted due to an error). This leaves the parent free
to handle other requests. But this solution is kind of lame: fork()
is relatively expensive, and we have to keep a cap on the number of
child processes to keep from swamping the system.

What we do now is grab control of the service transport handle and XDR
handle from the RPC library and send the records one at a time ourselves
instead of letting the RPC library do it. We send a record, then go
back to the svc_run() loop and select() on the socket. If select() says
we can still write data, we send the next record. Then we call
svc_getreqset() and handle other RPCs and loop around again. This way,
we can handle other RPCs between records.

We manage multiple YPPROC_ALL requests using a circular queue. When a
request is done, we dequeue it and destroy the handle. We also tag
each request with a ttl which is decremented whevever we run the queue
and a handle isn't serviced. This lets us nuke requests that have sat
idle for too long (if we didn't do this, we might run out of socket
descriptors.)

Now all I have to do is come up with an async resolver, and ypserv
won't need to fork() at all. :)

Note: these changes should not go into 2.2 unless they get a very
throrough shakedown before the final cutoff date.

Eek! When I added the YP_INTERDOMAIN and YP_SECURE support, I documented
and set the B and S variables here, but I forgot to actually add them to
the master.passwd and hosts.* targets. In other words, they weren't being
passed to yp_mkdb as needed.

This needs to go into 2.2; it doesn't break things a lot, but it leaves
your master.passwd maps available to unprivileged users without you
realizing it.

Add support for handling the YP_SECURE and YP_INTERDOMAIN keys from
any maps that may have them. If the YP_SECURE key is present, ypserv
will only allow access to the map from clients on reserved ports.
If the YP_INTERDOMAIN key is present, the server will do DNS lookups
for hostnames that it can't find in hosts.byname or hosts.byaddr.
This is the same as the -d flag (which is retained for backwards
compatibility) but it can be set on a per-map/per-domain basis.

Also modified /var/yp/Makefile to add YP_INTERDOMAIN to the hosts.*
maps and YP_SECURE to master.passwd.* maps by default.

In ypxfr_callback(), the failure to create an RPC CLIENT * handle for
the callback is a fatal error for this function; return immediatlely if
this happens. Also make the "failed to establish callback handle" error
mesaage print the IP address of the target callback host.

Close a potential security hole: if yp_access() is passed a map name,
have it check to see that it doesn't contain any '/' characters. This
prevents possible silliness like ypcat "../../../kernel". We already
test the domain name for this in yp_validdomain(), and ypserv itself
tests the map name in yp_open_db(), but it doesn't hurt to be paranoid
and test for it in the generic access routine too. rpc.ypxfrd does not
test the map name for slashes, but it does call yp_access() with the
map name, so this removes a potential vulnerability from there.

Also make the tests for IPPORT_RESERVED a little more selective: make
sure it trips when map == master.passwd.*, prog == YPPROC and proc ==
YPPROC_XFR, and prog == YPXFRD_FREEBSD_PROG and proc == YPXFRD_GETMAP.
Also use IPPORT_RESERVED instead of hard-coded value.

Toss the mkaliases script into the attic and remove its install
target from the Makefile. We don't need it anymore, and it was
broken anyway.

Hmm, well, whaddya know? ypserv was making calls to the undocumented
private internal _gethostbydnsname() resolver functions..

Use the .Fx macro where appropriate.

was missing @

Fix the services.byname target so that it creates search keys for the
aliases of the "official" names as well, because now that getportbyname()
does a yp match, it no longer found the entries under the alias.
This broke rsh(1), because it looks up "shell/tcp" while the official
name in /etc/services is "cmd/tcp".

Fix typo in last commit, it seems that a hash comments out a \ at the end
of line.

Also, fix existing bug in ethers.byname, it was passing an unknown option
to yppush. This appears to have been a cut/paste slip intended for a
$(DBLOAD) command above it.

Add sample rules for amd.host, mostly from the AMD docs, but tweaked to fit
the FreeBSD Makefile.yp structure by me. This allows you to have a single
amd map for all machines in a cluster.

In /etc/sysconfig, it would look something like:
amdflags="-p -a /net -c 1800 -l syslog /host amd.host"

'mkaliases' is broken (strips spaces) so don't use it,
perhaps it can be removed altogether.
corrected typos

Re-implement the DB handle cache using a circular queue and manage
it with the CIRCLEQ macros. This simplifies the code a little, makes
it somewhat easier to read, and may be a little faster. (Actually I think
the performace is about the same.)

Also, in the non DB_CACHE case, save copies of data returned from
the database library in a static buffer, just in case we decide to use
it after the database has been closed. Technically, the memory that the
data pointers refer to belongs to the DB package and we can't count on
it being there after the database has been closed -- the DB package
frees its buffers. (With DB_CACHE #defined the databases are held
open so the buffers remain valid.) I don't think any of the utilities
that use the dblookup module have had any problems with this yet, but
there's no sense in taking any chances.

Toss old mknetid script into the attic.

Adjust things slightly to support the new mknetid program.

Makefile.yp:

- Add a 'pushpw' target that only yppushes the various passwd maps
and sends a YPPROC_CLEAR to the local ypserv. This will be used by
rpc.yppasswdd once I merge in the in-place update changes.

yp_access.c:

- Make the yp_access() function print RPC program and procedure numbers
that it doesn't know about in literal form. This will allow it to work
with other prgrams that it doesn't know about, like rpc.ypxfrd I'm going
to import shortly.

yp_dblookup.c:

- Take out the __inline keywords. They weren't really helping me anyway.

- Somehow I broke yp_next() when DB_CACHE wasn't #defined. Fix it.

- Also fix potential case where yp_next() might loop forever; make sure
it checks the return values of all the (dbp->seq)()/R_NEXT calls that
it does as well as comparing keys.

Small touch-ups -- no functional changes.

Fix some comments to reflect reality (in some cases I made changes
to code but not to the comments).

Change some instances of 'inline' to '__inline' to pacify
gcc -ansi -pedantic.

Use rcsid strings more consistently.

Make 'oldaddr' static in yp_access().

Use strcpy()/strcat() in yp_open_db_cache() instead of snprintf().
(Seems to be a little faster this way.)

Performance enhancements (I hope) and new stuff:

yp_dblookup.c:

- Implement database handle caching. What this means is that instead
of opening and closing map databases for each request, we open a
database and save the handle (and, if requested, the key index)
in an array. This saves a bit of overhead on things like repeated
YPPROC_NEXT calls, such as you'd get from getpwent(). Normally,
each YPPROC_NEXT would require open()ing the database, seeking
to the location supplied by the caller (which is time consuming with
hash databases as the R_CURSOR flag doesn't work), reading the
data, close()ing the database and then shipping the data off to
the caller. The system call overhead is prohibitive, especially
with very large maps. By caching the handle to an open database,
we elimitate at least the open()/close() system calls, as well
as the associated DB setup and tear-down operations, for a large
percentage of the time. This improves performance substantially at
the cost of consuming a little more memory than before.

Note that all the caching support is surrounded by #ifdef DB_CACHE
so that this same source module can still be used by other programs
that don't need it.

- Make yp_open_db() call yp_validdomain(). Doing it here saves cycles
when caching is enabled since a hit on the map cache list by
definition means that the domain being referenced is valid.

- Also make yp_open_db() check for exhaustion of file descriptors,
just in case.

yp_server.c:

- Reorganize things a little to take advantage of the database
handle caching. Add a call to yp_flush_all() in ypproc_clear_2_svc().

- Remove calls to yp_validdomain() from some of the service procedures.
yp_validdomain() is called inside yp_open_db() now, so procedures that
call into the database package don't need to use yp_validdomain()
themselves.

- Fix a bogosity in ypproc_maplist_2_svc(): don't summarily initiallize
the result.maps pointer to NULL. This causes yp_maplist_free()
to fail and leaks memory.

- Make ypproc_master_2_svc() copy the string it gets from the database
package into a private static buffer before trying to NUL terminate it.
This is necessary with the DB handle caching: stuffing a NUL into the
data returned by DB package will goof it up internally.

yp_main.c:

- Stuff for DB handle caching: call yp_init_dbs() to clear the
handle array and add call to yp_flush_all() to the SIGHUP
signal handler.

Makefile.yp:

- Reorganize to deal with database caching. yp_mkdb(8) can now be used
to send a YPPROC_CLEAR signal to ypserv(8). Call it after each map
is created to refresh ypserv's cache.

- Add support for mail.alias map.
Contributed by Mike Murphy (mrm@sceard.com).

- Make default location for the netgroups source file be /var/yp/netgroup
instead of /etc/netgroup.

mkaliases:

- New file: script to generate mail.alias map.
Contributed by Mike Murphy (mrm@sceard.com).

Makefile:

- Install Makefile.yp as /var/yp/Makefile.dist and link it to
/var/yp/Makefile only if /var/yp/Makefile doesn't already exist.
Suggested by Peter Wemm.

- Install new mkaliases script in /usr/libexec along with mknetid.

- Use somewhat saner approach to generating rpcgen-dependent files
as suggested by Garrett Wollman.

Fix for memory leak: specify NULL as openinfo parameter when calling
dbopen() to open an NIS map.

Testing with very large maps (e.g. a sample password database with 31,000+
entries) has shown that ypserv will leak memory (ps shows RSS and VSZ
growing to 4000 pages or more) when performing repeated yp_next()s or
a yp_all(). The problem with yp_all() is not immediately obvious since
the ypproc_all service is handled in a child process which exits once
the transfer is finished, but with repeated yp_next()s (like what you
get when you use getpwent() to scroll through the password database),
the parent ypserv grows to enormous size and never shrinks again.

It seems this is related to the HASHINFO parameters I used in yp_dblookup.c,
which I actually stole from pwd_mkdb. Calling dbopen() with the default
parameters (specifying openinfo as NULL) fixes the problem.

I still need to see how this impacts the other NIS tools. I'm also
considering changing from hash to btree databases: the hash database
method doesn't support R_CURSOR, which means yp_next_record() has to
do a lot of ugly work in order to reach an arbitrary location in the
database.

Correct some man page cross references and file location references.

Initialize a few more 'result' members in a few places (both in the v2
and v2 service procedures). Also fixed the formatting in a few places
to keep everything under 80 columns.

In ypproc_master_2_svc(), don't leave result.peer uninitialized when
returning an error.

sense of 'mask' was reversed in default case of no securenets file.

Typo (vi -> v1).

Add support for NIS v1 client procedures. The following procedures
are currently implemented:

YPOLDPROC_NULL
YPOLDPROC_DOMAIN
YPOLDPROC_DOMAIN_NONACK
YPOLDPROC_FIRST
YPOLDPROC_NEXT
YPOLDPROC_MATCH
YPOLDPROC_POLL

These are all implemented as wrappers around their v2 counterparts.

The YPOLDPROC_PUSH, PULL and GET procedures are not implemented since
a) I couldn't figure out exactly what to have them do, and b) I
suspect they're used for doing map transfers between master and
slave servers, which we already do using the v2 protocol anyway.
This means we can server NIS v1 clients but can't be a master or
slave with NIS v1-only servers. I think I'll get over it. :)

The -k (sunos_4_kludge) flag and associated code has been removed
since it is no longer needed.

Also tweaked yp_access() to handle both sets of procedures and
updated the man page.

Duh: remember to #include <stdlib.h> to pick up declarations for
malloc() and free().

Add real securenets support. By default, ypserv now uses /var/yp/securenets
in the same was as the SunOS ypserv (same format, described in ypserv man
page). If the user wants tcpwrapper style access control, they can
recompile ypserv to use that instead. This way we get securenets without
having to ship libwrap.a and tcpd.h with core FreeBSD distribution.

If /var/yp/securenets doesn't exist, ypserv allows all connections.

Correct a bunch of man page cross references and generally
try and silence "manck".

ncurses, rpc, and some of the gnu stuff are still a big mess, however.

Makefile.yp:
- Improve support for multiple domains. (In preparation for new rpc.yppasswdd.)

yp_dblookup.c:
- Improve error reporting: be more selective as to what error code
we return when a (dbp->get) fails.

Found an instance of yp_error() in yp_maplist_create() that had two %s
tokens but only one argument; fixed by putting in missing argument.

Fix a bunch of spelling errors in a bunch of man pages.

Change private version of verr() to __verr() and make it static to
avoid potential clash with library function of the same name.

Update pointer to yppush.

(And now, on to rpc.yppasswdd...)

More changes brought about by testing of yppush (which is almost finished):

In yp_server.c:

- Modify ypproc_xfr_2_svc() so that it sends both a return status and
a yppush callback (if necessary: normally ypxfr is supposed to send the
callback once it's done transfering a map, but if we can't get ypxfr
off the ground for some reason, we have to send it here instead) and
do it in the right order: have to send the reply to the ypproc_xfr
request first, then send callback. This requires us to cheat a bit:
you're supposed to just return() and let the RPC dispatcher send
the reply for you, but we wouldn't be able to send the callback message
if we did that, so we have to call svc_sendreply() ourselves, then
send the callback, and then return NULL so that the RPC dispatcher
won't call svc_sendreply() itself.

- Also modify ypproc_xfr_2_svc() so that it doesn't invoke ypxfr with
the -f flag: this overrides the order number checks, which prevents
us from ever refusing maps that aren't newer than then ones we already
have.

In yp_access.c:

- Fix a typo in the TCP_WRAPPER support code (which is #ifdef'ed out
by default): a close paren somehow vanished into the ether.

A few small tweaks related to ypxfr:

- Add a ypxfr_callback() function that we can use to signal failure to
yppush(8) in the event that we can't fork()/exec() ypxfr(8). yppush
only checks the return status from YPPROC_XFR enough to determine
that the RPC succeded: it relies on its callback service to figure
out whether or not the transfer actually worked.

- Give yp_dblookup.c its own debug variable (ypdb_debug) so that DB
access debugging messages can be turned on or off independent of the
program's global debug messages.

- Have the Makefile rpcgen the ypushresp_xfr_1() client stub for us and
nuke the unneeded rule for yp_xdr.c that I left in by mistake (the XDR
filters live in libc now).

Fixed building in obj directory.

This commit was generated by cvs2svn to compensate for changes in r12891,
which included commits to RCS files with non-trunk default branches.