Copy stable/10@r272459 to releng/10.1 as part of
the 10.1-RELEASE process.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r265847:
Fix sem_unlink(3) to properly invalidate the semaphores name cache.

PR: standards/189353

MFC r265845:
Style.

MFC r263107:

To avoid missing a chance to cancel thread, call _pthread_testcancel at the
beginning of _sem_timedwait.

Submitted by: Eric van Gyzen < eric at vangyzen dot net >

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Make more code be protected by internal mutex, and now it is fork-safe, in
error case, the file exclusive lock is now released as soon as possible,
in previous code, child process can still hold the exclusive lock.

Simplify code by using flag O_EXLOCK.

PR: kern/175674

libc: Use O_CLOEXEC for various internal file descriptors.

This fixes a race condition where another thread may fork() before CLOEXEC
is set, unintentionally passing the descriptor to the child process.

This commit only adds O_CLOEXEC flags to open() or openat() calls where no
fcntl(fd, F_SETFD, FD_CLOEXEC) follows. The separate fcntl() call still
leaves a race window so it should be fixed later.

sem_open: Make sure to fail an O_CREAT|O_EXCL open, even if that semaphore
is already open in this process.

If the named semaphore is already open, sem_open() only increments a
reference count and did not take the flags into account (which otherwise
happens by passing them to open()). Add an extra check for O_CREAT|O_EXCL.

PR: kern/166706
Reviewed by: davidxu
MFC after: 10 days

In sem_post, the field _has_waiters is no longer used, because some
application destroys semaphore after sem_wait returns. Just enter
kernel to wake up sleeping threads, only update _has_waiters if
it is safe. While here, check if the value exceed SEM_VALUE_MAX and
return EOVERFLOW if this is true.

Revert previous change. It is an incomplete change from old branch. :-(

Use version 2 of semaphore provided by kernel umtx code, now if there is
no waiters, we still increase and decrease count in user mode without
entering kernel, once there is a waiter, sem_post will enter kernel to
increase count and wake thread up, this is atomicy and allow us to
gracefully destroy semaphore after sem_wait returned.

In revision 231989, we pass a 16-bit clock ID into kernel, however
according to POSIX document, the clock ID may be dynamically allocated,
it unlikely will be in 64K forever. To make it future compatible, we
pack all timeout information into a new structure called _umtx_time, and
use fourth argument as a size indication, a zero means it is old code
using timespec as timeout value, but the new structure also includes flags
and a clock ID, so the size argument is different than before, and it is
non-zero. With this change, it is possible that a thread can sleep
on any supported clock, though current kernel code does not have such a
POSIX clock driver system.

Insert read memory barriers.

To support stack unwinding for cancellation points, add -fexceptions flag
for them, two functions _pthread_cancel_enter and _pthread_cancel_leave
are added to let thread enter and leave a cancellation point, it also
makes it possible that other functions can be cancellation points in
libraries without having to be rewritten in libthr.

preserve errno when processing error cases.

Also call sem_module_init in sem_close to initialize mutex
with some attributes.

Return SEM_FAILED instead of NULL, though there are same, but the
SEM_FAILED is more suitable name.
In function, sem_close(), always set errno on error.

Don't forget to use fourth argument if O_CREAT is set in argument oflag.
The fourth specifies initial value for the semaphore.

More cleanup, remove _libc prefix because libthr no longer has stubs
referencing them.

Don't check has_waiters twice, inline some small functions.
performance result on my machine:
mutex Elapsed: 902115 us; per iteration: 90 ns.
semaphore Elapsed: 958780 us; per iteration: 95 ns.

Use umtx to implement process sharable semaphore, to make this work,
now type sema_t is a structure which can be put in a shared memory area,
and multiple processes can operate it concurrently.
User can either use mmap(MAP_SHARED) + sem_init(pshared=1) or use sem_open()
to initialize a shared semaphore.
Named semaphore uses file system and is located in /tmp directory, and its
file name is prefixed with 'SEMD', so now it is chroot or jail friendly.
In simplist cases, both for named and un-named semaphore, userland code
does not have to enter kernel to reduce/increase semaphore's count.
The semaphore is designed to be crash-safe, it means even if an application
is crashed in the middle of operating semaphore, the semaphore state is
still safely recovered by later use, there is no waiter counter maintained
by userland code.
The main semaphore code is in libc and libthr only has some necessary stubs,
this makes it possible that a non-threaded application can use semaphore
without linking to thread library.
Old semaphore implementation is kept libc to maintain binary compatibility.
The kernel ksem API is no longer used in the new implemenation.

Discussed on: threads@