Copy stable/10@r272459 to releng/10.1 as part of
the 10.1-RELEASE process.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Merge from tbemd:

Convert from using MACHINE_ARCH to MACHINE_CPUARCH. Hoist path statement
up into the top Makefile rather than repeating it on every arch Makefile.

For un-prototyped static inline functions declared in pthread_md.h on
ia64, powerpc, and sparc64, use ANSI function headers and specifically
indicate the lack of arguments with 'void'. Otherwise, warnings are
generated at WARNS=3, leading to a compile failure with -Werror.

WARNS=3'ify.

Stylize:
o avoid using a global register variable.
o redefine struct ia64_tp as a union. We don't have to get to the
fields themselves. We just need it to be of the right size with
the right alignment.

The ucontext is 16-byte aligned, which means that struct tcb is
16-byte aligned. Consequently, struct tcb is a multiple of 16
bytes in size. We need to make sure there's no padding after
struct ppc32_tp. We do this by explicitly adding the necessary
padding in front of it.

Stylize. Introduce ppc_{get|set}_tp() and ppc_{get|set}_tcb() to
abstract the magic that happens when deriving one or the other.

Implement TLS.

In order to let new binutils can compile it, replace movl with
movw for segment saving and restoring.

Submitted by: Diego 'Flameeyes' Petteno flameeyes at gentoo dot org

Fix a race condition introduced when redzones were added. Use an
atomic operation to return and adjust the stack.

Submitted by: luoqi

Remove an unused variable.

Clean out the leftovers from the i386_set_gsbase() TLS conversion.

Like on libthr, there is an i386_set_gsbase() stub implementation here
to avoid libc.so.5 issues. This should likely be a weak symbol and I
expect this will be fixed soon.

Approved by: re

Remove the special _amd64_set_gsbase() code for #ifdef COMPAT_32BIT, now
that the amd64 kernel implements i386_get/set_gsbase(). All the rest of
the ldt backwards compat code should go away soon.

Use the i386_set_gsbase() syscall if it is implemented in the kernel.

This is a little hairy here because the allocation and usage of this
functionality is split into two places in libpthread.

Use the new atomic_cmpset_32().

Bring in a more healthy version of the libpthread for arm, which uses
ARM_TP_ADDRESS.

i386_set_ldt() is not available when running 32 bit binaries on amd64
kernels. Use the recently exposed direct-set routines instead. This is
only activated for when we compile i386 support libraries on amd64.

Cosmetic tweaks to reduce diffs to the i386 counterpart.

Partial support of KSE for arm.

_tcb_ctor takes two args.

Add missing brackets. It was committed from wrong tree.

gcc -O2 cleanup. tested for a long time.

Reviewed by: deischen

Fix compile, s/tp_dtv/tp_tdv/g.

Bring PPC up to date with latest TLS changes.

1. Add macro DTV_OFFSET to calculate dtv offset in tcb.
2. Export symbols needed by debugger.

Add TLS support for i386 and amd64.

Save context in kernel fashion, so it can be restored by
kse_switchin syscall.

Remove unused field.

Macro optimize, this increases context switch speed about 2% on my
athlon64 machine.

PPC MD bits for KSE. Runs test cases OK. Crippled to 1:1 mode for
the time being.

Copy lwp id to thread mailbox.

Call kse_switchin to switch context when being debugged.

kse_switchin ABI was changed in kernel.

Avoid clobbering the red zone when running on the new context's stack in
_amd64_restore_context().

Arm bits for libpthread. It has no chances to work and should be considered
as stubs.

Simplify the contexts created by the kernel and remove the related
flags. We now create asynchronous contexts or syscall contexts only.
Syscall contexts differ from the minimal ABI dictated contexts by
having the scratch registers saved and restored because that's where
we keep the syscall arguments and syscall return values.
Since this change affects KSE, have it use kse_switchin(2) for the
"new" syscall context.

Apply a second fix for stack alignment with libkse. This time, enter the
UTS with the stack correctly aligned. Also, while here, use an indirect
jump rather than the pushq/ret hack.

This fixes threaded apps that use floating point for me, although
it hasn't solved all the problems. It is an improvement though.
Preservation of the 128 byte red zone hasn't been resolved yet.

Approved by: re (scottl)

Eliminate two pushl by using call instruction directly, this really
helps branch predict a lot for INTEL P4.

Approved by: re (scottl)

Use amd64_set_fsbase() instead of calling sysarch() directly.

Update context code for my last ABI breakage of mcontext. I'm worried
about the fpu code here. It should be using fxsave/fxrstor instead of
saving/restoring the control word. The SSE registers are used a lot in
gcc generated code on amd64. I'm not sure how this all fits together
though.

Don't forget to initialize the fake tcb when the kcb is allocated.

Reverse the order of the first two arguments to _sparc64_enter_uts().
The first argument is the UTS function, the second argument is the
first argument to the UTS function. Who's on first.

Convert a couple of hardcoded values to constants. Make thr_getcontext()
return 0 when called the first time, and return 1 when resumed by
thr_setcontext().

Add preliminary sparc64 support to libpthread. This does not
yet work, but hopefully someone familiar with the sparc64
port can pick up the reins.

Submitted by: jake
With mods by: deischen

Fix FPU state restoring bug by jumping to right position.

Make KSE_STACKSIZE machine dependent by moving it from thr_kern.c to
pthread_md.h. This commit only moves the definition; it does not
change it for any of the platforms. This more easily allows 64-bit
architectures (in particular) to pick a slightly larger stack size.

_ia64_break_setcontext() now takes a mcontext_t. While here, define
THR_SETCONTEXT as PANIC(). The THR_SETCONTEXT macro is currently not
used, which means that the definition we had could be wrong, overly
pessimistic or unknowingly right. I don't like the odds...

The new _ia64_break_setcontext() and corresponding kernel fixes make
KSE mostly usable. There's still a case where we don't properly
restore a context and end up with a NaT consumption fault (typically
an indication for not handling NaT collection points correctly),
but at least now mutex_d works...

Stop using the setcontext() syscall to restore an async context.
Instead use the break instruction with an immediate specially
created for us.

Remove a comment that questioned why the size of the FPU
state for amd64 was twice as large as necessary. Peter
recently fixed this, so the comment no longer applies.

Also, since the size of struct mcontext changed, adjust
the threads library version of get&set context to match.

FYI, any change layout/size change to any arch's struct
mcontext will likely need some minor changes in libpthread.

Don't assume sizeof(long) = sizeof(int) on x86; use int
instead of long types for low-level locks.

Add prototypes for some internal libc functions that are
wrapped by the library as cancellation points.

Add memory barriers to alpha atomic swap functions (submitted
by davidxu).

Requested by: bde

Don't forget to set kcb_self.

Grok async contexts. When a thread is interrupted and an upcall
happens, the context of the interrupted thread is exported to
userland. Unlike most contexts, it will be an async context and
we cannot easily use our existing functions to set such a
context.
To avoid a lot of complexity that may possibly interfere with
the common case, we simply let the kernel deal with it. However,
we don't use the EPC based syscall path to invoke setcontext(2).
No, we use the break-based syscall path. That way the trapframe
will be compatible with the context we're trying to restore and
we save the kernel a lot of trouble. The kind of trouble we did
not want to go though ourselves...

However, we also need to set the threads mailbox and there's no
syscall to help us out. To avoid creating a new syscall, we use
the context itself to pass the information to the kernel so that
the kernel can update the mailbox. This involves setting a flag
(_MC_FLAGS_KSE_SET_MBOX) and setting ifa (the address) and isr
(the value).

Fix a typo. s/Line/Like/

Avoid a level of indirection to get from the thread pointer to the
TCB. We know that the thread pointer points to &tcb->tcb_tp, so all
we have to do is subtract offsetof(struct tcb, tcb_tp) from the
thread pointer to get to the TCB. Any reasonably smart compiler will
translate accesses to fields in the TCB as negative offsets from TP.

In _tcb_set() make sure the fake TCB gets a pointer to the current
KCB, just like any other TCB. This fixes a NULL-pointer dereference
in _thr_ref_add() when it tried to get the current KSE.

Define the static TLS as an array of long double. This will guarantee
that the TLS is 16-byte aligned, as well as guarantee that the thread
pointer is 16-byte aligned as it points to struct ia64_tp. Likewise,
struct tcb and struct ksd are also guaranteed to be 16-byte aligned
(if they weren't already).

Use auto LDT allocation for i386.

Rethink the MD interfaces for libpthread to account for
archs that can (or are required to) have per-thread registers.

Tested on i386, amd64; marcel is testing on ia64 and will
have some follow-up commits.

Reviewed by: davidxu

Define THR_GETCONTEXT and THR_SETCONTEXT in terms of the userland
context functions. We don't need to enter the kernel anymore. The
contexts are compatible (ie a context created by getcontext() can
be restored by _ia64_restore_context()).

While here, make the use of THR_ALIGNBYTES and THR_ALIGN a no-op.
They are going to be removed anyway.

o In _ia64_save_context() clear the return registers except for r8.
We write 1 for r8 in the context so that _ia64_restore_context()
will return with a non-zero value. _ia64_save_context() always
return 0.
o In _ia64_restore_context(), don't restore the thread pointer. It
is not normally part of the context. Also, restore the return
registers. We get called for contexts created by getcontext(),
which means we have to restore all the syscall return values.

-15 is incorrect to be used to align stack to 16 bytes, use ~15 instead.

Take the same approach for i386 as that for ia64 and amd64. Use
the userland version of [gs]etcontext to switch between a thread
and the UTS scheduler (and back again). This also fixes a bug
in i386 _thr_setcontext() which wasn't properly restoring the
context.

Reviewed by: davidxu

Fix some typos, correctly jump into UTS.

Fix typo.

Add missing arguments to _amd64_restore_context() when called from
THR_SETCONTEXT().

Add some very beta amd64 bits. These will also need some tweaking.

Implement _ia64_save_context() and _ia64_restore_context(). Both
functions are derived from the swapctx() and restorectx() (resp)
from sys/ia64/ia64/context.s. The code is expected to be 99%
correct, but has not yet been tested.

Note that with these functions operating on mcontext_t, we also
created the foundation upon which we can implement getcontext(2)
and setcontext(2) replacements. It's not guaranteed that the use
of these syscalls and _ia64_{save|restore}_context() on the same
uicontext_t is actually going to work. Replacing the syscalls is
now trivially achieved.

This commit completes the ia64 port of libpthread itself (modulo
testing and bugfixes).

Implement _ia64_enter_uts(). The purpose of this function is to switch
the register stack and memory stack and call the function given to it.

While here, provide empty, non-working, stubs for the context functions
(_ia64_save_context() and _ia64_restore_context()) so that anyone can at
least compile libkse from CVS sources. Real implementations will follow
soon.

Implement _thr_enter_uts() and _thr_switch() as inline functions to
minimize the amount and complexity of assembly code that needs to be
written. This way the core functionality is spread over 3 elementary
functions that don't have to do anything that can more easily and
more safely be done in C. As such, assembly code will only have to
know about the definition of mcontext_t.
The runtime cost of not having these functions being inlined is less
important than the cleanliness and maintainability of the code at
this stage of the implementation.

Untangle the inter-dependency of kse types and ksd types/functions
by moving the definition of struct ksd to pthread_md.h and removing
the inclusion of ksd.h from thr_private.h (which has the definition
of struct kse and kse_critical_t). This allows ksd.h to have inline
functions that use struct kse and kse_critical_t and generally
yields a cleaner implementation at the cost of not having all ksd
related types/definitions in one header.

Implement the ksd functionality on ia64 by using inline functions
and permanently remove ksd.c from the ia64 specific makefile.

This change does not clean up the i386 specific version of ksd.h.

NOTE: The ksd code on ia64 abuses the tp register in the same way
as it is abused in libthr in that it is incompatible with the
runtime specification. This will be address when support for TLS
hits the tree.

Define THR_{G|S}ETCONTEXT to expand to {g|s}etcontext(2).
Define THR_ALIGN to align at 16-byte boundaries.

Implement atomic_swap_{int|long|ptr}. Define atomic_swap_ptr as a
macro that expands to atomic_swap_long() to avoid compiler warnings
caused by incompatible pointer passing.

Move the machine specific files from sys/Makefile.inc and put them
in a machine specific makefile. While here, sort the sub-directories
in Makefile and remove _atomic_lock.S from all makefiles.

Move the mailbox to the beginning of the thread and align the
thread so that the context (SSE FPU state) is also aligned.

Sorry folks; I accidentally committed a patch from what I was working
on a couple of days ago. This should be the most recent changes.

Noticed by: davidxu

Add architecture dependent atomic ops (atomic_swap only), KSE specific
data, and userland versions of [gs]etcontext().

Modify the UTS entry and exit functions to account of FPU validity
and format.

In _thread_enter_uts, fix eflags saving bug.
In _thread_switch, set current thread pointer in kse mailbox
only after all registers copied out of thread mailbox, kernel will do
upcall at trap time, if set current thread pointer before loading all
registers from thread mailbox, at trap time, the thread mailbox data
will be overwritten by kernel, result is junk data is loaded into CPU.

Adjust code for new mailbox format.

Reviewed by: deischen, mini

Use KSE to schedule threads.