Remove data dependency barrier from atomic_load_* functions

This makes the atomic_load_* functions relaxed in terms of memory
ordering. Now it should be acceptable to use these functions in
assertions.

The need of the data dependency barrier is conditioned to usage.
The barrier is unnecessary for the control decisions that cond_wait()
and refcnt_finalize() make. READ_ONCE() and SMR_PTR_GET() use the
barrier so that loaded pointers would work as excepted in lock-free
contexts (some Alpha CPUs have a data cache design that can cause
unusual load-load reordering if not synchronized properly).

OK bluhm@

Declare membar_datadep_consumer() inline to fix build on sparc64.

Provide atomic load and store functions for int and long. FreeBSD
also uses these names. This implements a complete interface for
atomic operations, such functions can be used for every access.
They provide compiler barriers, but no CPU memory barriers. This
is consistent with our other atomic operations.
OK mvs@ visa@

Add macros READ_ONCE() and WRITE_ONCE(). They constrain optimizations
done by the compiler.

OK guenther@

Add membar_enter_after_atomic(9) and membar_exit_before_atomic(9) APIs to
allow important optimizations on architectures where atomic instructions
include and implied memory barrier.

ok mikeb@, visa@, mpi@

add explicit casts for going from volatile void * to void * volatile *.

guenther says void * is less special in c++, and atomic.h now leaks into
c++ ports.

reported by naddy

make atomic cas and swap take volatile pointers like theyre documented to.

make the cas and swap ptr implementation take volatile void * and cast
to a void **ish thing inside so they match the defined API and dont
confuse the gcc builtins.

ok guenther@

get the args right for the fallback atomic_{cas,swap}_ptr
implementations, and then cast the pointer to the right type for
the gcc builtins.

add an MI api for atomic ops in the kernel. this api has its origins in
solaris, but we found it via netbsd.

this provides an MI implementation by using gccs atomic builtins. if an
arch wants to provide a better implementation it can simply provide the
specific bit of the api it wants to replace as a macro.

we're being conservative with this API and only providing it in the
kernel, and only for uints, ulongs, and pointers.

discussed at n2k14 and with hackers at length.
ok kettenis@

Declare membar_datadep_consumer() inline to fix build on sparc64.

Provide atomic load and store functions for int and long. FreeBSD
also uses these names. This implements a complete interface for
atomic operations, such functions can be used for every access.
They provide compiler barriers, but no CPU memory barriers. This
is consistent with our other atomic operations.
OK mvs@ visa@

Add macros READ_ONCE() and WRITE_ONCE(). They constrain optimizations
done by the compiler.

OK guenther@

Add membar_enter_after_atomic(9) and membar_exit_before_atomic(9) APIs to
allow important optimizations on architectures where atomic instructions
include and implied memory barrier.

ok mikeb@, visa@, mpi@

add explicit casts for going from volatile void * to void * volatile *.

guenther says void * is less special in c++, and atomic.h now leaks into
c++ ports.

reported by naddy

make atomic cas and swap take volatile pointers like theyre documented to.

make the cas and swap ptr implementation take volatile void * and cast
to a void **ish thing inside so they match the defined API and dont
confuse the gcc builtins.

ok guenther@

get the args right for the fallback atomic_{cas,swap}_ptr
implementations, and then cast the pointer to the right type for
the gcc builtins.

add an MI api for atomic ops in the kernel. this api has its origins in
solaris, but we found it via netbsd.

this provides an MI implementation by using gccs atomic builtins. if an
arch wants to provide a better implementation it can simply provide the
specific bit of the api it wants to replace as a macro.

we're being conservative with this API and only providing it in the
kernel, and only for uints, ulongs, and pointers.

discussed at n2k14 and with hackers at length.
ok kettenis@

Provide atomic load and store functions for int and long. FreeBSD
also uses these names. This implements a complete interface for
atomic operations, such functions can be used for every access.
They provide compiler barriers, but no CPU memory barriers. This
is consistent with our other atomic operations.
OK mvs@ visa@

Add macros READ_ONCE() and WRITE_ONCE(). They constrain optimizations
done by the compiler.

OK guenther@

Add membar_enter_after_atomic(9) and membar_exit_before_atomic(9) APIs to
allow important optimizations on architectures where atomic instructions
include and implied memory barrier.

ok mikeb@, visa@, mpi@

add explicit casts for going from volatile void * to void * volatile *.

guenther says void * is less special in c++, and atomic.h now leaks into
c++ ports.

reported by naddy

make atomic cas and swap take volatile pointers like theyre documented to.

make the cas and swap ptr implementation take volatile void * and cast
to a void **ish thing inside so they match the defined API and dont
confuse the gcc builtins.

ok guenther@

get the args right for the fallback atomic_{cas,swap}_ptr
implementations, and then cast the pointer to the right type for
the gcc builtins.

add an MI api for atomic ops in the kernel. this api has its origins in
solaris, but we found it via netbsd.

this provides an MI implementation by using gccs atomic builtins. if an
arch wants to provide a better implementation it can simply provide the
specific bit of the api it wants to replace as a macro.

we're being conservative with this API and only providing it in the
kernel, and only for uints, ulongs, and pointers.

discussed at n2k14 and with hackers at length.
ok kettenis@

Add macros READ_ONCE() and WRITE_ONCE(). They constrain optimizations
done by the compiler.

OK guenther@

Add membar_enter_after_atomic(9) and membar_exit_before_atomic(9) APIs to
allow important optimizations on architectures where atomic instructions
include and implied memory barrier.

ok mikeb@, visa@, mpi@

add explicit casts for going from volatile void * to void * volatile *.

guenther says void * is less special in c++, and atomic.h now leaks into
c++ ports.

reported by naddy

make atomic cas and swap take volatile pointers like theyre documented to.

make the cas and swap ptr implementation take volatile void * and cast
to a void **ish thing inside so they match the defined API and dont
confuse the gcc builtins.

ok guenther@

get the args right for the fallback atomic_{cas,swap}_ptr
implementations, and then cast the pointer to the right type for
the gcc builtins.

add an MI api for atomic ops in the kernel. this api has its origins in
solaris, but we found it via netbsd.

this provides an MI implementation by using gccs atomic builtins. if an
arch wants to provide a better implementation it can simply provide the
specific bit of the api it wants to replace as a macro.

we're being conservative with this API and only providing it in the
kernel, and only for uints, ulongs, and pointers.

discussed at n2k14 and with hackers at length.
ok kettenis@

Add membar_enter_after_atomic(9) and membar_exit_before_atomic(9) APIs to
allow important optimizations on architectures where atomic instructions
include and implied memory barrier.

ok mikeb@, visa@, mpi@

add explicit casts for going from volatile void * to void * volatile *.

guenther says void * is less special in c++, and atomic.h now leaks into
c++ ports.

reported by naddy

make atomic cas and swap take volatile pointers like theyre documented to.

make the cas and swap ptr implementation take volatile void * and cast
to a void **ish thing inside so they match the defined API and dont
confuse the gcc builtins.

ok guenther@

get the args right for the fallback atomic_{cas,swap}_ptr
implementations, and then cast the pointer to the right type for
the gcc builtins.

add an MI api for atomic ops in the kernel. this api has its origins in
solaris, but we found it via netbsd.

this provides an MI implementation by using gccs atomic builtins. if an
arch wants to provide a better implementation it can simply provide the
specific bit of the api it wants to replace as a macro.

we're being conservative with this API and only providing it in the
kernel, and only for uints, ulongs, and pointers.

discussed at n2k14 and with hackers at length.
ok kettenis@