Copy stable/10@r272459 to releng/10.1 as part of
the 10.1-RELEASE process.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Merge r264984

Retire smp_active. It was racey and caused demonstrated problems with
the cpufreq code. Replace its use with smp_started. There's at least
one userland tool that still looks at the kern.smp.active sysctl, so
preserve it but point it to smp_started as well.

Obtained from: Netflix, Inc.

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Replace kernel virtual address space allocation with vmem. This provides
transparent layering and better fragmentation.

- Normalize functions that allocate memory to use kmem_*
- Those that allocate address space are named kva_*
- Those that operate on maps are named kmap_*
- Implement recursive allocation handling for kmem_arena in vmem.

Reviewed by: alc
Tested by: pho
Sponsored by: EMC / Isilon Storage Division

- Add a BIT_FFS() macro and use it to replace cpusetffs_obj()

Discussed with: attilio
Sponsored by: EMC / Isilon Storage Division

With retirement of cpumask_t and usage of cpuset_t for representing a
mask of CPUs, pc_other_cpus and pc_cpumask become highly inefficient.

Remove them and replace their usage with custom pc_cpuid magic (as,
atm, pc_cpumask can be easilly represented by (1 << pc_cpuid) and
pc_other_cpus by (all_cpus & ~(1 << pc_cpuid))).

This change is not targeted for MFC because of struct pcpu members
removal and dependency by cpumask_t retirement.

MD review by: marcel, marius, alc
Tested by: pluknet
MD testing by: marcel, marius, gonzo, andreast

etire the cpumask_t type and replace it with cpuset_t usage.

This is intended to fix the bug where cpu mask objects are
capped to 32. MAXCPU, then, can now arbitrarely bumped to whatever
value. Anyway, as long as several structures in the kernel are
statically allocated and sized as MAXCPU, it is suggested to keep it
as low as possible for the time being.

Technical notes on this commit itself:
- More functions to handle with cpuset_t objects are introduced.
The most notable are cpusetobj_ffs() (which calculates a ffs(3)
for a cpuset_t object), cpusetobj_strprint() (which prepares a string
representing a cpuset_t object) and cpusetobj_strscan() (which
creates a valid cpuset_t starting from a string representation).
- pc_cpumask and pc_other_cpus are target to be removed soon.
With the moving from cpumask_t to cpuset_t they are now inefficient
and not really useful. Anyway, for the time being, please note that
access to pcpu datas is protected by sched_pin() in order to avoid
migrating the CPU while reading more than one (possible) word
- Please note that size of cpuset_t objects may differ between kernel
and userland. While this is not directly related to the patch itself,
it is good to understand that concept and possibly use the patch
as a reference on how to deal with cpuset_t objects in userland, when
accessing kernland members.
- KTR_CPUMASK is changed and now is represented through a string, to be
set as the example reported in NOTES.

Please additively note that no MAXCPU is bumped in this patch, but
private testing has been done until to MAXCPU=128 on a real 8x8x2(htt)
machine (amd64).

Please note that the FreeBSD version is not yet bumped because of
the upcoming pcpu changes. However, note that this patch is not
targeted for MFC.

People to thank for the time spent on this patch:
- sbruno, pluknet and Nicholas Esborn (nick AT desert DOT net) tested
several revision of the patches and really helped in improving
stability of this work.
- marius fixed several bugs in the sparc64 implementation and reviewed
patches related to ktr.
- jeff and jhb discussed the basic approach followed.
- kib and marcel made targeted review on some specific part of the
patch.
- marius, art, nwhitehorn and andreast reviewed MD specific part of
the patch.
- marius, andreast, gonzo, nwhitehorn and jceel tested MD specific
implementations of the patch.
- Other people have made contributions on other patches that have been
already committed and have been listed separately.

Companies that should be mentioned for having participated at several
degrees:
- Yahoo! for having offered the machines used for testing on big
count of CPUs.
- The FreeBSD Foundation for having sponsored my devsummit attendance,
which has been instrumental.
- Sandvine for having offered offices and infrastructure during
development.

(I really hope I didn't forget anyone, if it happened I apologize in
advance).

On multi-core, multi-threaded PPC systems, it is important that the threads
be brought up in the order they are enumerated in the device tree (in
particular, that thread 0 on each core be brought up first). The SLIST
through which we loop to start the CPUs has all of its entries added with
SLIST_INSERT_HEAD(), which means it is in reverse order of enumeration
and so AP startup would always fail in such situations (causing a machine
check or RTAS failure). Fix this by changing the SLIST into an STAILQ,
and inserting new CPUs at the end.

Reviewed by: jhb

Allow the platform code to return a bitmask of running cores rather than just
a number of cores, this allows for a sparse set of CPUs. Implement support
for sparse core masks on Octeon.

XXX jeff@ suggests that all_cpus should include cores that are offline or
running other applications/OSes, so the platform API should be further
extended to allow us to set all_cpus to include all cores that are
physically-present as opposed to only those that are running FreeBSD.

Submitted by: Bhanu Prakash (with modifications)
Reviewed by: jchandra
Glanced at by: kib, jeff, jhb

Remove bogus usage of INTR_FAST. "Fast" interrupts are now indicated by
registering a filter handler rather than a threaded handler. Also remove
a bogus use of INTR_MPSAFE for a filter.

Fix double ;;

Port r212559 to mips.

Do not explicitly enable interrupts in smp_init_secondary() because it
renders any spinlock protected code after that point to run with
interrupts enabled. This is because the processor is executing in the
context of idlethread whose 'md_spinlock_count' is already set to 1.

Instead just let sched_throw() re-enable interrupts when it releases
the spinlock.

The original powerpc commit log for r212559 is available here:
http://svn.freebsd.org/viewvc/base?view=revision&revision=212559

Refactor timer management code with priority to one-shot operation mode.
The main goal of this is to generate timer interrupts only when there is
some work to do. When CPU is busy interrupts are generating at full rate
of hz + stathz to fullfill scheduler and timekeeping requirements. But
when CPU is idle, only minimum set of interrupts (down to 8 interrupts per
second per CPU now), needed to handle scheduled callouts is executed.
This allows significantly increase idle CPU sleep time, increasing effect
of static power-saving technologies. Also it should reduce host CPU load
on virtualized systems, when guest system is idle.

There is set of tunables, also available as writable sysctls, allowing to
control wanted event timer subsystem behavior:
kern.eventtimer.timer - allows to choose event timer hardware to use.
On x86 there is up to 4 different kinds of timers. Depending on whether
chosen timer is per-CPU, behavior of other options slightly differs.
kern.eventtimer.periodic - allows to choose periodic and one-shot
operation mode. In periodic mode, current timer hardware taken as the only
source of time for time events. This mode is quite alike to previous kernel
behavior. One-shot mode instead uses currently selected time counter
hardware to schedule all needed events one by one and program timer to
generate interrupt exactly in specified time. Default value depends of
chosen timer capabilities, but one-shot mode is preferred, until other is
forced by user or hardware.
kern.eventtimer.singlemul - in periodic mode specifies how much times
higher timer frequency should be, to not strictly alias hardclock() and
statclock() events. Default values are 2 and 4, but could be reduced to 1
if extra interrupts are unwanted.
kern.eventtimer.idletick - makes each CPU to receive every timer interrupt
independently of whether they busy or not. By default this options is
disabled. If chosen timer is per-CPU and runs in periodic mode, this option
has no effect - all interrupts are generating.

As soon as this patch modifies cpu_idle() on some platforms, I have also
refactored one on x86. Now it makes use of MONITOR/MWAIT instrunctions
(if supported) under high sleep/wakeup rate, as fast alternative to other
methods. It allows SMP scheduler to wake up sleeping CPUs much faster
without using IPI, significantly increasing performance on some highly
task-switching loads.

Tested by: many (on i386, amd64, sparc64 and powerc)
H/W donated by: Gheorghe Ardelean
Sponsored by: iXsystems, Inc.

Add a new ipi_cpu() function to the MI IPI API that can be used to send an
IPI to a specific CPU by its cpuid. Replace calls to ipi_selected() that
constructed a mask for a single CPU with calls to ipi_cpu() instead. This
will matter more in the future when we transition from cpumask_t to
cpuset_t for CPU masks in which case building a CPU mask is more expensive.

Submitted by: peter, sbruno
Reviewed by: rookie
Obtained from: Yahoo! (x86)
MFC after: 1 month

Update MIPS timer code (except RMI) to utilize new MI event timer
infrastructure.

Reviewed by: neel

Merge jmallett@'s n64 work into HEAD - changeset 4

Re-write tlb operations in C with a simpler API.
Update callers to use the new API.

Changes from http://svn.freebsd.org/base/user/jmallett/octeon

Approved by: rrs(mentor), jmallett

Adds JC's cleanup patches that fix it so
we call an platform dependant topo function as
well as clean up all the XLR specific ifdefs around
smp platform init.

Obtained from: JC

This pushes all of JC's patches that I have in place. I
am now able to run 32 cores ok.. but I still will hang
on buildworld with a NFS problem. I suspect I am missing
a patch for the netlogic rge driver.

JC check and see if I am missing anything except your
core-mask changes

Obtained from: JC

o) Add SMP support for Octeon using U-Boot to launch all the processors at the
same time.
o) Remove some unused trivial uart functions from octeon_machdep now that the
uart part is fully working and they are unused.
o) Use __func__ instead of __FUNCTION__.
o) Use intr_*() instead of other routines that do the same thing.
o) Remove some duplicate printfs from the Octeon port, as well as duplicate
setting of Maxmem.
o) Use the right frequency divider on Octeon.
o) Use PCPU_GET(cpuid) consistently to get the cpuid of the running core.
o) Remove some unused macros in the Octeon port.
o) Use mips_sync() around use of the global dpcpu, whose value may not be
visible to APs at first.
o) When loading the first thread's stack, use macros to make the code correct
for n64 as well.
o) Remove stub, do-nothing FAU init/enable/disable functions from the RGMX
driver.

o) Use inline functions to access coprocessor 0 registers rather than external
ones implemented using assembly.
o) Use TRAPF_USERMODE() consistently rather than USERMODE(). Eliminate
<machine/psl.h> as a result.
o) Use intr_*() rather than *intr(), consistently.
o) Use register_t instead of u_int in some trap code.
o) Merge some more endian-related macros to machine/asm.h from NetBSD.
o) Add PTR_LI macro, which loads an address with the correct sign-extension for
a pointer.
o) Restore interrupts when bailing out due to an excessive IRQ in
nexus_setup_intr().
o) Remove unused functions from psraccess.S.
o) Enter temporary virtual entries for large memory access into the page tables
rather than simply hoping they stay resident in the TLB and we don't need to
do a refill for them.
o) Abstract out large memory mapping setup/teardown using some macros.
o) Do mips_dcache_wbinv_range() when using temporary virtual addresses just
like we do when we can use the direct map.

Make the ddb command "show tlb" SMP friendly.

It now accepts an argument to dump out the tlb of a particular cpu.

Stash the context of the running thread at the time an IPI_STOP is received
in 'stoppcbs[]'. We use the 'stoppcbs[]' context to generate the backtrace
of such stopped threads.

SMP support for the mips port.

The platform that supports SMP currently is a SWARM with a dual-core Sibyte
processor. The kernel config file to use is SWARM_SMP.

Reviewed by: imp, rrs

Set the svn:eol-style = native and svn:mime-type = text/plain
properties on all files in this tree.

Submitted by: rpaulo@

* Completely Remove the option STOP_NMI from the kernel. This option
has proven to have a good effect when entering KDB by using a NMI,
but it completely violates all the good rules about interrupts
disabled while holding a spinlock in other occasions. This can be the
cause of deadlocks on events where a normal IPI_STOP is expected.
* Adds an new IPI called IPI_STOP_HARD on all the supported architectures.
This IPI is responsible for sending a stop message among CPUs using a
privileged channel when disponible. In other cases it just does match a
normal IPI_STOP.
Right now the IPI_STOP_HARD functionality uses a NMI on ia32 and amd64
architectures, while on the other has a normal IPI_STOP effect. It is
responsibility of maintainers to eventually implement an hard stop
when necessary and possible.
* Use the new IPI facility in order to implement a new userend SMP kernel
function called stop_cpus_hard(). That is specular to stop_cpu() but
it does use the privileged channel for the stopping facility.
* Let KDB use the newly introduced function stop_cpus_hard() and leave
stop_cpus() for all the other cases
* Disable interrupts on CPU0 when starting the process of APs suspension.
* Style cleanup and comments adding

This patch should fix the reboot/shutdown deadlocks many users are
constantly reporting on mailing lists.

Please don't forget to update your config file with the STOP_NMI
option removal

Reviewed by: jhb
Tested by: pho, bz, rink
Approved by: re (kib)

Implement a facility for dynamic per-cpu variables.
- Modules and kernel code alike may use DPCPU_DEFINE(),
DPCPU_GET(), DPCPU_SET(), etc. akin to the statically defined
PCPU_*. Requires only one extra instruction more than PCPU_* and is
virtually the same as __thread for builtin and much faster for shared
objects. DPCPU variables can be initialized when defined.
- Modules are supported by relocating the module's per-cpu linker set
over space reserved in the kernel. Modules may fail to load if there
is insufficient space available.
- Track space available for modules with a one-off extent allocator.
Free may block for memory to allocate space for an extent.

Reviewed by: jhb, rwatson, kan, sam, grehan, marius, marcel, stas

Remove ipi_all() and ipi_self() as the former hasn't been used at
all to date and the latter also is only used in ia64 and powerpc
code which no longer serves a real purpose after bring-up and just
can be removed as well. Note that architectures like sun4u also
provide no means of implementing IPI'ing a CPU itself natively
in the first place.

Suggested by: jhb
Reviewed by: arch, grehan, jhb

White space nit.

FreeBSD/mips port. The FreeBSD/mips port targets mips32, mips64,
mips32r2 and mips64r2 (and close relatives) processors. There
presently is support for ADMtek ADM5120, A mips 4Kc in a malta board,
the RB533 routerboard (based on IDT RC32434) and some preliminary
support for sibtye/broadcom designs. Other hardware support will be
forthcomcing.

This port boots multiuser under gxemul emulating the malta board and
also bootstraps on the hardware whose support is forthcoming...

Oleksandr Tymoshenko, Wojciech Koszek, Warner Losh, Olivier Houchard,
Randall Stewert and others that have contributed to the mips2 and/or
mips2-jnpr perforce branches. Juniper contirbuted a generic mips port
late in the life cycle of the misp2 branch. Warner Losh merged the
mips2 and Juniper code bases, and others list above have worked for
the past several months to get to multiuser.

In addition, the mips2 work owe a debt to the trail blazing efforts of
the original mips branch in perforce done by Juli Mallett.