MFC r315851: move thread switch tracing from mi_switch to sched_switch

MFC r315075: trace thread running state when a thread is run for the first time

MFC r312426: fix a thread preemption regression in schedulers introduced
in r270423

MFC 308564: Don't place threads on the run queue after waking up other CPUs.

The other CPU might resume and see a still-empty runq and go back to
sleep before sched_add() adds the thread to the runq. This results
in a lost wakeup and a potential hang if the system is otherwise
completely idle.

The race originated due to a micro-optimization (my fault) in 4BSD in
that it avoided putting a thread on the run queue if the scheduler was
going to preempt to the new thread. To avoid complexity while fixing
this race, just drop this optimization. 4BSD now always sets the
"owepreempt" flag when a preemption is warranted and defers the actual
preemption to the thread_unlock of the caller the same as ULE.

MFC 303503: Don't treat NOCPU as a valid CPU to CPU_ISSET.

If a thread is created bound to a cpuset it might already be bound before
its very first timeslice, and td_lastcpu will be NOCPU in that case.

MFC r298819:

sys/kern: spelling fixes in comments.

MFC 286256:
kgdb uses td_oncpu to determine if a thread is running and should use
a pcb from stoppcbs[] rather than the thread's PCB. However, exited threads
retained td_oncpu from the last time they ran, and newborn threads had their
CPU fields cleared to zero during fork and thread creation since they are
in the set of fields zeroed when threads are setup. To fix, explicitly
update the CPU fields for exiting threads in sched_throw() to reflect the
switch out and reset the CPU fields for new threads in sched_fork_thread()
to NOCPU.

MFC r282213:

Add kern.racct.enable tunable and RACCT_DISABLED config option.
The point of this is to be able to add RACCT (with RACCT_DISABLED)
to GENERIC, to avoid having to rebuild the kernel to use rctl(8).

MFC r282901:

Build GENERIC with RACCT/RCTL support by default. Note that it still
needs to be enabled by adding "kern.racct.enable=1" to /boot/loader.conf.

Note those two are MFC-ed together, because the latter one changes the
name of RACCT_DISABLED option to RACCT_DEFAULT_TO_DISABLED. Should have
committed the renaming separately...

Relnotes: yes
Sponsored by: The FreeBSD Foundation

MFC r270423:
Restore pre-r239157 handling of sched_yield(), when thread time slice
was aborted, allowing other threads to run. Without this change thread
is just rescheduled again, that was illustrated by provided test tool.

PR: 192926
Submitted by: eric@vangyzen.net
Approved by: re (marius)

MFC r260043:
The arguments to sched:::off-cpu are the thread and associated process of
the thread selected to run, not the currently running thread.

MFC r258622: dtrace sdt: remove the ugly sname parameter of SDT_PROBE_DEFINE

MFC 286256:
kgdb uses td_oncpu to determine if a thread is running and should use
a pcb from stoppcbs[] rather than the thread's PCB. However, exited threads
retained td_oncpu from the last time they ran, and newborn threads had their
CPU fields cleared to zero during fork and thread creation since they are
in the set of fields zeroed when threads are setup. To fix, explicitly
update the CPU fields for exiting threads in sched_throw() to reflect the
switch out and reset the CPU fields for new threads in sched_fork_thread()
to NOCPU.

MFC r282213:

Add kern.racct.enable tunable and RACCT_DISABLED config option.
The point of this is to be able to add RACCT (with RACCT_DISABLED)
to GENERIC, to avoid having to rebuild the kernel to use rctl(8).

MFC r282901:

Build GENERIC with RACCT/RCTL support by default. Note that it still
needs to be enabled by adding "kern.racct.enable=1" to /boot/loader.conf.

Note those two are MFC-ed together, because the latter one changes the
name of RACCT_DISABLED option to RACCT_DEFAULT_TO_DISABLED. Should have
committed the renaming separately...

Relnotes: yes
Sponsored by: The FreeBSD Foundation

MFC r270423:
Restore pre-r239157 handling of sched_yield(), when thread time slice
was aborted, allowing other threads to run. Without this change thread
is just rescheduled again, that was illustrated by provided test tool.

PR: 192926
Submitted by: eric@vangyzen.net
Approved by: re (marius)

MFC r260043:
The arguments to sched:::off-cpu are the thread and associated process of
the thread selected to run, not the currently running thread.

MFC r258622: dtrace sdt: remove the ugly sname parameter of SDT_PROBE_DEFINE