Copy stable/10@r272459 to releng/10.1 as part of
the 10.1-RELEASE process.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r261095,r263464,r263752,r264189

r263464,r263752,r275189:

Mask out SRR1 bits that aren't exported to the MSR.

This appears to fix a strange condition with X on 32-bit PowerBooks I
observed, caused by one of these bits getting set in the mcontext, but
not set in the thread, which is a symptom of another problem, more
difficult to diagnose. Since these bits aren't exported anyway, this
change makes it more explicit that the bits aren't MSR-related in SRR1.

r261095:

Fix 32-bit signal handling on ppc64. This was broken when the
PSL_USERSTATIC macro was changed. Since copying 64-bit srr1 into
32-bit srr1 drops the upper 32 bits, any bits set in the context were
dropped, meaning the context check fails. Since 32-bit set_context()
can't change those bits anyway, copy the ones from the current context
(td->td_frame) before calling set_context().

Approved by: re
Relnotes: yes (Affects 10-stable, but not 10.0-release)

MFC r258078,258079

Increase the stack size for ppc64 from 4 pages to 8.

I found a stack overflow when a coredump was taken onto a ZFS volume with
heavy network activity. 2 DSI traps, plus one DECR trap, along with several
function calls in the stack, overflowed the 4 pages. 8 page stack fixes this.

Discussed with: nwhitehorn
Approved by: re
Relnotes: yes

Merge the changes to pmap_enter(9) for sleep-less operation (requested
by flag). The ia64 pmap.c changes are direct commit, since ia64 is
removed on head.

MFC r269368 (by alc):
Retire PVO_EXECUTABLE.

MFC r269728:
Change pmap_enter(9) interface to take flags parameter and superpage
mapping size (currently unused).

MFC r269759 (by alc):
Update the text of a KASSERT() to reflect the changes in r269728.

MFC r269822 (by alc):
Change {_,}pmap_allocpte() so that they look for the flag
PMAP_ENTER_NOSLEEP instead of M_NOWAIT/M_WAITOK when deciding whether
to sleep on page table page allocation.

MFC r270151 (by alc):
Replace KASSERT that no PV list locks are held with a conditional
unlock.

Reviewed by: alc
Approved by: re (gjb)
Sponsored by: The FreeBSD Foundation

MFC r257854 (discussed with alc@)

As of r257209, all architectures have defined
VM_KMEM_SIZE_SCALE. In other words, every architecture is now
auto-sizing the kmem arena. This revision changes kmeminit() so
that the definition of VM_KMEM_SIZE_SCALE becomes mandatory and
the definition of VM_KMEM_SIZE becomes optional.

Replace or eliminate all existing definitions of VM_KMEM_SIZE.
With auto-sizing enabled, VM_KMEM_SIZE effectively became an
alternate spelling for VM_KMEM_SIZE_MIN on most architectures.
Use VM_KMEM_SIZE_MIN for clarity.

MFC r257738, r259202, r258410, r260288, r260292, r260294, r260320, r260323,
r260326, r260327, r260331, r260333, r260340, r260371, r260372, r260373,
r260374, r260375

Add common bus_space tag definition shared for most supported ARMv6/v7 SoCs.
Correct license statements to reflect the fact that these files were all
derived from sys/arm/mv/bus_space.c.

In pmap_unmapdev(), remember the size, and use that as an argument to
kva_free(), or we'd end up always passing it a size of 0

In pmap_mapdev(), first check whether a static mapping exists,

Convert TI static device mapping to use the new arm_devmap_add_entry(),

Use the common armv6 fdt_bus_tag defintion for tegra instead of a local copy.

Eliminate use of fdt_immr_addr(), it's not needed for tegra

Convert lpc from using fdt_immr style to arm_devmap_add_entry() to make
static device mappings.

Retire machine/fdt.h as a header used by MI code, as its function is now
obsolete. This involves the following pieces:
- Remove it entirely on PowerPC, where it is not used by MD code either
- Remove all references to machine/fdt.h in non-architecture-specific code
(aside from uart_cpu_fdt.c, shared by ARM and MIPS, and so is somewhat
non-arch-specific).
- Fix code relying on header pollution from machine/fdt.h includes
- Legacy fdtbus.c (still used on x86 FDT systems) now passes resource
requests to its parent (nexus). This allows x86 FDT devices to allocate
both memory and IO requests and removes the last notionally MI use of
fdtbus_bs_tag.
- On those architectures that retain a machine/fdt.h, unused bits like
FDT_MAP_IRQ and FDT_INTR_MAX have been removed.

Add #include <machine/fdt.h> to a few files that used to get it via pollution

Enable the mv cesa security/crypto device by providing the required property
in the dts source, and adding the right devices to the kernel config.

Remove dev/fdt/fdt_pci.c, which was code specific to Marvell ARM SoCs,
related to setting up static device mappings. Since it was only used by
arm/mv/mv_pci.c, it's now just static functions within that file, plus
one public function that gets called only from arm/mv/mv_machdep.c.

Switch RPi to using arm_devmap_add_entry() to set up static device mapping.

Allow 'no static device mappings' to potentially work.

Don't try to find a static mapping before calling pmap_mapdev(), that logic
is now part of pmap_mapdev() and doesn't need to be duplicated here.

Switch a10 to using arm_devmap_add_entry() to set up static device mapping.

MFC r258800, r258802, r258805, r258806, r258807, r258851, r258857,
r259199, r259484, r259513, r259514, r259516

The kernel stack guard pages are only below the stack pointer, not above.

Remove unnecessary double-setting of the thread's onfault state in
copyinstr().

Open Firmware mandates that certain cross-references, in particular those
in /chosen, be ihandles. The ePAPR spec makes those cross-reference phandles,
since FDT has no concept of ihandles. Have the OF FDT CI module interpret
queries about ihandles as cross-reference phandles.

Real OF systems have an ihandle under /chosen/stdout, not a phandle. Use
the right type.

Rearchitect platform memory map parsing to make it less
Open Firmware-centric.

Remove fdtbus_bs_tag definition, which is now obsolete. The remainder of
this file is also slated for future demolition.

Return the correct IEEE 1275 code for "nextprop".

Use the common Open Firmware PCI interrupt routing code instead of the
duplicate version in dev/fdt.

Configure interrupt sense based on device tree information.

Simplify the ofw_bus_lookup_imap() API slightly: make it allocate maskbuf
internally instead of requiring the caller to allocate it.

MFC r258259, r258798, r259010

Unify handling of illegal instruction faults between AIM and Book-E.

Make uart_cpu_powerpc work on both FDT and OFW systems.

Fix debug printfs in FPU_EMU to compile on powerpc64 and enable it for
powerpc64.

MFC r258247, r258250, r258257

Remove a pointless #ifdef AIM. This is just PPC64 specific, including
64-bit Book-E.

Make single precision floating point arithmetic actually work

Split the function of the PCB_FPU flags into two: PCB_FPU now indicates that
the actual FPU is enabled, while PCB_FPREGS indicates that the FPU state
structure in the PCB is valid.

MFC r257995, r258244, r258246,

Rename the "bare" platform "mpc85xx"
Also turn "bare" into a truly bare platform

Move CCSR discovery into the platform module

There is no reason Book-E needs to save XER and CTR on context switches.

MFC r258002, r258024, r258027, r258051, r258052, r258243, r258244, r258002,
r258024, r258027, r258051, r258052, r258243,

Follow up r223485, which made AIM use the ABI thread pointer instead of
PCPU fields for curthread, by doing the same to Book-E.

Use the same implementation of copyinout.c for both AIM and Book-E.

Actually add IOMMU domain to the list of known mappings.

Following the approach with ACPI DMAR on x86, split IOMMU handling into
a variant PCI bus instead of trying to shoehorn it into the PCI host bridge
adapter.

Make sure that TLB1 mappings are aligned correctly.

MFC r257180, r257195, r257196, r257198, r257209, r257295

Add some extra sanity checking and checks to printf format specifiers.

Try even harder to find a console before giving up.

Make devices with registers into the KVA region work reliably.

Turn on VM_KMEM_SIZE_SCALE on 32-bit as well as 64-bit PowerPC.

Return NOKEY instead of 0 if there are no more key presses queued.

MFC r257161, r257169, r257178, r257190, r257191

Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces.

Fix concurrency issues with TLB1 updates and make pmap_kextract() search
TLB1 mappings as well

Interrelated improvements to early boot mappings:
- Remove explicit requirement that the SOC registers be found except as an
optimization (although the MPC85XX LAW drivers still require they be found
externally, which should change).
- Remove magic CCSRBAR_VA value.
- Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and
systems not in real-mode or global 1:1 maps in early boot.
- Allow pmap_mapdev() on Book-E to reissue previous addresses if the
area is already mapped. Additionally have it check all mappings, not
just the CCSR area.

Add some extra sanity checking and checks to printf format specifiers.

Bump initial TLB size. The kernel is not necessarily less than 16 MB

Handle (in a slightly ugly way) ePAPR-type loaders that just place a
device tree into r3.

MFC r257162, r257175

The old trap.h (then trap_aim.h) actually had trap ID codes for Book-E CPUs.
Use it universally. Book-E traps may also need revisiting due to the
introduction of fixed-offset traps and the deprecation of IVORs in POWER
ISA 2.06, but that's very much an issue for another day.

MFC r257115, r257116, r257117

Remove dead and duplicated code.

MFC r256994, r257016, r257055, r257059, r257060, r257075

Add two new interfaces to ofw_bus:
- ofw_bus_map_intr()
Maps an (iparent, IRQ) tuple to a system-global interrupt number in some
platform dependent way. This is meant to be implemented as a replacement
for [FDT_]MAP_IRQ() that is an MI interface that knows about the bus
hierarchy.
- ofw_bus_config_intr()
Configures an interrupt (previously mapped) based on firmware sense flags.
This replaces manual interpretation of the sense field in bus drivers and
will, in a follow-up, allow that interpretation to be redirected to the PIC
drivers where it belongs. This will eventually replace the tables in
/sys/dev/fdt/fdt_ARCH.c

The PowerPC/AIM code has been converted to use these globally, with an
implementation in terms of MAP_IRQ() and powerpc_config_intr(), assuming
OpenPIC, at the bus root in nexus(4). The ofw_bus_config_intr() will shortly
be integrated into pic_if.m and bounced through nexus into the PIC tree.

Factor out MI portions of the PowerPC nexus device into /sys/dev/ofw. The
sparc64 driver will be modified to use this shortly.

Allow PIC drivers to translate firmware sense codes for themselves. This
is designed to replace the tables in dev/fdt/fdt_ARCH.c, but will not
happen quite yet.

Do not map IRQs twice. This fixes PowerPC/FDT systems with multiple PICs,
which would try to treat the previously-mapped interrupts from
fdt_decode_intr() as interrupt line numbers on the same parent PIC.

Remove some of the code required for supporting ssm(4) on SPARC in favor
of a more PowerPC/FDT-focused design. Whenever SPARC64 is integrated
into this rework, this should be (trivially) revisited.

MFC r263998:

Rename __wchar_t so it no longer conflicts with __wchar_t from clang 3.4
-fms-extensions.

MFC r260607, r260610, r260934:

r260607:
The onyx codec works also as module, so add it.

r260610:
Described in the man page but not implemented. Here it comes,
atomic_swap_32/64. The latter only for powerpc64.

r260934:
Fix the resource information for the i2s-a node on certain G5 PowerMacs.
This is the first step needed to get the snapper codec working on those
machines.
The second step is to enable the corresponding I2S device and its clock.

Tested on machines where the snapper codec was already working, a G4 PowerBook
and a PowerMac9,1 with a Shasta based macio.
The PowerMac7,2/7,3 with a K2 based macio can now also play sound.

MFC r263289: Update NetBSD Foundation copyrights to 2-clause BSD

The NetBSD Foundation states "Third parties are encouraged to change the
license on any files which have a 4-clause license contributed to the
NetBSD Foundation to a 2-clause license."

This change removes clauses 3 and 4 from copyright / license blocks that
list The NetBSD Foundation as the only copyright holder.

Sponsored by: The FreeBSD Foundation

MFC r261342

Add hwpmc(4) support for the PowerPC 970 class processors, direct events.
This also fixes asserts on removal of the module for the mpc74xx.

The PowerPC 970 processors have two different types of events: direct events
and indirect events. Thus far only direct events are supported. I included
some documentation in the driver on how indirect events work, but support is
for the future.

MFC r261309

Unbreak non-SMP builds. This was broken by r259284. Also, reorganize the
code introduced in that revision a bit.

MFC r259284,r259287

Add PMU-based CPU frequency scalling. This is used on most Titanium
PowerBooks.

MFC r259394,r259395,r259699

r259394:
Rebase the PMC indices at 1, since PMC_SOFT is at 0.

r259395,r259699:
Add userland PMC backtracing, and use the PMC trapframe macros for kernel
backtraces.

MFC r257991, r257992, 257993, 258504

r257991:
Consolidate Apple firmware hacks and improve them by switching on the
presence of mac-io devices in the tree, which uniquely identifies Apple
hardware.

r257992:
Allow OF_decode_addr() to also be able to map resources on big-endian
devices. To this end, make PCI device detection rely on the device_type
field rather than name, as per the standard.

r257993:

Make tsec work with the device tree present on the RB800. The previous code
assumed that the MDIO bus was a direct child of the Ethernet interface. It
may not be and indeed on many device trees is not. While here, add proper
locking for MII transactions, which may be on a bus shared by several MACs.

r258504:

Save and restore the trap vectors when doing OF calls on pSeries machines.

It turned out that on pSeries machines the call into OF modified the trap
vectors and this made further behaviour unpredictable.

With this commit I'm now able to boot multi user on a network booted
environment on my IntelliStation 285. This is a POWER5+ machine.

MFC r259007

Increase PHYS_AVAIL_SZ because on pSeries machines we can have many logical
regions which represent the total amount of memory. The size of these regions
is not the physical size of the chip but it is a logical one and it is given
by the OpenFirmware, it is selectable at boot time and varies between 16MB and
256MB in my case. There is an 'automatic' option which would select the size as
64MB in case you have around 16GB of RAM.
To make sure we can allocate RAM with the automatic option bump this value
of PHYS_AVAIL_SZ to 256.

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Implement GET_STACK_USAGE() on PowerPC. This implementation is identical
to that on x86 and sparc64.

Approved by: re (kib)

Rework handling of ofw_quiesce(), making it the responsibility of the
platform modules. Whether to call this function or not is highly machine
dependent: on some systems, it is required, while on others it breaks
everything. Platform modules are in a better position to figure this
out. This is required for POWER hypervisor SCSI to work correctly. There
are no functional changes on Powermac systems.

Approved by: re (kib)

Add POWER7+ and POWER8 to the CPU ID table.

Approved by: re (kib)

Raise artificial limits on number of CPUs and number of interrupts.

Approved by: re (kib)

Add hook called when every new processor is brought online -- including the
BSP -- so that platform modules have a chance to add the new CPU to any
internal bookkeeping.

Approved by: re (kib)

Use the canonical bits for wired, etc. in the PTE. This is important for
interactions with certain kinds of hypervisors that look into the PTEs
more closely than they should.

Approved by: re (kib)

Fix build with gcc. Move sf_buf_alloc()/sf_buf_free() declarations
to MD headers.

Also align the 32-bit PowerPC stacks.

Align stacks of kernel threads correctly at 16-byte boundaries rather than
making sure they are all misaligned at +8 bytes. This fixes clang builds
of powerpc64 kernels (aside from a required increase in KSTACK_PAGES which
will come later).

This commit from FreeBSD/powerpc64 with a clang-built kernel.

MFC after: 2 weeks

Newer versions of gcc define __INT64_C and __UINT64_C, so avoid
redefining them if gcc provides them.

Refactor PowerPC hwpmc(4) driver into generic and specific. More refactoring
will likely be done as more drivers are added, since AIM-compatible processors
have similar PMC configuration logic.

Remove duplicate definition of SPR MMCR0.

MFC after: 3 days

Revert r253748,253749

This WIP should not have been committed yet.

Pointyhat to: avg

put contents of cpu.h under _KERNEL

no userland-serviceable parts inside

MFC after: 20 days

Fix issues with zeroing and fetching the counters, on x86 and ppc64.
Issues were noted by Bruce Evans and are present on all architectures.

On i386, a counter fetch should use atomic read of 64bit value,
otherwise carry from the increment on other CPU could be lost for the
given fetch, making error of 2^32. If 64bit read (cmpxchg8b) is not
available on the machine, it cannot be SMP and it is enough to disable
preemption around read to avoid the split read.

On x86 the counter increment is not atomic on purpose, which makes it
possible for the store of the incremented result to override just
zeroed per-cpu slot. The effect would be a counter going off by
arbitrary value after zeroing. Perform the counter zeroing on the
same processor which does the increments, making the operations
mutually exclusive. On i386, same as for the fetching, if the
cmpxchg8b is not available, machine is not SMP and we disable
preemption for zeroing.

PowerPC64 is treated the same as amd64.

For other architectures, the changes made to allow the compilation to
succeed, without fixing the issues with zeroing or fetching. It
should be possible to handle them by using the 64bit loads and stores
atomic WRT preemption (assuming the architectures also converted from
using critical sections to proper asm). If architecture does not
provide the facility, using global (spin) mutex would be non-optimal
but working solution.

Noted by: bde
Sponsored by: The FreeBSD Foundation

Pad the PCPU MD struct, to satisfy an assert added with the projects/counters
branch import.

PR: ports/179173,ports/179164

Fix the PowerPC Book-E register definitions used by the remote GDB
protocol.

Obtained from: Juniper Networks, Inc.

Rename VM_NDOMAIN into MAXMEMDOM and move it into machine/param.h in
order to match the MAXCPU concept. The change should also be useful
for consolidation and consistency.

Sponsored by: EMC / Isilon storage division
Obtained from: jeff
Reviewed by: alc

Since UMA_ZONE_PCPU zones put a constraint on sizeof(struct pcpu), declared
as CTASSERT in MI pcpu.h, stop including all possible mutually exclusive
PCPU_MD_FIELDS fields into LINT kernels, due to brekaing
aforementioned CTASSERT.

Fix build for AIM 64bit.

Merge from projects/counters: counter(9).

Introduce counter(9) API, that implements fast and raceless counters,
provided (but not limited to) for gathering of statistical data.

See http://lists.freebsd.org/pipermail/freebsd-arch/2013-April/014204.html
for more details.

In collaboration with: kib
Reviewed by: luigi
Tested by: ae, ray
Sponsored by: Nginx, Inc.

Merge from projects/counters:

Pad struct pcpu so that its size is denominator of PAGE_SIZE. This
is done to reduce memory waste in UMA_PCPU_ZONE zones.

Sponsored by: Nginx, Inc.

Eliminate an unused #define.

Introduce PLATFORMMETHOD_END and use it.

Allow this file to be used in LOCORE sections of the kernel.

Move the prototype for savectx from cpu.h to pcb.h, as it is on other
platforms, as well as putting it in an #ifdef KERNEL block.

MFC after: 2 weeks

On Nintendo Wii CPUs, the mdp value will be garbage. Set it to NULL
so as to not confuse things.

Submitted by: Margarida Gouveia

Make the wchar_t type machine dependent.

This is required for ARM EABI. Section 7.1.1 of the Procedure Call for the
ARM Architecture (AAPCS) defines wchar_t as either an unsigned int or an
unsigned short with the former preferred.

Because of this requirement we need to move the definition of __wchar_t to
a machine dependent header. It also cleans up the macros defining the limits
of wchar_t by defining __WCHAR_MIN and __WCHAR_MAX in the same machine
dependent header then using them to define WCHAR_MIN and WCHAR_MAX
respectively.

Discussed with: bde

Implement mechanism to export some kernel timekeeping data to
usermode, using shared page. The structures and functions have vdso
prefix, to indicate the intended location of the code in some future.

The versioned per-algorithm data is exported in the format of struct
vdso_timehands, which mostly repeats the content of in-kernel struct
timehands. Usermode reading of the structure can be lockless.
Compatibility export for 32bit processes on 64bit host is also
provided. Kernel also provides usermode with indication about
currently used timecounter, so that libc can fall back to syscall if
configured timecounter is unknown to usermode code.

The shared data updates are initiated both from the tc_windup(), where
a fast task is queued to do the update, and from sysctl handlers which
change timecounter. A manual override switch
kern.timecounter.fast_gettime allows to turn off the mechanism.

Only x86 architectures export the real algorithm data, and there, only
for tsc timecounter. HPET counters page could be exported as well, but
I prefer to not further glue the kernel and libc ABI there until
proper vdso-based solution is developed.

Minimal stubs neccessary for non-x86 architectures to still compile
are provided.

Discussed with: bde
Reviewed by: jhb
Tested by: flo
MFC after: 1 month

Reserve AT_TIMEKEEP auxv entry for providing usermode the pointer to
timekeeping information.

MFC after: 1 week

The page flag PGA_WRITEABLE is set and cleared exclusively by the pmap
layer, but it is read directly by the MI VM layer. This change introduces
pmap_page_is_write_mapped() in order to completely encapsulate all direct
access to PGA_WRITEABLE in the pmap layer.

Aesthetics aside, I am making this change because amd64 will likely begin
using an alternative method to track write mappings, and having
pmap_page_is_write_mapped() in place allows me to make such a change
without further modification to the MI VM layer.

As an added bonus, tidy up some nearby comments concerning page flags.

Reviewed by: kib
MFC after: 6 weeks

Extract vendor specific Book-E pieces into separate files and have a common
skeleton (maybe we should kobj-tize this one day).

Note the PPC4xx bit is not connected to the build yet.

Obtained from: AppliedMicro, Semihalf.

Let us manage differences of Book-E PowerPC variations i.e. vendor /
implementation specific vs. the common architecture definition.

Bring PPC4XX defines (PSL, SPR, TLB). Note the new definitions under
BOOKE_PPC4XX are not used in the code yet.

This change set is not supposed to affect existing E500 support, it's just
another reorg step before bringing support for E500mc, E5500 and PPC465.

Obtained from: AppliedMicro, Freescale, Semihalf

Provide SPR definitions for newer Book-E (E500mc, E5500, PPC465).

Obtained from: Freescale, Semihalf.

Unify SPR defines formatting, no funtional changes.

Update HID defines for E500mc and E5500 CPU cores.

Obtained from: Freescale, Semihalf

Add a missing " to get closer to compiling.

Atomic operation acquire barriers also need to be isync on 64-bit systems.

Revert isync for ILP32 to sync as per my original change that I discussed
with Nathan. Leave __ATOMIC_ACQ as an isync as per Nathan.

MFp4 bz_ipv6_fast:

in_cksum.h required ip.h to be included for struct ip. To be
able to use some general checksum functions like in_addword()
in a non-IPv4 context, limit the (also exported to user space)
IPv4 specific functions to the times, when the ip.h header is
present and IPVERSION is defined (to 4).

We should consider more general checksum (updating) functions
to also allow easier incremental checksum updates in the L3/4
stack and firewalls, as well as ponder further requirements by
certain NIC drivers needing slightly different pseudo values
in offloading cases. Thinking in terms of a better "library".

Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems

Reviewed by: gnn (as part of the whole)
MFC After: 3 days

Fix physical address type to vm_paddr_t.

Fix the memory barriers for CPUs that do not like lwsync and wedge or cause
exceptions early enough during boot that the kernel will do ithe same.
Use lwsync only when compiling for LP64 and revert to the more proven isync
when compiling for ILP32. Note that in the end (i.e. between revision 222198
and this change) ILP32 changed from using sync to using isync. As per Nathan
the isync is needed to make sure I/O accesses are properly serialized with
locks and isync tends to be more effecient than sync.

While here, undefine __ATOMIC_ACQ and __ATOMIC_REL at the end of the file
so as not to leak their definitions.

Discussed with: nwhitehorn

Replace the list of PVOs owned by each PMAP with an RB tree. This simplifies
range operations like pmap_remove() and pmap_protect() as well as allowing
simple operations like pmap_extract() not to involve any global state.
This substantially reduces lock coverages for the global table lock and
improves concurrency.

Fix final bugs in memory barriers on PowerPC:
- Use isync/lwsync unconditionally for acquire/release. Use of isync
guarantees a complete memory barrier, which is important for serialization
of bus space accesses with mutexes on multi-processor systems.
- Go back to using sync as the I/O memory barrier, which solves the same
problem as above with respect to mutex release using lwsync, while not
penalizing non-I/O operations like a return to sync on the atomic release
operations would.
- Place an acquisition barrier around thread lock acquisition in
cpu_switchin().

Add a convenience macro for the returns_twice attribute, and apply it to
the prototypes of the appropriate functions (getcontext, savectx,
setjmp, sigsetjmp and vfork).

MFC after: 2 weeks

Switch the default I/O memory barrier to eieio, as it should be. This
does not appear to cause any problems due to fixes elsewhere.

MFC after: 2 months

Fix copy-and-paste error in r230400.

MFC after: 3 days

Provide a clearer split between read/write and acquire/release barriers.
This should really, actually be correct now.

Correctly specify assembler constrains for synchronization instructions.

MFC after: 3 days

Clarify what we are doing in r234583 a little better: eieio and isync do
not provide general barriers, but only barriers in the context of the
atomic sequences here. As such, make them private and keep the global
*mb() routines using a variant of sync.

On non-64-bit systems (which generally don't have lwsync), use eieio and
isync to implement read and write barriers, following Appendix B.2 of
Book II of the architecture manual. This provides a 25% speed increase
to fork() on the PowerPC G4.

Use lwsync to provide memory barriers on systems that support it instead
of sync (lwsync is an alternate encoding of sync on systems that do not
support it, providing graceful fallback). This provides more than an order
of magnitude reduction in the time required to acquire or release a mutex.

MFC after: 2 months

Remove dead code. The routines in atomic.S did not work properly anyway, and
were everywhere unused. If we turn out to need them, they should be
reimplemented.

MFC after: 2 weeks

Replace eieio; sync for creating bus-space memory barriers with sync.
sync performs a strict superset of the functions of eieio, so using both
is redundant. While here, expand bus barriers to all bus_space operations,
since many drivers do not correctly use bus_space_barrier().

In principle, we can also replace sync just with eieio, for a significant
performance increase, but it remains to be seen whether any poorly-written
drivers currently depend on the side effects of sync to properly function.

MFC after: 1 week

Organize some members of ucontext_t in the same order they are in the
trap frame. These are usually not used, and so this changes very little.

MFC after: 5 days

We don't need kcopy() in any of the remaining places it is used, so
remove it.

MFC after: 2 weeks

Give the kernel pmap lock a different name than user pmap locks. It has
(slightly) different semantics and renaming it prevents a (harmless)
WITNESS warning during bootup for 32-bit kernels on 64-bit CPUs.

MFC after: 5 days

- Rename VM_MEMATTR_UNCACHED to VM_MEMATTR_WEAK_UNCACHEABLE on x86 to
be less ambiguous and more clearly identify what it means. This
attribute is what Intel refers to as UC-, and it's only difference
relative to normal UC memory is that a WC MTRR will override a UC-
PAT entry causing the memory to be treated as WC, whereas a UC PAT
entry will always override the MTRR.
- Remove the VM_MEMATTR_UNCACHED alias from powerpc.

Allow multiple inclusion of trap.h. This has always been broken, but
until recently never caused problems.

Add software PMC support.

New kernel events can be added at various location for sampling or counting.
This will for example allow easy system profiling whatever the processor is
with known tools like pmcstat(8).

Simultaneous usage of software PMC and hardware PMC is possible, for example
looking at the lock acquire failure, page fault while sampling on
instructions.

Sponsored by: NETASQ
MFC after: 1 month

Add casts to __uint16_t to the __bswap16() macros on all arches which
didn't already have them. This is because the ternary expression will
return int, due to the Usual Arithmetic Conversions. Such casts are not
needed for the 32 and 64 bit variants.

While here, add additional parentheses around the x86 variant, to
protect against unintended consequences.

MFC after: 2 weeks

Restore proper dot symbol creation for assembly files in the kernel build case.
Without this patch we were not able to see the assembly function.
Only the function descriptor was visible.

- Distinguish between user-land and kernel when creating the ENTRY() point of
assembly source.
- Make the ENTRY() macro more readable, replace the .align directive with the
gas platform independant .p2align directive.
- Create an END()macro for later use to provide traceback tables on powerpc64.

Replace the assembler macro WEAK_ALIAS with a new macro WEAK_REFERENCE which
has the same API as __weak_reference(). Give 'x' in SYS.h a more meaningful
name.

Tested on 32- and 64-bit PowerMac.

Reviewed by: bde

Add C11 macros describing subnormal numbers to float.h.

Reviewed by: bde

This commit adds profiling support for powerpc64. Now we can do application
profiling and kernel profiling. To enable kernel profiling one has to build
kgmon(8). I will enable the build once I managed to build and test powerpc
(32-bit) kernels with profiling support.

- add a powerpc64 PROF_PROLOGUE for _mcount.
- add macros to avoid adding the PROF_PROLOGUE in certain assembly entries.
- apply these macros where needed.
- add size information to the MCOUNT function.

MFC after: 3 weeks, together with r230291

Add parentheses where required. Without them, `sizeof LDBL_MAX'
is a syntax error and shouldn't be, while `1 FLT_ROUNDS' isn't a
syntax error and should be. Thanks to bde for the examples.

Fix the value of float_t to match what is implied by FLT_EVAL_METHOD.

Change the definition of FLT_EVAL_METHOD from 1 to 0. A value of 1 implies
that the compiler promotes floats to double precision in computations, but
inspection of the output of a cross-compiler indicates that this isn't the
case on powerpc.

Rework SLB trap handling so that double-faults into an SLB trap handler are
possible, and double faults within an SLB trap handler are not. The result
is that it possible to take an SLB fault at any time, on any address, for
any reason, at any point in the kernel.

This lets us do two important things. First, it removes the (soft) 16 GB RAM
ceiling on PPC64 as well as any architectural limitations on KVA space.
Second, it lets the kernel tolerate poorly designed hypervisors that
have a tendency to fail to restore the SLB properly after a hypervisor
context switch.

MFC after: 6 weeks

Introduce internal macros for __U/INT64_C to define the U/INT64_MAX/MIN
values properly. The previous definition only worked if __STDC_LIMIT_MACROS
and __STDC_CONSTANT_MACROS were defined at the same time.

Implement hwpmc counting PMC support for PowerPC G4+ (MPC745x/MPC744x).
Sampling is in progress.

Approved by: nwhitehorn (mentor)
MFC after: 9.0-RELEASE

Replace __signed by signed.

The signed keyword is an integral part of the C syntax. There's no need
to use __signed.

Increase the available virtual address space for user programs on PowerPC
AIM systems to 4 GB on 32-bit systems and 2^64 bytes on 64-bit systems.
VM_MAXUSER_ADDRESS remains at 2 GB on pending Book-E, pending review of
an increase to 3 GB by those more familiar with Book-E.

Keep track of PVO entries in each pmap, which allows much faster
pmap_remove() for large sparse requests. This can prevent pmap_remove()
operations on 64-bit process destruction or swapout that would take
several hundred times the lifetime of the universe to complete. This
behavior is largely indistinguishable from a hang.

Use a global __pure2 function instead of a global register variable for
curthread, like on x86 and sparc64. This makes the kernel somewhat more
clang friendly, which doesn't support global register variables.

People porting FreeBSD to new architectures ought not have to
implement a deprecated FPU control interface in addition to the
standard one. To make this clearer, further deprecate ieeefp.h
by not declaring the function prototypes except on architectures
that implement them already.

Currently i386 and amd64 implement the ieeefp.h interface for
compatibility, and for fp[gs]etprec(), which doesn't exist on
most other hardware. Powerpc, sparc64, and ia64 partially implement
it and probably shouldn't, and other architectures don't implement it
at all.

Remove unused define.

MFC after: 1 month

- Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag
to VPO_UNMANAGED (and also making the flag protected by the vm object
lock, instead of vm page queue lock).
- Mark the fake pages with both PG_FICTITIOUS (as it is now) and
VPO_UNMANAGED. As a consequence, pmap code now can use use just
VPO_UNMANAGED to decide whether the page is unmanaged.

Reviewed by: alc
Tested by: pho (x86, previous version), marius (sparc64),
marcel (arm, ia64, powerpc), ray (mips)
Sponsored by: The FreeBSD Foundation
Approved by: re (bz)

Add the possibility to specify from kernel configs MAXCPU value.
This patch is going to help in cases like mips flavours where you
want a more granular support on MAXCPU.

No MFC is previewed for this patch.

Tested by: pluknet
Approved by: re (kib)

Use the ABI-mandated thread pointer register (r2 for ppc32, r13 for ppc64)
instead of a PCPU field for curthread. This averts a race on SMP systems
with a high interrupt rate where the thread looking up the value of
curthread could be preempted and migrated between obtaining the PCPU
pointer and reading the value of pc_curthread, resulting in curthread being
observed to be the current thread on the thread's original CPU. This played
merry havoc with the system, in particular with mutexes. Many thanks to
jhb for helping me work this one out.

Note that Book-E is in principle susceptible to the same problem, but has
not been modified yet due to lack of Book-E hardware.

MFC after: 2 weeks

etire the cpumask_t type and replace it with cpuset_t usage.

This is intended to fix the bug where cpu mask objects are
capped to 32. MAXCPU, then, can now arbitrarely bumped to whatever
value. Anyway, as long as several structures in the kernel are
statically allocated and sized as MAXCPU, it is suggested to keep it
as low as possible for the time being.

Technical notes on this commit itself:
- More functions to handle with cpuset_t objects are introduced.
The most notable are cpusetobj_ffs() (which calculates a ffs(3)
for a cpuset_t object), cpusetobj_strprint() (which prepares a string
representing a cpuset_t object) and cpusetobj_strscan() (which
creates a valid cpuset_t starting from a string representation).
- pc_cpumask and pc_other_cpus are target to be removed soon.
With the moving from cpumask_t to cpuset_t they are now inefficient
and not really useful. Anyway, for the time being, please note that
access to pcpu datas is protected by sched_pin() in order to avoid
migrating the CPU while reading more than one (possible) word
- Please note that size of cpuset_t objects may differ between kernel
and userland. While this is not directly related to the patch itself,
it is good to understand that concept and possibly use the patch
as a reference on how to deal with cpuset_t objects in userland, when
accessing kernland members.
- KTR_CPUMASK is changed and now is represented through a string, to be
set as the example reported in NOTES.

Please additively note that no MAXCPU is bumped in this patch, but
private testing has been done until to MAXCPU=128 on a real 8x8x2(htt)
machine (amd64).

Please note that the FreeBSD version is not yet bumped because of
the upcoming pcpu changes. However, note that this patch is not
targeted for MFC.

People to thank for the time spent on this patch:
- sbruno, pluknet and Nicholas Esborn (nick AT desert DOT net) tested
several revision of the patches and really helped in improving
stability of this work.
- marius fixed several bugs in the sparc64 implementation and reviewed
patches related to ktr.
- jeff and jhb discussed the basic approach followed.
- kib and marcel made targeted review on some specific part of the
patch.
- marius, art, nwhitehorn and andreast reviewed MD specific part of
the patch.
- marius, andreast, gonzo, nwhitehorn and jceel tested MD specific
implementations of the patch.
- Other people have made contributions on other patches that have been
already committed and have been listed separately.

Companies that should be mentioned for having participated at several
degrees:
- Yahoo! for having offered the machines used for testing on big
count of CPUs.
- The FreeBSD Foundation for having sponsored my devsummit attendance,
which has been instrumental.
- Sandvine for having offered offices and infrastructure during
development.

(I really hope I didn't forget anyone, if it happened I apologize in
advance).

The POWER7 has only 32 SLB slots instead of 64, like other supported
64-bit PowerPC CPUs. Add infrastructure to support variable numbers of
SLB slots and move the user slot from 63 to 0, so that it is always
available.

MFpseries:
Renovate and improve the AIM Open Firmware support:
- Add RTAS (Run-Time Abstraction Services) support, found on all IBM systems
and some Apple ones
- Improve support for 32-bit real mode Open Firmware systems
- Pull some more OF bits over from the AIM directory
- Fix memory detection on IBM LPARs and systems with more than one /memory
node (by andreast@)

The P4080 has 8 cores. Bump MAXCPU to 8 to match.

o Add system versions for the P4040(E) and P4080(E).
o In bare_probe(), change the logic that determines the maximum
number of processors/cores into a switch statement and take
advantage of the fact that bit 3 of the SVR value indicates
whether we're running on a security enabled version. Since we
don't care about that here, mask the bit. All -E versions
are taken care of automatically.

o Swap the SVR numbers for MPC8533 & MPC8533E
o Add SVR defines for P1011(E), P1020(E), P2010(E) & P2020(E)

Merge r221614,221696,221737,221840 from largeSMP project branch:
Rewrite atomic operations for powerpc in order to achieve the following:
- Produce a type-clean implementation (in terms of functions arguments
and returned values) for the primitives.
- Fix errors with _long() atomics where they ended up with the wrong
arguments to be accepted.
- Follow the sys/type.h specifics that define the numbered types starting
from standard C types.
- Let _ptr() version to not auto-magically cast arguments, but leave
the burden on callers, as _ptr() atomic is intended to be used
relatively rarely.

Fix cfi in order to support the latest point.

In collabouration with: bde
Tested by: andreast, nwhitehorn, jceel
MFC after: 2 weeks

Move the ZERO_REGION_SIZE to a machine-dependent file, as on many
architectures (i386, for example) the virtual memory space may be
constrained enough that 2MB is a large chunk. Use 64K for arches
other than amd64 and ia64, with special handling for sparc64 due to
differing hardware.

Also commit the comment changes to kmem_init_zero_region() that I
missed due to not saving the file. (Darn the unfamiliar development
environment).

Arch maintainers, please feel free to adjust ZERO_REGION_SIZE as you
see fit.

Requested by: alc
MFC after: 1 week
MFC with: r221853

Remove duplicate definition of FIRSTARG.

Approved by: nwhitehorn (mentor)

Turn off default generation of userland dot symbols on powerpc64 now that
we have a binutils that supports it. Kernel dot symbols remain on to assist
DDB.

Remove pmap fields that are either unused or not fully implemented.

Discussed with: kib

Rename INTR_VEC to MAP_IRQ. From the OFW or FDT we obtain a
PIC handle with interrupt pin. This we map to the resource
called SYS_RES_IRQ.

Fix the interrupt code, broken 7 months ago. The interrupt framework
already supported nested PICs, but was limited to having a nested
AT-PIC only. With G5 support the need for nested OpenPIC controllers
needed to be added. This was done the wrong way and broke the MPC8555
eval system in the process.

OFW, as well as FDT, describe the interrupt routing in terms of a
controller and an interrupt pin on it. This needs to be mapped to a
flat and global resource: the IRQ. The IRQ is the same as the PCI
intline and as such needs to be representable in 8 bits. Secondly,
ISA support pretty much dictates that IRQ 0-15 should be reserved
for ISA interrupts, because of the internal workins of south bridges.
Both were broken.

This change reverts revision 209298 for a big part and re-implements
it simpler. In particular:
o The id() method of the PIC I/F is removed again. It's not needed.
o The openpic_attach() function has been changed to take the OFW
or FDT phandle of the controller as a second argument. All bus
attachments that previously used openpic_attach() as the attach
method of the device I/F now implement as bus-specific method
and pass the phandle_t to the renamed openpic_attach().
o Change powerpc_register_pic() to take a few more arguments. In
particular:
- Pass the number of IPIs specificly. The number of IRQs carved
out for a PIC is the sum of the number of int. pins and IPIs.
- Pass a flag indicating whether the PIC is an AT-PIC or not.
This tells the interrupt framework whether to assign IRQ 0-15
or some other range.
o Until we implement proper multi-pass bus enumeration, we have to
handle the case where we need to map from PIC+pin to IRQ *before*
the PIC gets registered. This is done in a similar way as before,
but rather than carving out 256 IRQs per PIC, we carve out 128
IRQs (124 pins + 4 IPIs). This is supposed to handle the G5 case,
but should really be fixed properly using multiple passes.
o Have the interrupt framework set root_pic in most cases and not
put that burden in PIC drivers (for the most part).
o Remove powerpc_ign_lookup() and replace it with powerpc_get_irq().
Remove IGN_SHIFT, INTR_INTLINE and INTR_IGN.

Related to the above, fix the Freescale PCI controller driver, broken
by the FDT code. Besides not attaching properly, bus numbers were
assigned improperly and enumeration was broken in general. This
prevented the AT PIC from being discovered and interrupt routing to
work properly. Consequently, the ata(4) controller stopped functioning.

Fix the driver, and FDT PCI support, enough to get the MPC8555CDS
going again. The FDT PCI code needs a whole lot more work.

No breakages are expected, but lackiong G5 hardware, it's possible
that there are unpleasant side-effects. At least MPC85xx support is
back to where it was 7 months ago -- it's amazing how badly support
can be broken in just 7 months...

Sponsored by: Juniper Networks

Introduce macro FDT_MAP_IRQ to map from an interrupt controller and
interrupt pin pair to a global IRQ number. When multiple PICs exist
on a board, the interrupt pin alone is not unique.

Add reader/writer lock around mem_range_attr_get() and mem_range_attr_set().
Compile sys/dev/mem/memutil.c for all supported platforms and remove now
unnecessary dev_mem_md_init(). Consistently define mem_range_softc from
mem.c for all platforms. Add missing #include guards for machine/memdev.h
and sys/memrange.h. Clean up some nearby style(9) nits.

MFC after: 1 month

Don't redefine MODINFOMD_BOOTINFO as MODINFOMD_DTBP. This
breaks support for older loaders. Add MODINFOMD_DTBP as
a new tag instead.

Enable shared page for the signal trampolines on PowerPC.

Reviewed and tested by: nwhitehorn

Fix handling of NX pages on capable CPUs. Thanks to kib for prodding me
in the right direction.

Move repeated MAXSLP definition from machine/vmparam.h to sys/vmmeter.h.
Update the outdated comments describing MAXSLP and the process
selection algorithm for swap out.

Comments wording and reviewed by: alc

We don't support any floating point types larger than double on
powerpc, so DECIMAL_DIG should be 17.

White space changes to align comments. The mips and powerpc _inttypes.h
are now exactly the same.

Approved by: kib (mentor)

Rename PRIreg helper macro to PRIptr to better reflect its use. Registers
and pointers don't always have the same size, e.g. the __mips_n32 ABI
(ILP32) has 64 bit registers but 32 bit pointers.

On mips introduce PRIptr to fix the format specifier for (u)intptr_t.

Prefix PRI64 and PRIptr with underscores because macro names starting with
PRI[a-zX] are reserved for future use.

Approved by: kib (mentor)

On mixed 32/64 bit architectures (mips, powerpc) use __LP64__ rather than
architecture macros (__mips_n64, __powerpc64__) when 64 bit types (and
corresponding macros) are different from 32 bit. [1]

Correct the type of INT64_MIN, INT64_MAX and UINT64_MAX.

Define (U)INTMAX_C as an alias for (U)INT64_C matching the type definition
for (u)intmax_t. Do this on all architectures for consistency.

Suggested by: bde [1]
Approved by: kib (mentor)

On 32 bit architectures define (u)int64_t as (unsigned) long long instead
of (unsigned) int __attribute__((__mode__(__DI__))). This aligns better
with macros such as (U)INT64_C, (U)INT64_MAX, etc. which assume (u)int64_t
has type (unsigned) long long.

The mode attribute was used because long long wasn't standardised until
C99. Nowadays compilers should support long long and use of the mode
attribute is discouraged according to GCC Internals documentation.

The type definition has to be marked with __extension__ to support
compilation with "-std=c89 -pedantic".

Discussed with: bde
Approved by: kib (mentor)

Fix types of some values in machine/_limits.h.

On some architectures UCHAR_MAX and USHRT_MAX had type unsigned int.
However, lacking integer suffixes for types smaller than int, their type
should correspond to that of an object of type unsigned char (or short)
when used in an expression with objects of type int. In that case unsigned
char (short) are promoted to int (i.e. signed) so the type of UCHAR_MAX and
USHRT_MAX should also be int.

Where MIN/MAX constants implicitly have the correct type the suffix has
been removed.

While here, correct some comments.

Reviewed by: bde
Approved by: kib (mentor)

Remove unused support for 64 bit long on 32 bit architectures.

It was used mainly to discover and fix some 64-bit portability problems
before 64-bit arches were widely available.

Discussed with: bde
Approved by: kib (mentor)

Add AT_STACKPROT elf aux vector. Will be used to inform rtld about the
initial stack protection set by the kernel image activator.

Import support for the Sony Playstation 3 using the OtherOS feature
available on firmwares 3.15 and earlier.

Caveats: Support for the internal SATA controller is currently missing,
as is support for framebuffer resolutions other than 720x480. These
deficiencies will be remedied soon.

Special thanks to Peter Grehan for providing the hardware that made this
port possible, and thanks to Geoff Levand of Sony Computer Entertainment
for advice on the LV1 hypervisor.

Memory can be laid out with large gaps on 64-bit PowerPC, so switch to
VM_PHYSSEG_SPARSE.

Switch which software-reserved bit is used to designate a locked PTE
to correspond to the definition used by the PAPR spec so that its PTE
insertion algorithm will properly respect it.

Add an abstraction layer to the 64-bit AIM MMU's page table manipulation
logic to support modifying the page table through a hypervisor. This
uses KOBJ inheritance to provide subclasses of the base 64-bit AIM MMU
class with additional methods for page table manipulation.

Many thanks to Peter Grehan for suggesting this design and implementing
the MMU KOBJ inheritance mechanism.

Provide a simple IOMMU framework on PowerPC, which is required to support
PPC hypervisors.

Revert r216134. This checkin broke platforms where bus_space are macros:
they need to be a single statement, and do { } while (0) doesn't work in this
situation so revert until a solution can be devised.

Disallow passing in a count of zero bytes to the bus_space(9) functions.

Passing a count of zero on i386 and amd64 for [I386|AMD64]_BUS_SPACE_MEM
causes a crash/hang since the 'loop' instruction decrements the counter
before checking if it's zero.

PR: kern/80980
Discussed with: jhb

Define bswap macros for constants to allow the compiler to pre-compute
byte-swapped versions of compile-time constants. This allows use of
bswap() and htole*() in initializers, which is required to cross-build
btxld.

Obtained from: sparc64

Add CPU support code for the IBM Cell Broadband Engine.

Add some platform KOBJ extensions and continue integrating PowerPC
hypervisor infrastructure support:
- Fix coexistence of multiple platform modules in the same kernel
- Allow platform modules to provide an SMP topology
- PowerPC hypervisors limit the amount of memory accessible in real mode.
Allow the platform modules to specify the maximum real-mode address,
and modify the bits of the kernel that need to allocate
real-mode-accessible buffers to respect this limits.

Use local TLB_UNLOCKED marker instead of MTX_UNOWNED for Book-E PowerPC trap
routines.

This unbreaks Book-E build after the recent machine/mutex.h removal.

While there move tlb_*lock() prototypes to machine/tlb.h.

Submitted by: jhb

- Remove <machine/mutex.h>. Most of the headers were empty, and the
contents of the ones that were not empty were stale and unused.
- Now that <machine/mutex.h> no longer exists, there is no need to allow it
to override various helper macros in <sys/mutex.h>.
- Rename various helper macros for low-level operations on mutexes to live
in the _mtx_* or __mtx_* namespaces. While here, change the names to more
closely match the real API functions they are backing.
- Drop support for including <sys/mutex.h> in assembly source files.

Suggested by: bde (1, 2)

Restructure the way the copyin/copyout segment is stored to prevent a
concurrency bug. Since all SLB/SR entries were invalidated during an
exception, a decrementer exception could cause the user segment to be
invalidated during a copyin()/copyout() without a thread switch that
would cause it to be restored from the PCB, potentially causing the
operation to continue on invalid memory. This is now handled by explicit
restoration of segment 12 from the PCB on 32-bit systems and a check in
the Data Segment Exception handler on 64-bit.

While here, cause copyin()/copyout() to check whether the requested
user segment is already installed, saving some pipeline flushes, and
fix the synchronization primitives around the mtsr and slbmte
instructions to prevent accessing stale segments.

MFC after: 2 weeks

Handle vector assist traps without a kernel panic, by setting denormalized
values to zero. A correct solution would involve emulating vector
operations on denormalized values, but this has little effect on accuracy
and is much less complicated for now.

MFC after: 2 weeks

Add a memory-range interface to /dev/mem on PowerPC using PAT attributes.
Unlike actual MTRR, this only controls the mapping attributes for
subsequent mmap() of /dev/mem. Nonetheless, the support is sufficiently
MTRR-like that Xorg can use it, which translates into an enormous increase
in graphics performance on PowerPC.

MFC after: 2 weeks

Add support for memory attributes (pmap_mapdev_attr() and friends) on
PowerPC/AIM. This is currently stubbed out on Book-E, since I have no
idea how to implement it there.

Split the SLB mirror cache into two kinds of object, one for kernel maps
which are similar to the previous ones, and one for user maps, which
are arrays of pointers into the SLB tree. This changes makes user SLB
updates atomic, closing a window for memory corruption. While here,
rearrange the allocation functions to make context switches faster.

Replace the SLB backing store splay tree used on 64-bit PowerPC AIM
hardware with a lockless sparse tree design. This marginally improves
the performance of PMAP and allows copyin()/copyout() to run without
acquiring locks when used on wired mappings.

Submitted by: mdf

Introduce inheritance into the PowerPC MMU kobj interface.

include/mmuvar.h - Change the MMU_DEF macro to also create the class
definition as well as define the DATA_SET. Add a macro, MMU_DEF_INHERIT,
which has an extra parameter specifying the MMU class to inherit methods
from. Update the comments at the start of the header file to describe the
new macros.

booke/pmap.c
aim/mmu_oea.c
aim/mmu_oea64.c - Collapse mmu_def_t declaration into updated MMU_DEF macro

The MMU_DEF_INHERIT macro will be used in the PS3 MMU implementation to
allow it to inherit the stock powerpc64 MMU methods.

Reviewed by: nwhitehorn

Refactor timer management code with priority to one-shot operation mode.
The main goal of this is to generate timer interrupts only when there is
some work to do. When CPU is busy interrupts are generating at full rate
of hz + stathz to fullfill scheduler and timekeeping requirements. But
when CPU is idle, only minimum set of interrupts (down to 8 interrupts per
second per CPU now), needed to handle scheduled callouts is executed.
This allows significantly increase idle CPU sleep time, increasing effect
of static power-saving technologies. Also it should reduce host CPU load
on virtualized systems, when guest system is idle.

There is set of tunables, also available as writable sysctls, allowing to
control wanted event timer subsystem behavior:
kern.eventtimer.timer - allows to choose event timer hardware to use.
On x86 there is up to 4 different kinds of timers. Depending on whether
chosen timer is per-CPU, behavior of other options slightly differs.
kern.eventtimer.periodic - allows to choose periodic and one-shot
operation mode. In periodic mode, current timer hardware taken as the only
source of time for time events. This mode is quite alike to previous kernel
behavior. One-shot mode instead uses currently selected time counter
hardware to schedule all needed events one by one and program timer to
generate interrupt exactly in specified time. Default value depends of
chosen timer capabilities, but one-shot mode is preferred, until other is
forced by user or hardware.
kern.eventtimer.singlemul - in periodic mode specifies how much times
higher timer frequency should be, to not strictly alias hardclock() and
statclock() events. Default values are 2 and 4, but could be reduced to 1
if extra interrupts are unwanted.
kern.eventtimer.idletick - makes each CPU to receive every timer interrupt
independently of whether they busy or not. By default this options is
disabled. If chosen timer is per-CPU and runs in periodic mode, this option
has no effect - all interrupts are generating.

As soon as this patch modifies cpu_idle() on some platforms, I have also
refactored one on x86. Now it makes use of MONITOR/MWAIT instrunctions
(if supported) under high sleep/wakeup rate, as fast alternative to other
methods. It allows SMP scheduler to wake up sleeping CPUs much faster
without using IPI, significantly increasing performance on some highly
task-switching loads.

Tested by: many (on i386, amd64, sparc64 and powerc)
H/W donated by: Gheorghe Ardelean
Sponsored by: iXsystems, Inc.

Fix the build after r212453. IPI_STATCLOCK declaration is still needed
for build, though not really used.

Submitted by: andreast

Update PowerPC event timer code to use new event timers infrastructure.

Reviewed by: nwitehorn
Tested by: andreast
H/W donated by: Gheorghe Ardelean

- Bump MAXCPU to 4. Tested on a quad G5 with both 32 and 64-bit kernels.
A make buildkernel -j4 uses ~360% CPU.
- Bracket the AP spinup printf with a mutex to avoid garbled output.
- Enable SMP by default on powerpc64.

Reviewed by: nwhitehorn

Restructure how reset and poweroff are handled on PowerPC systems, since
the existing code was very platform specific, and broken for SMP systems
trying to reboot from KDB.

- Add a new PLATFORM_RESET() method to the platform KOBJ interface, and
migrate existing reset functions into platform modules.
- Modify the OF_reboot() routine to submit the request by hand to avoid
the IPIs involved in the regular openfirmware() routine. This fixes
reboot from KDB on SMP machines.
- Move non-KDB reset and poweroff functions on the Powermac platform
into the relevant power control drivers (cuda, pmu, smu), instead of
using them through the Open Firmware backdoor.
- Rename platform_chrp to platform_powermac since it has become
increasingly Powermac specific. When we gain support for IBM systems,
we will grow a new platform_chrp.

Supply some useful information to the started image using ELF aux vectors.
In particular, provide pagesize and pagesizes array, the canary value
for SSP use, number of host CPUs and osreldate.

Tested by: marius (sparc64)
MFC after: 1 month

Update various places that store or manipulate CPU masks to use cpumask_t
instead of int or u_int. Since cpumask_t is currently u_int on all
platforms this should just be a cosmetic change.

Add a new ipi_cpu() function to the MI IPI API that can be used to send an
IPI to a specific CPU by its cpuid. Replace calls to ipi_selected() that
constructed a mask for a single CPU with calls to ipi_cpu() instead. This
will matter more in the future when we transition from cpumask_t to
cpuset_t for CPU masks in which case building a CPU mask is more expensive.

Submitted by: peter, sbruno
Reviewed by: rookie
Obtained from: Yahoo! (x86)
MFC after: 1 month

Improve hash coverage for kernel page table entries by modifying the kernel
ESID -> VSID map function. This makes ZFS run stably on PowerPC under
heavy loads (repeated simultaneous SVN checkouts and updates).

Very rough first cut at NUMA support for the physical page allocator. For
now it uses a very dumb first-touch allocation policy. This will change in
the future.
- Each architecture indicates the maximum number of supported memory domains
via a new VM_NDOMAIN parameter in <machine/vmparam.h>.
- Each cpu now has a PCPU_GET(domain) member to indicate the memory domain
a CPU belongs to. Domain values are dense and numbered from 0.
- When a platform supports multiple domains, the default freelist
(VM_FREELIST_DEFAULT) is split up into N freelists, one for each domain.
The MD code is required to populate an array of mem_affinity structures.
Each entry in the array defines a range of memory (start and end) and a
domain for the range. Multiple entries may be present for a single
domain. The list is terminated by an entry where all fields are zero.
This array of structures is used to split up phys_avail[] regions that
fall in VM_FREELIST_DEFAULT into per-domain freelists.
- Each memory domain has a separate lookup-array of freelists that is
used when fulfulling a physical memory allocation. Right now the
per-domain freelists are listed in a round-robin order for each domain.
In the future a table such as the ACPI SLIT table may be used to order
the per-domain lookup lists based on the penalty for each memory domain
relative to a specific domain. The lookup lists may be examined via a
new vm.phys.lookup_lists sysctl.
- The first-touch policy is implemented by using PCPU_GET(domain) to
pick a lookup list when allocating memory.

Reviewed by: alc

When compat32 binary asks for the value of hw.machine_arch, report the
name of 32bit sibling architecture instead of the host one. Do the
same for hw.machine on amd64.

Add a safety belt debug.adaptive_machine_arch sysctl, to turn the
substitution off.

Reviewed by: jhb, nwhitehorn
MFC after: 2 weeks

Eliminate FDT_IMMR_VA define.

This removes platform dependencies from <machine>/fdt.h for the benfit of
portability.

Remove obsolete code that sets SHMMAXPGS to a tiny value by default
on PowerPC.

MFppc64:

Kernel sources for 64-bit PowerPC, along with build-system changes to keep
32-bit kernels compiling (build system changes for 64-bit kernels are
coming later). Existing 32-bit PowerPC kernel configurations must be
updated after this change to specify their architecture.

Unify ABI-related bits of the Book-E and AIM machdep routines
(exec_setregs, etc.) in order to simplify the addition of 64-bit support,
and possible future extension of the Book-E code to handle hard floating
point and Altivec.

MFC after: 1 month

MFppc64:

Provide ELF definitions for 64-bit PowerPC. This unbreaks the powerpc
loader build.

Convert Freescale PowerPC platforms to FDT convention.

The following systems are affected:

- MPC8555CDS
- MPC8572DS

This overhaul covers the following major changes:

- All integrated peripherals drivers for Freescale MPC85XX SoC, which are
currently in the FreeBSD source tree are reworked and adjusted so they
derive config data out of the device tree blob (instead of hard coded /
tabelarized values).

- This includes: LBC, PCI / PCI-Express, I2C, DS1553, OpenPIC, TSEC, SEC,
QUICC, UART, CFI.

- Thanks to the common FDT infrastrucutre (fdtbus, simplebus) we retire
ocpbus(4) driver, which was based on hard-coded config data.

Note that world for these platforms has to be built WITH_FDT.

Reviewed by: imp
Sponsored by: The FreeBSD Foundation

Remove the unneeded header <machine/intr.h>.

MFia64:
When compiling with profiling, we define PROF for userspace and GPROF
for the kernel.

Fix profiling (part 1):
o Functions are 4-byte aligned for Book-E.
o We get compiled with -DPROF and not -DGPROF if profiling
is enabled.

Configure interrupts on SMP systems to be distributed among all online
CPUs by default, and provide a functional version of BUS_BIND_INTR().
While here, fix some potential concurrency problems in the interrupt
handling code.

Missed commit in r209310: the IRQ number in INTR_VEC() should have
parantheses around it to allow arithmetic expressions to be passed.

Submitted by: Andreas Tobler

Provide for multiple, cascaded PICs on PowerPC systems, and extend the
OFW interrupt map interface to also return the device's interrupt parent.

MFC after: 8.1-RELEASE

Initial loader(8) support for Flattened Device Tree.

o This is disabled by default for now, and can be enabled using WITH_FDT at
build time.

o Tested with ARM and PowerPC.

Reviewed by: imp
Sponsored by: The FreeBSD Foundation

Reorganize syscall entry and leave handling.

Extend struct sysvec with three new elements:
sv_fetch_syscall_args - the method to fetch syscall arguments from
usermode into struct syscall_args. The structure is machine-depended
(this might be reconsidered after all architectures are converted).
sv_set_syscall_retval - the method to set a return value for usermode
from the syscall. It is a generalization of
cpu_set_syscall_retval(9) to allow ABIs to override the way to set a
return value.
sv_syscallnames - the table of syscall names.

Use sv_set_syscall_retval in kern_sigsuspend() instead of hardcoding
the call to cpu_set_syscall_retval().

The new functions syscallenter(9) and syscallret(9) are provided that
use sv_*syscall* pointers and contain the common repeated code from
the syscall() implementations for the architecture-specific syscall
trap handlers.

Syscallenter() fetches arguments, calls syscall implementation from
ABI sysent table, and set up return frame. The end of syscall
bookkeeping is done by syscallret().

Take advantage of single place for MI syscall handling code and
implement ptrace_lwpinfo pl_flags PL_FLAG_SCE, PL_FLAG_SCX and
PL_FLAG_EXEC. The SCE and SCX flags notify the debugger that the
thread is stopped at syscall entry or return point respectively. The
EXEC flag augments SCX and notifies debugger that the process address
space was changed by one of exec(2)-family syscalls.

The i386, amd64, sparc64, sun4v, powerpc and ia64 syscall()s are
changed to use syscallenter()/syscallret(). MIPS and arm are not
converted and use the mostly unchanged syscall() implementation.

Reviewed by: jhb, marcel, marius, nwhitehorn, stas
Tested by: marcel (ia64), marius (sparc64), nwhitehorn (powerpc),
stas (mips)
MFC after: 1 month

Provide missing members for Book-E pmap (and fix build).

Add support for the U4 PCI-Express bridge chipset used in late-generation
Powermac G5 systems. MSI and several other things are not presently
supported.

The U3/U4 internal device support portions of this change were contributed
by Andreas Tobler.

MFC after: 1 week

On Alan's advice, rather than do a wholesale conversion on a single
architecture from page queue lock to a hashed array of page locks
(based on a patch by Jeff Roberson), I've implemented page lock
support in the MI code and have only moved vm_page's hold_count
out from under page queue mutex to page lock. This changes
pmap_extract_and_hold on all pmaps.

Supported by: Bitgravity Inc.

Discussed with: alc, jeffr, and kib

Style: use #define<TAB> instead of #define<SPACE>.

Noted by: bde, pluknet gmail com
MFC after: 11 days

Move the constants specifying the size of struct kinfo_proc into
machine-specific header files. Add KINFO_PROC32_SIZE for struct
kinfo_proc32 for architectures providing COMPAT_FREEBSD32. Add
CTASSERT for the size of struct kinfo_proc32.

Submitted by: pluknet
Reviewed by: imp, jhb, nwhitehorn
MFC after: 2 weeks

The NetBSD Foundation has granted permission to remove clause 3 and 4 from
the software.

Obtained from: NetBSD

The NetBSD Foundation has granted permission to remove clause 3 and 4 from
their software.

Obtained from: NetBSD

Close a race involving the OEA64 scratchpage. When the scratch page's
physical address is changed, there is a brief window during which its PTE
is invalid. Since moea64_set_scratchpage_pa() does not and cannot hold
the page table lock, it was possible for another CPU to insert a new PTE
into the scratch page's PTEG slot during this interval, corrupting both
mappings.

Solve this by creating a new flag, LPTE_LOCKED, such that
moea64_pte_insert will avoid claiming locked PTEG slots even if they
are invalid. This change also incorporates some additional paranoia
added to solve things I thought might be this bug.

Reported by: linimon

Reduce KVA pressure on OEA64 systems running in bridge mode by mapping
UMA segments at their physical addresses instead of into KVA. This emulates
the direct mapping behavior of OEA32 in an ad-hoc way. To make this work
properly required sharing the entire kernel PMAP with Open Firmware, so
ofw_pmap is transformed into a stub on 64-bit CPUs.

Also implement some more tweaks to get more mileage out of our limited
amount of KVA, principally by extending KVA into segment 16 until the
beginning of the first OFW mapping.

Reported by: linimon

Bump limits on PowerPC. This allows large executables like parts of LLVM
to function.

Reviewed by: grehan
Obtained from: NetBSD
MFC after: 2 weeks

Add a CPU features framework on PowerPC and simplify CPU setup a little
more. This provides three new sysctls to user space:
hw.cpu_features - A bitmask of available CPU features
hw.floatingpoint - Whether or not there is hardware FP support
hw.altivec - Whether or not Altivec is available

PR: powerpc/139154
MFC after: 10 days

Turn on NAP mode on G5 systems, and refactor the HID0 setup code a little.
This makes my G5 Xserve sound slightly less like it is filled with
howling banshees.

Add SMP support on U3-based G5 systems. This does not yet work perfectly:
at least on my Xserve, getting the decrementer and timebase on APs to tick
requires setting up a clock chip over I2C, which is not yet done.

While here, correct the 64-bit tlbie function to set the CPU to 64-bit
mode correctly.

Hardware donated by: grehan

o Introduce vm_sync_icache() for making the I-cache coherent with
the memory or D-cache, depending on the semantics of the platform.
vm_sync_icache() is basically a wrapper around pmap_sync_icache(),
that translates the vm_map_t argumument to pmap_t.
o Introduce pmap_sync_icache() to all PMAP implementation. For powerpc
it replaces the pmap_page_executable() function, added to solve
the I-cache problem in uiomove_fromphys().
o In proc_rwmem() call vm_sync_icache() when writing to a page that
has execute permissions. This assures that when breakpoints are
written, the I-cache will be coherent and the process will actually
hit the breakpoint.
o This also fixes the Book-E PMAP implementation that was missing
necessary locking while trying to deal with the I-cache coherency
in pmap_enter() (read: mmu_booke_enter_locked).

The key property of this change is that the I-cache is made coherent
*after* writes have been done. Doing it in the PMAP layer when adding
or changing a mapping means that the I-cache is made coherent *before*
any writes happen. The difference is key when the I-cache prefetches.

Define architectural load bases for PIE binaries. Addresses were selected
by looking at the bases used for non-relocatable executables by gnu ld(1),
and adjusting it slightly.

Discussed with: bz
Reviewed by: kan
Tested by: bz (i386, amd64), bsam (linux)
MFC after: some time

Add a new sysctl for reporting all of the supported page sizes.

Reviewed by: jhb
MFC after: 3 weeks

Get rid of the _NO_NAMESPACE_POLLUTION kludge by creating an
architecture specific include file containing the _ALIGN*
stuff which <sys/socket.h> needs.

* Completely Remove the option STOP_NMI from the kernel. This option
has proven to have a good effect when entering KDB by using a NMI,
but it completely violates all the good rules about interrupts
disabled while holding a spinlock in other occasions. This can be the
cause of deadlocks on events where a normal IPI_STOP is expected.
* Adds an new IPI called IPI_STOP_HARD on all the supported architectures.
This IPI is responsible for sending a stop message among CPUs using a
privileged channel when disponible. In other cases it just does match a
normal IPI_STOP.
Right now the IPI_STOP_HARD functionality uses a NMI on ia32 and amd64
architectures, while on the other has a normal IPI_STOP effect. It is
responsibility of maintainers to eventually implement an hard stop
when necessary and possible.
* Use the new IPI facility in order to implement a new userend SMP kernel
function called stop_cpus_hard(). That is specular to stop_cpu() but
it does use the privileged channel for the stopping facility.
* Let KDB use the newly introduced function stop_cpus_hard() and leave
stop_cpus() for all the other cases
* Disable interrupts on CPU0 when starting the process of APs suspension.
* Style cleanup and comments adding

This patch should fix the reboot/shutdown deadlocks many users are
constantly reporting on mailing lists.

Please don't forget to update your config file with the STOP_NMI
option removal

Reviewed by: jhb
Tested by: pho, bz, rink
Approved by: re (kib)

Add support to the virtual memory system for configuring machine-
dependent memory attributes:

Rename vm_cache_mode_t to vm_memattr_t. The new name reflects the
fact that there are machine-dependent memory attributes that have
nothing to do with controlling the cache's behavior.

Introduce vm_object_set_memattr() for setting the default memory
attributes that will be given to an object's pages.

Introduce and use pmap_page_{get,set}_memattr() for getting and
setting a page's machine-dependent memory attributes. Add full
support for these functions on amd64 and i386 and stubs for them on
the other architectures. The function pmap_page_set_memattr() is also
responsible for any other machine-dependent aspects of changing a
page's memory attributes, such as flushing the cache or updating the
direct map. The uses include kmem_alloc_contig(), vm_page_alloc(),
and the device pager:

kmem_alloc_contig() can now be used to allocate kernel memory with
non-default memory attributes on amd64 and i386.

vm_page_alloc() and the device pager will set the memory attributes
for the real or fictitious page according to the object's default
memory attributes.

Update the various pmap functions on amd64 and i386 that map pages to
incorporate each page's memory attributes in the mapping.

Notes: (1) Inherent to this design are safety features that prevent
the specification of inconsistent memory attributes by different
mappings on amd64 and i386. In addition, the device pager provides a
warning when a device driver creates a fictitious page with memory
attributes that are inconsistent with the real page that the
fictitious page is an alias for. (2) Storing the machine-dependent
memory attributes for amd64 and i386 as a dedicated "int" in "struct
md_page" represents a compromise between space efficiency and the ease
of MFCing these changes to RELENG_7.

In collaboration with: jhb

Approved by: re (kib)

Cleanup ALIGNED_POINTER:
o add to platforms where it was missing (arm, i386, powerpc, sparc64, sun4v)
o define as "1" on amd64 and i386 where there is no restriction
o make the type returned consistent with ALIGN
o remove _ALIGNED_POINTER
o make associated comments consistent

Reviewed by: bde, imp, marcel
Approved by: re (kensmith)

Correct the #endif comment.

Noticed by: jmallett
Approved by: re (kib)

This change is the next step in implementing the cache control functionality
required by video card drivers. Specifically, this change introduces
vm_cache_mode_t with an appropriate VM_CACHE_DEFAULT definition on all
architectures. In addition, this changes adds a vm_cache_mode_t parameter
to kmem_alloc_contig() and vm_phys_alloc_contig(). These will be the
interfaces for allocating mapped kernel memory and physical memory,
respectively, with non-default cache modes.

In collaboration with: jhb

Fix copy/paste typo in last revision. PMC0 control should be shifted 8
bits, not 6, on the PPC 970.

Teach cpu_est_clockrate() about the G5's slightly different PMC. This
allows the boot messages to include the CPU speed and makes possible
the forthcoming cpufreq support for the PPC 970.

Initial version of the sec(4) driver for the integrated security engine found
in Freescale system-on-chip devices.

The following algorithms and schemes are currently supported:
- 3DES, AES, DES
- MD5, SHA1, SHA256, SHA384, SHA512

Reviewed by: philip
Obtained from: Freescale, Semihalf

Provide 64-bit big endian bus space operations for PowerPC. They are required
for the upcoming sec(4) driver.

Submitted by: Piotr Ziecik
Obtained from: Semihalf

Initial support for SMP on PowerPC MPC85xx.

Tested with Freescale dual-core MPC8572DS development system.

Obtained from: Freescale, Semihalf

Improve style(9)

PowerPC common SMP startup and time base rework.

- make mftb() shared, rewrite in C, provide complementary mttb()
- adjust SMP startup per the above, additional comments, minor naming
changes
- eliminate redundant TB defines, other minor cosmetics

Reviewed by: marcel, nwhitehorn
Obtained from: Freescale, Semihalf

Factor out platform dependent things unrelated to device drivers into a
new platform module. These are probed in early boot, and have the
responsibility of determining the layout of physical memory, determining
the CPU timebase frequency, and handling the zoo of SMP mechanisms
found on PowerPC.

Reviewed by: marcel, raj
Book-E parts by: raj

Add suppport for ISA and ISA interrupts to make the ATA
controller in the VIA southbridge functional in the CDS
(Configurable Development System) for MPC85XX.
The embedded USB controllers look operational but the
interrupt steering is still wrong.

Remove PTE_FAKE and PTE_ISFAKE().

Centralize setting HID0/1 for E500. Rename HID defines which are specific
to E500 rather than shared within Book-E family.

Obtained from: Freescale, Semihalf

Lower VM_MAX_KERNEL_ADDRESS to 0xf8000000. We actually have
devices below CCSRBAR_VA, which overlap with KVA if that's
out limit.

Don't conditionally define CACHE_LINE_SHIFT, as we anticipate sizing
a fair number of static data structures, making this an unlikely
option to try to change without also changing source code. [1]

Change default cache line size on ia64, sparc64, and sun4v to 128
bytes, as this was what rtld-elf was already using on those
platforms. [2]

Suggested by: bde [1], jhb [2]
MFC after: 2 weeks

Add description and cautionary note regarding CACHE_LINE_SIZE.

MFC after: 2 weeks
Suggested by: alc

For each architecture, define CACHE_LINE_SHIFT and a derived
CACHE_LINE_SIZE constant. These constants are intended to
over-estimate the cache line size, and be used at compile-time
when a run-time tuning alternative isn't appropriate or
available.

Defaults for all architectures are 64 bytes, except powerpc
where it is 128 bytes (used on G5 systems).

MFC after: 2 weeks
Discussed on: arch@

Fix a typo in the SRR1 comparison for program exceptions. While here,
replace magic numbers with constants to keep this from happening again.

Without this fix, some programs would occasionally get SIGTRAP instead
of SIGILL on an illegal instruction. This affected Altivec detection
in pixman, and possibly other software.

Reported by: Andreas Tobler
MFC after: 1 week

Rework the way we get the cacheline size. Instead of having a table of
CPUs known to use 128 byte cache lines and defaulting to 32, use the dcbz
instruction to measure it. Also make dcbz behave the way you would
expect on PPC 970.

Add sysarch.h. It's included by drm(4).

First round of cleanups. There's a lot of NetBSDism in this header.

PowerPC, meet kernel core dumps. The support is based
on a generic dumper that creates an ELF core file and
uses PMAP functions to scan and iterate over memory
chunks, as well as handle memory mappings used during
dumping.
the PMAP layer can choose to return physical memory
chunks or virtual memory chunks. For minidumps, the
chunks should be virtual.

The default MMU I/F implementation for the scan_md()
method returns NULL. Thus, when a PMAP implementation
does not implement the required methods, an empty
core file is created. Here, empty means having an ELF
header only.

Obtained from: Juniper Networks

Add support for 64-bit PowerPC CPUs operating in the 64-bit bridge mode
provided, for example, on the PowerPC 970 (G5), as well as on related CPUs
like the POWER3 and POWER4.

This also adds support for various built-in hardware found on Apple G5
hardware (e.g. the IBM CPC925 northbridge).

Reviewed by: grehan

Add AT_EXECPATH ELF auxinfo entry type. The value's a_ptr is a pointer
to the full path of the image that is being executed.
Increase AT_COUNT.

Remove no longer true comment about types used in Linux ELF binaries,
listed types contain FreeBSD-specific entries.

Reviewed by: kan

Make MPC85xx LAW handling and reset routines aware of the MPC8548 variant.

Inspired by discussion with Alexey V Fedorov on freebsd-powerpc@.

Make Book-E debug register state part of the PCB context.

Previously, DBCR0 flags were set "globally", but this leads to problems
because Book-E fine grained debug settings work only in conjuction with the
debug master enable bit in MSR: in scenarios when the DBCR0 was set with
intention to debug one process, but another one with MSR[DE] set got
scheduled, the latter would immediately cause debug exceptions to occur upon
execution of its own code instructions (and not the one intended for
debugging).

To avoid such problems and properly handle debugging context, DBCR0 state
should be managed individually per process.

Submitted by: Grzegorz Bernacki gjb ! semihalf dot com
Reviewed by: marcel

Add Altivec support for supported CPUs. This is derived from the FPU support
code, and also reducing the size of trapcode to fit inside a 32 byte handler
slot.

Reviewed by: grehan
MFC after: 2 weeks

Rework BookE pmap towards multi-core support.

o Eliminate tlb0[] (a s/w copy of TLB0)
- The table contents cannot be maintained reliably in multiple MMU
environments, where asynchronous events (invalidations from other cores)
can change our local TLB0 contents underneath.
- Simplify and optimize TLB flushing: system wide invalidations are
performed using tlbivax instruction (propagates to other cores), for
local MMU invalidations a new optimized routine (assembly) is introduced.

o Improve and simplify TID allocation and management.
- Let each core keep track of its TID allocations.
- Simplify TID recycling, eliminate dead code.
- Drop the now unused powerpc/booke/support.S file.

o Improve page tables management logic.

o Simplify TLB1 manipulation routines.

o Other improvements and polishing.

Obtained from: Freescale, Semihalf

Modularize the Open Firmware client interface to allow run-time switching
of OFW access semantics, in order to allow future support for real-mode
OF access and flattened device frees. OF client interface modules are
implemented using KOBJ, in a similar way to the PPC PMAP modules.

Because we need Open Firmware to be available before mutexes can be used on
sparc64, changes are also included to allow KOBJ to be used very early in
the boot process by only using the mutex once we know it has been initialized.

Reviewed by: marius, grehan

Improve MPC85XX helper routines.

- Move CCSR accessors to the shared MPC85XX area
- Simplify SVR version subfield handling
- Adjust OCP

AT_DEBUG and AT_BRK were OBE like 10 years ago, so retire them.

Reviewed by: peter

Adapt parts of the sparc64 Open Firmware bus enumeration code (in particular,
the code for parsing interrupt maps) to PowerPC and reflect their new MI
status by moving them to the shared dev/ofw directory.

This commit also modifies the OFW PCI enumeration procedure on PowerPC to
allow the bus to find non-firmware-enumerated devices that Apple likes to add,
and adds some useful Open Firmware properties (compat and name) to the pnpinfo
string of children on OFW SBus, EBus, PCI, and MacIO links. Because of the
change to PCI enumeration on PowerPC, X has started working again on PPC
machines with Grackle hostbridges.

Reviewed by: marius
Obtained from: sparc64

Unbreak previous commit.

- bump __FreeBSD version to reflect added buf_ring, memory barriers,
and ifnet functions

- add memory barriers to <machine/atomic.h>
- update drivers to only conditionally define their own

- add lockless producer / consumer ring buffer
- remove ring buffer implementation from cxgb and update its callers

- add if_transmit(struct ifnet *ifp, struct mbuf *m) to ifnet to
allow drivers to efficiently manage multiple hardware queues
(i.e. not serialize all packets through one ifq)
- expose if_qflush to allow drivers to flush any driver managed queues

This work was supported by Bitgravity Inc. and Chelsio Inc.

Define LDBL_EPSILON, LDBL_MAX and LDBL_MIN as long double constants.

Submitted by: Andreas Tobler <andreast-list@fgznet.ch>
Reviewed by: das@

Fix compilation in the case when kernel doesn't have KDB ebabled.
subr_kdb.c still references breakpoint() in this case.

Add support for little-endian compilations to this file.

Add support for kernel profiling for both AIM and BookE.

Obtained from: Juniper Networks, Inc (BookE support).

Declare btext and etext. Needed by sys/kern/subr_prof.c for
for kernel profiling.

Convert PowerPC AIM PCI and nexus busses to standard OFW bus interface. This
simplifies certain device attachments (Kauai ATA, for instance), and makes
possible others on new hardware.

On G5 systems, there are several otherwise standard PCI devices
(Serverworks SATA) that will not allow their interrupt properties to be
written, so this information must be supplied directly from Open Firmware.

Obtained from: sparc64

Remove ipi_all() and ipi_self() as the former hasn't been used at
all to date and the latter also is only used in ia64 and powerpc
code which no longer serves a real purpose after bring-up and just
can be removed as well. Note that architectures like sun4u also
provide no means of implementing IPI'ing a CPU itself natively
in the first place.

Suggested by: jhb
Reviewed by: arch, grehan, jhb

Expand the DBDMA API to allow setting device-dependent control bits. While
here, clean up and document this a little.

Submitted by: Marco Trillo
MFC after: 1 week

Allow the cacheline size on PowerPC to be set at runtime. This is essential for
supporting 64-bit CPUs, which often have 128-byte cache lines instead of the
standard 32.

In preparation for PowerPC G5 support, allow PVO objects to contain page
table entries for both the 32-bit and 64-bit AIM MMUs.

Change the DBDMA API to allow DBDMA registers in a subregion of a resource. This is necessary to allow future support of DMA for the various Apple on-board ATA controllers.

MFC after: 1 week

In powerpc_get_pcpup(), make the inline assembly statement
volatile so that the compiler won't perform CSE. For SMP,
this may result in us accessing the wrong PCPU and as such
results in a bogus curthread value.

Note that getting curthread is not quite MP-safe in the sense
that it requires two instructions that aren't performed
atomically. The first instruction gets the address of the PCPU
structure and the second instruction dereferences that pointer
to get curthread. If a thread is switched-out in between these
instructions and switched-in on a different CPU, we still get
the wrong curthread.

Remove the tracing from the AP startup. The AP is known
to start and the tracing can interfere with AP startup.
Instead, use the available space in the reset vector
for the initial stack.

o Remove SPR_TSR & SPR_TCR for AIM.
o Remove SPR_HID2.
o Add more SPR_L3CR bit definitions.

Remove restore_intr(). We have intr_restore()...

Add powerpc_sync() as an inline function.

Export 'struct pcpu' to userland w/o requiring _KERNEL. A few ports
already define _KERNEL to get to this and I'm about to add hooks to
libkvm to access per-CPU data.

MFC after: 1 week

Remove the unused M_MEMDEV from the kernel.

The M_MEMDEV memory allocation pool does not seem to be used. We can
live without it.

Approved by: philip (mentor)

Remove the unused major/minor numbers from iodev and memdev.

Now that st_rdev is being automatically generated by the kernel, there
is no need to define static major/minor numbers for the iodev and
memdev. We still need the minor numbers for the memdev, however, to
distinguish between /dev/mem and /dev/kmem.

Approved by: philip (mentor)

Add support for Apple's Descriptor-Based DMA (DBDMA) engine. The DMA
engine is usful to various existing drivers, such as ata(4) and scc(4),
and is used bhy the soon to be added bm(4).

Submitted by: Nathan Whitehorn

MFp4: SMP support

MFp4: SMP support

Remove mfsvr():
o The function is defined unconditionally but depends on SPR_SVR,
which is defined conditionally.
o spr.h defines mfspr() and mtspr(), which is no worse to use.

Improve handling of Local Access Windows on MPC85xx systems:

- detect number of LAWs in run time and initalize accordingly
- introduce decode windows target IDs used in MPC8572
- other minor updates

Obtained from: Freescale, Semihalf

Move System Revision defines to a bit better place, add MPC8572 systems IDs.

Fix copy-n-paste typos in free text.

Reimplement atomic_add, atomic_clear, atomic_set and atomic_subtract
so that all implemented variants have proper prototypes. The 8-bit,
16-bit and 64-bit variants are not implemented.

This really fixes the current build breakages caused by type casting
and struct aliasing rules.

Quick fix for the kernel build breakage in netgraph and the
aliasing warning in libthr. A more elaborate fix is in the
works that makes sure that all variants have proper inline
functions with proper types.

Align functions to 16-byte boundaries due to profiling granularity.

When building a kernel module, define MAXCPU the same as SMP so
that modules work with and without SMP.

The "free-lance" timer in the i8254 is only used for the speaker
these days, so de-generalize the acquire_timer/release_timer api
to just deal with speakers.

The new (optional) MD functions are:
timer_spkr_acquire()
timer_spkr_release()
and
timer_spkr_setfreq()

the last of which configures the timer to generate a tone of a given
frequency, in Hz instead of 1/1193182th of seconds.

Drop entirely timer2 on pc98, it is not used anywhere at all.

Move sysbeep() to kern/tty_cons.c and use the timer_spkr*() if
they exist, and do nothing otherwise.

Remove prototypes and empty acquire-/release-timer() and sysbeep()
functions from the non-beeping archs.

This eliminate the need for the speaker driver to know about
i8254frequency at all. In theory this makes the speaker driver MI,
contingent on the timer_spkr_*() functions existing but the driver
does not know this yet and still attaches to the ISA bus.

Syscons is more tricky, in one function, sc_tone(), it knows the hz
and things are just fine.

In the other function, sc_bell() it seems to get the period from
the KDMKTONE ioctl in terms if 1/1193182th second, so we hardcode
the 1193182 and leave it at that. It's probably not important.

Change a few other sysbeep() uses which obviously knew that the
argument was in terms of i8254 frequency, and leave alone those
that look like people thought sysbeep() took frequency in hertz.

This eliminates the knowledge of i8254_freq from all but the actual
clock.c code and the prof_machdep.c on amd64 and i386, where I think
it would be smart to ask for help from the timecounters anyway [TBD].

Make remote GDB work for AIM processors. For BookE, the kernel
will have a special section, named .PPC.EMB.apuinfo, which will
tell GDB that a BookE processor is targeted and which will
result in GDB using a different register definition. In order
to support remote GDB for BookE, we need the GDB stub in the
kernel look for that section and use the BookE definitions.

Implement atomic_fetchadd_long() for all architectures and document it.

Reviewed by: attilio, jhb, jeff, kris (as a part of the uidinfo_waitfree.patch)

Obtain TSEC h/w address from the parent bus (OCP) and not rely blindly on what
might be currently programmed into the registers.

Underlying firmware (U-Boot) would typically program MAC address into the
first unit only, and others are left uninitialized. It is now possible to
retrieve and program MAC address for all units properly, provided they were
passed on in the bootinfo metadata.

Reviewed by: imp, marcel
Approved by: cognet (mentor)

For AIM, have cpu_idle() set MSR_POW when the powerpc_pow_enabled
variable is set. On my Mac Mini this puts the CPU in NAP mode when
the kernel is idle and, any technical or environmental reasons
aside, avoids that I have to listen to the fan all day :-)

Add support for the BUS_CONFIG_INTR() method to the platform and to
openpic(4). Make use of it in ocpbus(4). On the MPC85xxCDS, IRQ0:4
are active-low.

Add a catch-all for PCPU_MD_FIELDS. While we expect this to be
used in the kernel only (by virtue of checking for _KERNEL),
ports like lsof (part of gtop) cheat. It sets _KERNEL, but does
not set either AIM or E500. As such, PCPU_MD_FIELDS didn't get
defined and the build broke.
The catch-all is to define PCPU_MD_FIELDS with a dummy integer
when at the end of line we ended up without a definition for it.

Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family.

The PQ3 is a high performance integrated communications processing system
based on the e500 core, which is an embedded RISC processor that implements
the 32-bit Book E definition of the PowerPC architecture. For details refer
to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E

This port was tested and successfully run on the following members of the PQ3
family: MPC8533, MPC8541, MPC8548, MPC8555.

The following major integrated peripherals are supported:

* On-chip peripherals bus
* OpenPIC interrupt controller
* UART
* Ethernet (TSEC)
* Host/PCI bridge
* QUICC engine (SCC functionality)

This commit brings the main functionality and will be followed by individual
drivers that are logically separate from this base.

Approved by: cognet (mentor)
Obtained from: Juniper, Semihalf
MFp4: e500

Rework and extend PowerPC headers definitons towards Book-E/e500 CPUs support.

Approved by: cognet (mentor)
Obtained from: Juniper, Semihalf
MFp4: e500

Unify and generalize PowerPC headers, adjust AIM code accordingly.

Rework of this area is a pre-requirement for importing e500 support (and
other PowerPC core variations in the future). Mainly the following
headers are refactored so that we can cover for low-level differences between
various machines within PowerPC architecture:

<machine/pcpu.h>
<machine/pcb.h>
<machine/kdb.h>
<machine/hid.h>
<machine/frame.h>

Areas which use the above are adjusted and cleaned up.

Credits for this rework go to marcel@

Approved by: cognet (mentor)
MFp4: e500

Teach PowerPC CPU identification routines to recognize e500 cores. Fix style
issues in this area.

Approved by: cognet (mentor)
MFp4: e500

Let PowerPC world optionally build with -msoft-float. For FPU-less PowerPC
variations (e500 currently), this provides a gcc-level FPU emulation and is an
alternative approach to the recently introduced kernel-level emulation
(FPU_EMU).

Approved by: cognet (mentor)
MFp4: e500

Define the bootinfo structure for FreeBSD. It is not used on
AIM, but it's used for BookE.

Add PIC support for IPIs. When registering an interrupt handler,
the PIC also informs the platform at which IRQ level it can start
assigning IPIs, since this can depend on the number of IRQs
supported for external interrupts.

Add configuration knobs for the superpage reservation system. Initially,
the reservation will only be enabled on amd64.

Redefine bus_space_tag_t on PowerPC from a 32-bit integral to
a pointer to struct bus_space. The structure contains function
pointers that do the actual bus space access.

The reason for this change is that previously all bus space
accesses were little endian (i.e. had an explicit byte-swap
for multi-byte accesses), because all busses on Macs are little
endian.
The upcoming support for Book E, and in particular the E500
core, requires support for big-endian busses because all
embedded peripherals are in the native byte-order.

With this change, there's no distinction between I/O port
space and memory mapped I/O. PowerPC doesn't have I/O port
space. Busses assign tags based on the byte-order only.
For that purpose, two global structures exist (bs_be_tag and
bs_le_tag), of which the address can be taken to get a valid
tag.

Obtained from: Juniper, Semihalf

Rename OEA to AIM. The former means nothing as it applies to all
processors (it's the PowerPC Operating Environment Architecture).
AIM designates the processors made by the Apple-IBM-Motorola
alliance and those we typically support.

While here, remove the NetBSD option IPKDB. It's not an option
used by us. Also, PPC_HAVE_FPU is not used by us either. Remove
that too.

Obtained from: Juniper, Semihalf

Add stubs to unbreak LINT.

Break out stack(9) from ddb(4):

- Introduce per-architecture stack_machdep.c to hold stack_save(9).
- Introduce per-architecture machine/stack.h to capture any common
definitions required between db_trace.c and stack_machdep.c.
- Add new kernel option "options STACK"; we will build in stack(9) if it is
defined, or also if "options DDB" is defined to provide compatibility
with existing users of stack(9).

Add new stack_save_td(9) function, which allows the capture of a stacktrace
of another thread rather than the current thread, which the existing
stack_save(9) was limited to. It requires that the thread be neither
swapped out nor running, which is the responsibility of the consumer to
enforce.

Update stack(9) man page.

Build tested: amd64, arm, i386, ia64, powerpc, sparc64, sun4v
Runtime tested: amd64 (rwatson), arm (cognet), i386 (rwatson)

Define atomic_readandclear_ptr.

Implement the _long functions using u_long rather than trying to
cast as uint32_t which is defined as unsigned int. gcc doesn't want to
consider that there might not be much difference between an int and
a long on a 32 bit architecture.

Define atomic_cmpset_acq_long and atomic_cmpset_rel_long so that
they use casts rather than just assuming that the compiler will DTRT
without complaining.

Split decr_init() into two, with the section that reads the timebase
frequency from OpenFirmware moved out and into a routine that is called
from cpu_startup().

This allows correct reporting of the CPU clockspeed when printing out
CPU information at boot time.

Reported by: numerous
Reviewed by: marcel
MFC after: 1 day

Change the management of cached pages (PQ_CACHE) in two fundamental
ways:

(1) Cached pages are no longer kept in the object's resident page
splay tree and memq. Instead, they are kept in a separate per-object
splay tree of cached pages. However, access to this new per-object
splay tree is synchronized by the _free_ page queues lock, not to be
confused with the heavily contended page queues lock. Consequently, a
cached page can be reclaimed by vm_page_alloc(9) without acquiring the
object's lock or the page queues lock.

This solves a problem independently reported by tegge@ and Isilon.
Specifically, they observed the page daemon consuming a great deal of
CPU time because of pages bouncing back and forth between the cache
queue (PQ_CACHE) and the inactive queue (PQ_INACTIVE). The source of
this problem turned out to be a deadlock avoidance strategy employed
when selecting a cached page to reclaim in vm_page_select_cache().
However, the root cause was really that reclaiming a cached page
required the acquisition of an object lock while the page queues lock
was already held. Thus, this change addresses the problem at its
root, by eliminating the need to acquire the object's lock.

Moreover, keeping cached pages in the object's primary splay tree and
memq was, in effect, optimizing for the uncommon case. Cached pages
are reclaimed far, far more often than they are reactivated. Instead,
this change makes reclamation cheaper, especially in terms of
synchronization overhead, and reactivation more expensive, because
reactivated pages will have to be reentered into the object's primary
splay tree and memq.

(2) Cached pages are now stored alongside free pages in the physical
memory allocator's buddy queues, increasing the likelihood that large
allocations of contiguous physical memory (i.e., superpages) will
succeed.

Finally, as a result of this change long-standing restrictions on when
and where a cached page can be reclaimed and returned by
vm_page_alloc(9) are eliminated. Specifically, calls to
vm_page_alloc(9) specifying VM_ALLOC_INTERRUPT can now reclaim and
return a formerly cached page. Consequently, a call to malloc(9)
specifying M_NOWAIT is less likely to fail.

Discussed with: many over the course of the summer, including jeff@,
Justin Husted @ Isilon, peter@, tegge@
Tested by: an earlier version by kris@
Approved by: re (kensmith)

Revamp the interrupt handling in support of INTR_FILTER. This includes:
o Revamp the PIC I/F to only abstract the PIC hardware. The
resource handling has been moved to nexus, where it belongs.
o Include EOI and MASK+EOI methods to the PIC I/F in support of
INTR_FILTER.
o With the allocation of interrupt resources and setup of
interrupt handlers in the common platform code we can delay
talking to the PIC hardware after enumeration of all devices.
Introduce a call to powerpc_intr_enable() in configure_final()
to achieve that and have powerpc_setup_intr() only program the
PIC when !cold.
o As a consequence of the above, remove all early_attach() glue
from the OpenPIC and Heathrow PIC drivers and have them
register themselves when they're found during enumeration.
o Decouple the interrupt vector from the interrupt request line.
Allocate vectors increasingly so that they can be used for
the intrcnt index as well. Extend the Heathrow PIC driver to
translate between IRQ and vector. The OpenPIC driver already
has the support for vectors in hardware.

Approved by: re (blanket)

Eliminate <machine/interruptvar.h> as it has only a single
prototype. In the future that prototype will not be needed
at all anyway, but for now it's moved to intr_machdep.h.

Approved by: re (blanket)

Add prototype for trap().

Approved by: re (blanket)

Cast the arguments to atomic_*_ptr() when mapping it to atomic_*_32()
This is a minimal fix.

Approved by: re (kensmith)

Enable the new physical memory allocator.

This allocator uses a binary buddy system with a twist. First and
foremost, this allocator is required to support the implementation of
superpages. As a side effect, it enables a more robust implementation
of contigmalloc(9). Moreover, this reimplementation of
contigmalloc(9) eliminates the acquisition of Giant by
contigmalloc(..., M_NOWAIT, ...).

The twist is that this allocator tries to reduce the number of TLB
misses incurred by accesses through a direct map to small, UMA-managed
objects and page table pages. Roughly speaking, the physical pages
that are allocated for such purposes are clustered together in the
physical address space. The performance benefits vary. In the most
extreme case, a uniprocessor kernel running on an Opteron, I measured
an 18% reduction in system time during a buildworld.

This allocator does not implement page coloring. The reason is that
superpages have much the same effect. The contiguous physical memory
allocation necessary for a superpage is inherently colored.

Finally, the one caveat is that this allocator does not effectively
support prezeroed pages. I hope this is temporary. On i386, this is
a slight pessimization. However, on amd64, the beneficial effects of
the direct-map optimization outweigh the ill effects. I speculate
that this is true in general of machines with a direct map.

Approved by: re

Add kdb_cpu_sync_icache(), intended to synchronize instruction
caches with data caches after writing to memory. This typically
is required to make breakpoints work on ia64 and powerpc. For
those architectures the function is implemented.

Rework the PCPU_* (MD) interface:
- Rename PCPU_LAZY_INC into PCPU_INC
- Add the PCPU_ADD interface which just does an add on the pcpu member
given a specific value.

Note that for most architectures PCPU_INC and PCPU_ADD are not safe.
This is a point that needs some discussions/work in the next days.

Reviewed by: alc, bde
Approved by: jeff (mentor)

Eliminate some unused definitions that came from NetBSD.

Eliminate an unused definition.

Define every architecture as either VM_PHYSSEG_DENSE or
VM_PHYSSEG_SPARSE depending on whether the physical address space is
densely or sparsely populated with memory. The effect of this
definition is to determine which of two implementations of
vm_page_array and PHYS_TO_VM_PAGE() is used. The legacy
implementation is obtained by defining VM_PHYSSEG_DENSE, and a new
implementation that trades off time for space is obtained by defining
VM_PHYSSEG_SPARSE. For now, all architectures except for ia64 and
sparc64 define VM_PHYSSEG_DENSE. Defining VM_PHYSSEG_SPARSE on ia64
allows the entirety of my Itanium 2's memory to be used. Previously,
only the first 1 GB could be used. Defining VM_PHYSSEG_SPARSE on
sparc64 allows USIIIi-based systems to boot without crashing.

This change is a combination of Nathan Whitehorn's patch and my own
work in perforce.

Discussed with: kmacy, marius, Nathan Whitehorn
PR: 112194

Push down the implementation of PCPU_LAZY_INC() into the machine-dependent
header file. Reimplement PCPU_LAZY_INC() on amd64 and i386 making it
atomic with respect to interrupts.

Reviewed by: bde, jhb

o break newbus api: add a new argument of type driver_filter_t to
bus_setup_intr()

o add an int return code to all fast handlers

o retire INTR_FAST/IH_FAST

For more info: http://docs.freebsd.org/cgi/getmsg.cgi?fetch=465712+0+current/freebsd-current

Reviewed by: many
Approved by: re@

Remove stale header.

MFC after: 3 days

Remove 3rd clause, renumber, ok per email

Implement bus_space_map().

Fix gdb issue where the i-cache was not being updated when a breakpoint
was written into a user's address space. The fix is to modify uiomove_fromphys
to sync the icache when an executable user-space page is written into.

Alan Cox suggested that there should probably be a higher-level interface
to this in the ptrace code, but agreed that this is an OK short-term solution.

Files changed:

pmap.h - declaration of pmap_page_executable()
pmap_dispatch.c - pass through the page_executable call to the mmu object
mmu_oea.c - implement the page_executable method by examining the PTE_EXEC
field in the vm_page_t
uio_machdep.c - in uiomove_fromphys(), if the op was a UIO_WRITE to user-space,
and if the page is executable, sync the icache since this is at the least
a breakpoint-write from gdb.

Reported by: marcel
Tested by: marcel, grehan on g3+g4
Discussed with: alc
MFC after: 2 weeks

PR:
Submitted by:
Reviewed by:
Approved by:
Obtained from:
MFC after:
Security:
Move the relocation definitions to the common elf header so that DTrace
can use them on one architecture targeted to a different one.

Add the additional ELF types defines in Sun's "Linker and Libraries"
manual.

First part of a little cleanup in the calendar/timezone/RTC handling.

Move relevant variables to <sys/clock.h> and fix #includes as necessary.

Use libkern's much more time- & spamce-efficient BCD routines.

Use __builtin_va_start instead of __builtin_stdarg_start. GCC4 obsoletes
the former and __builtin_va_start was present in all GCC version 3.1 and
later.

Eliminate unused definitions. (They came from NetBSD.)

Discussed with: cognet, grehan, marcel

Add skeletal support for GDB. In particular gdb_cpu_getreg() needs
implementing to make GDB support usable.

o Move the prototype of mem_valid() from ofw_machdep.h to md_var.h.
This avoids that mem.c has to include ofw_machdep.h, including
all OFW related headers.
o Provide a stub for OF_decode_addr(), which is used by low-level
console drivers to obtain a tag and handle given a OFW phandle.
This is different from sparc64, where a fake bus tag needs to be
created explicitly.

Forward declare struct trapframe.

Clean out sysctl machdep.* related defines.

The cmos clock related stuff should really be in MI code.

Add a dummy implementation of bus_space_map().

Add user-space profiling support. Kernel profiling still todo.

Obtained from: NetBSD

Forward-declare struct trapframe to allow the aic module to compile.

Tweak how the MD code calls the fooclock() methods some. Instead of
passing a pointer to an opaque clockframe structure and requiring the
MD code to supply CLKF_FOO() macros to extract needed values out of the
opaque structure, just pass the needed values directly. In practice this
means passing the pair (usermode, pc) to hardclock() and profclock() and
passing the boolean (usermode) to hardclock_cpu() and hardclock_process().
Other details:
- Axe clockframe and CLKF_FOO() macros on all architectures. Basically,
all the archs were taking a trapframe and converting it into a clockframe
one way or another. Now they can just extract the PC and usermode values
directly out of the trapframe and pass it to fooclock().
- Renamed hardclock_process() to hardclock_cpu() as the latter is more
accurate.
- On Alpha, we now run profclock() at hz (profhz == hz) rather than at
the slower stathz.
- On Alpha, for the TurboLaser machines that don't have an 8254
timecounter, call hardclock() directly. This removes an extra
conditional check from every clock interrupt on Alpha on the BSP.
There is probably room for even further pruning here by changing Alpha
to use the simplified timecounter we use on x86 with the lapic timer
since we don't get interrupts from the 8254 on Alpha anyway.
- On x86, clkintr() shouldn't ever be called now unless using_lapic_timer
is false, so add a KASSERT() to that affect and remove a condition
to slightly optimize the non-lapic case.
- Change prototypeof arm_handler_execute() so that it's first arg is a
trapframe pointer rather than a void pointer for clarity.
- Use KCOUNT macro in profclock() to lookup the kernel profiling bucket.

Tested on: alpha, amd64, arm, i386, ia64, sparc64
Reviewed by: bde (mostly)

GC some unused frame types.

Approved by: grehan

- Cleanup whitespace and extra ()s in vtophys() macros.
- Move vtophys() macros next to vtopte() where vtopte() exists to match
comments above vtopte().
- Remove references to the alternate address space in the comment above
vtopte(). amd64 never had the alternate address space, and i386 lost it
prior to PAE support being added.
- s/entires/entries/ in comments.

Reviewed by: alc

Drop _MACHINE_ARCH and _MACHINE defines (not to be confused with
MACHINE_ARCH and MACHINE). Their purpose was to be able to test
in cpp(1), but cpp(1) only understands integer type expressions.
Using such unsupported expressions introduced a number of subtle
bugs, which were discovered by compiling with -Wundef.

Add definitions for 64-bit PTEs

Insert a layer of indirection to the pmap code, using a kobj for
the interface. This allows run-time selection of MMU code, based
on CPU-type detection, or tunable-overrides when testing new code.

Pre-requisite for G5 support.

conf/files.powerpc
- remove pmap.c
- add mmu_if.h, mmu_oea.c, pmap_dispatch.c

powerpc/include/mmuvar.h
- definitions for MMU implementations

powerpc/include/pmap.h
- remove pmap_pte_spill declaration
- add pmap_mmu_install declaration
- size the phys_avail array
- pmap_bootstrapped is now global-scope

powerpc/powerpc/machdep.c
- call kobj_machdep_init early in the boot sequence to allow
kobj usage prior to SI_SUB_LOCK
- install the OEA pmap code. This will be moved to CPU-specific
init code in the future.

powerpc/powerpc/mmu_if.m
- Kobj MMU interface definitions

powerpc/powerpc/pmap_dispatch.c
- central dispatch for pmap calls
- contains the global mmu kobj and the routine to locate the
the mmu implementation and init the kobj

Reorganize the interrupt handling code a bit to make a few things cleaner
and increase flexibility to allow various different approaches to be tried
in the future.
- Split struct ithd up into two pieces. struct intr_event holds the list
of interrupt handlers associated with interrupt sources.
struct intr_thread contains the data relative to an interrupt thread.
Currently we still provide a 1:1 relationship of events to threads
with the exception that events only have an associated thread if there
is at least one threaded interrupt handler attached to the event. This
means that on x86 we no longer have 4 bazillion interrupt threads with
no handlers. It also means that interrupt events with only INTR_FAST
handlers no longer have an associated thread either.
- Renamed struct intrhand to struct intr_handler to follow the struct
intr_foo naming convention. This did require renaming the powerpc
MD struct intr_handler to struct ppc_intr_handler.
- INTR_FAST no longer implies INTR_EXCL on all architectures except for
powerpc. This means that multiple INTR_FAST handlers can attach to the
same interrupt and that INTR_FAST and non-INTR_FAST handlers can attach
to the same interrupt. Sharing INTR_FAST handlers may not always be
desirable, but having sio(4) and uhci(4) fight over an IRQ isn't fun
either. Drivers can always still use INTR_EXCL to ask for an interrupt
exclusively. The way this sharing works is that when an interrupt
comes in, all the INTR_FAST handlers are executed first, and if any
threaded handlers exist, the interrupt thread is scheduled afterwards.
This type of layout also makes it possible to investigate using interrupt
filters ala OS X where the filter determines whether or not its companion
threaded handler should run.
- Aside from the INTR_FAST changes above, the impact on MD interrupt code
is mostly just 's/ithread/intr_event/'.
- A new MI ddb command 'show intrs' walks the list of interrupt events
dumping their state. It also has a '/v' verbose switch which dumps
info about all of the handlers attached to each event.
- We currently don't destroy an interrupt thread when the last threaded
handler is removed because it would suck for things like ppbus(8)'s
braindead behavior. The code is present, though, it is just under
#if 0 for now.
- Move the code to actually execute the threaded handlers for an interrrupt
event into a separate function so that ithread_loop() becomes more
readable. Previously this code was all in the middle of ithread_loop()
and indented halfway across the screen.
- Made struct intr_thread private to kern_intr.c and replaced td_ithd
with a thread private flag TDP_ITHREAD.
- In statclock, check curthread against idlethread directly rather than
curthread's proc against idlethread's proc. (Not really related to intr
changes)

Tested on: alpha, amd64, i386, sparc64
Tested on: arm, ia64 (older version of patch by cognet and marcel)

Add a new atomic_fetchadd() primitive that atomically adds a value to a
variable and returns the previous value of the variable.

Tested on: i386, alpha, sparc64, arm (cognet)
Reviewed by: arch@
Submitted by: cognet (arm)
MFC after: 1 week

Stop using the '+' constraint modifier with inline assembly. The '+'
constraint is actually only allowed for register operands. Instead, use
separate input and output memory constraints.

Education from: alc
Reviewed by: alc
Tested on: i386, alpha
MFC after: 1 week

Move MINSIGSTKSZ from <machine/signal.h> to <machine/_limits.h> and rename
it to __MINSIGSTKSZ. Define MINSIGSTKSZ in <sys/signal.h>.

This is done in order to use MINSIGSTKSZ for the macro PTHREAD_STACK_MIN
in <pthread.h> (soon <limits.h>) without having to include the whole
<sys/signal.h> header.

Discussed with: bde

Convert the atomic_ptr() operations over to operating on uintptr_t
variables rather than void * variables. This makes it easier and simpler
to get asm constraints and volatile keywords correct.

MFC after: 3 days
Tested on: i386, alpha, sparc64
Compiled on: ia64, powerpc, amd64
Kernel toolchain busted on: arm

Unbreak the PowerPC GENERIC build.

Reviewed by: delphij

MFP4:

- Implement sampling modes and logging support in hwpmc(4).

- Separate MI and MD parts of hwpmc(4) and allow sharing of
PMC implementations across different architectures.
Add support for P4 (EMT64) style PMCs to the amd64 code.

- New pmcstat(8) options: -E (exit time counts) -W (counts
every context switch), -R (print log file).

- pmc(3) API changes, improve our ability to keep ABI compatibility
in the future. Add more 'alias' names for commonly used events.

- bug fixes & documentation.

Remove bus_{mem,p}io.h and related code for a micro-optimization on i386
and amd64. The optimization is a trivial on recent machines.

Reviewed by: -arch (imp, marcel, dfr)

Add empty header (except of the multiple-inclusion protection) to
get hwpmc(4) to compile on this platform.

Break out the definition of bus_space_{tag,handle}_t and a few other types
into _bus.h to help with name space polution from including all of bus.h.
In a few days, I'll commit changes to the MI code to take advantage of thse
sepration (after I've made sure that these changes don't break anything in
the main tree, I've tested in my trees, but you never know...).

Suggested by: bde (in 2002 or 2003 I think)
Reviewed in principle by: jhb

Divorce critical sections from spinlocks. Critical sections as denoted by
critical_enter() and critical_exit() are now solely a mechanism for
deferring kernel preemptions. They no longer have any affect on
interrupts. This means that standalone critical sections are now very
cheap as they are simply unlocked integer increments and decrements for the
common case.

Spin mutexes now use a separate KPI implemented in MD code: spinlock_enter()
and spinlock_exit(). This KPI is responsible for providing whatever MD
guarantees are needed to ensure that a thread holding a spin lock won't
be preempted by any other code that will try to lock the same lock. For
now all archs continue to block interrupts in a "spinlock section" as they
did formerly in all critical sections. Note that I've also taken this
opportunity to push a few things into MD code rather than MI. For example,
critical_fork_exit() no longer exists. Instead, MD code ensures that new
threads have the correct state when they are created. Also, we no longer
try to fixup the idlethreads for APs in MI code. Instead, each arch sets
the initial curthread and adjusts the state of the idle thread it borrows
in order to perform the initial context switch.

This change is largely a big NOP, but the cleaner separation it provides
will allow for more efficient alternative locking schemes in other parts
of the kernel (bare critical sections rather than per-CPU spin mutexes
for per-CPU data for example).

Reviewed by: grehan, cognet, arch@, others
Tested on: i386, alpha, sparc64, powerpc, arm, possibly more

Refactor the bus_dma header files so that the interface is described in
sys/bus_dma.h instead of being copied in every single arch. This slightly
reorders a flag that was specific to AXP and thus changes the ABI there.
The interface still relies on bus_space definitions found in <machine/bus.h>
so it cannot be included on its own yet, but that will be fixed at a later
date. Add an MD <machine/bus_dma.h> for ever arch for consistency and to
allow for future MD augmentation of the API. sparc64 makes heavy use of
this right now due to its different bus_dma implemenation.

netchild's mega-patch to isolate compiler dependencies into a central
place.

This moves the dependency on GCC's and other compiler's features into
the central sys/cdefs.h file, while the individual source files can
then refer to #ifdef __COMPILER_FEATURE_FOO where they by now used to
refer to #if __GNUC__ > 3.1415 && __BARC__ <= 42.

By now, GCC and ICC (the Intel compiler) have been actively tested on
IA32 platforms by netchild. Extension to other compilers is supposed
to be possible, of course.

Submitted by: netchild
Reviewed by: various developers on arch@, some time ago

Use a common multi-inclusion protection, and add such a
protection to alpha/include/exec.h.

Convert bus_space_barrier() into a null inline function rather than an
empty macro to avoid many compile warnings in the USB code.

- add definitions for MPC7447A/7448 (i.e. miniMac)
- expand MPC745X_P macro to include these

Obtained from: NetBSD

HID0 updates:
- updated relevant models for High BAT enable bit
- fixed bug in BHTCLR/XAEN constants
- added LRSTK and FOLD bits

- change all u_int_XX to uint_XX
- cast param for atomic_subtract_long, since Netgraph uses it.

Add bus_dmamap_load_mbuf_sg() to powerpc.

/* -> /*- for license, minor formatting changes

Create a new definition, PSL_KERNSET, which is used for setting the
MSR in kernel mode. Redefine PSL_USERSET in terms of this by or'ing
in PSL_PR.

Remove UAREA_PAGES.

Reviewed by: arch@

Implement TLS relocations for powerpc.

Approved by: grehan (mentor)

Move the kernel-specific logic to adjust frompc from MI to MD. For
these two reasons:
1. On ia64 a function pointer does not hold the address of the first
instruction of a functions implementation. It holds the address
of a function descriptor. Hence the user(), btrap(), eintr() and
bintr() prototypes are wrong for getting the actual code address.
2. The logic forces interrupt, trap and exception entry points to
be layed-out contiguously. This can not be achieved on ia64 and is
generally just bad programming.

The MCOUNT_FROMPC_USER macro is used to set the frompc argument to
some kernel address which represents any frompc that falls outside
the kernel text range. The macro can expand to ~0U to bail out in
that case.
The MCOUNT_FROMPC_INTR macro is used to set the frompc argument to
some kernel address to represent a call to a trap or interrupt
handler. This to avoid that the trap or interrupt handler appear to
be called from everywhere in the call graph. The macro can expand
to ~0U to prevent adjusting frompc. Note that the argument is selfpc,
not frompc.

This commit defines the macros on all architectures equivalently to
the original code in sys/libkern/mcount.c. People can take it from
here...

Compile-tested on: alpha, amd64, i386, ia64 and sparc64
Boot-tested on: i386

Add pmap locking to many of the functions.

Many thanks to Andrew Gallatin for resolving a powerpc-specific
initialization problem in my original patch.

Tested by: gallatin@

Add /dev/mem and /dev/kmem to powerpc.

Approved by: grehan (mentor)

Always isync after a mtmsr. While perhaps not strictly necessary for PSL_EE
bit banging according to the OEA, it's better to be conservative than
having to continually audit uses of this inline.

Instead of calling ia32_pause() conditionally on __i386__ or __amd64__
being defined, define and use a new MD macro, cpu_spinwait(). It only
expands to something on i386 and amd64, so the compiled code should be
identical.

Name of the macro found by: jhb
Reviewed by: jhb

Remove race condition between reading of MSR, setting md_savecrit,
and setting MSR. This was most evident with the idle proc running
with interrupts disabled and causing a lockup. Switch over to the
i386 style which does things in the right order.

debug assisted by: gallatin, and the invaluable KTR option.

Add comment explaining struct reg and struct fpreg must match the trapframe.

Approved by: grehan (mentor)

Pass a thread argument into cpu_critical_{enter,exit}() rather than
dereference curthread. It is called only from critical_{enter,exit}(),
which already dereferences curthread. This doesn't seem to affect SMP
performance in my benchmarks, but improves MySQL transaction throughput
by about 1% on UP on my Xeon.

Head nodding: jhb, bmilekic

Properly implement kdb_cpu_{set|clear}_singlestep to allow DDB to
continue from breakpoints.

Let ddb know powerpc is big endian so as to make ddb output
human readable.

Obtained from: sparc64/include/db_machdep.h

Update the callframe structure to leave space for the frame pointer
and saved link register as per the ABI call sequence. Update code
that uses this (fork_trampoline etc) to use the correct genassym'd
offsets.

This fixes the 'invalid LR' message when backtracing kernel
threads in DDB.

Fix printing of long doubles to match the size that
gcc is using. This fixes devstat consumers (like vmstat, iostat,
systat) so they don't print crazy zillion digit numbers for
disk transfers and bandwidth.

According to gcc, long doubles are 64-bits, rather than 128 bits
like the SVR4 ABI spec wants them to be.. Note that MacOSX also treats
long doubles as 64-bits, and not 128 bits, so we are in good company.

Reviewed by: das
Approved by: grehan

Make FLT_ROUNDS correctly reflect the dynamic rounding mode.

Use the version field to identify the partial context used by
KSE process-scope threads.

Gratuitous namechange to avoid low-level association with ddb.

Add prototype for KDB's makectx routine

Remove old NetBSD-derived unused code and stuff that is now obsolete
due to KDB.

DDB -> KDB, and rename low-level trap handler to avoid name conflict.

kdb.h for PowerPC. Stubs for now.

In the spirit of amd64/include/stdarg.h rev 1.6; add __va_copy
(but keep it conditional on __ISO_C_VISIBLE >= 1999.

Why? Our out /usr/src/contrib assumes it, and more than a few ports have
an autoconf that looks for __va_copy because it is available on glibc.
It is critical that we use it on PowerPC. It generally isn't a problem
for i386 and its ilk because those platforms can get away with cheating
the C standard, using a plain assignment.

Moved most of the "MI" definitions and declarations from <machine/profile.h>
to <sys/gmon.h>. Cleaned them up a little by not attempting to ifdef
for incomplete and out of date support for GUPROF in userland, as in
the sparc64 version.

<stdint.h> should define WINT_M{AX,IN} independent from whether WCHAR_MIN is
defined. Otherwise first including <wchar.h> and then <stdint.h> leads to no
WINT_M{AX,IN} at all.

PR: 64956
Approved by: das (mentor)

Hide FLT_EVAL_METHOD and DECIMAL_DIG in pre-C99 compilation
environments.

PR: 63935
Submitted by: Stefan Farfeleder <stefan@fafoe.narf.at>

Include <machine/pte.h> since it has been removed from <machine/param.h>.

<machine/pte.h> has no business being here. Finally exposed by F77 build
failure.

MFamd64
Simplify the sf_buf implementation. In short, make it a veneer
over the direct virtual-to-physical mapping.

Remove avail_end. It is not used.

Remove advertising clause from University of California Regent's
license, per letter dated July 22, 1999 and email from Peter Wemm,
Alan Cox and Robert Watson.

Approved by: core, peter, alc, rwatson

Remove avail_start on those platforms that no longer use it. (Only amd64
does anything with it beyond simple initialization.)

Introduce the cpumask_t type. The purpose of the type is to create a
level of abstraction for any and all CPU mask and CPU bitmap variables
so that platforms have the ability to break free from the hard limit
of 32 CPUs, simply because we don't have more bits in an u_int. Note
that the type is not supposed to solve massive parallelism, where
the number of CPUs can be larger than the width of the widest integral
type. As such, cpumask_t is not supposed to be a compound type. If
such would be necessary in the future, we can deal with the issues
then and there. For now, it can be assumed that the type is integral
and unsigned.

With this commit, all MD definitions start off as u_int. This allows
us to phase-in cpumask_t at our leasure without breaking anything.
Once cpumask_t is used consistently, platforms can switch to wider
(or smaller) types if such would be beneficial (or not; whatever :-)

Compile-tested on: i386

Fix syntax errors and wrong function prototypes in several MD header
files when using non-GNUC compilers.

PR: kern/58515
Submitted by: Stefan Farfeleder <stefan@fafoe.narf.at>
Approved by: grog (mentor), obrien

Increase kernel VA from 256Mb to 512Mb by shifting the segment used
for user copyinout down to 12, and keeping segments 13/14 for
kernel VA.

It would be nice to have more available, but segments lower than
this are reserved for either memory or 1:1 mapped device i/o,
and seg 15 is OpenFirmware ROM. Also, the effort to keep OpenFirmware
available for callbacks limits the use of VA-mapped segments.
Fortunately UMA_MD_SMALL_ALLOC takes away a lot of VM pressure.

Obtained from: NetBSD

Kernel changes for libthr (and probably libpthread).

include/ucontext.h
- remove trapframe and switch over to 'generic' description of machine
state. Include version field to help with future modifications.
Include floating point and altivec state, and hopefully align
correctly

powerpc/copyinout.c
- fill out casuptr() sync primitive, required by kern_umtx.c

powerpc/machdep.c
- shifted proc0/thread0/pcpu setup to before cninit, since
syscons -> make_dev -> devlock requires a valid curthread
- implemented get_mcontext/set_mcontext
- recast sendsig/sigreturn to use get/set_mcontext and new
ucontext struct. floating point now saved
- TODO: save/restore altivec state

powerpc/vm_machdep.c
- implemented cpu_thread_setup/cpu_set_upcall/cpu_set_upcall_kse
- eliminated trailing whitespace

Submitted by: Suleiman Souhlal <refugee@segfaulted.com>, ucontext by grehan

Add sys file required for IEEE fp functions.

Submitted by: Suleiman Souhlal <refugee@segfaulted.com>

Interrupt statistics, vmstat -i now works.

Submitted by: Suleiman Souhlal <refugee@segfaulted.com>
Slightly modified by: grehan
Derived from: i386

Cleaned up param.h:

- culled long-dead #define's
- segment register defs moved to sr.h
- NPMAPS moved to pmap.h
- KERNBASE moved to vmparam.h
- removed include of <machine/cpu.h> and fixed src files that
relied on this.

Modifying segment register code no longer causes gcc rebuilds :-)

Definitions for MPC7457 CPU type and HID0 bits

Move temporary register save area from exception-vector memory to
per-CPU memory. This allows for interrupt handling on multiple CPUs.

Obtained from: NetBSD

Implement UMA_MD_SMALL_ALLOC, since the BAT registers allow direct
addressing of memory. Makes a substantial improvement for apps that
stress the limited amount of KVM on PPC (e.g. untarring the ports tree).

uma_machdep.c stolen from amd64/ia64.

A syscons implementation using the 8-bit framebuffer set up by
OpenFirmware. Not at all optimized, but provides a PC-style
user-experience.

Tested on revA imac, B&W G3, 2k iBook, and G4 eMac.

Update 128-bit long double constants to match what is expected
by libc

Make the OpenPic driver bus-independent, with attachments for
the MacIO chip and PSIM's IOBus. Bus-specific drivers should
use the identify method to attach themselves to nexus so
interrupt can be allocated before the h/w is probed. The
'early attach' routine in openpic is used for this stage
of boot. When h/w is probed, the openpic can be attached
properly. It will enable interrupts allocated prior to
this.

Remove hard-coded knowledge of specific OFW devices. Use bus_generic_probe
and add_child entry point to allow devices to use the identify
method to add themselves if need be (e.g. openpic, syscons).
Export interrupt-controller-add routine for extern int cntlr drivers.
Eliminate recursive OFW device-tree walk and only iterate the
top-level ala sparc64. Allow child devices to set the device
type with write_ivars.

Step 1 of many in removing the hard-dependency on OpenFirmware.

GC the unused <machine/kse.h> file.

- removed obsolete ppc_exit/ppc_boot functions
- OpenFirmware returns overlapping memory regions. Use a simple
brute force algorithm to merge these into non-overlapping
regions. This fixes bugs in reporting of available memory
and also prevents pages from being added twice in the VM system.

Use the "shut-down" and "reset-all" Forth procedures to halt and
reboot, as calling OF_exit() just hangs a mac.

FreeBSD on my G4 800Mhz mac behaves identically to OSX for halt
and reboot now.

Reviewed by: grehan (who also supplied the concept and sample code)

Make breakpoint() actually break into ddb.

Reviewed by: grehan

Widen the enable/disable helper function's argument in line with the
ithread_create() changes etc. This should be mostly a NOP.

- Modify alpha's sf_buf implementation to use the direct virtual-to-
physical mapping.
- Move the sf_buf API to its own header file; make struct sf_buf's
definition machine dependent. In this commit, we remove an
unnecessary field from struct sf_buf on the alpha, amd64, and ia64.
Ultimately, we may eliminate struct sf_buf on those architecures
except as an opaque pointer that references a vm page.

Move pmap_resident_count() from the MD pmap.h to the MI pmap.h.
Add a definition of pmap_wired_count().
Add a definition of vmspace_wired_count().

Reviewed by: truckman
Discussed with: peter

DELAY must be a routine, not a macro definition.

Add sysentvec->sv_fixlimits() hook so that we can catch cases on 64 bit
systems where the data/stack/etc limits are too big for a 32 bit process.

Move the 5 or so identical instances of ELF_RTLD_ADDR() into imgact_elf.c.

Supply an ia32_fixlimits function. Export the clip/default values to
sysctl under the compat.ia32 heirarchy.

Have mmap(0, ...) respect the current p->p_limits[RLIMIT_DATA].rlim_max
value rather than the sysctl tweakable variable. This allows mmap to
place mappings at sensible locations when limits have been reduced.

Have the imgact_elf.c ld-elf.so.1 placement algorithm use the same
method as mmap(0, ...) now does.

Note that we cannot remove all references to the sysctl tweakable
maxdsiz etc variables because /etc/login.conf specifies a datasize
of 'unlimited'. And that causes exec etc to fail since it can no
longer find space to mmap things.

_MACHINE/_MACHINE_ARCH shouldn't be quoted. Found by trying to
compile the isp driver.

Standardize idempotentcy ifdefs. Consistently use _MACHINE_VARARGS_H_
symbol.

Further cleanup <machine/cpu.h> and <machine/md_var.h>: move the MI
prototypes of cpu_halt(), cpu_reset() and swi_vm() from md_var.h to
cpu.h. This affects db_command.c and kern_shutdown.c.

ia64: move all MD prototypes from cpu.h to md_var.h. This affects
madt.c, interrupt.c and mp_machdep.c. Remove is_physical_memory().
It's not used (vm_machdep.c).

alpha: the MD prototypes have been left in cpu.h with a comment
that they should be there. Moving them is left for later. It was
expected that the impact would be significant enough to be done in
a seperate commit.

powerpc: MD prototypes left in cpu.h. Comment added.

Suggested by: bde
Tested with: make universe (pc98 incomplete)

Expand inline the relevant parts of src/COPYRIGHT for Matt Dillon's
copyrighted files.

Approved by: Matt Dillon

- Since td_critnest is now initialized in MI code, it doesn't have to be
set in cpu_critical_fork_exit() anymore.
- As far as I can tell, cpu_thread_link() has never been used, not even
when it was originally added, so remove it.

Deal with GCC annoyingly defining _BIG_ENDIAN.

Deal with 'options KSTACK_PAGES' being a global option.

- Introduce a new busdma flag BUS_DMA_ZERO to request for zero'ed
memory in bus_dmamem_alloc(). This is possible now that
contigmalloc() supports the M_ZERO flag.
- Remove the locking of Giant around calls to contigmalloc() since
contigmalloc() now grabs Giant itself.

Mega busdma API commit.

Add two new arguments to bus_dma_tag_create(): lockfunc and lockfuncarg.
Lockfunc allows a driver to provide a function for managing its locking
semantics while using busdma. At the moment, this is used for the
asynchronous busdma_swi and callback mechanism. Two lockfunc implementations
are provided: busdma_lock_mutex() performs standard mutex operations on the
mutex that is specified from lockfuncarg. dftl_lock() is a panic
implementation and is defaulted to when NULL, NULL are passed to
bus_dma_tag_create(). The only time that NULL, NULL should ever be used is
when the driver ensures that bus_dmamap_load() will not be deferred.
Drivers that do not provide their own locking can pass
busdma_lock_mutex,&Giant args in order to preserve the former behaviour.

sparc64 and powerpc do not provide real busdma_swi functions, so this is
largely a noop on those platforms. The busdma_swi on is64 is not properly
locked yet, so warnings will be emitted on this platform when busdma
callback deferrals happen.

If anyone gets panics or warnings from dflt_lock() being called, please
let me know right away.

Reviewed by: tmm, gibbs

Remove unused bootpath[] variable. It conflicted with a declaration
in the sunlabel utility, causing build problems.

Migrate the thread stack management functions from the machine-dependent
to the machine-independent parts of the VM. At the same time, this
introduces vm object locking for the non-i386 platforms.

Two details:

1. KSTACK_GUARD has been removed in favor of KSTACK_GUARD_PAGES. The
different machine-dependent implementations used various combinations
of KSTACK_GUARD and KSTACK_GUARD_PAGES. To disable guard page, set
KSTACK_GUARD_PAGES to 0.

2. Remove the (unnecessary) clearing of PG_ZERO in vm_thread_new. In
5.x, (but not 4.x,) PG_ZERO can only be set if VM_ALLOC_ZERO is passed
to vm_page_alloc() or vm_page_grab().

Bring back bus_dmasync_op_t. It is now a typedef to an int, though the
BUS_DMASYNC_ definitions remain as before. The does not change the ABI,
and reverts the API to be a bit more compatible and flexible. This has
survived a full 'make universe'.

Approved by: re (bmah)

sys/sys/limits.h:

- Fix visibilty test for LONG_BIT and WORD_BIT. `#if defined(__FOO_VISIBLE)'
is alays wrong because __FOO_VISIBLE is always defined (to 0 for
invisibility).

sys/<arch>/include/limits.h
sys/<arch>/include/_limits.h:

- Style fixes.

Submitted by: bde
Reviewed by: bsdmike
Approved by: re (scottl)

Style fixes.
Remove DBL_DIG, DBL_MIN, DBL_MAX and their FLT_ counterparts, they
were marked for deprecation ever since SUSv1 at least.
Only define ULLONG_MIN/MAX and LLONG_MAX if long long type is
supported.
Restore a lost comment in MI _limits.h file and remove it from
sys/limits.h where it does not belong.

Slight reorg and added AMD64 support. A couple of the MODINFOMD_* values
that were added to sparc64 and later powerpc, really should have been in
the MI area. But changing that now with insufficient preperation will
just cause too much pain.

Move MD_FETCH() to the MI sys/linker.h file to avoid another two copies
of it.

Deprecate machine/limits.h in favor of new sys/limits.h.
Change all in-tree consumers to include <sys/limits.h>

Discussed on: standards@
Partially submitted by: Craig Rodrigues <rodrigc@attbi.com>

Add a new sys/limits.h file which in turn depends on machine/_limits.h
to get actual constant values. This is in preparation for machine/limits.h
retirement.

Discussed on: standards@
Submitted by: Craig Rodrigues <rodrigc@attbi.com> (*)
Modified by: kan

- Convert NetBSD-derived macros to inline functions for better
type-checking and future debug code.
- Remove sparse addressing hack, since the only consumer, the macio ATA
driver, doesn't require it anymore.

I deserve a big pointy hat for having missed all those references
to bus_dmasync_op_t in my last commit.

- Add vm_paddr_t, a physical address type. This is required for systems
where physical addresses larger than virtual addresses, such as i386s
with PAE.
- Use this to represent physical addresses in the MI vm system and in the
i386 pmap code. This also changes the paddr parameter to d_mmap_t.
- Fix printf formats to handle physical addresses >4G in the i386 memory
detection code, and due to kvtop returning vm_paddr_t instead of u_long.

Note that this is a name change only; vm_paddr_t is still the same as
vm_offset_t on all currently supported platforms.

Sponsored by: DARPA, Network Associates Laboratories
Discussed with: re, phk (cdevsw change)

Made the prototypes for pmap_kenter and pmap_kremove MD. These functions
are machine dependent because they are not required to update the tlb when
mappings are added or removed, and doing so is machine dependent.
In addition, an implementation may require that pages mapped with pmap_kenter
have a backing vm_page_t, which is not necessarily true of all physical
pages, and so may choose to pass the vm_page_t to pmap_kenter instead of the
physical address in order to make this requirement clear.

Correctly set BUS_SPACE_MAXSIZE in all the busdma backends.
It was bogusly set to 64 * 1024 or 128 * 1024 because it was
bogusly reused in the BUS_DMAMAP_NSEGS definition.

Doh. Forgot to remove _KERNEL version.

Expose powerpc_mb() to user-space. Currently needed for atomic.h users,
this may go away in the future.

Implement fpclassify():
o Add a MD header private to libc called _fpmath.h; this header
contains bitfield layouts of MD floating-point types.
o Add a MI header private to libc called fpmath.h; this header
contains bitfield layouts of MI floating-point types.
o Add private libc variables to lib/libc/$arch/gen/infinity.c for
storing NaN values.
o Add __double_t and __float_t to <machine/_types.h>, and provide
double_t and float_t typedefs in <math.h>.
o Add some C99 manifest constants (FP_ILOGB0, FP_ILOGBNAN, HUGE_VALF,
HUGE_VALL, INFINITY, NAN, and return values for fpclassify()) to
<math.h> and others (FLT_EVAL_METHOD, DECIMAL_DIG) to <float.h> via
<machine/float.h>.
o Add C99 macro fpclassify() which calls __fpclassify{d,f,l}() based
on the size of its argument. __fpclassifyl() is never called on
alpha because (sizeof(long double) == sizeof(double)), which is good
since __fpclassifyl() can't deal with such a small `long double'.

This was developed by David Schultz and myself with input from bde and
fenner.

PR: 23103
Submitted by: David Schultz <dschultz@uclink.Berkeley.EDU>
(significant portions)
Reviewed by: bde, fenner (earlier versions)

Export the ns_per_tick variable through md_var.h rather than by declaring
it extern in cpu.c.

Add cpu.c. This contains one exported function, cpu_setup(), which handles
setup of and printing information about cpus.

Obtained from: NetBSD (parts)

- Update spr.h
- Add hid.h

Obtained from: NetBSD

NOTE: This undoes some changes I'd made to prefix the processor name defines
with PVR_. This was due to my original decision to use MPC750 as a cpu name.
With this changed, the PVR_ change is no longer required.

Add an inline function wrapper for the mfpvf (Move From Processor Version
Register) instruction.

Not all cpus are MPC750s. Replace the MPC750 cpu option with OEA. This
stands for Operating Environment Architecture and is the specification that
all of the MPC6xx, MPC7xx, MPC7xxx and IBM7xx CPUs adhere to.

- Rename the "powerpc" timecounter to the "decrementer" timecounter.
- Initialise it earlier.

- Introduce a flags value into the interrupt handler structure.
- Copy the flags passed to inthand_add into the flags value.
- If the interrupt is INTR_FAST, re-enable the irq after running the handler.

Rework of how memory resources are discovered and dealt with in macio.
- Store the OpenFirmware "reg" property in the macio ivars.
- Use a struct to define the structure of a "reg" property entry.
- Discover all memory ranges, not just the first.
- In ata_macio, manage our own range and hand out our own allocations using
bus_space_subregion.
- Fix bus_space_subregion to handle subregions of sparse maps.

Put the right fix in. Instead of deleting the declaration of __FBSDID, we
undef it before this definition.

Remove a duplicate definition of the __FBSDID macro.

include cdefs.h so as to unbreak the libc build

Stub profile.h, required for userland builds.

Approved by: Benno

<machine/ieee.h>, taken from sparc64

Approved by: Benno

Fix bugs with operand ordering and unnecessary sync/eieio ops. Mostly
obtained from Alpha atomic.h

Approved by: Benno

Allow the MD frame definition to be seen in. Required for truss/ptrace.

Approved by: Benno

RAIDframe requires LONG_BIT

Approved by: Benno

Prepended underscores to macro local vars, avoiding gcc "declaration
shadows global" warning

Approved by: benno

Change definition of int64 to avoid gcc3.2.1 complaints. Taken from i386

Approved by: benno

- define HAS_STREAM_METHODS correctly
- dmamap_load_mbuf/load_uio prototypes

Submitted partly by: Hiten Pandya <hiten@unixdaemons.com>

Correct typos, mostly s/ a / an / where appropriate. Some whitespace cleanup,
especially in troff files.

MB_LEN_MAX is not MD, move it to the MI limits.h.

Split 4.x and 5.x signal handling so that we can keep 4.x signal
handling clean and functional as 5.x evolves. This allows some of the
nasty bandaids in the 5.x codepaths to be unwound.

Encapsulate 4.x signal handling under COMPAT_FREEBSD4 (there is an
anti-foot-shooting measure in place, 5.x folks need this for a while) and
finish encapsulating the older stuff under COMPAT_43. Since the ancient
stuff is required on alpha (longjmp(3) passes a 'struct osigcontext *'
to the current sigreturn(2), instead of the 'ucontext_t *' that sigreturn
is supposed to take), add a compile time check to prevent foot shooting
there too. Add uniform COMPAT_43 stubs for ia64/sparc64/powerpc.

Tested on: i386, alpha, ia64. Compiled on sparc64 (a few days ago).
Approved by: re

Add the USER_SR segment register to pcb state. Initialize correctly,
and save/restore during a context switch.

The USER_SR could be overwritten when the current thread was switched
out with a faulting copyin/copyout.

Approved by: Benno

The a.out md_coredump stuff isn't referenced anywhere anymore, and
hasn't been filled in for ages.. Nuked.

Add standards visibility conditionals. Change any uses of sigset_t to
struct __sigset to avoid depending on objects from <sys/signal.h>.

Add conditionals to allow va_list to be defined in other headers.

o Add conditionals to allow va_list to be defined in other headers.
o Standardize on _MACHINE_STDARG_H_ to allow multiple header includes.
o Restrict the definition of va_copy() to C99 environments.

Fix namespace issues by using visibility conditionals from
<sys/cdefs.h>.

Define _MACHINE.

style(9) <machine/setjmp.h> headers so they look mostly the same.

Clean up ddb warnings/errors and enable in GENERIC

Approved by: benno
Motivated by: gallatin

PIC_GOTOFF is OBE.

Be careful not to define GCC-specific optimizations in the non-GCC
case.

Removed osigframe. No need for COMPAT_43 signal bin-compat in PPC.

Approved by: benno

psim device support

Approved by: benno

<machine/types.> -> <sys/types.h>

Approved by: benno

Fix clearing of recoverable exception MSR bit when disabling
interrupts

Approved by: benno

Additional machdep sysctl constants needed for userland utils

Approved by: benno

Added sparse address support, required by the macio ATA device

Approved by: benno

Fixed branch labels

Approved by: benno

Implement C99's va_copy() macro.

Initiate deorbit burn for the i386-only a.out related support. Moves are
under way to move the remnants of the a.out toolchain to ports. As the
comment in src/Makefile said, this stuff is deprecated and one should not
expect this to remain beyond 4.0-REL. It has already lasted WAY beyond
that.

Notable exceptions:
gcc - I have not touched the a.out generation stuff there.
ldd/ldconfig - still have some code to interface with a.out rtld.
old as/ld/etc - I have not removed these yet, pending their move to ports.
some includes - necessary for ldd/ldconfig for now.

Tested on: i386 (extensively), alpha

Now that _BSD_CLK_TCK_ and _BSD_CLOCKS_PER_SEC_ are the same on all
architectures, move the definition directly into <time.h> and finish
the removal of <machine/ansi.h>.

Change hw.physmem and hw.usermem to unsigned long like they used to be
in the original hardwired sysctl implementation.

The buf size calculator still overflows an integer on machines with large
KVA (eg: ia64) where the number of pages does not fit into an int. Use
'long' there.

Change Maxmem and physmem and related variables to 'long', mostly for
completeness. Machines are not likely to overflow 'int' pages in the
near term, but then again, 640K ought to be enough for anybody. This
comes for free on 32 bit machines, so why not?

Renamed poorly named setregs to exec_setregs. Moved its prototype to
imgact.h with the other exec support functions.

Since arm and powerpc aren't far enough to set stathz, take a
preemptive strike and change _BSD_CLK_TCK_ and _BSD_CLOCKS_PER_SEC_
to 128.

Approved by: benno

Move several MI types from <machine/_types.h> to <sys/_types.h>.
These types are unlikely to ever become very MD. They include:
clockid_t, ct_rune_t, fflags_t, intrmask_t, mbstate_t, off_t, pid_t,
rune_t, socklen_t, timer_t, wchar_t, and wint_t.

While moving them, make a few adjustments (submitted by bde):
o __ct_rune_t needs to be precisely `int', not necessarily __int32_t,
since the arg type of the ctype functions is int.
o __rune_t, __wchar_t and __wint_t inherit this via a typedef of
__ct_rune_t.
o Some minor wording changes in the comment blocks for ct_rune_t and
mbstate_t.

Submitted by: bde (partially)

o Merge <machine/ansi.h> and <machine/types.h> into a new header
called <machine/_types.h>.
o <machine/ansi.h> will continue to live so it can define MD clock
macros, which are only MD because of gratuitous differences between
architectures.
o Change all headers to make use of this. This mainly involves
changing:
#ifdef _BSD_FOO_T_
typedef _BSD_FOO_T_ foo_t;
#undef _BSD_FOO_T_
#endif
to:
#ifndef _FOO_T_DECLARED
typedef __foo_t foo_t;
#define _FOO_T_DECLARED
#endif

Concept by: bde
Reviewed by: jake, obrien

o Introduce pmap_page_is_mapped(). Its purpose is to obsolete
the PG_MAPPED flag.

Create a new header <machine/_stdint.h> for storing MD parts of
<stdint.h>. Previously, parts were defined in <machine/ansi.h> and
<machine/limits.h>. This resulted in two problems:
(1) Defining macros in <machine/ansi.h> gets in the way of that
header only defining types.
(2) Defining C99 limits in <machine/limits.h> adds pollution to
<limits.h>.

Remove label_t and physadr, which seem to have never been used in
FreeBSD.

Submitted by: bde

Add setjmp (needed for DDB).

Add DDB support.

Metadata definitions.

Add the OF_getetheraddr function required by if_gem.

Changes for KSE3.

Submitted by: Peter Grehan <peterg@ptree32.com.au>

Add this file, which I forgot in a previous commit.

This relates to the trap/interrupt cleanup.

Submitted by: Peter Grehan <peterg@ptree32.com.au>

1) Add busdma machdep code.
2) Add bus_pio.h and bus_memio.h (which do nothing).

Submitted by: Peter Grehan <peterg@ptree32.com.au> (1)

Driver for OpenPIC compatible interrupt controllers.
It's fairly PowerMac specific at the moment, but that should be fixable.

- Add the "compatible" property to the list that we keep in ivars.
- Add interrupt alloc/setup/teardown/dealloc support, via whichever PIC
OpenFirmware gives us.

Add interrupt handling support code.

I've tried to make this fairly platform-independant as some PowerPC platforms
may not have openpic-style interrupt controllers. This may not have the best
performance but it works for now.

Move __offsetof() macro from <machine/ansi.h> to <sys/cdefs.h>. It's
hardly MD, since all our platforms share the same macro. It's not
really compiler dependent either, but this helps in reducing
<machine/ansi.h> to only type definitions.

Since printf(3) now supports the `j' conversion specifier, use that
when printing intmax_t and uintmax_t.

Forgotten by: mike
Noticed by: bde

Add an inline to call eieio.

("Enforce In-order Execution of I/O". I am not making this up.)

We don't need to clear RI in the MSR when entering a critical section.

in_cksum et al.

Submitted by: Peter Grehan <peterg@ptree32.com.au>

Implement vtophys()

Add pmap_mapdev and pmap_unmapdev.

To quote Peter:

The case in cpu_switch() where there isn't a higher priority thread
(choosethread() == curthread) uses r4 as the PCB context pointer. However, the
use of r4 after the label L2 is incorrect, since it was probably trashed by
the call to choosethread, and in any case was set up to curthread at the start
of the routine.

This condition will occur when an interrupt thread schedules a netisr, which
is a lower priority thread.

Another (probably unnecessary) difference is that I was paranoid about
register trashing, so I decided to save r2 and r13 as well.

Submitted by: Peter Grehan <peterg@ptree32.com.au>

Make vm_pindex_t 64-bit on all platforms. This is necessary to avoid
overflows with the large file sizes that UFS2 permits.

Reviewed by: dillon, alc, tegge

Move the "- 1" into the RQB_FFS(mask) macro itself so that
implementations can provide a base zero ffs function if they wish.
This changes
#define RQB_FFS(mask) (ffs64(mask))
foo = RQB_FFS(mask) - 1;
to
#define RQB_FFS(mask) (ffs64(mask) - 1)
foo = RQB_FFS(mask);
On some platforms we can get the "- 1" for free, eg: those that use the
C code for ffs64().

Reviewed by: jake (in principle)

Move the definition of ElfN_Hashelt to common headers. The only platform
which has a different definition for this is alpha.

- Move macros that represent where syscall args are kept in a trapframe from
trap.c to frame.h
- Use the macros in vm_machdep.c:cpu_fork() to set up the trap frame of the
new thread.

Move the kcopy() function from trap.c to machdep.c. Add a prototype.

Get the correct memory regions from OpenFirmware. We were getting the
"available" ranges, not the "physical" ranges. Clean up some of the
bootstrap code in the process.

Submitted by: Peter Grehan <peterg@ptree32.com.au>