Merge r267738 from stable/9:

Remove send-pr and fix up all references to it. Replace it with a
stub send-pr directing people towards the web site.

Approved by: re (gjb), bugmeister

Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC the mpr(4) driver for LSI's 12Gb SAS cards.

This includes r265236, r265237, r265241, r265261, r265386, r265424, and
r265473.

------------------------------------------------------------------------
r265236 | ken | 2014-05-02 14:25:09 -0600 (Fri, 02 May 2014) | 51 lines

Bring in the mpr(4) driver for LSI's MPT3 12Gb SAS controllers.

This is derived from the mps(4) driver, but it supports only the 12Gb
IT and IR hardware including the SAS 3004, SAS 3008 and SAS 3108.

Some notes about this driver:
o The 12Gb hardware can do "FastPath" I/O, and that capability is included in
this driver.

o WarpDrive functionality has been removed, since it isn't supported in
the 12Gb driver interface.

o The Scatter/Gather list handling code is significantly different between
the 6Gb and 12Gb hardware. The 12Gb boards support IEEE Scatter/Gather
lists.

Thanks to LSI for developing and testing this driver for FreeBSD.

share/man/man4/mpr.4:
mpr(4) man page.

sys/dev/mpr/*:
mpr(4) driver files.

sys/modules/Makefile,
sys/modules/mpr/Makefile:
Add a module Makefile for the mpr(4) driver.

sys/conf/files:
Add the mpr(4) driver.

sys/amd64/conf/GENERIC,
sys/i386/conf/GENERIC,
sys/mips/conf/OCTEON1,
sys/sparc64/conf/GENERIC:
Add the mpr(4) driver to all config files that currently
have the mps(4) driver.

sys/ia64/conf/GENERIC:
Add the mps(4) and mpr(4) drivers to the ia64 GENERIC
config file.

sys/i386/conf/XEN:
Exclude the mpr module from building here.

Submitted by: Steve McConnell <Stephen.McConnell@avagotech.com>
Tested by: Chris Reeves <chrisr@spectralogic.com>
Sponsored by: LSI, Spectra Logic
Relnotes: LSI 12Gb SAS driver mpr(4) added

------------------------------------------------------------------------
------------------------------------------------------------------------
r265237 | ken | 2014-05-02 14:36:20 -0600 (Fri, 02 May 2014) | 8 lines

Add the mpr(4) man page to the man4 Makefile.

This should have been included in r265236.

Submitted by: Steve McConnell <Stephen.McConnell@avagotech.com>
Sponsored by: LSI, Spectra Logic

------------------------------------------------------------------------
------------------------------------------------------------------------
r265241 | brueffer | 2014-05-02 15:14:28 -0600 (Fri, 02 May 2014) | 2 lines

Use our standard SYNOPSIS wording; perform some cleanup while here.

------------------------------------------------------------------------
------------------------------------------------------------------------
r265261 | brueffer | 2014-05-03 05:15:28 -0600 (Sat, 03 May 2014) | 2 lines

Add a missing colon.

------------------------------------------------------------------------
------------------------------------------------------------------------
r265386 | ken | 2014-05-05 13:53:03 -0600 (Mon, 05 May 2014) | 15 lines

Adjust #if statements inside mprsas_send_smpcmd() to more accurately
reflect when unmapped I/O support was added.

For FreeBSD 10, it arrived just prior to __FreeBSD_version 1000028.
For FreeBSD 9, it arrived just prior to __FreeBSD_version 902001.

Also, fix compiler warnings in mprsas_send_smpcmd() that happen in the
i386 PAE build for non-unmapped I/O builds. These were fixed in mps(4)
in revision 241145, but didn't make it into the mpr(4) driver. This
change should only affect FreeBSD versions outside the above revisions,
and thus doesn't affect head.

Sponsored by: Spectra Logic Corporation

------------------------------------------------------------------------
------------------------------------------------------------------------
r265424 | ken | 2014-05-06 00:18:43 -0600 (Tue, 06 May 2014) | 33 lines

Fix a problem with async notifications in the mpr(4) driver.

This problem only occurs on versions of FreeBSD prior to the recent CAM
locking changes. (i.e. stable/9 and older versions of stable/10) This
change should be a no-op for head and stable/10.

If a path isn't specified, xpt_register_async() will create a fully
wildcarded path and acquire a lock (the XPT lock in older versions,
and via xpt_path_lock() in newer versions) to call xpt_action() for the
XPT_SASYNC_CB CCB. It will then drop the lock and if the requested event
includes AC_FOUND_DEVICE or AC_PATH_REGISTERED, it will get the caller up
to date with any device arrivals or path registrations.

The issue is that before the locking changes, each SIM lock would get
acquired in turn during the EDT tree traversal process. If a path is
specified for xpt_register_async(), it won't acquire and drop its own lock,
but instead expects the caller to hold its own SIM lock. That works for
the first part of xpt_register_async(), but causes a recursive lock
acquisition once the EDT traversal happens and it comes to the SIM in
question. And it isn't possible to call xpt_action() without holding a SIM
lock.

The locking changes fix this by using the XPT topology lock for EDT
traversal, so it is no longer an issue to hold the SIM lock while calling
xpt_register_async().

The solution for FreeBSD versions before the locking changes is to request
notification of all device arrivals (so we pass a NULL path into
xpt_register_async()) and then filter out the arrivals that are not ours.

Sponsored by: Spectra Logic Corporation

------------------------------------------------------------------------
------------------------------------------------------------------------
r265473 | ken | 2014-05-06 16:13:38 -0600 (Tue, 06 May 2014) | 7 lines

Change the device name for mpr(4) from /dev/mpr_N to /dev/mprN.

This is more consistent with the existing mps(4) behavior.

Reviewed by: Steve McConnell <stephen.mcconnell@avagotech.com>

------------------------------------------------------------------------

Submitted by: Steve McConnell <Stephen.McConnell@avagotech.com>
Tested by: Chris Reeves <chrisr@spectralogic.com>
Sponsored by: LSI, Spectra Logic
Relnotes: LSI 12Gb SAS driver mpr(4) added

MFC r263289: Update NetBSD Foundation copyrights to 2-clause BSD

The NetBSD Foundation states "Third parties are encouraged to change the
license on any files which have a 4-clause license contributed to the
NetBSD Foundation to a 2-clause license."

This change removes clauses 3 and 4 from copyright / license blocks that
list The NetBSD Foundation as the only copyright holder.

Sponsored by: The FreeBSD Foundation

MFC vm_page_alloc_contig()

This function will be used by TTM, a memory manager used by the Radeon
KMS driver.

Compared to HEAD, the type of the "boundary" argument to several functions
is left as "unsigned long" (instead of "vm_paddr_t" in HEAD) to prevent an
API change in a stable branch.

The following revisions were merged in this single commit:

MFC r226928:
Eliminate vm_phys_bootstrap_alloc(). It was a failed attempt at
eliminating duplicated code in the various pmap implementations.

Micro-optimize vm_phys_free_pages().

Introduce vm_phys_free_contig(). It is fast routine for freeing an
arbitrary number of physically contiguous pages. In particular, it
doesn't require the number of pages to be a power of two.

Use "u_long" instead of "unsigned long".

Bruce Evans (bde@) has convinced me that the "boundary" parameters
to kmem_alloc_contig(), vm_phys_alloc_contig(), and
vm_reserv_reclaim_contig() should be of type "vm_paddr_t" and not
"u_long". Make this change.

MFC r227012:
Add support for VM_ALLOC_WIRED and VM_ALLOC_ZERO to vm_page_alloc_freelist()
and use these new options in the mips pmap.

Wake up the page daemon in vm_page_alloc_freelist() if the number of free
and cached pages becomes too low.

Tidy up vm_page_alloc_init(). In particular, add a comment about an
important restriction on its use.

Tested by: jchandra@

MFC r227072:
Simplify the implementation of the failure case in kmem_alloc_attr().

MFC r227127:
Wake up the page daemon in vm_page_alloc_freelist() if it couldn't
allocate the requested page because too few pages are cached or free.

Document the VM_ALLOC_COUNT() option to vm_page_alloc() and
vm_page_alloc_freelist().

Make style changes to vm_page_alloc() and vm_page_alloc_freelist(),
such as using a variable name that more closely corresponds to the
comments.

MFC r227568:
Refactor the code that performs physically contiguous memory allocation,
yielding a new public interface, vm_page_alloc_contig(). This new function
addresses some of the limitations of the current interfaces, contigmalloc()
and kmem_alloc_contig(). For example, the physically contiguous memory that
is allocated with those interfaces can only be allocated to the kernel vm
object and must be mapped into the kernel virtual address space. It also
provides functionality that vm_phys_alloc_contig() doesn't, such as wiring
the returned pages. Moreover, unlike that function, it respects the low
water marks on the paging queues and wakes up the page daemon when
necessary. That said, at present, this new function can't be applied to all
types of vm objects. However, that restriction will be eliminated in the
coming weeks.

From a design standpoint, this change also addresses an inconsistency
between vm_phys_alloc_contig() and the other vm_phys_alloc*() functions.
Specifically, vm_phys_alloc_contig() manipulated vm_page fields that other
functions in vm/vm_phys.c didn't. Moreover, vm_phys_alloc_contig() knew
about vnodes and reservations. Now, vm_page_alloc_contig() is responsible
for these things.

Reviewed by: kib
Discussed with: jhb

MFC r255194 and prerequisites r229494,229496:

Introduce internal macros for __U/INT64_C to define the U/INT64_MAX/MIN
values properly. The previous definition only worked if __STDC_LIMIT_MACROS
and __STDC_CONSTANT_MACROS were defined at the same time.

Newer versions of gcc define __INT64_C and __UINT64_C, so avoid
redefining them if gcc provides them.

Applying r255194 fixes stable/9 host to allow source upgrading to the newer
versions with usr.bin/dtc included in bootstrap tools.

Approved by: imp (MFC to stable/9)

MFC r250576 (by eadler):
Fix several typos

PR: kern/176054

MFC r241082:
Inherit USB mode from RootHUB port where the USB device is connected.
Only RootHUB ports can be dual mode. Disallow OTG ports on external HUBs.
This simplifies some checks in the USB controller drivers.

MFC r251703 (by attilio):
- Add a BIT_FFS() macro and use it to replace cpusetffs_obj()

MFC: r241374

Add an unified macro to deny ability from the compiler to reorder
instruction loads/stores at its will.
The macro __compiler_membar() is currently supported for both gcc and
clang, but kernel compilation will fail otherwise.

Reviewed by: bde, kib
Discussed with: dim, theraven

Merge the second part of the unmapped I/O changes. This enables the
infrastructure in the block layer and UFS filesystem as well as a few
drivers. The list of MFC revisions is long, so I won't quote changelogs.

r248508,248510,248511,248512,248514,248515,248516,248517,248518,
248519,248520,248521,248550,248568,248789,248790,249032,250936

Submitted by: kib
Approved by: kib
Obtained from: Netflix

Big MFC of the physbio changes necessary for unmapped I/O. These changes
have been in production at Netflix for several months with significant
success.

MFC r246713:

Reform the busdma API so that new types may be added without modifying
every architecture's busdma_machdep.c. It is done by unifying the
bus_dmamap_load_buffer() routines so that they may be called from MI
code. The MD busdma is then given a chance to do any final processing
in the complete() callback.

MFC r249538:
Some compilers issue a warning when wider integer is casted to narrow
pointer. Supposedly shut down the warning by casting through
uintptr_t.

MFC r251479:
Simplify the checking of flags for cam_periph_mapmem(). This gets rid of
a lot of code redundancy and grossness at very minor expense.

MFC r251837:
MFC r251842:
Add infrastructure for doing compatibility shims, as has been sorely
needed for the last 10 years. Far too much of the internal API is
exposed, and every small adjustment causes applications to stop working.
To kick this off, bump the API version to 0x17 as should have been done
with r246713, but add shims to compensate. Thanks to the shims, there
should be no visible change in application behavior.

Submitted by: kib, jeffr
Approved by: kib
Obtained from: Netflix

MFC r240177:

Dynamically allocate the S/G lists passed to callback routines rather than
allocating them on the stack of various bus_dmamap_load*() functions. The
S/G lists are stored in the DMA tags. This matches the implementation on
all other platforms.

Submitted by: jhb
Approved by: marius, adrian
Obtained from: Netflix

MFC r242465:
Free the dma map -after- it's checked, not before. Or you'll be
potentially referencing already-freed memory.

MFC r242466:
Drop this from 500 to 128, to save a little space on memory constrained
platforms.

Submitted by: adrian
Obtained from: Netflix

Tidy up some CVS workarounds.

MFC r249449:

Fix undefined behaviour in several gpio_pin_setflags() routines (under
sys/arm and sys/mips), squelching the clang 3.3 warnings about this.

Noticed by: tinderbox and many irate spectators
Submitted by: Luiz Otavio O Souza <loos.br@gmail.com>
PR: kern/177759

MFC r248280:
Add pmap function pmap_copy_pages().

MFC: r227309 (partial)

Mark all SYSCTL_NODEs static that have no corresponding SYSCTL_DECLs.

The SYSCTL_NODE macro defines a list that stores all child-elements of
that node. If there's no SYSCTL_DECL macro anywhere else, there's no
reason why it shouldn't be static.

Clean some 'svn:executable' properties in the tree.

Submitted by: Christoph Mallon

While here, merge some other mergeinfo properties that
were left behind from my commits

/head/include:r241008,241141,241181
/head/contrib/gcc:r244776,244792
/head/cddl:r238457,238509,238558

MFC r241844:
remove duplicate semicolons where possible.

Approved by: cperciva (implicit)

MFC r237433:
Implement mechanism to export some kernel timekeeping data to
usermode, using shared page.

MFC r237474:
Stop updating the struct vdso_timehands from even handler executed in
the scheduled task from tc_windup(). Do it directly from tc_windup in
interrupt context.

MFC r237168
The page flag PGA_WRITEABLE is set and cleared exclusively by the pmap
layer, but it is read directly by the MI VM layer. This change introduces
pmap_page_is_write_mapped() in order to completely encapsulate all direct
access to PGA_WRITEABLE in the pmap layer.

MFC r240317
Simplify pmap_unmapdev().

Don't set PTE_W on the page table entry in pmap_kenter{,_attr}() on MIPS.

MFC r233122,r237086,r237228,r237264,r237290,r237404,r237414,r237513,r237551,
r237592,r237604,r237623,r237684,r237733,r237813,r237855,r238124,r238126,
r238163,r238414,r238610,r238889,r238970,r239072,r239137,r240126 (all by alc):

Add fine-grained PV chunk and list locking to the amd64 pmap, enabling
concurrent execution of the following functions on different pmaps:

pmap_change_wiring()
pmap_copy()
pmap_enter()
pmap_enter_object()
pmap_enter_quick()
pmap_page_exists_quick()
pmap_page_is_mapped()
pmap_protect()
pmap_remove()
pmap_remove_pages()

Requested and approved by: alc

MFH r236751: document sha256 / sha512 support
MFH r236892: remove mention of auth.conf from programs that don't use it
MFH r236963: remove dead code relating to auth.conf
MFH r236965 r236966 r236967 r237005 r237006 r237011: retire auth.conf

Approved by: re

Merge r231712 from head to stable/9:

When initialising the CP0 status register during boot on 64-bit MIPS,
set all three of the kernel, supervisor, and user-mode 64-bit mode
flags. While FreeBSD does not currently use the supervisor ring (and
hence this is effectively a NOP on most systems), doing this avoids
triggering an exception on 64-bit MIPS CPUs that don't support 32-bit
compatibility mode, and therefore don't allow clearing the SX bit.

Reviewed by: gonzo
Sponsored by: DARPA, SRI International

Approved by: re (kib)

MFC r235941:

in_cksum.h required ip.h to be included for struct ip. To be
able to use some general checksum functions like in_addword()
in a non-IPv4 context, limit the (also exported to user space)
IPv4 specific functions to the times, when the ip.h header is
present and IPVERSION is defined (to 4).

Approved by: re

MFC r237430:
Reserve AT_TIMEKEEP auxv entry for providing usermode the pointer to
timekeeping information.

MFC r233628, r234598, r235229, r235831, r226986.

Add software PMC support.

New kernel events can be added at various location for sampling or counting.
This will for example allow easy system profiling whatever the processor is
with known tools like pmcstat(8).

Simultaneous usage of software PMC and hardware PMC is possible, for example
looking at the lock acquire failure, page fault while sampling on
instructions.

Sponsored by: NETASQ

Toss bogus mergeinfo.

MFC r235634

Fix and update battery status bits according to linux driver

MFC r234785:

Add a convenience macro for the returns_twice attribute, and apply it to
the prototypes of the appropriate functions (getcontext, savectx,
setjmp, sigsetjmp and vfork).

MFC r228631: kern cons: introduce infrastructure for console grabbing by
kernel

MFC 233670,233671:
- Use VM_MEMATTR_UNCACHEABLE for the constant for UC memory rather than
VM_MEMATTR_UNCACHED on mips.
- Rename VM_MEMATTR_UNCACHED to VM_MEMATTR_WEAK_UNCACHEABLE on x86 to
be less ambiguous and more clearly identify what it means. An alias
from VM_MEMATTR_WEAK_UNCACHEABLE to VM_MEMATTR_WEAK_UNCACHED remains
on x86 to preserve the KPI.
- Remove the VM_MEMATTR_UNCACHED alias from powerpc.

Merge r232250 from head:

Correct capitalization of "Hz" in user-visible text (manpages, printf(),
etc).

MFC 228962:
Use curthread rather than PCPU_GET(curthread). 'curthread' uses
special-case optimizations on several platforms and is preferred.

MFC: r228005

Change another instance of amd(4) to esp(4) missed in r227006 (MFC'ed to
stable/9 in r227305).

Submitted by: Garrett Cooper

MFC r229765 - Fix the ar724x shift calculation when writing to the PCI config space.

MFC r227458, r226436:

- change "is is" to "is" or "it is"
- change "the the" to "the"
- other typo fixes

Approved by: lstewart

MFC r226112:
Remove unused define.

MFC r226065:
Convert MIPS to the syscallenter/syscallret system call sequence handlers.

MFC r227701, r227847 and r227849:
Move the device_delete_all_children() function from usb_util.c
to kern/subr_bus.c. Simplify this function so that it no longer
depends on malloc() to execute. Rename device_delete_all_children()
into device_delete_children(). Identify a few other places where
it makes sense to use device_delete_children().

MFC r229086, r228483 and r228640:
Implement better support for USB controller suspend and resume.
Fix some build issues in avr32dci.c

MFC r226173, r227843, r227848 and r227908:
Use DEVMETHOD_END to mark end of device methods.
Remove superfluous device methods.
Add some missing __FBSBID() macros.

MFC r226478:
Close a race where SIOCGIFMEDIA ioctl get inconsistent link status.
Because driver is accessing a common MII structure in
mii_pollstat(), updating user supplied structure should be done
before dropping a driver lock.

Reported by: Karim (fodillemlinkarimi <> gmail dot com)

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

In order to maximize the re-usability of kernel code in user space this
patch modifies makesyscalls.sh to prefix all of the non-compatibility
calls (e.g. not linux_, freebsd32_) with sys_ and updates the kernel
entry points and all places in the code that use them. It also
fixes an additional name space collision between the kernel function
psignal and the libc function of the same name by renaming the kernel
psignal kern_psignal(). By introducing this change now we will ease future
MFCs that change syscalls.

Reviewed by: rwatson
Approved by: re (bz)

Fix a zyd(4) comment typo that was copy+pasted into most kernel config files.

PR: 160276
Submitted by: MATSUMIYA Ryo <matsumiya@mma.club.uec.ac.jp>
Approved by: re (kib)
MFC after: 1 week

Split the vm_page flags PG_WRITEABLE and PG_REFERENCED into atomic
flags field. Updates to the atomic flags are performed using the atomic
ops on the containing word, do not require any vm lock to be held, and
are non-blocking. The vm_page_aflag_set(9) and vm_page_aflag_clear(9)
functions are provided to modify afalgs.

Document the changes to flags field to only require the page lock.

Introduce vm_page_reference(9) function to provide a stable KPI and
KBI for filesystems like tmpfs and zfs which need to mark a page as
referenced.

Reviewed by: alc, attilio
Tested by: marius, flo (sparc64); andreast (powerpc, powerpc64)
Approved by: re (bz)

MIPS XLP platform code update.

* Update the hardware access register definitions and functions to bring
them in line with other Netlogic software.
* Update the platform bus to use PCI even for on-chip devices. Add a dummy
PCI driver to ignore on-chip devices which do not need driver.
* Provide memory and IRQ resource allocation code for on-chip devices
which cannot get it from PCI config.
* add support for on-chip PCI and USB interfaces.
* update conf files, enable pci and retain old MAXCPU until we can support
>32 cpus.

Approved by: re(kib), jmallett

Fix a typo in cb_dumpdata() about wrongly calling wdog_kern_pat().

Submitted by: Andrew Boyer <aboyer at averesystems dot com>
Approved by: re (kib)

- Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag
to VPO_UNMANAGED (and also making the flag protected by the vm object
lock, instead of vm page queue lock).
- Mark the fake pages with both PG_FICTITIOUS (as it is now) and
VPO_UNMANAGED. As a consequence, pmap code now can use use just
VPO_UNMANAGED to decide whether the page is unmanaged.

Reviewed by: alc
Tested by: pho (x86, previous version), marius (sparc64),
marcel (arm, ia64, powerpc), ray (mips)
Sponsored by: The FreeBSD Foundation
Approved by: re (bz)

Make octeon_ap_boot 64 bits to handle MAXCPU up to 64.
Better construction of CPU mask in platform_cpu_mask().
Release cores still in reset when platform_start_ap() is called.

Reviewed by: imp, marcel
Obtained from: Juniper Networks, Inc
Author: Andrew Duane
Approved by: re (kib)

Fix for arm and mips case the size of storage for sintrcnt/sintrnames.
It seems that "info as" is not much precise on what expect by pseudo-op
.word, by the way.

No MFC is previewed for this patch.

Tested by: andreast, pluknet
Approved by: re (kib)

Bump MAXCPU for amd64, ia64 and XLP mips appropriately.
From now on, default values for FreeBSD will be 64 maxiumum supported
CPUs on amd64 and ia64 and 128 for XLP. All the other architectures
seem already capped appropriately (with the exception of sparc64 which
needs further support on jalapeno flavour).

Bump __FreeBSD_version in order to reflect KBI/KPI brekage introduced
during the infrastructure cleanup for supporting MAXCPU > 32. This
covers cpumask_t retiral too.

The switch is considered completed at the present time, so for whatever
bug you may experience that is reconducible to that area, please report
immediately.

Requested by: marcel, jchandra
Tested by: pluknet, sbruno
Approved by: re (kib)

On 64 bit architectures size_t is 8 bytes, thus it should use an 8 bytes
storage.
Fix the sintrcnt/sintrnames specification.

No MFC is previewed for this patch.

Reported, reviewed and tested by: marcel
Approved by: re (kib)

Add the possibility to specify from kernel configs MAXCPU value.
This patch is going to help in cases like mips flavours where you
want a more granular support on MAXCPU.

No MFC is previewed for this patch.

Tested by: pluknet
Approved by: re (kib)

- Remove the eintrcnt/eintrnames usage and introduce the concept of
sintrcnt/sintrnames which are symbols containing the size of the 2
tables.
- For amd64/i386 remove the storage of intr* stuff from assembly files.
This area can be widely improved by applying the same to other
architectures and likely finding an unified approach among them and
move the whole code to be MI. More work in this area is expected to
happen fairly soon.

No MFC is previewed for this patch.

Tested by: pluknet
Reviewed by: jhb
Approved by: re (kib)

Comment out KDB/DDB related options for N32 kernel.

DDB files have warnings in this ABI which will break the kernel build
unless Werror is ignored

Reported by: bz

Fix compilation of mips kernels adding apropriate #ifdefs after r224115.

Pointed out by: Luiz Otavio O Souza (lists.br gmail.com)

MIPS changes for Netlogic XLP support.

This patch adds support for the Netlogic XLP mips64 processors in
the common MIPS code. The changes are :

- Add CPU_NLM processor type
- Add cases for CPU_NLM, mostly were CPU_RMI is used.
- Update cache flush changes for CPU_NLM
- Add kernel build configuration files for xLP.

In collaboration with: Prabhath Raman <prabhathpr at netlogicmicro com>

Approved by: bz(re), jmallett, imp(mips)

Add MIPS platform files for Netlogic XLP SoC.

Processor, UART, PIC and Messaging Network code. Also add
sys/mips/nlm/hal for on-chip device registers.

In collaboration with: Prabhath Raman <prabhathpr at netlogicmicro com>

Approved by: bz(re), jmallett, imp(mips)

Support compiling MIPS elf trampoline with a different ABI.

Allow changing the trampoline ABI with makeoptions, this will allow
us to have a trampoline with a different ABI from the kernel.
Useful in cases where we have to boot a 64 bit kernel from a
bootloader which supports only 32 bit or vice versa.

Approved by: bz(re), jmallett, imp

The i8259 controller is initialized incorrectly on MALTA. It writes
mask bits to control register and control bits to mask register.

The former causes ICW1_RESET|ICW1_LTIM combination to be written to
control register, which on QEMU results in "level sensitive irq not
supported" error.

Submitted by: Robert Millan <rmh@debian.org>

Include device rt in RT305X config.

Approved by: adrian (mentor)

Revert it back. Too many changes for one commit.

Approved by: adrian (mentor)

Include device rt in RT305X config and little cleanup.

Approved by: adrian (mentor)

With retirement of cpumask_t and usage of cpuset_t for representing a
mask of CPUs, pc_other_cpus and pc_cpumask become highly inefficient.

Remove them and replace their usage with custom pc_cpuid magic (as,
atm, pc_cpumask can be easilly represented by (1 << pc_cpuid) and
pc_other_cpus by (all_cpus & ~(1 << pc_cpuid))).

This change is not targeted for MFC because of struct pcpu members
removal and dependency by cpumask_t retirement.

MD review by: marcel, marius, alc
Tested by: pluknet
MD testing by: marcel, marius, gonzo, andreast

When iterating over a paging queue, explicitly check for PG_MARKER, instead
of relying on zeroed memory being interpreted as an empty PV list.

Reviewed by: kib

Add a new option, OBJPR_NOTMAPPED, to vm_object_page_remove(). Passing this
option to vm_object_page_remove() asserts that the specified range of pages
is not mapped, or more precisely that none of these pages have any managed
mappings. Thus, vm_object_page_remove() need not call pmap_remove_all() on
the pages.

This change not only saves time by eliminating pointless calls to
pmap_remove_all(), but it also eliminates an inconsistency in the use of
pmap_remove_all() versus related functions, like pmap_remove_write(). It
eliminates harmless but pointless calls to pmap_remove_all() that were being
performed on PG_UNMANAGED pages.

Update all of the existing assertions on pmap_remove_all() to reflect this
change.

Reviewed by: kib

Remove duplicate header includes

etire the cpumask_t type and replace it with cpuset_t usage.

This is intended to fix the bug where cpu mask objects are
capped to 32. MAXCPU, then, can now arbitrarely bumped to whatever
value. Anyway, as long as several structures in the kernel are
statically allocated and sized as MAXCPU, it is suggested to keep it
as low as possible for the time being.

Technical notes on this commit itself:
- More functions to handle with cpuset_t objects are introduced.
The most notable are cpusetobj_ffs() (which calculates a ffs(3)
for a cpuset_t object), cpusetobj_strprint() (which prepares a string
representing a cpuset_t object) and cpusetobj_strscan() (which
creates a valid cpuset_t starting from a string representation).
- pc_cpumask and pc_other_cpus are target to be removed soon.
With the moving from cpumask_t to cpuset_t they are now inefficient
and not really useful. Anyway, for the time being, please note that
access to pcpu datas is protected by sched_pin() in order to avoid
migrating the CPU while reading more than one (possible) word
- Please note that size of cpuset_t objects may differ between kernel
and userland. While this is not directly related to the patch itself,
it is good to understand that concept and possibly use the patch
as a reference on how to deal with cpuset_t objects in userland, when
accessing kernland members.
- KTR_CPUMASK is changed and now is represented through a string, to be
set as the example reported in NOTES.

Please additively note that no MAXCPU is bumped in this patch, but
private testing has been done until to MAXCPU=128 on a real 8x8x2(htt)
machine (amd64).

Please note that the FreeBSD version is not yet bumped because of
the upcoming pcpu changes. However, note that this patch is not
targeted for MFC.

People to thank for the time spent on this patch:
- sbruno, pluknet and Nicholas Esborn (nick AT desert DOT net) tested
several revision of the patches and really helped in improving
stability of this work.
- marius fixed several bugs in the sparc64 implementation and reviewed
patches related to ktr.
- jeff and jhb discussed the basic approach followed.
- kib and marcel made targeted review on some specific part of the
patch.
- marius, art, nwhitehorn and andreast reviewed MD specific part of
the patch.
- marius, andreast, gonzo, nwhitehorn and jceel tested MD specific
implementations of the patch.
- Other people have made contributions on other patches that have been
already committed and have been listed separately.

Companies that should be mentioned for having participated at several
degrees:
- Yahoo! for having offered the machines used for testing on big
count of CPUs.
- The FreeBSD Foundation for having sponsored my devsummit attendance,
which has been instrumental.
- Sandvine for having offered offices and infrastructure during
development.

(I really hope I didn't forget anyone, if it happened I apologize in
advance).

Add support for True IDE mode to the Octeon CF driver. This mode is
signalled when the attribute address for the CF is 0 in the octeon
sysinfo structure. In this mode, the DATA port is 16-bits, but the
other ports are 8-bits, but on a 16-bit bus (so you have to access it
a short at a time, but only believe the lower byte). See the code for
more details on this slightly odd arrangement. I'm still not 100%
happy with the abstractions here on many levels (starting with the
globals for these settings, on down to no bus_space use, etc), but the
driver had these problems before the change.

Also, clean up the code a bit to make this support easier, and the
code a bit easier to read. I tried to follow existing style, but may
have missed a few spots. Add some comments.

Fix probe/attach routine to return a proper error for the simulator.

With this change, my EBH5200 eval board now recognizes the CF well
enough to boot to the login prompt. Before it would say it never
became ready. My CN3010-EVB-HS5 still boots properly. My older
CN3860-based board won't load the 64-bit kernel, either before or
after the change, and I didn't chase that down.

On multi-core, multi-threaded PPC systems, it is important that the threads
be brought up in the order they are enumerated in the device tree (in
particular, that thread 0 on each core be brought up first). The SLIST
through which we loop to start the CPUs has all of its entries added with
SLIST_INSERT_HEAD(), which means it is in reverse order of enumeration
and so AP startup would always fail in such situations (causing a machine
check or RTAS failure). Fix this by changing the SLIST into an STAILQ,
and inserting new CPUs at the end.

Reviewed by: jhb

Merge r221846 from largeSMP project branch:
Fix arguments passing to _long() version of atomic function for mips.

The native implementation is bogus in that regard and offers the same
problem solved for powerpc as r222198, but mips' guys just wanted a
small and self-contained patch for mips rather than rewriting the
whole support.

Reviewed by: art, imp
Tested by: gonzo
MFC after: 2 weeks

Move the ZERO_REGION_SIZE to a machine-dependent file, as on many
architectures (i386, for example) the virtual memory space may be
constrained enough that 2MB is a large chunk. Use 64K for arches
other than amd64 and ia64, with special handling for sparc64 due to
differing hardware.

Also commit the comment changes to kmem_init_zero_region() that I
missed due to not saving the file. (Darn the unfamiliar development
environment).

Arch maintainers, please feel free to adjust ZERO_REGION_SIZE as you
see fit.

Requested by: alc
MFC after: 1 week
MFC with: r221853

Use new option names for NFS client and NFS server

See UPDATING entry for 20110427

Fix GPIO_MAXPINS calculation for the AR71xx, AR724x, AR913x SoC.

Submitted by: Luiz Otavio O Souza <loos.br@gmail.com>

Add nvram2env (untested) and geom_map (tested) to PB92.

- Remove attempts to implement setting of BMCR_LOOP/MIIF_NOLOOP
(reporting IFM_LOOP based on BMCR_LOOP is left in place though as
it might provide useful for debugging). For most mii(4) drivers it
was unclear whether the PHYs driven by them actually support
loopback or not. Moreover, typically loopback mode also needs to
be activated on the MAC, which none of the Ethernet drivers using
mii(4) implements. Given that loopback media has no real use (and
obviously hardly had a chance to actually work) besides for driver
development (which just loopback mode should be sufficient for
though, i.e one doesn't necessary need support for loopback media)
support for it is just dropped as both NetBSD and OpenBSD already
did quite some time ago.
- Let mii_phy_add_media() also announce the support of IFM_NONE.
- Restructure the PHY entry points to use a structure of entry points
instead of discrete function pointers, and extend this to include
a "reset" entry point. Make sure any PHY-specific reset routine is
always used, and provide one for lxtphy(4) which disables MII
interrupts (as is done for a few other PHYs we have drivers for).
This includes changing NIC drivers which previously just called the
generic mii_phy_reset() to now actually call the PHY-specific reset
routine, which might be crucial in some cases. While at it, the
redundant checks in these NIC drivers for mii->mii_instance not being
zero before calling the reset routines were removed because as soon
as one PHY driver attaches mii->mii_instance is incremented and we
hardly can end up in their media change callbacks etc if no PHY driver
has attached as mii_attach() would have failed in that case and not
attach a miibus(4) instance.
Consequently, NIC drivers now no longer should call mii_phy_reset()
directly, so it was removed from EXPORT_SYMS.
- Add a mii_phy_dev_attach() as a companion helper to mii_phy_dev_probe().
The purpose of that function is to perform the common steps to attach
a PHY driver instance and to hook it up to the miibus(4) instance and to
optionally also handle the probing, addition and initialization of the
supported media. So all a PHY driver without any special requirements
has to do in its bus attach method is to call mii_phy_dev_attach()
along with PHY-specific MIIF_* flags, a pointer to its PHY functions
and the add_media set to one. All PHY drivers were updated to take
advantage of mii_phy_dev_attach() as appropriate. Along with these
changes the capability mask was added to the mii_softc structure so
PHY drivers taking advantage of mii_phy_dev_attach() but still
handling media on their own do not need to fiddle with the MII attach
arguments anyway.
- Keep track of the PHY offset in the mii_softc structure. This is done
for compatibility with NetBSD/OpenBSD.
- Keep track of the PHY's OUI, model and revision in the mii_softc
structure. Several PHY drivers require this information also after
attaching and previously had to wrap their own softc around mii_softc.
NetBSD/OpenBSD also keep track of the model and revision on their
mii_softc structure. All PHY drivers were updated to take advantage
as appropriate.
- Convert the mebers of the MII data structure to unsigned where
appropriate. This is partly inspired by NetBSD/OpenBSD.
- According to IEEE 802.3-2002 the bits actually have to be reversed
when mapping an OUI to the MII ID registers. All PHY drivers and
miidevs where changed as necessary. Actually this now again allows to
largely share miidevs with NetBSD, which fixed this problem already
9 years ago. Consequently miidevs was synced as far as possible.
- Add MIIF_NOMANPAUSE and mii_phy_flowstatus() calls to drivers that
weren't explicitly converted to support flow control before. It's
unclear whether flow control actually works with these but typically
it should and their net behavior should be more correct with these
changes in place than without if the MAC driver sets MIIF_DOPAUSE.

Obtained from: NetBSD (partially)
Reviewed by: yongari (earlier version), silence on arch@ and net@

Add in the work-around for the AR8316 gige switch PHY.

This allows the ethernet to function on the PB92 board.

Submitted by: Luiz Otavio O Souza

Some AR724x PCIe fixes, which should wrap up the first round
of endian-ness issues with the AR724x.

From Luiz:

* Fix the bus space tag used so endian-ness is correctly handled;
* Only do the workaround for the AR7240; AR7241/AR7242 (PB92)
don't require this

From me:

* Add a read flush from openwrt

Submitted by: Luiz Otavio O Souza

An example configuration file for the Atheros PB92.

TODO:

* pcie doesn't yet work properly, I'm working on it.
* if_arge doesn't work - it's likely the phy mask is wrong; again I'm working on it.

The AR724x SoC's require the irq status line to be acked/cleared.

This allows console IO to occur correctly once the kernel is in multi-user
mode.

Submitted by: Luiz Otavio O Souza

Call the DDR FIFO flush method when IP2 interrupts occur.

Flip off debugging for now.

Add some initial PCIe bridge support for the AR724x chipsets.

This is reported to work on the AR7240 based Ubiquiti Rocket M5
but I haven't tested it on that hardware. I also don't yet have
it fully working on the AR7242 based development board here;
probe/attach functions but the register space resource looks like
the endian-ness is wrong (0x10000000 instead of 0x00001000).o

Further digging will be required.

Submitted by: Luiz Otavio O Souza

In preparation for the AR724x PCIe bus code, make the AR71xx PCI bus
glue require 'device ar71xx_pci' .

Users of the AR71xx board configs will now require this for functioning
PCI:

device pci
device ar71xx_pci

Add a missing DDR FIFO method for the ar71xx.

Change rman_manage_region() to actually honor the rm_start and rm_end
constraints on the rman and reject attempts to manage a region that is out
of range.
- Fix various places that set rm_end incorrectly (to ~0 or ~0u instead of
~0ul).
- To preserve existing behavior, change rman_init() to set rm_start and
rm_end to allow managing the full range (0 to ~0ul) if they are not set by
the caller when rman_init() is called.

Tidy up the naming of the ip2 DDR flush routine, and add an inline
accessor method (which is currently unused) in there.

Add the watchdogs patting during the (shutdown time) disk syncing and
disk dumping.
With the option SW_WATCHDOG on, these operations are doomed to let
watchdog fire, fi they take too long.

I implemented the stubs this way because I really want wdog_kern_*
KPI to not be dependant by SW_WATCHDOG being on (and really, the option
only enables watchdog activation in hardclock) and also avoid to
call them when not necessary (avoiding not-volountary watchdog
activations).

Sponsored by: Sandvine Incorporated
Discussed with: emaste, des
MFC after: 2 weeks

Re-enable the wireless build parameters for the AR9130 WMAC.

* enable 11n
* add ath_ahb so the AHB<->ath glue is linked in
* disable descriptor order swapping, it isn't needed here
* disable interrupt mitigation, it isn't supported here

Add the IP2 DDR flush handlers.

These aren't yet used in the interrupt handler path but should be.

- Add shim to simplify migration to the CAM-based ATA. For each new adaX
device in /dev/ create symbolic link with adY name, trying to mimic old ATA
numbering. Imitation is not complete, but should be enough in most cases to
mount file systems without touching /etc/fstab.
- To know what behavior to mimic, restore ATA_STATIC_ID option in cases
where it was present before.
- Add some more details to UPDATING.

Switch the GENERIC kernels for all architectures to the new CAM-based ATA
stack. It means that all legacy ATA drivers are disabled and replaced by
respective CAM drivers. If you are using ATA device names in /etc/fstab or
other places, make sure to update them respectively (adX -> adaY,
acdX -> cdY, afdX -> daY, astX -> saY, where 'Y's are the sequential
numbers for each type in order of detection, unless configured otherwise
with tunables, see cam(4)).

ataraid(4) functionality is now supported by the RAID GEOM class.
To use it you can load geom_raid kernel module and use graid(8) tool
for management. Instead of /dev/arX device names, use /dev/raid/rX.

We don't need to call EOWRITE4(sc, EHCI_USBINTR, 0) directly from each EHCI
bus driver at detach, hence ehci_detach() does exactly this since r199718.

Submitted by: Luiz Otavio O Souza
MFC after: 7 days
Approved by: thompsa (mentor)

if_arge has had a strange bug that only appears during high traffic
levels. TX would hang, RX wouldn't. A bit of digging showed the interface
send queue was full, but IFF_DRV_OACTIVE was clear and the hardware TX
queue was empty.

It turns out that there wasn't a check to drain the interface send
queue once hardware TX had completed, so if the interface send queue
had filled up in the meantime, subsequent packets would be dropped
by the higher layers and if_start (and thus arge_start()) would never
be called.

The fix is simple - call arge_start_locked() in the software interrupt
handler after the hardware TX queue has been handled or a TX underrun
occured. This way the interface send queue gets drained.

* Add some more debugging to if_arge
* Make doubly sure that IFF_DRV_OACTIVE is set if the hardware TX queue is full

Put the ARGE_DEBUG behind a kernel config option.

Begin fleshing out a functioning debugging setup for if_arge.

I'm seeing TX hangs when doing large amounts of TX traffic;
an interface reset fixes it. This will hopefully help me identify
why.

- Correct EHCI interrupt disabling at detach.

Submitted by: Luiz Otavio O Souza
MFC after: 7 days
Approved by: thompsa (mentor)

Import the initial CPU support for the MIPS RALink RT305x SoC.
This is a MIPS4KC CPU with various embedded peripherals, including
wireless and ethernet support.

This commit includes the platform, UART, ethernet MAC and GPIO support.
The interrupt-driven GPIO code is disabled for now pending GPIO changes
from the submitter.

Submitted by: Aleksandr Rybalko <ray@dlink.ua>

Add in some missing flags in the EHCI initialisation code,
needed to get USB working on the AR913x/AR724x.

Commit configuration files for the AR913x SoC and the TP-Link WR-1043ND.

The AR91XX_BASE{,.hints} files define all the common parameters for
the AR913x SoC systems.

The TP-1043ND config file adds local parameters.

A handful of the openwrt devices use a MAC address that's at a hard-coded
offset in the flash.

Some devices (eg the TPLink WR-1043ND) don't have a flash environment
partition which can be queried for the current board settings.

This particular workaround allows for image creators to use a hint
to set the base MAC address. For example:

hint.arge.0.eeprommac=0x1f01fc00

Break out the ath PCI logic into a separate device/module.

Introduce the AHB glue for Atheros embedded systems. Right now it's
hard-coded for the AR9130 chip whose support isn't yet in this HAL;
it'll be added in a subsequent commit.

Kernel configuration files now need both 'ath' and 'ath_pci' devices; both
modules need to be loaded for the ath device to work.

Implement AR724x USB initialisation code.

This (again) still requires an offset for the AR913x/AR724x before USB will
function.

Submitted by: Luiz Otavio O Souzau <loos.br@gmail.com>

The previous commit didn't completely rename this to what it should be.

Refactor out the ar71xx mac address code into something that's
just for Redboot.

At some point we're going to need to build options for different
boot environments - for example, the UBoot setups I've seen simply
have the MAC address hard-coded at a fixed location in flash.
The OpenWRT support simply yanks the if_arge MAC directly from that
in code, rather than trying to find a uboot environment to pull it
from.

Add an option - AR71XX_REALMEM - which overrides the amount of
memory detected from Redboot, or overrides the "otherwise" case
if no Redboot information was found.

Some AR71XX platforms don't use Redboot (eg TP-LINK devices using
UBoot; some later Ubiquiti devices which apparently also use
UBoot) and at least one plain out lies - the Ubiquiti LS-SR71A
Redboot says there's 16mb of RAM when in fact there's 32mb.

A more "clean" solution will be needed at a later date.

Add some missing flags needed for AR913x/AR724x USB to correctly operate.

The AR913x/AR724x USB lives at a different offset to the AR71xx
USB, so this needs to be either adjusted for in a subsequent
commit, or updated in hints for kernels compiled for those
platforms.

Submitted by: Luiz Otavio O Souzau <loos.br@gmail.com>

For now remove options FLOWTABLE from the remaining GENERIC kernel
configurations and make it opt-in for those who want it. LINT will
still build it.

While it may be a perfect win in some scenarios, it still troubles users
(see PRs) in general cases. In addition we are still allocating resources
even if disabled by sysctl and still leak arp/nd6 entries in case of
interface destruction.

Discussed with: qingli (2010-11-24, just never executed)
Discussed with: juli (OCTEON1)
PR: kern/148018, kern/155604, kern/144917, kern/146792
MFC after: 2 weeks

o) Set MAC addresses starting at the MAC base for all management ports, not
just the MR-730.
o) Start MAC addresses for the non-management ports after the management ports.

Submitted by: Bhanu Prakash (with modifications)

o) Tear down receive interrupt on detach.

Submitted by: Bhanu Prakash

o) Clean up FPA pools on module unload.
o) Allocate output buffer pool based on available output queues.

Submitted by: Bhanu Prakash (with modifications)

o) Properly size caches and TLB on Octeon.
o) Make COP0_SYNC do nothing on Octeon, which is fully interlocked.

Submitted by: Bhanu Prakash (with modifications)

* Add wireless MAC reset, in prep for bringing over AR9130 support.
* Whilst I'm here, reformat to fit inside 80 characters.

Add the missing AR724x DDR flush routines for if_arge0.

Submitted by: Luiz Otavio O Souza

Fix the TX underrun status reset; remove a now unused variable.

Submitted by: Luiz Otavio O Souza

Commit FIFO configuration fixes from OpenWRT. This fixes performance
issues with if_arge on the AR913x and AR724x.

Reference: https://dev.openwrt.org/ticket/6754
Submitted by: Luiz Otavio O Souza

Extend struct sysvec with new method sv_schedtail, which is used for an
explicit process at fork trampoline path instead of eventhadler(schedtail)
invocation for each child process.

Remove eventhandler(schedtail) code and change linux ABI to use newly added
sysvec method.

While here replace explicit comparing of module sysentvec structure with the
newly created process sysentvec to detect the linux ABI.

Discussed with: kib

MFC after: 2 Week

Increase NKPT in case of n32 and n64 to support more physical memory.

On n32, vm_page_startup() needs more virtual mem to map vm_page structs.
The new value of 256 will allow us to support 16GB RAM.

Use new thread's stack to invoke pmap_activate in cpu_switch()

Restore the SP from the new thread's PCB before calling pmap_activate.
Remove some old FIXME comments.

Use correct types and fromats for physical address

- Use vm_paddr_t for pa in pmap_steal_memory()
- Use uintmax_t and %jx to ensure that physical address are printed
correctly in cpu_startup() and pmap_bootstrap()

Use vm_offset_t for virtual addresses.

The unsigned int type used now is incorrect in n64 compilation.

Fix typos - remove duplicate "the".

PR: bin/154928
Submitted by: Eitan Adler <lists at eitanadler.com>
MFC after: 3 days

Remove pmap fields that are either unused or not fully implemented.

Discussed with: kib

The Lanner MR-730 uses the first two MACs at its MAC base for the 10/100
management ports, and gigabit ports start at an offset of 2 from the MAC
base.

Allow the platform code to return a bitmask of running cores rather than just
a number of cores, this allows for a sparse set of CPUs. Implement support
for sparse core masks on Octeon.

XXX jeff@ suggests that all_cpus should include cores that are offline or
running other applications/OSes, so the platform API should be further
extended to allow us to set all_cpus to include all cores that are
physically-present as opposed to only those that are running FreeBSD.

Submitted by: Bhanu Prakash (with modifications)
Reviewed by: jchandra
Glanced at by: kib, jeff, jhb

o) Cavium Octeon doesn't need nop barriers.
o) Have mips_wblush just do syncw, not sync on Cavium Octeon.
o) Add support for reading and writing some Octeon-specific registers.
NB: Some of these are not entirely Octeon-specific.

Submitted by: Bhanu Prakash

Replace __LP64__ with __mips_n64. This partly reverts r217147.

Requested by: jmallett, imp
Approved by: kib (mentor)

Put the general logic for being a CPU hog into a new function
should_yield(). Use this in various places. Encapsulate the common
case of check-and-yield into a new function maybe_yield().

Change several checks for a magic number of iterations to use
should_yield() instead.

MFC after: 1 week

Implement sf_buf using direct map (XKPHYS) in MIPS n64.

- Provide trivial implementation of sf_buf_alloc(), sf_buf_free(),
sf_buf_kva() and sf_buf_page() using direct map for n64.
- uio_machdep.c - use macros so that the direct map will be used in
case of n64.

Reviewed by: imp (earlier version)
Obtained from: jmallett (user/jmallett/octeon)

Make MSGBUF_SIZE kernel option a loader tunable kern.msgbufsize.

Submitted by: perryh pluto.rain.com (previous version)
Reviewed by: jhb
Approved by: kib (mentor)
Tested by: universe

If there is no WQE available for a packet that needs segmentation, drop it
and return.

Remove some compile-time options from the driver, particularly async IOBDMA
support which is unused on FreeBSD and which complicates working on the
code now, can easily be added back later by someone determined to use it.

Use simplified ldscripts rather than specific ones

Move 'cpu CPU_RMI' to std.xlr, this is common for all XLR cpus.

Re-format XLR configuartion files and remove obsolete options.

ldscript and conf cleanup for MIPS

- Remove sys/conf/ldscript.mips.64 and sys/conf/ldscript.mips.n32 and use
ldscript.mips for all ABIs. The default OUTPUT_FORMAT of the toolchain
is correct.
- Remove LDSCRIPT_NAME entires from XLR n32 and n64 conf files.
- Remove TARGET_BIG_ENDIAN from XLR conf files.
- Fix machine entry in XLRN32

Fixes in the XLR platform code

- Fix bug in pic.h, assign reg variable, before return.
- In xlr_pci.c, need to ignore the result of mmio read.

Fix build by changing format for size_t to %jd

Introduce signed and unsigned version of CTLTYPE_QUAD, renaming
existing uses. Rename sysctl_handle_quad() to sysctl_handle_64().

Fix format of physical addresses; this fixes the n32 build.

Don't do a device_identify to add uart0, it's already hinted.

Reported by: imp

For architectures not using direct map , and requiring real KVA page for
sf buf allocation, use wakeup() instead of wakeup_one() to notify sf
buffer waiters about free buffer.

sf_buf_alloc() calls msleep(PCATCH) when SFB_CATCH flag was given,
and for simultaneous wakeup and signal delivery, msleep() returns
EINTR/ERESTART despite the thread was selected for wakeup_one(). As
result, we loose a wakeup, and some other waiter will not be woken up.

Reported and tested by: az
Reviewed by: alc, jhb
MFC after: 1 week

Remove empty dev_mem_md_init() stubs.

Save the CPU model, the board and the CPU clock rate so they are reported by
the approrpiate sysctl.

Kill redundant cpu line

Make cpu_model public (otherwise there's no way to set it) and bump it
to 80 characters.
Add hw.board to export board information, if known, from the mips kernel.

Add reader/writer lock around mem_range_attr_get() and mem_range_attr_set().
Compile sys/dev/mem/memutil.c for all supported platforms and remove now
unnecessary dev_mem_md_init(). Consistently define mem_range_softc from
mem.c for all platforms. Add missing #include guards for machine/memdev.h
and sys/memrange.h. Clean up some nearby style(9) nits.

MFC after: 1 month

Support for 64 bit PTEs on n32 and n64 compilation.

In n32 and n64, add support for physical address above 4GB by having
64 bit page table entries and physical addresses. Major changes are:
- param.h: update PTE sizes, masks and shift values to support 64 bit PTEs.
- param.h: remove DELAY(), mips_btop(same as atop), mips_ptob (same as
ptoa), and reformat.
- param.h: remove casting to unsigned long in trunc_page and round_page
since this will be used on physical addresses.
- _types.h: have 64 bit __vm_paddr_t for n32.
- pte.h: update TLB LO0/1 access macros to support 64 bit PTE
- pte.h: assembly macros for PTE operations.
- proc.h: md_upte is now 64 bit for n32 and n64.
- exception.S and swtch.S: use the new PTE macros for PTE operations.
- cpufunc.h: TLB_LO0/1 registers are 64bit for n32 and n64.
- xlr_machdep.c: Add memory segments above 4GB to phys_avail[] as they are
supported now.

Reviewed by: jmallett (earlier version)

Cleanup physical address and PTE types on MIPS.

1. Use vm_paddr_t for physical addresses.

There are a few places in the MIPS platform code where vm_offset_t is
used for physical addresses, change these to use vm_paddr_t:
- phys_avail[], physmem_desc[] arrays
- pmap_mapdev(), page_is_managed(), is_cacheable_mem() pmap_map() args
- local variables of various pmap functions

2. Change init_pte_prot() return from int to pt_entry_t, as this can be
64 bit when using 64 bit TLB entries.

3. Update printing of pt_entry_t and of vm_paddr_t to use 'j' format with
uintmax_t. This will be useful later if we plan to use 64bit phsical addr
on 32 bit n32 compilation.

Reviewed by: imp

The message ring interrupt needs to be enabled for all cpus, not just
the ones which run the message ring handler.

Some bits of the interrupt mask are part of the status register which is
saved with the process context, and these bits are initialized from the
cpu on which the process is created. This means that all the processes
should have the same value for these interrupt mask bits, so that the
interrupt mask remains the same regardless of what thread is scheduled
on the cpu.

Submitted by: Sriram Gorti (srgorti at netlogicmicro dot com)

Remove unneeded includes of <sys/linker_set.h>. Other headers that use
it internally contain nested includes.

Reviewed by: bde

Initialize PCIe buses and add preliminary support for 64-bit BARs.

Count output bytes and packets.

Shorten device name so it fits into vmstat -i.

o) Free mbufs in error cases.
o) Clear/acknowledge receive interrupt at end of thread. This gives the
management interfaces performance on the order of 100Mbps rather than
the previous level of 10Mbps on my MR-730.

o) Expand the CIU driver to be aware of newly-allocated parts of the IRQ range.
o) Add 'octm', a trivial driver for the 10/100 management ports found on some
Octeon systems.
o) Make the Simple Executive's management port helper routines compile on
FreeBSD (namely by not doing math on void pointers.)
o) Add a cvmx_mgmt_port_sendm routine to the Simple Executive to send an mbuf
so there is only one copy in the transmit path, rather than having to first
copy the mbuf to an intermediate buffer and then copy that to the Simple
Executive's transmit ring.
o) Properly work out MII addresses of management ports on the Lanner MR-730.
XXX The MR-730 also needs some patches to the MII read/write routines, but
this is sufficient for now. Media detection will be fixed in the future
when I can spend more time reading the vendor-supplied patches.

Now that we correctly enable rx interrupts on all cores, performance has gotten
quite awful, because e.g. 4 packets will come in and get processed on 4
different cores at the same time, really battling with the TCP stack quite
painfully. For now, just run one task at a time.

This gets performance up in most cases to where it was before the correctness
fixes that got interrupts to run on all cores (except in high-load TCP transmit
cases where all we're handling receive for is ACKs) and in some cases it's
better now. What would be ideal would be to use a more advanced interrupt
mitigation strategy and possibly to use different workqueue groups per port for
multi-port systems, and so on, but this is a fine stopgap.

o) Remove some unused local definitions of IP protocol numbers.
o) Use CPU_FOREACH rather than a miscrafted for loop with an off-by-one to
enable workq interrupts on all cores.

Move repeated MAXSLP definition from machine/vmparam.h to sys/vmmeter.h.
Update the outdated comments describing MAXSLP and the process
selection algorithm for swap out.

Comments wording and reviewed by: alc

Add missing ar91xx definition for the WMAC reset control.

White space changes to align comments. The mips and powerpc _inttypes.h
are now exactly the same.

Approved by: kib (mentor)

Rename PRIreg helper macro to PRIptr to better reflect its use. Registers
and pointers don't always have the same size, e.g. the __mips_n32 ABI
(ILP32) has 64 bit registers but 32 bit pointers.

On mips introduce PRIptr to fix the format specifier for (u)intptr_t.

Prefix PRI64 and PRIptr with underscores because macro names starting with
PRI[a-zX] are reserved for future use.

Approved by: kib (mentor)

On mixed 32/64 bit architectures (mips, powerpc) use __LP64__ rather than
architecture macros (__mips_n64, __powerpc64__) when 64 bit types (and
corresponding macros) are different from 32 bit. [1]

Correct the type of INT64_MIN, INT64_MAX and UINT64_MAX.

Define (U)INTMAX_C as an alias for (U)INT64_C matching the type definition
for (u)intmax_t. Do this on all architectures for consistency.

Suggested by: bde [1]
Approved by: kib (mentor)

On 32 bit architectures define (u)int64_t as (unsigned) long long instead
of (unsigned) int __attribute__((__mode__(__DI__))). This aligns better
with macros such as (U)INT64_C, (U)INT64_MAX, etc. which assume (u)int64_t
has type (unsigned) long long.

The mode attribute was used because long long wasn't standardised until
C99. Nowadays compilers should support long long and use of the mode
attribute is discouraged according to GCC Internals documentation.

The type definition has to be marked with __extension__ to support
compilation with "-std=c89 -pedantic".

Discussed with: bde
Approved by: kib (mentor)

Fix types of some values in machine/_limits.h.

On some architectures UCHAR_MAX and USHRT_MAX had type unsigned int.
However, lacking integer suffixes for types smaller than int, their type
should correspond to that of an object of type unsigned char (or short)
when used in an expression with objects of type int. In that case unsigned
char (short) are promoted to int (i.e. signed) so the type of UCHAR_MAX and
USHRT_MAX should also be int.

Where MIN/MAX constants implicitly have the correct type the suffix has
been removed.

While here, correct some comments.

Reviewed by: bde
Approved by: kib (mentor)

Remove unused support for 64 bit long on 32 bit architectures.

It was used mainly to discover and fix some 64-bit portability problems
before 64-bit arches were widely available.

Discussed with: bde
Approved by: kib (mentor)

Add AT_STACKPROT elf aux vector. Will be used to inform rtld about the
initial stack protection set by the kernel image activator.

Remove bogus usage of INTR_FAST. "Fast" interrupts are now indicated by
registering a filter handler rather than a threaded handler. Also remove
a bogus use of INTR_MPSAFE for a filter.

Correct an 8-year-old typo which reliably leads to typo after typo today:

The macros here for generating coprocessor 0 accessors are named like:

MIPS_RDRW32_COP0

That macro would produce mips_rd_<register>() and mips_wr_<register>()
inlines to access the specified register by name from C. The problem is that
the R and the W were swapped in the macros originally; it was meant to be named
RDWR because it generated mips_rd_* and mips_wr_* functions, but was instead
spelled RDRW, which nobody should be expected to get right by anything other
than copy and paste.

It's too many consonants in a row to keep straight anyway, so just prefer e.g.:

MIPS_RW32_COP0

While here, add a missing #undef.

Remove OCTEON1-32. Since moving to the Cavium SDK, we've lost our support for
running an o32 kernel safely, and would have to add interrupt disabling and
reenabling to a bunch of macros in the Simple Executive sources to support it.

The only reason one would run an o32 kernel on Octeon would be to run o32 world,
which is better worked towards by adding o32 binary compatibility to n64 kernels
along with, eventually, supporting multilib systems so o32 binaries can run
alongside n32 and n64 ones.

Discussed with: imp

o) Add MIPS_COP_0_EXC_PC accessors to <machine/cpufunc.h>.
o) Make the octeon_wdog driver work on multi-CPU systems and to also print more
information on NMI that may aid debugging. Simplify and clean up internal
API and structure.

o) Unmask Central Interrupt Unit interrupts on APs, too.
o) There's no good reason to make the low bits of the ebase the core
number. While per-CPU exception bases would be nice, for now we just
need to make ebase the same on all cores.

o) When trying to determine whether the pcpu pointer is a managed address, check
not just that it is greater than the minimal kernel virtual address, but also
that it is less than the maximal kernel virtual address. On n64 kernels, the
pcpup comes out of a direct-mapped address that, with an unsigned compare, is
rather greater than the minimal kernel virtual address.
o) Turn the panic if interrupts are disabled in cpu_idle into a KASSERT since on
other architectures it's behind INVARIANTS anyway.
o) Add a check that not all interrupts are masked, too.
o) Add cpu_idleclock() and cpu_activeclock() use to cpu_idle as is done on other
architectures.

When allocating memory from bootmem for the kernel to use, try to leave about
2MB of memory in the bootmem allocator for the SDK to use internally at a later
point. It'd be nice if there were some functions we could call before
allocating memory to let various facilities reserve some memory, but for now
this seems sufficient. Previously some unfortunate systems could give up all
(or at least most) of their memory to the kernel from bootmem, and then
allocating command queues for packet output and the like would fail later in
the boot process (which in turn would lead to crashes even later.)

Reported by: kan

Include std.sentry5 from the SENTRY5 kernel configuration. This was
apparently missed in r215270.

o) Add support for the Lanner MR-321X/MR-325, which is just a modified MR-320.
o) On the Lanner MR-730, disable PCIe lane swap, per vendor.

- include argument should be in quotes

Make the ELF trampoline binary ELF executable (and do some cleanup).

- Remove the -shared flag for the trampoline binary, generate an
ELF executable instead of a shared object.
- No need to generate tmphack.S, move the code to sys/mips/mips/inckern.S
- No need generate opt_kernname.h, KERNNAME can be passed with -D

Reviewed by: gonzo, imp

Updates for I2C devices on XLR engg boards.

- ds1374u : use multi-byte write.
- at24co2n, max6657: remove mutex, iicbus has the necessary locking.

Submitted by: Sreekanth M. S. (kanthms at netlogicmicro com)

fix mips build breakage introduced in r216375: atomic_store_int doesn't exists

1) 32-bit assignment are expected to always be atomic.
2) Release/acquire memory barrier semantics doesn't seem to be needed here.
So a simple assignment can be used.

Remove unused port_set_counter() while here, it also used to mis-use
atomic_set_int().

Reported by: jhb
Pointyhat to: avg
MFC after: 3 weeks

I2C drivers for XLR/XLS processors.

- Major update to xlr_i2c.c: do multi-byte ops correctly, remove unnecessary
code, add mutex to protect bus operations, style(9) fixes.
- Drivers for I2C devices on XLR/XLS engineering boards, ds1374u RTC, max6657
temparature sensor and at24co2n EEPROM.

Submitted by: Sreekanth M. S. (kanthms at netlogicmicro com)

fix atomic_set_xxx misuse in rge

It seems that atomic_set_xxx and atomic_store_xxx were confused.

Reviewed by: jhb (general issue)
MFC after: 3 weeks

Fix compilation when DEBUG option is enabled.

- remove unused code in mips/rmi/xlr_pci.c
- remove unused variable in mips/rmi/dev/nlge/if_nlge.c
- fix reference to old function in mips/mips/pmap.c

Reported by: Prabhath Raman (prabhath at netlogicmicro com)

- dump_avail layout should be sequence of [start, end)
pairs, not <start, size>.

Spotted by: alc@

- Populate dump_avail with proper values from phys_avail

swi_vm() for mips.

UMA_MD_SMALL_ALLOC for mips.

Implement uma_small_alloc() and uma_small_free() for mips that allocates
pages from direct mapped memory. Uses the same mechanism as the page table
page allocator, so that we allocate from KSEG0 in 32 bit, and from XKPHYS
on 64 bit.

Reviewed by: alc, jmallett

Fix double ;;

1. Fix off by one errors in calls to MIPS_DIRECT_MAPPABLE, reported by alc@
2. Remove unnecessary #defines from vmparam.h

Submitted by: alc (2)
Reviewed by: alc (1)

Fixup for r216141, dump_add_page needs to be non-static now.
Add it to sys/mips/include/md_var.h, make dump_drop_page non-static too
for completeness.

Revert r216134. This checkin broke platforms where bus_space are macros:
they need to be a single statement, and do { } while (0) doesn't work in this
situation so revert until a solution can be devised.

Disallow passing in a count of zero bytes to the bus_space(9) functions.

Passing a count of zero on i386 and amd64 for [I386|AMD64]_BUS_SPACE_MEM
causes a crash/hang since the 'loop' instruction decrements the counter
before checking if it's zero.

PR: kern/80980
Discussed with: jhb

Add interrupt describing and binding to CIU.

Run all poll requests through a single function that can either do the generic
link state polling or media-specific ones, while avoidiing changing link state
on interfaces that use miibus; this substantially speeds up link time on
interface (re)initialization.

Display some Octeon 2 features and a feature for distinguishing between PCIe
implementations.

Don't free the work queue entry that we're using to hold the scatter-gather
list on exit from the transmit path. The scatter-gather list itself can be
asynchronously DMAed to the transmit hardware, and we could actually lock up
the transmitter if any of a number of races around this were lost.

Instead, let the PKO free the scatter-gather list when it is done with it, and
use the "i" bit in each segment of the packet to avoid having them go into the
FPA.

This fixes an unrecoverable transmit stall under transmit load.

MFC after: 3 days

Merge Cavium Octeon SDK 2.0 Simple Executive; this brings some fixes and new
facilities as well as support for the Octeon 2 family of SoCs.

XXX Note that with our antediluvian assembler, we can't support some Octeon 2
instructions and fall back to using the old ones instead.

- Add watchdog driver for Cavium Octeon. At the moment only
UP systems are supported.

o) Remove some commented out or unimplemented code.
o) Remove some options that are configurable on Linux but not FreeBSD.
o) Centralize open/poll/stop routines for XAUI and SGMII and use the common
uninit routine directly rather than providing a wrapper for it. The init
functions for these interfaces are now identical and the common init routine
could merge in setting those function pointers except that some hardware
seems to use no open/poll/stop method?

Fix issue noted by alc while reviewing r215938:
The current implementation of vm_page_alloc_freelist() does not handle
order > 0 correctly. Remove order parameter to the function and use it
only for order 0 pages.

Submitted by: alc

Set MACHINE_ARCH based on ABI and endianness.

Reviewed by: imp

Remove unused and broken code to implement POW send and POW-only devices; a
separate POW driver makes more sense, generally.

Use if_transmit to avoid ifq locking in transmit path.

Use if_transmit method instead of if_start, this avoids the interface queue lock.
Use the hardware IP checksum verification on receive.

Submitted by: Sriram Gorti (srgorti at netlogicmicro dot com)

After some off-list discussion, revert a number of changes to the
DPCPU_DEFINE and VNET_DEFINE macros, as these cause problems for various
people working on the affected files. A better long-term solution is
still being considered. This reversal may give some modules empty
set_pcpu or set_vnet sections, but these are harmless.

Changes reverted:

------------------------------------------------------------------------
r215318 | dim | 2010-11-14 21:40:55 +0100 (Sun, 14 Nov 2010) | 4 lines

Instead of unconditionally emitting .globl's for the __start_set_xxx and
__stop_set_xxx symbols, only emit them when the set_vnet or set_pcpu
sections are actually defined.

------------------------------------------------------------------------
r215317 | dim | 2010-11-14 21:38:11 +0100 (Sun, 14 Nov 2010) | 3 lines

Apply the STATIC_VNET_DEFINE and STATIC_DPCPU_DEFINE macros throughout
the tree.

------------------------------------------------------------------------
r215316 | dim | 2010-11-14 21:23:02 +0100 (Sun, 14 Nov 2010) | 2 lines

Add macros to define static instances of VNET_DEFINE and DPCPU_DEFINE.

Apply the STATIC_VNET_DEFINE and STATIC_DPCPU_DEFINE macros throughout
the tree.

Remove the 'machine mips' from DEFAULTS. Put the proper 'machine mips
mipsel' or 'machine mips mipseb' into the config file (with a few 64's
tossed in for good measure). This will let us build the proper
kernels with different worlds as part of make universe.

Switch to the new network driver nlge, the old rge driver is deprecated now.

rename SWARM_COMMON to std.SWARM so universe doesn't try to build it standalone

- Remove <machine/mutex.h>. Most of the headers were empty, and the
contents of the ones that were not empty were stale and unused.
- Now that <machine/mutex.h> no longer exists, there is no need to allow it
to override various helper macros in <sys/mutex.h>.
- Rename various helper macros for low-level operations on mutexes to live
in the _mtx_* or __mtx_* namespaces. While here, change the names to more
closely match the real API functions they are backing.
- Drop support for including <sys/mutex.h> in assembly source files.

Suggested by: bde (1, 2)

Fix typos.

PR: bin/148894
Submitted by: olgeni

- Provide more registers for GDB

- Add minidump support for FreeBSD/mips

Adjust the order of operations in spinlock_enter() and spinlock_exit() to
work properly with single-stepping in a kernel debugger. Specifically,
these routines have always disabled interrupts before increasing the nesting
count and restored the prior state of interrupts after decreasing the nesting
count to avoid problems with a nested interrupt not disabling interrupts
when acquiring a spin lock. However, trap interrupts for single-stepping
can still occur even when interrupts are disabled. Now the saved state of
interrupts is not saved in the thread until after interrupts have been
disabled and the nesting count has been increased. Similarly, the saved
state from the thread cannot be read once the nesting count has been
decreased to zero. To fix this, use temporary variables to store interrupt
state and shuffle it between the thread's MD area and the appropriate
registers.

In cooperation with: bde
MFC after: 1 month

Don't attach the PCI bus driver if the board we're being run on has PCIe. The
two are mutually-exclusive on Octeon.

Declare the CF GEOM class so that g_modevent will get called, the class will
be recorded, etc. This fixes libgeom on Octeon.

Network driver updates
- Fix network driver issue on a XLS eval board (major# 8).
- Fix issue uncovered by r213475 in check for XGMII

Submitted by: Sriram Gorti (srgorti at netlogicmicro dot com)

On uniprocessor, warn and fixup hardware cpu mask if more than on CPU
is enabled by the bootloader.

Converted the remainder of the NIC drivers to use the mii_attach()
introduced in r213878 instead of mii_phy_probe(). Unlike r213893 these
are only straight forward conversions though.

Reviewed by: yongari

Convert the PHY drivers to honor the mii_flags passed down and convert
the NIC drivers as well as the PHY drivers to take advantage of the
mii_attach() introduced in r213878 to get rid of certain hacks. For
the most part these were:
- Artificially limiting miibus_{read,write}reg methods to certain PHY
addresses; we now let mii_attach() only probe the PHY at the desired
address(es) instead.
- PHY drivers setting MIIF_* flags based on the NIC driver they hang
off from, partly even based on grabbing and using the softc of the
parent; we now pass these flags down from the NIC to the PHY drivers
via mii_attach(). This got us rid of all such hacks except those of
brgphy() in combination with bce(4) and bge(4), which is way beyond
what can be expressed with simple flags.

While at it, I took the opportunity to change the NIC drivers to pass
up the error returned by mii_attach() (previously by mii_phy_probe())
and unify the error message used in this case where and as appropriate
as mii_attach() actually can fail for a number of reasons, not just
because of no PHY(s) being present at the expected address(es).

Reviewed by: jhb, yongari

Keep polling at 50hz as long as link state is changing.

o) Make it possible to attach a PHY directly to an octe device rather than
using miibus, since for some devices that use multiple addresses on the bus,
going through miibus may be unclear, and for devices that are not standard
MII PHYs, miibus may throw a fit, necessitating complicated interfaces to
fake the interface that it expects during probe/attach.
o) Make the mv88e61xx SMI interface in octe attach a PHY directly and fix some
mistakes in the code that resulted from trying too hard to present a nice
interface to miibus.
o) Add a PHY driver for the mv88e61xx. If attached (it is optional in kernel
compiles so the default behavior of having a dumb switch is preserved) it
will place the switch in a VLAN-tagging mode such that each physical port
has a VLAN associated with it and interfaces for the VLANs can be created to
address or bridge between them.
XXX It would be nice for this to be part of a single module including the
SMI interface, and for it to fit into a generic switch configuration
framework and for it to use DSA rather than VLANs, but this is a start
and gives some sense of the parameters of such frameworks that are not
currently present in FreeBSD. In lieu of a switch configuration
interface, per-port media status and VLAN settings are in a sysctl tree.
XXX There may be some minor nits remaining in the handling of broadcast,
multicast and unknown destination traffic. It would also be nice to go
through and replace the few remaining magic numbers with macros at some
point in the future.
XXX This has only been tested with the MV88E6161, but it should work with
minimal or no modification on related switches, so support for probing
them was included.

Thanks to Pat Saavedra of TELoIP and Rafal Jaworowski of Semihalf for their
assistance in understanding the switch chipset.

PCI fix for XLR C revision chips, limit DMA address to the first 2GB
physical address. Adds a dma tag to the XLR/XLS pci bus with the
lowaddr if the CPU happens to be a XLR C rev.

Submitted by: Sreekanth M. S. (kanthms at netlogicmicro dot com))

XLR/XLS network driver (nlge) updates:

- nlge_ioctl handles IFF_UP and IFF_PROMISC flags
- Translate table code, to enable flow based CPU assignment added
disabled by default (can be enabled by a tunable).
- Changed signature of nlge_port_disable to make it consistent with nlge_port_enable
- Removed TXCSUM and VLAN_HW_TAGGING from i/f capabilities.

Submitted by: Sriram Gorti (srgorti at netlogicmicro dot com)

Fix n64 compile.

Network driver support for more XLR/XLS chip versions.

Obtained from: Sriram Gorti (srgorti at netlogicmicro dot com)

Clear any pending interrupts after disabling FMN interrupts. Add debugging
sysctl and counters for message ring threads (intial version). Update
watermark values, and and decrease the maximum threads to 3 (this will leave
a few CPUs for other processes)

Minor comment fix in nlge.

Fix asm for XKPHYS read for o32, constriants needed for input registers
that are clobbered. Fixes a crash when compiled without DEBUG

Update message ring handling code for XLR/XLS

- Wakeup multiple threads per core using message ring watermark interrupts.
- Update message ring handler registration, use the real device station id
for registering interrupts.
- rge/nlge: update for the new message ring registration code.
- rge/nlge: use 2 message ring stations for incoming packets, this will
allow more messages to be queued.
- nlge: comment fixes, remove unused variable
- style and whitespace fixes

o) Allow devices to override the MDIO read and write functions presented to
the miibus attached to octe interfaces.
o) Add an SMI/MDIO interface to the MV88E61XX and use it for the switch PHY on
the Lanner MR-320. An actual driver for the switch PHY will come later.
Note that for now it intercepts and fakes MII_BMSR reads to prevent the
miibus from talking to anything but the switch itself.

Rather than shifting offsets by three, set register offset to 3. All our
bus interface does that's special here now is to use a 64-bit register size.
In theory, uart(4) ought to support a regsz as well as regshft and support
64-bit registers directly.

Also use the UART class's range rather than a hand-coded 1024 for the address
range.

Use ABI-aware macros for setting up a fake frame.

Remove extra cpu setting and commented-out devices, some of which don't exist.

Make the OCTEON1 configuration look like a normal GENERIC configuration. In
particular, add PCI and USB buses and most devices. Also move to using ULE,
adding INET6, turning on WITNESS by default, etc.

- Fix values of CS1_EN and CS2_EN flags
- Unbreak kernel build by fixing naming convention of
GPIO_FUNC flags

Spotted by: Luiz Otavio O Souza, Andrew Thompson

AR71XX_GPIO_* defines were introduced by adrian@ a while ago,
remove duplicated.

Add GPIO bus to config and hints. Also add sample gpioled device.

Add AR71XX GPIO bus driver.

Give devices lots of time to settle around programming BARs and command
registers. Without this, the settings do not seem to stick for Atheros NICs in
the PCI slot of the Lanner MR-320.

o) Program the Lanner MR-320 for 32-bit mode, too.
o) Give a virtual address for I/O ports on n64.
o) On the Portwell CAM-0100, return the right IRQ for the on-board SATA.
o) Except on bridges, only set PORTEN and MEMEN on devices that have I/O or
memory BARs respectively.
o) Disable PORTEN and MEMEN while reprogramming BARs.
o) On the Lanner MR-955, set the Tx DMA power register for the on-board Promise
SATA controller.

XLS B0 revision PCI support and related changes.

- XLS B0 and later revision chips have PCIe link 2 & 3 mapped to different
PIC interrupts. Update pic.h, board.h and xlr_pci.c to reflect this.
- remove debug prints in xlr_pci.c
- add more processor IDs to board.h, add function xlr_is_xls_b0()
- some style(9) and whitespace fixes

o) Send mbufs to BPF listeners from within cvm_oct_xmit().
o) Pin receive threads when they're running since we do access some core-local
resources.

Handle link updates in a task.

Make the vast majority of Simple Executive files standard. Perhaps some of
these could be made dependent on either of the octusb or octe options, but
making them standard fixes a number of option combinations that were previously
broken.

o) Add bus_teardown_intr for pci and ciu. This allows the Promise SATA
driver to try to switch interrupt handlers at setup. It's not a very
good implementation of bus_teardown_intr, though.
o) Set cache line size and latency timers for PCI devices per Linux.

Flesh out PCI bus support some:
o) Reset and configure the bus from scratch rather than expecting U-Boot to
do it for us. Values and configuration from Linux, U-Boot and comments
in the Cavium Simple Executive sources.
o) Do a resource assignment and bus numbering pass in the absence of a PCI
BIOS or firmware that will do it for us.
XXX This has to be the third or fourth instance of this in FreeBSD and
it would be nice to have it become part of the PCI bus driver itself,
like it is on Linux.
o) Fix interrupt mapping for and adjust bus configuration for the Lanner
MR-955, based on information provided by Lanner.

Enforce that 'pmap_kenter()' is only used to establish cacheable mappings.

Mappings with other cacheability attributes can be established, if needed,
by using 'pmap_kenter_attr()'.

Suggested by: jchandra, imp

n32 fixes for nlge.

Use correct define (__mips_n64) for n64 compilation and use correct
parameter type for xlr_paddr_lw() in n32.

Add 64 bit support to nlge, and additional fixes

- 64 bit fixes for ifnlge.c
- Use m_nextpkt to save mbuf vaddr on 64 bit, we cannot store the
64 bit vaddr in the 40bit freeback field.
- remove unused code and unnecessary variables.
- use xlr_io_mmio macro instead of adding io base address
- rewrite GPIO related code to fixup nlge using xlr_write_reg and DELAY
- support for engg boards major num 11 and 12
- add xlr_paddr_lw() to load 32bit value from physical address, fix
inline assembly
- style fixes

Fix to specify generic bus_add_child.

Don't use memory that can't be direct-mapped on !n64.

Fix octusb build.

Fixes for XLR network accelerator driver (nlge).

- Process some tx done messages in the transmit path, to ensure that
the XLR NA tx done FIFO does not overflow.
- Add a message ring handler API to process atmost a given number of
messages from a specified bucket mask. This will be used to process
the tx done messages
- Add a callout to restart transmit in the case transmit gets blocked.
- Update enable_msgring_int() and disable_msgring_int(), remove unused
args and make static.

Obtained from: Sriram Gorti (srgorti at netlogicmicro dot com)

Get rid of the unnecessary redirection of 'is_cacheable_mem()' to
'is_physical_memory()' through a macro.

Implement 'is_cacheable_mem()' directly instead.

Get rid of unused macros.

Make a note of which platforms the mac strings come from.

Suggested by: adrian

Use getenv to find the mac address since it could be in the bootloader
environment or command line and under different names.

RMI security accelerator driver update.

- Compile fixes for 9.0, the previous version of this driver was
for FreeBSD 6.
- Add virtual address field in OperationDescriptor_t, we cannot use
MIPS_PHYS_TO_KSEG0 on physical address.
- Fixes for new message ring API
- Remove unused sys/mips/rmi/dev/sec/stats.h
- Whitespace fixes

Update rge.h for r212758 - stats incremented using xlr_ldaddwu should be
unsigned int.

Network driver updates.

- Move RMI MIPS extension to atomic increment word (LDADDWU) to common
header file sys/mips/rmi/rmi_mips_exts.h
- Fix xlr_ldaddwu() for 64 bit, it is a 32 bit operation, use
unsigned int* instead of unsigned long* argument
- Provide dummy xlr_enable_kx/xlr_restore_kx for n32 and n64.
- Provide xlr_paddr_ld() instead of xlr_paddr_lw(), so that the
descriptor formats are same for 32 and 64 bit
- update nlge and rge for the changes

These changes are also needed by the security driver which will be
added later.

Add 64-bit SWARM board kernel configs.

Factor out the common parts of the swarm board in SWARM_COMMON and start
including that in SWARM and SWARM_SMP kernel configs.

Make the meaning of the 'mask' argument to 'set_intr_mask(mask)' consistent
with the meaning of IM bits in the status register.

Reviewed by: jmallett, jchandra

Port r212559 to mips.

Do not explicitly enable interrupts in smp_init_secondary() because it
renders any spinlock protected code after that point to run with
interrupts enabled. This is because the processor is executing in the
context of idlethread whose 'md_spinlock_count' is already set to 1.

Instead just let sched_throw() re-enable interrupts when it releases
the spinlock.

The original powerpc commit log for r212559 is available here:
http://svn.freebsd.org/viewvc/base?view=revision&revision=212559

Enforce that pmap_mapdev() always returns uncacheable mappings.

Reviewed by: imp, jchandra, jmallett

TARGET_64BIT isn't needed anymore, GC it (partial merge from tbemd).

sys/mips/rmi/msgring.h - fixes and clean up.

- Remove sync from msgrng_send, sync needs to be called just once before
sending.
- Fix retry logic - don't reload registers when retrying in message_send,
also fix check for send pending fail.
- remove unused message_send_block_fast()
- merge message_receive_fast() to message_receive
- style(9) fixes, and comments
- rge and nlge updated for the sys/mips/rmi/msgring.h changes

bus_add_child method is needed now.

Refactor timer management code with priority to one-shot operation mode.
The main goal of this is to generate timer interrupts only when there is
some work to do. When CPU is busy interrupts are generating at full rate
of hz + stathz to fullfill scheduler and timekeeping requirements. But
when CPU is idle, only minimum set of interrupts (down to 8 interrupts per
second per CPU now), needed to handle scheduled callouts is executed.
This allows significantly increase idle CPU sleep time, increasing effect
of static power-saving technologies. Also it should reduce host CPU load
on virtualized systems, when guest system is idle.

There is set of tunables, also available as writable sysctls, allowing to
control wanted event timer subsystem behavior:
kern.eventtimer.timer - allows to choose event timer hardware to use.
On x86 there is up to 4 different kinds of timers. Depending on whether
chosen timer is per-CPU, behavior of other options slightly differs.
kern.eventtimer.periodic - allows to choose periodic and one-shot
operation mode. In periodic mode, current timer hardware taken as the only
source of time for time events. This mode is quite alike to previous kernel
behavior. One-shot mode instead uses currently selected time counter
hardware to schedule all needed events one by one and program timer to
generate interrupt exactly in specified time. Default value depends of
chosen timer capabilities, but one-shot mode is preferred, until other is
forced by user or hardware.
kern.eventtimer.singlemul - in periodic mode specifies how much times
higher timer frequency should be, to not strictly alias hardclock() and
statclock() events. Default values are 2 and 4, but could be reduced to 1
if extra interrupts are unwanted.
kern.eventtimer.idletick - makes each CPU to receive every timer interrupt
independently of whether they busy or not. By default this options is
disabled. If chosen timer is per-CPU and runs in periodic mode, this option
has no effect - all interrupts are generating.

As soon as this patch modifies cpu_idle() on some platforms, I have also
refactored one on x86. Now it makes use of MONITOR/MWAIT instrunctions
(if supported) under high sleep/wakeup rate, as fast alternative to other
methods. It allows SMP scheduler to wake up sleeping CPUs much faster
without using IPI, significantly increasing performance on some highly
task-switching loads.

Tested by: many (on i386, amd64, sparc64 and powerc)
H/W donated by: Gheorghe Ardelean
Sponsored by: iXsystems, Inc.

The functions in sys/mips/mips/psraccess.S can be implemented with
mips_rd_status/mips_wr_status. Implement them in mips/include/cpufunc.h,
and remove psraccess.S.

Reviewed by: neel, imp

bus_add_child: change type of order parameter to u_int

This reflects actual type used to store and compare child device orders.
Change is mostly done via a Coccinelle (soon to be devel/coccinelle)
semantic patch.
Verified by LINT+modules kernel builds.

Followup to: r212213
MFC after: 10 days

Compilation fix - when INVARIANTS are turned off.

Clean up and update sys/mips/rmi/rmi_mips_exts.h

- Provide 64 bit implementations for some macros. On n64 and n32,
don't split 64 bit values.
- No need for 32 bit ops for control registers.
- Fix few bugs (write control reg, write_c0_register64).
- Re-write EIRR/EIMR/CPUID operations using read_c0_registerXX, no
need of inline assembly.
- rename control reg access functions to avoid phnx, update callers.
- stlye/whitespace fixes.

Minor clean up for nlge - nlna_submit_rx_free_desc() can use the same desc
every time. Also merge couple of one-line functions into their caller.

nlge (alternate XLR GMAC driver) updates:

- Updates for the message ring clean up in r212321.
- Instead of dropping Tx packet on credit fail, retry send until it
succeeds.
- Fix freeing mbufs in case of P2P descriptors:
We cannot free the mbuf when the P2P descriptor freeback is received. The
mbuf may be still in use by the GMAC, since the P2P freeback indicates that
it read the P2D descriptors in the P2P message.
Now we free just the P2P descriptor when the P2P freeback message is
received. Another freeback P2D message has been added to the end of
the packet descriptors, the mbuf will be freed only when we received
this.

The P2P descriptor issue was reported by srgorti at netlogicmicro dot com.

Clean up fast message ring code for XLR.

Fix message ring send path:
- define msgrng_access_enable() which disables local interrupts
and enables message ring access. Also define msgrng_restore() which
restores interrupts
- remove all other msgrng enable/disable macros, no need of critical_enter
and other locking here.
- message_send() fixup: re-read status until pending bit clears
- message_send_retry() fixup: retry only few times with interrupts disabled
- Fix up message_send/message_send_retry callers - call
msgrng_access_enable() and msgrng_restore() correctly so that interrupts
are not disabled for long.
- removed unused and obsolete code from sys/mips/rmi/msgring.h
- some style fixes - more later

rge.c (XLR GMAC driver):
- updated for the message ring changes
- remove unused message_send_block()
- retry on credit failure, this is not a permanent failure when credits
are configured correctly. Add panic if credits are not available to
send for a long time.

PCIe updates for XLS.

Fix interrupt routing so that the irq returned is correct for XLR and
XLS. This also updates the MSI hack we had earlier - we still don't
really support MSI, but we support some drivers that use MSI, by providing
support for allocating one MSI per pci link - this MSI is directly
mapped to the link IRQ.

Whitespace fixes - indent with tabs instead of spaces.
Few other style(9) changes.

On boards with >512MB memory, the result of vtophys cannot be converted
to KSEG1 address - use pmap_mapdev/pmap_unmapdev instead.

Remove unused variable maxphys.

XLR/XLS hardware interrupts should be programmed level triggered at the
PIC. This should fix the interrupt releated issues seen after the
interrupt handling re-write for SMP.

Updates for the RMI MIPS platform code
- set cache_coherent_dma flag in cpuinfo for XLR, this will make sure that
BUS_DMA_COHERENT flag is handled correctly in busdma_machdep.c
- iodi.c, call device_get_name() just once
- clear RMI specific EIRR while intializing CPUs
- remove debug print in intr_machdep.c

Add the workaround for 4xx lite boards after it was lost in the last
board.c update.

Move debug.h to the the rge driver directory. rge.c is the only user of
debug.h. Remove debug.h references from other files.

Clean up header files in RMI platform code (sys/mips/rmi), and remove
unused files.

- remove clock.c and clock.h, these are not used after the new timer
code was added.
- remove duplicated include files, fix header file ordering, remove
some unneeded includes.
- rename mips/rmi/shared_structs.h which contains the RMI boot loader
interface to mips/rmi/rmi_boot_info.h. Remove unused files
mips/rmi/shared_structs_func.h and sys/mips/rmi/shared_structs_offsets.h
- merge mips/rmi/xlrconfig.h and mips/rmi/rmi_mips_exts.h, and remove
duplicated functions.
- nlge - minor change to remove unneeded argument.
- Add FreeBSD svn keyword for headers

Remove misleading comment in pte.h. MIPS PTE entries are software managed
and does not need atomics.

Submitted by: alc

Apply MIPS pmap clean up patch from alc@ (with minor change to KASSERT):

PMAP_DIAGNOSTIC was eliminated from amd64/i386, and, in fact, the
non-MIPS parts of the kernel, several years ago. Any of the interesting
checks were turned into KASSERT()s. Basically, the motivation was that
lots of people run with INVARIANTS but no one runs with DIAGNOSTIC.

panic strings needn't and shouldn't have a terminating newline.

Finally, there is one functional change. The sched_pin() in
pmap_remove_pages() is an artifact of the way we temporarily map page
table pages on i386. (The mappings are processor private. We don't do
a system-wide shootdown.) It isn't needed by MIPS.

Tested by: jchandra

Submitted by: alc

New driver nlge for XLR/XLS Network Accelerator. This will support the XGMAC
and XAUI 10G interfaces in addition RGMII/SGMII 1G interfaces. This driver
is work in progress.

board.c and board.h expanded to include more info.

Only one of rge and nlge can be enabled at a time, rge will be deprecated
when nlge stabilizes.

Submitted by: Sriram Gorti <srgorti at netlogicmicro com>

Initial code for XLR CompactFlash driver.

Submitted by: Sreekanth M. S. <kanthms at netlogicmicro com>

Revamp XLR interrupt handling, the previous scheme does not work well on
SMP.

We used to route all PIC based interrupts to cpu 0, and used the per-CPU
interrupt mask to enable/disable interrupts. But the interrupt threads can
run on any cpu on SMP, and the interrupt thread will re-enable the interrupts
on the CPU it runs on when it is done, and not on cpu0 where the PIC will
still send interrupts to.

The fix is move the disable/enable for PIC based interrupts to PIC, we will
ack on PIC only when the interrupt thread is done, and we do not use the
per-CPU interrupt mask.

The changes also introduce a way for subsystems to add a function that
will be called to clear the interrupt on the subsystem. Currently This is
used by the PCI/PCIe for doing additional work during the interrupt
handling.

Whitespace fixes in mips/include, remove unused 'struct tlb' from locore.h

PR: misc/147471

Provide timecounter based on XLR PIC timer.

- Use timer 7 in XLR PIC as a 32 counter
- provide pic_init_timer(), pic_set_timer(), pic_timer_count32() and
pic_timer_count() PIC timer operations.
- register this timer as platform_timecounter on rmi platform.

XLR PIC code update.
- Fix a bug in xlr_pic_init (use irq in PIC_IRQ_IS_EDGE_TRIGGERED)
- use new macro PIC_INTR_TO_IRQ() and PIC_IRT_x() in xlr_pic_init

XLR PIC code update and style(9) fixes.

- style(9) fixes to mips/rmi platform files
- update pic.h to add pic_setup_intr() and use pic_setup_intr() for setting
up interrupts which are routed thru PIC.
- remove rmi_spin_mutex_safe and haslock, and make sure that the functions
are called only after mutexes are available.

Rename on_chip.c to fmn.c, as the file has just the fast messaging network
code. The iodi.c has the bus for SoC devices, so the name on_chip.c is
misleading.

RMI XLR platform code clean-up.

- move PIC code to xlr_machdep.c
- move fast message ring code completely to on_chip.c
- move memory initialization to a new function xlr_mem_init()
- style fixes

Sync up XLR tick.c with the new MIPS tick.c

Migrate if_arge to use the PLL cpuops.

This has been lightly tested on the AR7161 and AR9132.

Implement PLL generalisation in preparation for use in if_arge.

* Add a function to write to the relevant PLL register
* Break out the PLL configuration for the AR71XX into the CPU ops,
lifted from if_arge.c.
* Add the AR91XX PLL configuration ops, using the AR91XX register
definitions.

add the PLL set functions to cpuops

Fix mistaken indenting.

Add some initial AR724X chipset support.

This is untested but should at least allow an AR724X to boot.

The current code is lacking the detail needed to expose the PCIe bus.
It is also lacking any NIC, PLL or flush/WB code.

Add initial Atheros AR91XX support.

This works well enough to bring a system up to single-user mode
using an MDROOT.

Known Issues:

* The EHCI USB doesn't currently work and will panic the kernel during
attach.
* The onboard ethernet won't work until the PLL routines have been
fleshed out and shoe-horned into if_arge.
* The WMAC device glue (and quite likely the if_ath support)
hasn't yet been implemented.

Add missing licence.

style(9) pick from imp@ .

Remove now unused 'reg'.

Initialise the USB system using cpuops rather than the AR71XX specific method.

Migrate the CPU reset path to use the new cpuops.

Remove the now-unused DDR flush register value.

Make the PCI initialisation path use the new cpuops rather than directly
programming the reset register.

Make if_arge use the new cpuops rather than hard coding the DDR flush registers.

Preparation work for supporting the AR91xx and AR724x.

* Implement a SoC probe function, from Linux, which determines the
SoC family, type and revision. This only probes the AR71xx series
SoC and (currently) panics on others.

* Migrate some of the AR71XX specific hardware init (USB device, determining
system frequencies) into using the cpuops introduced in an earlier commit.
Other SoC specific hardware stuff (per-device flush/WB, GPIO pin wiring,
Ethernet PLL setup, other things I've likely missed) will be introduced in
subsequent commits.

Reviewed by: imp@
Obtained from: (partially) Linux

MIPS n64 support - continued...

1. On n64, use XKPHYS to map page table pages instead of KSEG0. Maintain
just one freepages list on n64.

The changes are mainly to introduce MIPS_PHYS_TO_DIRECT(pa),
MIPS_DIRECT_TO_PHYS(), which will use KSEG0 in 32 bit compilation
and XKPHYS in 64 bit compilation.

2. Change macro based PMAP_LMEM_MAP1(), PMAP_LMEM_MAP2(), PMAP_LMEM_UNMAP()
to inline functions.

3. Introduce MIPS_DIRECT_MAPPABLE(pa), which will further reduce the cases
in which we will need to have a special case for 64 bit compilation.

4. Update CP0 hazard definitions for CPU_RMI - the cpu does not need any
nops

Reviewed by: neel

Add a DDR flush function, inspired by both Linux and if_arge.c.

Add a further register definition for USB device initialisation.

Obtained from: Linux

Bring over the first cut of the Atheros-specific SoC operations.

Each of these SoCs have different devices, different hardware initialisation
methods and, quite likely, different quirks. These functions will abstract
out the SoC differences and keep these differences out of the drivers (eg
USB init, if_arge, etc.)

Make return statements style(9) compliant in MIPS pmap.c

Reported by: neel

Reviewed by: neel (earlier version)

Import initial AR91XX and AR724X CPU register definitions.

Obtained from: Linux

Supply some useful information to the started image using ELF aux vectors.
In particular, provide pagesize and pagesizes array, the canary value
for SSP use, number of host CPUs and osreldate.

Tested by: marius (sparc64)
MFC after: 1 month

Rename TARGET_XLR_XLS to CPU_RMI to match other CPU_xxx definitions.
use CPU_RMI all XLR configurations. Update ident string for N32 and
N64 kernels.

SMP support in n64.

- Enable KX and UX bits on CPU startup for non-boot CPUs
- Keep the KX bit when in userspace - XTLB handler needs it to access
PCPU data
- revert r210638 partly - we don't need to enable KX on kernel entry
now

Reviewed by: jmallett, imp

Implement pmap changes suggested by alc@:

1. Move dirty bit emulation code that is duplicted for kernel and user
in trap.c to a function pmap_emulate_modified() in pmap.c.

2. While doing dirty bit emulation, it is not necessary to update the
TLB entry on all CPUs using smp_rendezvous(), we can just update the
TLB entry on the current CPU, and let the other CPUs update their TLB
entry lazily if they get an exception.

Reviewed by: alc, neel

Optimization for pmap_kenter(), call pmap_update_page() only when
necessary. On SMP, pmap_update_page() can be costly as it needs a
a smp_rendezvous().

Reviewed by: alc, neel
Obtained from: jmallett (http://svn.freebsd.org/base/user/jmallett/octeon)

Re-arrange the pmap calls that use smp_rendezvous() on SMP, so that their
per-cpu variants are also available to be called. The per-cpu variants
are needed for some later optimizations.

Also remove unnecessary casts, do some style fixes.

Reviewed by: alc, neel

Update various places that store or manipulate CPU masks to use cpumask_t
instead of int or u_int. Since cpumask_t is currently u_int on all
platforms this should just be a cosmetic change.

Fix for 64 bit compile, with SMP enabled.

Fix compilation when DDB disabled. Adds 'ifdef DDB' around
DB_SHOW_COMMAND definitions.

Add parentheses around the argument 'x' used in the __bswapXX(x) macros. Revert
r211130 in favor of this more general fix.

This fixes a compilation error for mips 64-bit little endian build.
libexec/rtld-elf/mips/reloc.c:196: warning: right shift count >= width of type

Suggested by: stefanf, jchandra, bde

loadandclear() for PTEs are not needed on MIPS. The PTEs are software
managed and we already take pmap lock for PTE operations(see r210922)

Reviewed by: alc

- Consolidate the the cache coherence attribute definitions in a single place.
Adapted from Juli's changes to pte.h in the octeon branch:
http://svn.freebsd.org/viewvc/base/user/jmallett/octeon/sys/mips/include/pte.h

- Set the KX and UX bits in the status register for n64 kernels.

Reviewed by: jmallett

Add a new ipi_cpu() function to the MI IPI API that can be used to send an
IPI to a specific CPU by its cpuid. Replace calls to ipi_selected() that
constructed a mask for a single CPU with calls to ipi_cpu() instead. This
will matter more in the future when we transition from cpumask_t to
cpuset_t for CPU masks in which case building a CPU mask is more expensive.

Submitted by: peter, sbruno
Reviewed by: rookie
Obtained from: Yahoo! (x86)
MFC after: 1 month

Fix issue reported by alc :

MIPS doesn't really need to use atomic_cmpset_int() in situations like
this because the software dirty bit emulation in trap.c acquires
the pmap lock. Atomics like this appear to be a carryover from i386
where the hardware-managed TLB might concurrently set the modified bit.

Reviewed by: alc

Fix the issue reported by alc:

pmap_page_wired_mappings() counts the number of pv entries for the
specified page that have the pv entry wired flag set to TRUE.
pmap_enter() correctly initializes this flag. However,
pmap_change_wiring() doesn't update the corresponding pv entry flag,
only the PTE. So, the count returned by pmap_page_wired_mappings()
will sometimes be wrong.

In the short term, the best fix would be to eliminate the pv entry
flag and use only the PTE. That flag is wasting non-trivial memory.

Remove pv_wired flag, and use PTE flag to count the wired mappings.

Reviewed by: alc

Remove redundant declaration of 'pcib_driver' class from sb_zbpci.c. This
causes a compilation error.

The declaration is provided by sys/dev/pci/pcib_private.h starting from r210864.

Use a signed integer to hold the address of a register.

This does the right thing by sign extending the address when compiling for
the n64 abi.

uint64_t is 'unsigned long' in n64 build, so compiler is unhappy if the
format specifier in printf is "%llu".

Use "%ju" instead.

- Add interrupts counter for PCI devices

Fix a race between clock_intr() and tick_ticker() when updating
'counter_upper' and 'counter_lower_last'. The race exists because
interrupts are enabled even though tick_ticker() executes in a
critical section.

Fix a bug in clock_intr() in how it updates the cached values of
'counter_upper' and 'counter_lower_last'. They are updated only
when the COUNT register rolls over. More interestingly it will *never*
update the cached values if 'counter_lower_last' happens to be zero.

Get rid of superfluous critical section in clock_intr(). There is no
reason to do this because clock_intr() executes in hard interrupt
context.

Switch back to using 'tick_ticker()' as the cpu ticker for Sibyte.

Reviewed by: jmallett, mav

Add 3 level page tables for MIPS in n64.

- 32 bit compilation will still use old 2 level page tables
- re-arrange pmap code so that adding another level is easier
- pmap code for 3 level page tables for n64
- update TLB handler to traverse 3 levels in n64

Reviewed by: jmallett

ithd_name no longer defined, use td_name. Fix compile with KTR enabled.

Use PTR_ADDU to change sp, so that it works for n64 too.

Use fuword32() to fetch instructions, this will work on both 32 and 64 bit
compilation.

n64 support - enable UX bit in STATUS for kernel and userspace.

- enable UX in kernel start, and kernel entry
- keep UX flag in cpu_fork and cpu_set_upcall
- enable UX for userspace

MIPS n64 support - support kstack in XKSEG.
- enable KX on entry from user-space, we need KX set to save to XKSEG
addresses.
- add MIPS_XKSEG_START to genassym.c
- Add n64 case for swth.S

Update rge driver for 64 bit kernel.
- stored virtual addresses should be 64bit
- physical memory can be directly accessed using XKPHYS pointers in 64 bit.
- no need to enable KX

Prepare for 3 level page tables for MIPS.

- Move page table second level shift and mask to param.h
- rename SEGOFSET to SEGMASK
- fix values for 64 bit maximum kernel and user addresses.

Add fuiword() in n64 for completeness.

MIPS 64 bit support. Define fuword64() for n64 compilation, fuword() should
be fuword64() in 64 bit.

Update MIPS _stdint.h for 64 bit. Initial 64 bit changes for profile.h.

Fix RQB_FFS for 64 bit, we need to use ffsl() for 64bit.

Use 'ifdef __mips_n64' instead of 'if defined' to be consistant with other
usage.

Fix build for o32 kernels.

The emulation of 'ld' and 'sd' instructions only works for ABIs that support
64-bit registers and the instructions 'ldl' and 'ldr' that operate on those
registers.

Reviewed by: jmallett

o) Subtract 64K from the default userland stack pointer. GCC generate code
that with a 32-bit ABI on a system with 64-bit registers can attempt to
access an invalid (well, kernel) memory address rather than the intended
user address for stack-relative loads and stores. Lowering the stack
pointer works around this. [1]
o) Make TRAP_DEBUG code conditional on the trap_debug variable. Make
trap_debug default to 0 instead of 1 now but make it possible to change it
at runtime using sysctl.
o) Kill programs that attempt an unaligned access of a kernel address. Note
that with some ABIs, calling useracc() is not sufficient since the register
may be 64-bit but vm_offset_t is 32-bit so a kernel address could be
truncated to what looks like a valid user address, allowing the user to
crash the kernel.
o) Clean up unaligned access emulation to support unaligned 16-bit and 64-bit
accesses. (For 16-bit accesses it was checking for user access to too much
memory (4 bytes) and there was no 64-bit support.) This still lacks support
for unaligned load-linked and store-conditional.

Reviewed by: [1] gonzo

Very rough first cut at NUMA support for the physical page allocator. For
now it uses a very dumb first-touch allocation policy. This will change in
the future.
- Each architecture indicates the maximum number of supported memory domains
via a new VM_NDOMAIN parameter in <machine/vmparam.h>.
- Each cpu now has a PCPU_GET(domain) member to indicate the memory domain
a CPU belongs to. Domain values are dense and numbered from 0.
- When a platform supports multiple domains, the default freelist
(VM_FREELIST_DEFAULT) is split up into N freelists, one for each domain.
The MD code is required to populate an array of mem_affinity structures.
Each entry in the array defines a range of memory (start and end) and a
domain for the range. Multiple entries may be present for a single
domain. The list is terminated by an entry where all fields are zero.
This array of structures is used to split up phys_avail[] regions that
fall in VM_FREELIST_DEFAULT into per-domain freelists.
- Each memory domain has a separate lookup-array of freelists that is
used when fulfulling a physical memory allocation. Right now the
per-domain freelists are listed in a round-robin order for each domain.
In the future a table such as the ACPI SLIT table may be used to order
the per-domain lookup lists based on the penalty for each memory domain
relative to a specific domain. The lookup lists may be examined via a
new vm.phys.lookup_lists sysctl.
- The first-touch policy is implemented by using PCPU_GET(domain) to
pick a lookup list when allocating memory.

Reviewed by: alc

mips/rmi/bus_space_rmi_pci.c is needed even when PCI is disabled. This
file really provides a bus that does byteswapping, and can be used by
non-PCI components too.

The changes in r210542 moved setting counter_upper and counter_lower_last
outside the critical section - fix that.

Reported by: mav

The count/compare values have to be tracked per CPU.

Reviewed by: mav

Fixup mips/rmi for the new mips timer code(r210403). This will get XLR
booting again.

The code is a copy of the mips/mips/tick.c with minor modifications for
XLR interrupt handling. Disable mips/rmi/clock.c for now, the PIC based
timer code will be added later.

Get N64 building by defining VM_FREELIST_DIRECT to be
VM_FREELIST_DEFAULT. I believe this is correct, since KX is set in
n64, and thus all RAM can be direct mapped.

Add some comments.

Update MIPS timer code (except RMI) to utilize new MI event timer
infrastructure.

Reviewed by: neel

Redo the page table page allocation on MIPS, as suggested by
alc@.

The UMA zone based allocation is replaced by a scheme that creates
a new free page list for the KSEG0 region, and a new function
in sys/vm that allocates pages from a specific free page list.

This also fixes a race condition introduced by the UMA based page table
page allocation code. Dropping the page queue and pmap locks before
the call to uma_zfree, and re-acquiring them afterwards will introduce
a race condtion(noted by alc@).

The changes are :
- Revert the earlier changes in MIPS pmap.c that added UMA zone for
page table pages.
- Add a new freelist VM_FREELIST_HIGHMEM to MIPS vmparam.h for memory that
is not directly mapped (in 32bit kernel). Normal page allocations will first
try the HIGHMEM freelist and then the default(direct mapped) freelist.
- Add a new function 'vm_page_t vm_page_alloc_freelist(int flind, int
order, int req)' to vm/vm_page.c to allocate a page from a specified
freelist. The MIPS page table pages will be allocated using this function
from the freelist containing direct mapped pages.
- Move the page initialization code from vm_phys_alloc_contig() to a
new function vm_page_alloc_init(), and use this function to initialize
pages in vm_page_alloc_freelist() too.
- Split the function vm_phys_alloc_pages(int pool, int order) to create
vm_phys_alloc_freelist_pages(int flind, int pool, int order), and use
this function from both vm_page_alloc_freelist() and vm_phys_alloc_pages().

Reviewed by: alc

Remove unused file. Replaced by tlb.c.

o) Add the "octusb" controller which supports the first port of the Octeon
on-board USB controller. It is not currently enabled because there are
known problems with device communication and until those are fixed I am not
certain that it won't destabilize the system. [1]
o) Add the "cryptocteon" opencrypto device based on the OCF device written by
David McCullough. It is not currently enabled because until support for
saving/restoring coprocessor 2 state on context switch is available, it runs
with interrupts disabled, which tends to pessimize performance over using a
software crypto facility. Tests using this driver which are not negatively
affected by it running with interrupts disabled show it to be substantially
faster than software for large blocks.

Submitted by: hps [1]

Update the port of FreeBSD to Cavium Octeon to use the Cavium Simple Executive
library:
o) Increase inline unit / large function growth limits for MIPS to accommodate
the needs of the Simple Executive, which uses a shocking amount of inlining.
o) Remove TARGET_OCTEON and use CPU_CNMIPS to do things required by cnMIPS and
the Octeon SoC.
o) Add OCTEON_VENDOR_LANNER to use Lanner's allocation of vendor-specific
board numbers, specifically to support the MR320.
o) Add OCTEON_BOARD_CAPK_0100ND to hard-wire configuration for the CAPK-0100nd,
which improperly uses an evaluation board's board number and breaks board
detection at runtime. This board is sold by Portwell as the CAM-0100.
o) Add support for the RTC available on some Octeon boards.
o) Add support for the Octeon PCI bus. Note that rman_[sg]et_virtual for IO
ports can not work unless building for n64.
o) Clean up the CompactFlash driver to use Simple Executive macros and
structures where possible (it would be advisable to use the Simple Executive
API to set the PIO mode, too, but that is not done presently.) Also use
structures from FreeBSD's ATA layer rather than structures copied from
Linux.
o) Print available Octeon SoC features on boot.
o) Add support for the Octeon timecounter.
o) Use the Simple Executive's routines rather than local copies for doing reads
and writes to 64-bit addresses and use its macros for various device
addresses rather than using local copies.
o) Rename octeon_board_real to octeon_is_simulation to reduce differences with
Cavium-provided code originally written for Linux. Also make it use the
same simplified test that the Simple Executive and Linux both use rather
than our complex one.
o) Add support for the Octeon CIU, which is the main interrupt unit, as a bus
to use normal interrupt allocation and setup routines.
o) Use the Simple Executive's bootmem facility to allocate physical memory for
the kernel, rather than assuming we know which addresses we can steal.
NB: This may reduce the amount of RAM the kernel reports you as having if
you are leaving large temporary allocations made by U-Boot allocated
when starting FreeBSD.
o) Add a port of the Cavium-provided Ethernet driver for Linux. This changes
Ethernet interface naming from rgmxN to octeN. The new driver has vast
improvements over the old one, both in performance and functionality, but
does still have some features which have not been ported entirely and there
may be unimplemented code that can be hit in everyday use. I will make
every effort to correct those as they are reported.
o) Support loading the kernel on non-contiguous cores.
o) Add very conservative support for harvesting randomness from the Octeon
random number device.
o) Turn SMP on by default.
o) Clean up the style of the Octeon kernel configurations a little and make
them compile with -march=octeon.
o) Add support for the Lanner MR320 and the CAPK-0100nd to the Simple
Executive.
o) Modify the Simple Executive to build on FreeBSD and to build without
executive-config.h or cvmx-config.h. In the future we may want to
revert part of these changes and supply executive-config.h and
cvmx-config.h and access to the options contained in those files via
kernel configuration files.
o) Modify the Simple Executive USB routines to support getting and setting
of the USB PID.

Move common macros into asm.h. Replace MIPS_CPU_NOP_DELAY with
HAZARD_DELAY. Move HAZARD_DELAY and ITLBNOPFIX into asm.h, for
possible later optimization...

Reviewed by: jmallet, jchandra

This file appears not to be used.

Use #define for get_cyclecount rather than inline function.
mips_rd_count() isn't defined in userland, and cpu.h is included there
in alias_scpt.h (maybe they don't need it in the first place).

Better description of this file

Fix for 64 bit compilation.

RMI bootloader passes argv[] and envp[] as an array of 32 bit pointers.
Convert the pointers to correct pointer type before use.

Move TLB definitions to tlb.h

This file has been unused for a while now...

Remove unused stuff from cpu.h.
Move inappropriate stuff in cpu.h elsewhere:
{s,g}et_intr_mask -> md_var.h
num_tlbentries -> tlb.h
Remove #define clockframe trapframe and fix clock, which was the only place
this was used.
All the rest of this stuff was unused.

# we're not quite minimal yet, since we duplicate a few status register things
# here...

Inspired by: bde@

We don't need sys/cdefs.h for __CONCAT here.

Remove i386-ish sysctls. Also, make the bootinfo sysctl OID_AUTO.

Remove one layer of indirection. No need to call cpu_throw which then
calls mips_cpu_call via an obfuscated assembler call. Instead, delete
the current cpu_throw, and rename mips_cpu_throw to cpu_throw. This
is nicer to the cache on each context switch (since fixed jumps can be
prefected, while jumps through a register can't). Incidentally, it
also saves about 5 or 6 instructions.

Reviewed by: jmallet@

Use cpuregs.h spellings over the cpu.h spellings.

Remove the unused part of cpu.h now that the rest of the tree has been
transitioned to use cpuregs.h spellings. Now we're only 4x too big,
according to the bde-ometer.

Prefer the cpuregs.h spellings of register and bit names over cpu.h.

union cpuprid is also unused now

Add INFO config register from mips32/64 land

Remove redunant machine/cpuregs.h include.
Also, spell things like in machine/cpuregs.h instead of machine/cpu.h.

Temp hack to N32 kernel: turn off debugger since n32 is too weird for ddb

Define break value for ddb.
Use int32/intptr casts for exception vector names.
Define MIPS_SR_INT_MASK again
Change MIPS_XKPHYS_CCA_* to MIPS_CCA_* since we can use them in many contexts
Minor gratuitous whitespace churn

Use the cpuregs.h spellings for these registers rather than the cpu.h
spelling.

cpu_id and fpu_id are unused, except to be set early in the boot code.
The problem with setting it there is that the last CPU to come up
wins, it seems. This also removes one more ifdef in locore.S, a noble
goal too. Since they are unused, and pollute cpu.h, remove them.

Submitted by: bde.h (cpu.h pollution)
Approved in theory by: jmallet@

Remove obsolete define "COPY_SIGCODE". This is unused in FreeBSD.

Submitted by: bde@

Merge jmallett@'s n64 work into HEAD

Initial set of ddb changes to support n64

Reviewed by: imp
Obtained from: jmallett (http://svn.freebsd.org/base/user/jmallett/octeon)

Merge jmallett@'s n64 work into HEAD.

Merge changes for initial n64 support in pmap.c. Use direct mapped (XKPHYS)
access for a lot of operations that earlier needed temporary mapping. Add
support for using XKSEG for kernel mappings.

Reviewed by: imp
Obtained from: jmallett (http://svn.freebsd.org/base/user/jmallett/octeon)

Merge jmallett@'s n64 work into HEAD

64 bit TLB definitions in pte.h

Reviewed by: imp
Obtained from: jmallett (http://svn.freebsd.org/base/user/jmallett/octeon)

Move KSEG address definitions from cpu.h to cpuregs.h with the other
definitions, add some XKPHYS related definitions for n64.

Reviewed by: imp

Fix XLR64, the previous version had the contents of file duplicated.

64 bit ld script for mips, and configuration file for 64 bit compilation
on XLR

Approved by: rrs

Use 64 bit type for rqb_word_t in n64 kernel.

Reviewed by: imp
Approved by: rrs

Enable KX bit, which is needed for 64 bit access, in status register
for XLR. Update exception handlers and other functions which set/change
status registers to preserve this.

Approved by: rrs

Add TX-path aligned/unaligned stats for if_arge.

64 bit compilation support XLR platform code.
Mostly changes to make casting between int and pointer and printing
64bit values safe for 32 and 64 bit compile.

Approved by: rrs

Address PR kern/148307 - fix if_ath TX mbuf alignment/size constraint checks

The existing code only checked the alignment of the first mbuf and
didn't enforce the size constraints.

This commit introduces a simple function to check the alignment and
size of all mbufs in the list. This fixes the initial issue in the
PR.

PR: kern/148307
Reviewed by: gonzo@

Fix minor issue in sys/mips/mips/cache_mipsNN.c, using int type for
virtual addr will break on 64 bit, change to vm_offset_t.

Reviewed by: imp@
Approved by: rrs (mentor)

Merge jmallett@'s n64 work into HEAD - changeset 8

Updated PTE/PDE macros from http://svn.freebsd.org/base/user/jmallett/octeon
Introduce pmap_segshift() macro, use pmap_segmap() in place of pmap_pde, and
remove pmap_pde().

Approved by: rrs (mentor)
Obtained from: jmallett@

Introduce a sysctl block for if_arge and, for now, a blank debug sysctl
placeholder for later.

Add in a missing FreeBSD ID string.

Fix the CS line definitions. These bits are for the CS2/CS1 lines
rather than CS1/CS0.

This has been tested on the Ubiqiti Routerstation Pro board.

Add support for XLS 108 chips - update board.c/board.h to setup GMAC
driver parameters correctly.

Approved by: rrs (mentor)
Obtained from: Sriram Gorti <srgorti@netlogicmicro.com>

Remove save/restore of PageMask in tlb.c functions introduced in r209243.
If we save/restore the PageMask, the value set by the bootloader will
persist, and will cause problems later in TLB exception handler.
This caused a crash in AR71xx boards.

Also fixes the EntryHi mask in pte.h

Reported by: Luiz Otavio O Souza <lists.br@gmail.com>
Tested by: Luiz Otavio O Souza <lists.br@gmail.com>

Approved by: rrs (mentor)

Move prototypes for kern_sigtimedwait() and kern_sigprocmask() to
<sys/syscallsubr.h> where all other kern_<syscall> prototypes live.

Add linker script and configuration file for n32 kernel.

Approved by: rrs (mentor)

Merge jmallett@'s n64 work into HEAD - changeset 7

Initial support for n32 and n64 ABIs from
http://svn.freebsd.org/base/user/jmallett/octeon

Changes are:
- syscall, exception and trap support for n32/n64 ABIs
- 64-bit address space defines
- _jmp_buf for n32/n64
- casts between registers and ptr/int updated to work on n32/n64

Approved by: rrs(mentor), jmallett

Comment about the shared pins I know about.

Merge jmallett@'s n64 work into HEAD - changeset 6

PTE flag cleanup from http://svn.freebsd.org/base/user/jmallett/octeon
- Rename PTE_xx flags to match their MIPS names
- Use the new pte_set/test/clear macros uniformly, instead of a mixture
of mips_pg_xxx(), pmap_pte_x() macros and direct access.
- Remove unused macros and defines from pte.h and pmap.c

Discussed on freebsd-mips@

Approved by: rrs(mentor), jmallett

AR71XX GPIO register definitions.

Reviewed by: gonzo@

Extend the AR71XX watchdog debugging and data.

* Add some per-device sysctl entries which record the watchdog state -
whether it is armed; whether the last reboot was due to the watchdog.
* Add a per-device sysctl debug flag to enable logging watchdog arming/
disarming.

Reviewed by: gonzo@

Add support for newer XLS chips/boards in the GMAC driver.
Improved processor id code in board.h, remove unnecessary macros.

Approved by: rrs(mentor)

Merge jmallett@'s n64 work into HEAD - changeset 5

Remove unnecessary locking and sched_pin() call while creating a temporary
mapping.

Changes from http://svn.freebsd.org/base/user/jmallett/octeon

Approved by: rrs (mentor), jmallett

Merge jmallett@'s n64 work into HEAD - changeset 4

Re-write tlb operations in C with a simpler API.
Update callers to use the new API.

Changes from http://svn.freebsd.org/base/user/jmallett/octeon

Approved by: rrs(mentor), jmallett

Relax one of the new assertions in pmap_enter() a little. Specifically,
allow pmap_enter() to be performed on an unmanaged page that doesn't have
VPO_BUSY set. Having VPO_BUSY set really only matters for managed pages.
(See, for example, pmap_remove_write().)

Reduce the scope of the page queues lock and the number of
PG_REFERENCED changes in vm_pageout_object_deactivate_pages().
Simplify this function's inner loop using TAILQ_FOREACH(), and shorten
some of its overly long lines. Update a stale comment.

Assert that PG_REFERENCED may be cleared only if the object containing
the page is locked. Add a comment documenting this.

Assert that a caller to vm_page_requeue() holds the page queues lock,
and assert that the page is on a page queue.

Push down the page queues lock into pmap_ts_referenced() and
pmap_page_exists_quick(). (As of now, there are no longer any pmap
functions that expect to be called with the page queues lock held.)

Neither pmap_ts_referenced() nor pmap_page_exists_quick() should ever
be passed an unmanaged page. Assert this rather than returning "0"
and "FALSE" respectively.

ARM:

Simplify pmap_page_exists_quick() by switching to TAILQ_FOREACH().

Push down the page queues lock inside of pmap_clearbit(), simplifying
pmap_clear_modify(), pmap_clear_reference(), and pmap_remove_write().
Additionally, this allows for avoiding the acquisition of the page
queues lock in some cases.

PowerPC/AIM:

moea*_page_exits_quick() and moea*_page_wired_mappings() will never be
called before pmap initialization is complete. Therefore, the check
for moea_initialized can be eliminated.

Push down the page queues lock inside of moea*_clear_bit(),
simplifying moea*_clear_modify() and moea*_clear_reference().

The last parameter to moea*_clear_bit() is never used. Eliminate it.

PowerPC/BookE:

Simplify mmu_booke_page_exists_quick()'s control flow.

Reviewed by: kib@

Don't set PG_WRITEABLE in init_pte_prot() (and thus pmap_enter()) unless
the page is managed.

Don't set the machine-independent layer's dirty field for the page being
mapped in init_pte_prot(). (The dirty field is only supposed to set when
a mapping is removed or write-protected and the page was managed and
modified.)

Determine whether or not to perform dirty bit emulation based on whether
or not the page is managed, i.e., pageable, not based on whether the page
is being mapped into the kernel address space. Nearly all of the kernel
address space consists of unmanaged pages, so this has neglible impact on
the overhead of dirty bit emulation for the kernel address space. However,
there can also exist unmanaged pages in the user address space. Previously,
dirty bit emulation was unnecessarily performed on these pages.

Tested by: jchandra@

Make vm_contig_grow_cache() extern, and use it when vm_phys_alloc_contig()
fails to allocate MIPS page table pages. The current usage of VM_WAIT in
case of vm_phys_alloc_contig() failure is not correct, because:

"There is no guarantee that any of the available free (or cached) pages
after the VM_WAIT will fall within the range of suitable physical
addresses. Every time this function sleeps and a single page is freed
(or cached) by someone else, this function will be reawakened. With
a little bad luck, you could spin indefinitely."

We also add low and high parameters to vm_contig_grow_cache() and
vm_contig_launder() so that we restrict vm_contig_launder() to the range
of pages we are interested in.

Reported by: alc

Reviewed by: alc
Approved by: rrs (mentor)

Fix a KASSERT() that was broken in r208665.

Reported by: jmallett

Eliminate a stale comment.

Merge portions of r208645 and supporting code from the i386 pmap:
When I pushed down the page queues lock into pmap_is_modified(), I created
an ordering dependence: A pmap operation that clears PG_WRITEABLE and calls
vm_page_dirty() must perform the call first. Otherwise, pmap_is_modified()
could return FALSE without acquiring the page queues lock because the page
is not (currently) writeable, and the caller to pmap_is_modified() might
believe that the page's dirty field is clear because it has not seen the
effect of the vm_page_dirty() call.

When I pushed down the page queues lock into pmap_is_modified(), I
overlooked one place where this ordering dependence is violated:
pmap_enter(). In a rare situation pmap_enter() can be called to replace a
dirty mapping to one page with a mapping to another page. (I say rare
because replacements generally occur as a result of a copy-on-write fault,
and so the old page is not dirty.) This change delays clearing PG_WRITEABLE
until after vm_page_dirty() has been called.

Fixing the ordering dependency also makes it easy to introduce a small
optimization: When pmap_enter() used to replace a mapping to one page with a
mapping to another page, it freed the pv entry for the first mapping and
later called the pv entry allocator for the new mapping. Now, pmap_enter()
attempts to recycle the old pv entry, saving two calls to the pv entry
allocator.

Simplify the inner loop of get_pv_entry(): While iterating over the page's
pv list, there is no point in checking whether or not the pv list is empty,
wait instead until the loop completes.

Fix lock order reversal, unlock page queue and pmap locks before
calling uma_zfree().
Also if needed, acquire page queue lock before modifying pte page
attributes.

Approved by: rrs (mentor)

Get rid of unused variable 'virtual_sys_start'.

If 'timer2hz' is zero then we don't need to call 'timer2clock()' directly. It
will be called automatically by 'timer1clock()'.

Do profiling as often as possible by running it as the same frequency as
'timer1hz'. The statistics clock is run as close to 128Hz as possible.

Pointed out by: mav@

Call VM_WAIT in pmap_ptpgzone_allocf() if M_WAITOK is set.
Removed unused variable.

Approved by: rrs (mentor)

Simplify clock interrupt handling on mips by using the new KPI - timer1clock()
and timer2clock().

Dynamically adjust the tick frequency depending on the value of 'hz'. Tested
with hz values of 100, 1000 and 2000.

- Fix kseg0 address calculation - it doesn't always start at
page boundary
- Add cache ops to ensure memory validity before/after
copy operation

Push down page queues lock acquisition in pmap_enter_object() and
pmap_is_referenced(). Eliminate the corresponding page queues lock
acquisitions from vm_map_pmap_enter() and mincore(), respectively. In
mincore(), this allows some additional cases to complete without ever
acquiring the page queues lock.

Assert that the page is managed in pmap_is_referenced().

On powerpc/aim, push down the page queues lock acquisition from
moea*_is_modified() and moea*_is_referenced() into moea*_query_bit().
Again, this will allow some additional cases to complete without ever
acquiring the page queues lock.

Reorder a few statements in vm_page_dontneed() so that a race can't lead
to an old reference persisting. This scenario is described in detail by a
comment.

Correct a spelling error in vm_page_dontneed().

Assert that the object is locked in vm_page_clear_dirty(), and restrict the
page queues lock assertion to just those cases in which the page is
currently writeable.

Add object locking to vnode_pager_generic_putpages(). This was the one
and only place where vm_page_clear_dirty() was being called without the
object being locked.

Eliminate an unnecessary vm_page_lock() around vnode_pager_setsize()'s call
to vm_page_clear_dirty().

Change vnode_pager_generic_putpages() to the modern-style of function
definition. Also, change the name of one of the parameters to follow
virtual memory system naming conventions.

Reviewed by: kib

Get rid of empty and unused KSEG0TEXT macros.

Fix mips kernel build breakage caused by revision 208504.

Roughly half of a typical pmap_mincore() implementation is machine-
independent code. Move this code into mincore(), and eliminate the
page queues lock from pmap_mincore().

Push down the page queues lock into pmap_clear_modify(),
pmap_clear_reference(), and pmap_is_modified(). Assert that these
functions are never passed an unmanaged page.

Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m:
Contrary to what the comment says, pmap_mincore() is not simply an
optimization. Without a complete pmap_mincore() implementation,
mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED
because only the pmap can provide this information.

Eliminate the page queues lock from vfs_setdirty_locked_object(),
vm_pageout_clean(), vm_object_page_collect_flush(), and
vm_object_page_clean(). Generally speaking, these are all accesses
to the page's dirty field, which are synchronized by the containing
vm object's lock.

Reduce the scope of the page queues lock in vm_object_madvise() and
vm_page_dontneed().

Reviewed by: kib (an earlier version)

Remove unused code in sys/mips/rmi :
- ehcireg.h,ehcivar.h : USB related files from old merge
- pcibus.c : was merged into xlr_pci.c earlier
- xlr_boot1_console.c : obsolete console code using bootloader hooks
- sys/mips/rmi/perfmon* : obsolete custom performance monitoring code

Approved by: rrs (mentor)

Reorganize syscall entry and leave handling.

Extend struct sysvec with three new elements:
sv_fetch_syscall_args - the method to fetch syscall arguments from
usermode into struct syscall_args. The structure is machine-depended
(this might be reconsidered after all architectures are converted).
sv_set_syscall_retval - the method to set a return value for usermode
from the syscall. It is a generalization of
cpu_set_syscall_retval(9) to allow ABIs to override the way to set a
return value.
sv_syscallnames - the table of syscall names.

Use sv_set_syscall_retval in kern_sigsuspend() instead of hardcoding
the call to cpu_set_syscall_retval().

The new functions syscallenter(9) and syscallret(9) are provided that
use sv_*syscall* pointers and contain the common repeated code from
the syscall() implementations for the architecture-specific syscall
trap handlers.

Syscallenter() fetches arguments, calls syscall implementation from
ABI sysent table, and set up return frame. The end of syscall
bookkeeping is done by syscallret().

Take advantage of single place for MI syscall handling code and
implement ptrace_lwpinfo pl_flags PL_FLAG_SCE, PL_FLAG_SCX and
PL_FLAG_EXEC. The SCE and SCX flags notify the debugger that the
thread is stopped at syscall entry or return point respectively. The
EXEC flag augments SCX and notifies debugger that the process address
space was changed by one of exec(2)-family syscalls.

The i386, amd64, sparc64, sun4v, powerpc and ia64 syscall()s are
changed to use syscallenter()/syscallret(). MIPS and arm are not
converted and use the mostly unchanged syscall() implementation.

Reviewed by: jhb, marcel, marius, nwhitehorn, stas
Tested by: marcel (ia64), marius (sparc64), nwhitehorn (powerpc),
stas (mips)
MFC after: 1 month

- Use ptpgzone zone to allocate page table pages irrespective of the amount of
memory on a platform. Tested on the Sibyte with 256MB and 1GB memory
configurations.

- Replace vtophys() with MIPS_KSEG0_TO_PHYS() to convert a page table
page's virtual address to physical. We can safely do this because
page table pages are allocated out of KSEG0.

- Add an assertion to verify that when a page table page is freed it
contains all zeroes. We can now use it after allocation without
zeroing it.

- Adjust the whitespace for the lines that output fields in 'show pcpu' in
DDB so that all the fields line up.
- Print out the tid of the per-CPU idlethread instead of the pid since
the idle process is now shared across all idle threads.

MFC after: 1 month

Changes to boot on a subset of threads on an XLR/XLS core.
- Adds re-partitioning TLB per core for enabled threads.
- Adds hardware thread id to cpuid mapping
- updates rge driver packet distribution and message ring handling
threads to be started based on hardware thread id.
- remove unused early debugging code to set control registers.
- coding style fixes

Approved by: rrs (mentor)

Add a device description.

Fix Sibyte SMP kernel breakage caused by r208249.

We need to include the header file that provides declaration of the
smp_topo_none() function.

Adds the file I forgot to add... that handles
the mpwait.S for RMI

Approved by: JC

Adds JC's cleanup patches that fix it so
we call an platform dependant topo function as
well as clean up all the XLR specific ifdefs around
smp platform init.

Obtained from: JC

On entry to pmap_enter(), assert that the page is busy. While I'm
here, make the style of assertion used by pmap_enter() consistent
across all architectures.

On entry to pmap_remove_write(), assert that the page is neither
unmanaged nor fictitious, since we cannot remove write access to
either kind of page.

With the push down of the page queues lock, pmap_remove_write() cannot
condition its behavior on the state of the PG_WRITEABLE flag if the
page is busy. Assert that the object containing the page is locked.
This allows us to know that the page will neither become busy nor will
PG_WRITEABLE be set on it while pmap_remove_write() is running.

Correct a long-standing bug in vm_page_cowsetup(). We cannot possibly
do copy-on-write-based zero-copy transmit on unmanaged or fictitious
pages, so don't even try. Previously, the call to pmap_remove_write()
would have failed silently.

This pushes all of JC's patches that I have in place. I
am now able to run 32 cores ok.. but I still will hang
on buildworld with a NFS problem. I suspect I am missing
a patch for the netlogic rge driver.

JC check and see if I am missing anything except your
core-mask changes

Obtained from: JC

Remove some stray ';'s

Submitted by: marc balmer

Push down the page queues into vm_page_cache(), vm_page_try_to_cache(), and
vm_page_try_to_free(). Consequently, push down the page queues lock into
pmap_enter_quick(), pmap_page_wired_mapped(), pmap_remove_all(), and
pmap_remove_write().

Push down the page queues lock into Xen's pmap_page_is_mapped(). (I
overlooked the Xen pmap in r207702.)

Switch to a per-processor counter for the total number of pages cached.

Eliminate dead code.

Fix DDB backtrace involving kernel modules.

We can no longer assume that all valid program counter values reside
within the kernel object file.

Add new tunable 'net.link.ifqmaxlen' to set default send interface
queue length. The default value for this parameter is 50, which is
quite low for many of today's uses and the only way to modify this
parameter right now is to edit if_var.h file. Also add read-only
sysctl with the same name, so that it's possible to retrieve the
current value.

MFC after: 1 month

Remove redundant checking of sc_leaving (uart_intr() already handles this).

Approved by: marcel

Enable AH_RXCFG_SDMAMW_4BYTES option. See NOTES file for why this is
workaround (WAR) is needed.

Put the -current debugging options back into AR71XX.

Bug in the memory mapping module. The wrong
physaddr was being used in the macro (1 should be
used not 2)...

Obtained from: JC

On Alan's advice, rather than do a wholesale conversion on a single
architecture from page queue lock to a hashed array of page locks
(based on a patch by Jeff Roberson), I've implemented page lock
support in the MI code and have only moved vm_page's hold_count
out from under page queue mutex to page lock. This changes
pmap_extract_and_hold on all pmaps.

Supported by: Bitgravity Inc.

Discussed with: alc, jeffr, and kib

Adapt i386 r207205 to mips: Clearing PV_TABLE_REF and setting the page's
PG_REFERENCED flag in pmap_protect() can't really be justified, so don't
do it.

Style: use #define<TAB> instead of #define<SPACE>.

Noted by: bde, pluknet gmail com
MFC after: 11 days

Resurrect pmap_is_referenced() and use it in mincore(). Essentially,
pmap_ts_referenced() is not always appropriate for checking whether or
not pages have been referenced because it clears any reference bits
that it encounters. For example, in mincore(), clearing the reference
bits has two negative consequences. First, it throws off the activity
count calculations performed by the page daemon. Specifically, a page
on which mincore() has called pmap_ts_referenced() looks less active
to the page daemon than it should. Consequently, the page could be
deactivated prematurely by the page daemon. Arguably, this problem
could be fixed by having mincore() duplicate the activity count
calculation on the page. However, there is a second problem for which
that is not a solution. In order to clear a reference on a 4KB page,
it may be necessary to demote a 2/4MB page mapping. Thus, a mincore()
by one process can have the side effect of demoting a superpage
mapping within another process!

Move the constants specifying the size of struct kinfo_proc into
machine-specific header files. Add KINFO_PROC32_SIZE for struct
kinfo_proc32 for architectures providing COMPAT_FREEBSD32. Add
CTASSERT for the size of struct kinfo_proc32.

Submitted by: pluknet
Reviewed by: imp, jhb, nwhitehorn
MFC after: 2 weeks

Most MIPS systems have a comparatively-sparse physical memory layout. Switch
to using the sparse physseg layout in the VM system.

Build some nops into CLEAR_STATUS here to make sure that the following
instructions can't be interrupted.

o) Remove default MAXMEM on SWARM; pmap can readily use lmem for >512M
physical addresses.
o) Set a local maxmem in sb_machdep.c to avoid trying to use pages over 2^64
under 32-bit ABIs. Our pmap needs corrected to use vm_paddr_t consistently,
then we can make vm_paddr_t 64-bit under 32-bit ABIs and add code in pmap
to limit phys_avail by the maximum PFN that a 32-bit PTE can hold.

Large memory mappings are always CPU local and always done with interrupts
disabled. Be doubly-sure that we don't try to do a TLB shootdown on SMP
systems for those mappings.

Submitted by: C. Jayachandran

Fix compilation error.

tick.c:298:5: error: "KDTRACE_HOOKS" is not defined

Change USB_DEBUG to #ifdef and allow it to be turned off. Previously this had
the illusion of a tunable setting but was always turned on regardless.

MFC after: 1 week

Add the necessary hooks for dtrace cyclic module.

Remove svn:executable prop.

Fix MALTA64 build.

Remove unused file.

o) Eliminate the "stand" frame and its use. Use CALLFRAME_* everywhere.
o) Use <machine/asm.h> macros for register-width, etc., rather than doing it
by hand in a few more assembly files.
o) Reduce diffs between various bits of TLB refill code in exception.S and
between interrupt processing code.
o) Use PTR_* to operate on registers that are pointers (e.g. sp).
o) Add and use a macro, CLEAR_PTE_SWBITS rather than using the
mysteriously-named WIRED_SHIFT to select bits to truncate when loading PTEs.
o) Don't doubly disable interrupts by moving zero to the status register,
especially since that has the nasty side-effect of taking us out of 64-bit
mode.
o) Use CLEAR_STATUS to disable interrupts the first time.
o) Keep SR_PX set as well as SR_[KSU]X when doing exception processing. This
is the bit that determines whether 64-bit operations are allowed.
o) Don't enable interrupts until configure_final(), like most other ports.

o) Fix XKPHYS physical address extraction. Also define cache coherency
attributes for XKPHYS.
o) Make coprocessor 0 accessor function macros for register+selector registers
take the full name so that e.g. (as done in this commit), prid selector 1
can be written through mips_wr_ebase() rather than mips_wr_prid1().
o) Allow for sign extension of 32-bit segment addresses.
o) Remove an unused MIPS-I register number.

o) Add a VM find-space option, VMFS_TLB_ALIGNED_SPACE, which searches the
address space for an address as aligned by the new pmap_align_tlb()
function, which is for constraints imposed by the TLB. [1]
o) Add a kmem_alloc_nofault_space() function, which acts like
kmem_alloc_nofault() but allows the caller to specify which find-space
option to use. [1]
o) Use kmem_alloc_nofault_space() with VMFS_TLB_ALIGNED_SPACE to allocate the
kernel stack address on MIPS. [1]
o) Make pmap_align_tlb() on MIPS align addresses so that they do not start on
an odd boundary within the TLB, so that they are suitable for insertion as
wired entries and do not have to share a TLB entry with another mapping,
assuming they are appropriately-sized.
o) Eliminate md_realstack now that the kstack will be appropriately-aligned on
MIPS.
o) Increase the number of guard pages to 2 so that we retain the proper
alignment of the kstack address.

Reviewed by: [1] alc
X-MFC-after: Making sure alc has not come up with a better interface.

Delete svn:executable prop.

o) Make pcb_onfault a pointer rather than an obscure integer value.
o) Mask off PAGE_MASK bits in pmap_update_page, etc., rather than modifying the
badvaddr in trapframe. Some nearby interfaces already did this.
o) Make PTEs "unsigned int" for now, not "unsigned long" -- we are only ready
for them to be 32-bit on 64-bit platforms.
o) Rather than using pmap_segmap and calculating the offset into the page table
by hand in trap.c, use pmap_pte().
o) Remove unused quad_syscall variable in trap.c.
o) Log things for illegal instructions like we do for bad page faults.
o) Various cast cleanups related to how to print registers.
o) When logging page faults, show the page table information not just for the
program counter, but for the fault address.
o) Modify support.S to use ABI-neutral macros for operating on pointers.
o) Consistently use CALLFRAME_SIZ rather than STAND_FRAME_SIZE, etc.
o) Remove unused insque/remque functions.
o) Remove some coprocessor 0 accessor functions implemented in assembly that
are unused and have inline assembly counterparts.

o) Add NPDEPG, like NPTEPG but for PDEs.
o) Remove NBPG, PGOFSET and PGSHIFT. Use the standard names.
o) Remove some unused macros and move things from param.h to vmparam.h that
belong in the latter. (Actually, all of the kernel segment values, virtual
addresses, etc., belong in one place, but this is a step in the right
direction.)

o) Add SMP support for Octeon using U-Boot to launch all the processors at the
same time.
o) Remove some unused trivial uart functions from octeon_machdep now that the
uart part is fully working and they are unused.
o) Use __func__ instead of __FUNCTION__.
o) Use intr_*() instead of other routines that do the same thing.
o) Remove some duplicate printfs from the Octeon port, as well as duplicate
setting of Maxmem.
o) Use the right frequency divider on Octeon.
o) Use PCPU_GET(cpuid) consistently to get the cpuid of the running core.
o) Remove some unused macros in the Octeon port.
o) Use mips_sync() around use of the global dpcpu, whose value may not be
visible to APs at first.
o) When loading the first thread's stack, use macros to make the code correct
for n64 as well.
o) Remove stub, do-nothing FAU init/enable/disable functions from the RGMX
driver.

o) Back out my previous change to SWARM; some of it was to address an issue
that turned out to be unrelated, and the rest was, as pointed out by Neel,
just wrong-headed.
o) Tweak mem.c to fix use of /dev/kmem for direct-mapped addresses.

o) Use inline functions to access coprocessor 0 registers rather than external
ones implemented using assembly.
o) Use TRAPF_USERMODE() consistently rather than USERMODE(). Eliminate
<machine/psl.h> as a result.
o) Use intr_*() rather than *intr(), consistently.
o) Use register_t instead of u_int in some trap code.
o) Merge some more endian-related macros to machine/asm.h from NetBSD.
o) Add PTR_LI macro, which loads an address with the correct sign-extension for
a pointer.
o) Restore interrupts when bailing out due to an excessive IRQ in
nexus_setup_intr().
o) Remove unused functions from psraccess.S.
o) Enter temporary virtual entries for large memory access into the page tables
rather than simply hoping they stay resident in the TLB and we don't need to
do a refill for them.
o) Abstract out large memory mapping setup/teardown using some macros.
o) Do mips_dcache_wbinv_range() when using temporary virtual addresses just
like we do when we can use the direct map.

o) Remove code related to VM_ALLOC_WIRED_TLB_PG_POOL, VM_KERNEL_ALLOC_OFFSET
and floating pages. They are unused and unsupported.

Adjust limits and formats for ABIs with 64-bit longs.

o) Use the direct map where possible for uiomove_fromphys, based on code from
sparc64.
o) Use uiomove_fromphys rather than the broken fpage mechanism for /dev/mem.
o) Update sf_buf allocator to not share buffers and to do a pmap_qremove when
done with an sf_buf so as to better track valid mappings.

o) Fix the intr_* functions to not spam the whole status register, just the IE
bit.
o) Remove some unused inlines.
o) Generate CP0 access functions for 64-bit TLB registers when building for
n64.
o) Add an inline function version of the COP0_SYNC macro.

Set KERNLOADADDR and TARGET_BIG_ENDIAN for SWARM.

Remove some unused header files.

Destroy the pmap 'pm_mutex' in pmap_release() otherwise we will panic
subsequently in pmap_pinit() with the following signature:

panic: lock "pmap" 0xc7878bc8 already initialized

This bug was uncovered by the changes made to vm_map.c in r206140.

Revert the vm_machdep.c part of r205072.

This causes a panic in vm_thread_dispose() when it tries to add this kstack
to the kstack cache. This happens only when 'td_kstack' is not (PAGE_SIZE * 2)
bytes aligned and we have unmapped the page at that address in cpu_thread_alloc.

Pointed out by: nwhitehorn@

Fix a bug where bus_dma_load_xxx() would not bounce misaligned buffers
due to rounding the buffer's physical address to the beginning of its
page. This fixes a panic in arge(4) when using PPPoE.

Reported by: Jakob van Santen <vansanten at wisc dot edu>
Reviewed by: gonzo
Obtained from: amd64

Add BUS_SPACE_UNRESTRICTED and define it to be ~0, just like all the
other platforms.

- Fix mutex type for miibus_mtx: it's not spinlock, it's def lock

Enable module builds now that the build completes for them. This
should get them into the universe rotation.

Replace sb_store64()/sb_load64() with mips3_sd()/mips3_ld() respectively.

Obtained from NetBSD.

Suggested by: jmallett@

Change the arguments of exec_setregs() so that it receives a pointer
to the image_params struct instead of several members of that struct
individually. This makes it easier to expand its arguments in the future
without touching all platforms.

Reviewed by: jhb

Fix periodic "t_delta 16.01359db7eb5eb3c0 too long" messages on the console by
accounting for the "lost time" between when the timer interrupt fired
and when clock_intr() actually started executing.