Revert r330897:

This was intended to be a non-functional change. It wasn't. The commit
message was thus wrong. In addition it broke arm, and merged crypto
related code.

Revert with prejudice.

This revert skips files touched in r316370 since that commit was since
MFCed. This revert also skips files that require $FreeBSD$ property
changes.

Thank you to those who helped me get out of this mess including but not
limited to gonzo, kevans, rgrimes.

Requested by: gjb (re)

Partial merge of the SPDX changes

These changes are incomplete but are making it difficult
to determine what other changes can/should be merged.

No objections from: pfg

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Convert remaining {g,s}etenv->kern_{g,s}etenv
left over from r273174.

Reported by: bz

Follow up to r225617. In order to maximize the re-usability of kernel code
in userland rename in-kernel getenv()/setenv() to kern_setenv()/kern_getenv().
This fixes a namespace collision with libc symbols.

Submitted by: kmacy
Tested by: make universe

Remove platform APIs which are not used by any code and which had only stub
implementations or no implementation on all platforms.

Some of these functions might be good ideas, but their semantics were unclear
given the lack of implementation, and an unlucky porter could be fooled into
trying to implement them or, worse, being baffled when something like
platform_trap_enter() failed to be called.

etire the cpumask_t type and replace it with cpuset_t usage.

This is intended to fix the bug where cpu mask objects are
capped to 32. MAXCPU, then, can now arbitrarely bumped to whatever
value. Anyway, as long as several structures in the kernel are
statically allocated and sized as MAXCPU, it is suggested to keep it
as low as possible for the time being.

Technical notes on this commit itself:
- More functions to handle with cpuset_t objects are introduced.
The most notable are cpusetobj_ffs() (which calculates a ffs(3)
for a cpuset_t object), cpusetobj_strprint() (which prepares a string
representing a cpuset_t object) and cpusetobj_strscan() (which
creates a valid cpuset_t starting from a string representation).
- pc_cpumask and pc_other_cpus are target to be removed soon.
With the moving from cpumask_t to cpuset_t they are now inefficient
and not really useful. Anyway, for the time being, please note that
access to pcpu datas is protected by sched_pin() in order to avoid
migrating the CPU while reading more than one (possible) word
- Please note that size of cpuset_t objects may differ between kernel
and userland. While this is not directly related to the patch itself,
it is good to understand that concept and possibly use the patch
as a reference on how to deal with cpuset_t objects in userland, when
accessing kernland members.
- KTR_CPUMASK is changed and now is represented through a string, to be
set as the example reported in NOTES.

Please additively note that no MAXCPU is bumped in this patch, but
private testing has been done until to MAXCPU=128 on a real 8x8x2(htt)
machine (amd64).

Please note that the FreeBSD version is not yet bumped because of
the upcoming pcpu changes. However, note that this patch is not
targeted for MFC.

People to thank for the time spent on this patch:
- sbruno, pluknet and Nicholas Esborn (nick AT desert DOT net) tested
several revision of the patches and really helped in improving
stability of this work.
- marius fixed several bugs in the sparc64 implementation and reviewed
patches related to ktr.
- jeff and jhb discussed the basic approach followed.
- kib and marcel made targeted review on some specific part of the
patch.
- marius, art, nwhitehorn and andreast reviewed MD specific part of
the patch.
- marius, andreast, gonzo, nwhitehorn and jceel tested MD specific
implementations of the patch.
- Other people have made contributions on other patches that have been
already committed and have been listed separately.

Companies that should be mentioned for having participated at several
degrees:
- Yahoo! for having offered the machines used for testing on big
count of CPUs.
- The FreeBSD Foundation for having sponsored my devsummit attendance,
which has been instrumental.
- Sandvine for having offered offices and infrastructure during
development.

(I really hope I didn't forget anyone, if it happened I apologize in
advance).

Allow the platform code to return a bitmask of running cores rather than just
a number of cores, this allows for a sparse set of CPUs. Implement support
for sparse core masks on Octeon.

XXX jeff@ suggests that all_cpus should include cores that are offline or
running other applications/OSes, so the platform API should be further
extended to allow us to set all_cpus to include all cores that are
physically-present as opposed to only those that are running FreeBSD.

Submitted by: Bhanu Prakash (with modifications)
Reviewed by: jchandra
Glanced at by: kib, jeff, jhb

Fix format of physical addresses; this fixes the n32 build.

Support for 64 bit PTEs on n32 and n64 compilation.

In n32 and n64, add support for physical address above 4GB by having
64 bit page table entries and physical addresses. Major changes are:
- param.h: update PTE sizes, masks and shift values to support 64 bit PTEs.
- param.h: remove DELAY(), mips_btop(same as atop), mips_ptob (same as
ptoa), and reformat.
- param.h: remove casting to unsigned long in trunc_page and round_page
since this will be used on physical addresses.
- _types.h: have 64 bit __vm_paddr_t for n32.
- pte.h: update TLB LO0/1 access macros to support 64 bit PTE
- pte.h: assembly macros for PTE operations.
- proc.h: md_upte is now 64 bit for n32 and n64.
- exception.S and swtch.S: use the new PTE macros for PTE operations.
- cpufunc.h: TLB_LO0/1 registers are 64bit for n32 and n64.
- xlr_machdep.c: Add memory segments above 4GB to phys_avail[] as they are
supported now.

Reviewed by: jmallett (earlier version)

The message ring interrupt needs to be enabled for all cpus, not just
the ones which run the message ring handler.

Some bits of the interrupt mask are part of the status register which is
saved with the process context, and these bits are initialized from the
cpu on which the process is created. This means that all the processes
should have the same value for these interrupt mask bits, so that the
interrupt mask remains the same regardless of what thread is scheduled
on the cpu.

Submitted by: Sriram Gorti (srgorti at netlogicmicro dot com)

- dump_avail layout should be sequence of [start, end)
pairs, not <start, size>.

Spotted by: alc@

- Populate dump_avail with proper values from phys_avail

On uniprocessor, warn and fixup hardware cpu mask if more than on CPU
is enabled by the bootloader.

Update message ring handling code for XLR/XLS

- Wakeup multiple threads per core using message ring watermark interrupts.
- Update message ring handler registration, use the real device station id
for registering interrupts.
- rge/nlge: update for the new message ring registration code.
- rge/nlge: use 2 message ring stations for incoming packets, this will
allow more messages to be queued.
- nlge: comment fixes, remove unused variable
- style and whitespace fixes

Clean up and update sys/mips/rmi/rmi_mips_exts.h

- Provide 64 bit implementations for some macros. On n64 and n32,
don't split 64 bit values.
- No need for 32 bit ops for control registers.
- Fix few bugs (write control reg, write_c0_register64).
- Re-write EIRR/EIMR/CPUID operations using read_c0_registerXX, no
need of inline assembly.
- rename control reg access functions to avoid phnx, update callers.
- stlye/whitespace fixes.

XLR/XLS hardware interrupts should be programmed level triggered at the
PIC. This should fix the interrupt releated issues seen after the
interrupt handling re-write for SMP.

Updates for the RMI MIPS platform code
- set cache_coherent_dma flag in cpuinfo for XLR, this will make sure that
BUS_DMA_COHERENT flag is handled correctly in busdma_machdep.c
- iodi.c, call device_get_name() just once
- clear RMI specific EIRR while intializing CPUs
- remove debug print in intr_machdep.c

Clean up header files in RMI platform code (sys/mips/rmi), and remove
unused files.

- remove clock.c and clock.h, these are not used after the new timer
code was added.
- remove duplicated include files, fix header file ordering, remove
some unneeded includes.
- rename mips/rmi/shared_structs.h which contains the RMI boot loader
interface to mips/rmi/rmi_boot_info.h. Remove unused files
mips/rmi/shared_structs_func.h and sys/mips/rmi/shared_structs_offsets.h
- merge mips/rmi/xlrconfig.h and mips/rmi/rmi_mips_exts.h, and remove
duplicated functions.
- nlge - minor change to remove unneeded argument.
- Add FreeBSD svn keyword for headers

Revamp XLR interrupt handling, the previous scheme does not work well on
SMP.

We used to route all PIC based interrupts to cpu 0, and used the per-CPU
interrupt mask to enable/disable interrupts. But the interrupt threads can
run on any cpu on SMP, and the interrupt thread will re-enable the interrupts
on the CPU it runs on when it is done, and not on cpu0 where the PIC will
still send interrupts to.

The fix is move the disable/enable for PIC based interrupts to PIC, we will
ack on PIC only when the interrupt thread is done, and we do not use the
per-CPU interrupt mask.

The changes also introduce a way for subsystems to add a function that
will be called to clear the interrupt on the subsystem. Currently This is
used by the PCI/PCIe for doing additional work during the interrupt
handling.

Provide timecounter based on XLR PIC timer.

- Use timer 7 in XLR PIC as a 32 counter
- provide pic_init_timer(), pic_set_timer(), pic_timer_count32() and
pic_timer_count() PIC timer operations.
- register this timer as platform_timecounter on rmi platform.

XLR PIC code update.
- Fix a bug in xlr_pic_init (use irq in PIC_IRQ_IS_EDGE_TRIGGERED)
- use new macro PIC_INTR_TO_IRQ() and PIC_IRT_x() in xlr_pic_init

XLR PIC code update and style(9) fixes.

- style(9) fixes to mips/rmi platform files
- update pic.h to add pic_setup_intr() and use pic_setup_intr() for setting
up interrupts which are routed thru PIC.
- remove rmi_spin_mutex_safe and haslock, and make sure that the functions
are called only after mutexes are available.

RMI XLR platform code clean-up.

- move PIC code to xlr_machdep.c
- move fast message ring code completely to on_chip.c
- move memory initialization to a new function xlr_mem_init()
- style fixes

Fix for 64 bit compile, with SMP enabled.

Fixup mips/rmi for the new mips timer code(r210403). This will get XLR
booting again.

The code is a copy of the mips/mips/tick.c with minor modifications for
XLR interrupt handling. Disable mips/rmi/clock.c for now, the PIC based
timer code will be added later.

Fix for 64 bit compilation.

RMI bootloader passes argv[] and envp[] as an array of 32 bit pointers.
Convert the pointers to correct pointer type before use.

64 bit compilation support XLR platform code.
Mostly changes to make casting between int and pointer and printing
64bit values safe for 32 and 64 bit compile.

Approved by: rrs

Changes to boot on a subset of threads on an XLR/XLS core.
- Adds re-partitioning TLB per core for enabled threads.
- Adds hardware thread id to cpuid mapping
- updates rge driver packet distribution and message ring handling
threads to be started based on hardware thread id.
- remove unused early debugging code to set control registers.
- coding style fixes

Approved by: rrs (mentor)

Adds JC's cleanup patches that fix it so
we call an platform dependant topo function as
well as clean up all the XLR specific ifdefs around
smp platform init.

Obtained from: JC

This pushes all of JC's patches that I have in place. I
am now able to run 32 cores ok.. but I still will hang
on buildworld with a NFS problem. I suspect I am missing
a patch for the netlogic rge driver.

JC check and see if I am missing anything except your
core-mask changes

Obtained from: JC

Its possible that our RMI box has memory extending
above 4Gig. If so when we add the base address with
the size we will wrap. So for now we just ignore
such memory and only use what we can. When we
get 64 bit working then we will be much better ;->

Changes the msg ring so its a filter not a
handler. Somehow rrs missed this.. Thanks
to JC for catching this ;-)

Obtained from: JC (jayachandranc@netlogicmicro.com

Fix up the msg ring driver a bit tighter
so that we don't loose an interrupt which
we appeared to be doing.

Changes the order of the setting the int happened (inside
the lock).

This hopefully will fix the network problem I was seeing.
Basically the msg ring interrupt was being re-enabled
inside a spinlock as the thread set it self up for rescheduling.
This won't work since inside the re-enable is another
spin lock.. which means on return from the reenable
the interrupts have been reenabled. Thus you would
get a clock int and end up panicing holding a spin
lock to long :-o

Set the svn:eol-style = native and svn:mime-type = text/plain
properties on all files in this tree.

Submitted by: rpaulo@

Place proper svn:keywords tag on all these files. They were created
somehow without them on projects/mips, and that mistake was propigated
over to head.

Submitted by: rpaulo@

Merge from projects/mips to head by hand:

Copy over the support files (except sys/conf and sys/mips/conf) for
RMI XLR processor support. This port has been contributed by RMI and
brought up to date by Randal Stewart (rrs@). This port is a work in
progress, and there might still be significant changes. The port
makes it to multi-user, but is still early beta.

Rename mips_pcpu_init to mips_pcpu0_init since it applies only to the
BSP. Provide a missing prototype.

Centralize initialization of pcpu, and set curthread early...

Fixes so kdb works.

Ok With this commit we actually get through
the mi_startup (or to the last of it).. and
hit a panic after :

uart0: <16550 or compatible> on iodi0
Trap cause = 2 (TLB miss....)

I did have to take the pci bus OUT of the
build to get this far, hit a cache error with
the PCI code in. Interesting thing is the machine
reboots too ;-)

ok we now get so that the uart init's and we can print. We
cannot set baud rate as they did in 6.4, this hoses things and
we loose our 38400 default term.

We now lock somewhere in tcinit.

adds rmi specific mips extensions file and makes sure
the includes point to the new place.

White space changes

> Description of fields to fill in above: 76 columns --|
> PR: If a GNATS PR is affected by the change.
> Submitted by: If someone else sent in the change.
> Reviewed by: If someone else reviewed your modification.
> Approved by: If you needed approval for this commit.
> Obtained from: If the change is from a third party.
> MFC after: N [day[s]|week[s]|month[s]]. Request a reminder email.
> Security: Vulnerability reference (one per line) or description.
> Empty fields above will be automatically removed.

M rmi/xls_ehci.c
M rmi/clock.h
M rmi/xlr_pci.c
M rmi/perfmon.h
M rmi/uart_bus_xlr_iodi.c
M rmi/perfmon_percpu.c
M rmi/iodi.c
M rmi/pcibus.c
M rmi/perfmon_kern.c
M rmi/perfmon_xlrconfig.h
M rmi/pcibus.h
M rmi/tick.c
M rmi/xlr_boot1_console.c
M rmi/debug.h
M rmi/uart_cpu_mips_xlr.c
M rmi/xlrconfig.h
M rmi/interrupt.h
M rmi/xlr_i2c.c
M rmi/shared_structs.h
M rmi/msgring.c
M rmi/iomap.h
M rmi/ehcireg.h
M rmi/msgring.h
M rmi/shared_structs_func.h
M rmi/on_chip.c
M rmi/pic.h
M rmi/xlr_machdep.c
M rmi/ehcivar.h
M rmi/board.c
M rmi/clock.c
M rmi/shared_structs_offsets.h
M rmi/perfmon_utils.h
M rmi/board.h
M rmi/msgring_xls.c
M rmi/intr_machdep.c

more Updates on the RMI code close to compiling now ;-)

Fix build from r198563 (again). Sigh.

Fix build from r198563

Use init_static_kenv() and setenv() to simplify the environment string handling.

More initial RMI files. Note that these so far do NOT
compile and many of them may disappear. For example
the xlr_boot1_console.c is old code that is ifdef'd out.
I will clean these sorts of things up as I make progress
on the port. So far the only thing I have I think straightened
out is the bits around the interupt handling... and hey that
may be broke ;-)