MFC r302504, r302666, r302668, r302932, r302933:
Add emulation for Intel e1000 (e82545) network adapter.

The code was successfully tested with FreeBSD, Linux, Solaris and Windows
guests. This interface is predictably slower (about 2x) then virtio-net,
but it is very helpful for guests not supporting virtio-net by default.

Thanks to Jeremiah Lott and Peter Grehan for doing original heavy lifting.

MFC r302459: Allow AHCI controller to support up to 32 arbitrary devices.

While old syntax is still supported, new syntax looks like this:

-s 3,ahci,hd:/dev/zvol/XXX,hd:/dev/zvol/YYY,cd:/storage/ZZZ.iso

Sponsored by: iXsystems, Inc.

MFC r284539, r284630, r284688, r284877, r285217, r285218,
r286837, r286838, r288470, r288522, r288524, r288826,
r289001

Pull in bhyve bug fixes and changes to allow UEFI booting.
This provides Windows support.

Tested on Intel and AMD with:
- Arch Linux i386+amd64 (kernel 4.3.3)
- Ubuntu 15.10 server 64-bit
- FreeBSD-CURRENT/amd64 20160127 snap
- FreeBSD 10.2 i386+amd64
- OpenBSD 5.8 i386+amd64
- SmartOS latest
- Windows 10 build 1511'

Huge thanks to Yamagi Burmeister who submitted the patch
and did the majority of the testing.

r284539 - bootrom mem allocation support
r284630 - Add SO_REUSEADDR when starting debug port
r284688 - Fix a regression in "movs" emulation
r284877 - verify_gla() non-zero segment base fix
r285217 - Always assert DCD and DSR in the uart
r285218 - devmem nodes moved to /dev/vmm.io/
r286837 - Add define for SATA Check-Power-Mode
r286838 - Add simple (no-op) SATA cmd emulations
r288470 - Increase virtio-blk indirect descs
r288522 - Firmware guest query interface
r288524 - Fix post-test typo
r288826 - Clean up SATA unimplemented cmd msg
r289001 - Add -l option to specify userboot path

Submitted by: Yamagi Burmeister
Approved by: re (kib)

MFC: r286409

Manpage cleanup.

- new sentence -> new line
- fix manpage references
- fix macro usage
- fix a typo

MFC r286621, r286622: Fix couple minor typos.

MFC r282209:
Emulate the 'bit test' instruction.

MFC r282259:
Re-implement RTC current time calculation to eliminate the possibility of
losing time.

MFC r282281:
Advertise the MTRR feature via CPUID and emulate the minimal set of MTRR MSRs.

MFC r282284:
When an instruction cannot be decoded just return to userspace so bhyve(8)
can dump the instruction bytes.

MFC r282287:
Don't require <sys/cpuset.h> to be always included before <machine/vmm.h>.

MFC r282296:
Emulate MSR_SYSCFG which is accessed by Linux on AMD cpus when MTRRs are
enabled.

MFC r282301:
Relax limits when transitioning a vector from the IRR to the ISR and also
when extinguishing it from the ISR in response to an EOI.

MFC r282335:
Advertise an additional memory BAR in the "dummy" device emulation.

MFC r282336:
Emulate machine check related MSRs to allow guest OSes like Windows to boot.

MFC r282351:
Don't advertise the Intel SMX capability to the guest.

MFC r282407:
Emulate the 'CMP r/m8, imm8' instruction.

MFC r282519:
Add macros for AMD-specific bits in MSR_EFER: LMSLE, FFXSR and TCE.

MFC r282520:
Emulate guest writes to EFER_MSR properly.

MFC r282558:
Deprecate the 3-way return values from vm_gla2gpa() and vm_copy_setup().

MFC r282571:
Check 'td_owepreempt' and yield the vcpu thread if it is set.

MFC r282595:
Allow byte reads of AHCI registers.

MFC r282784:
Handling indirect descriptors is a capability of the host and not one that
needs to be negotiated. Use the host capabilities field and not the negotiated
field when verifying that indirect descriptors are supported.

MFC r282788:
Allow configuration of the sector size advertised to the guest.

MFC r282865:
Set the subvendor field in config space to the vendor ID. This is required
by the Windows virtio drivers to correctly match a device.

MFC r282922:
Bump the size of the blockif scatter-gather list to 67.

MFC r283075:
Fix off-by-one in array index bounds check. bhyveload would allow you to
create 33 entries on an array that only has 32 slots

MFC r283168:
Temporarily revert r282922 which bumped the max descriptors.

MFC r283255:
Emulate the "CMP r/m, reg" instruction (opcode 39H).

MFC r283256:
Add an option "--get-vmcs-exit-inst-length" to display the instruction length
of the instruction that caused the VM-exit.

MFC r283264:
Change the header type of the emulated host-bridge from type 1 to type 0.

MFC r283293:
Don't rely on the 'VM-exit instruction length' field in the VMCS to always
have an accurate length on an EPT violation.

MFC r283299:
Remove bogus verification of instruction length after instruction decode.

MFC r283308:
Exceptions don't deliver an error code in real mode.

MFC r283657:
Fix non-deterministic delays when accessing a vcpu that was in "running" or
"sleeping" state.

MFC r283973:
Use tunable 'hw.vmm.svm.features' to disable specific SVM features even
though they might be available in hardware. Use tunable 'hw.vmm.svm.num_asids'
to limit the number of ASIDs used by the hypervisor.

MFC r284046:
Fix regression in 'verify_gla()' with the RIP-relative addressing mode.

MFC r284174:
Support guest writes to the TSC by enabling the "use TSC offsetting"
execution control.

MFC r276428:
Replace bhyve's minimal RTC emulation with a fully featured one in vmm.ko.

MFC r276432:
Initialize all fields of 'struct vm_exception exception' before passing it
to vm_inject_exception().

MFC r276763:
Clear blocking due to STI or MOV SS in the hypervisor when an instruction is
emulated or when the vcpu incurs an exception.

MFC r277149:
Clean up usage of 'struct vm_exception' to only to communicate information
from userspace to vmm.ko when injecting an exception.

MFC r277168:
Fix typo (missing comma).

MFC r277309:
Make the error message explicit instead of just printing the usage if the
virtual machine name is not specified.

MFC r277310:
Simplify instruction restart logic in bhyve.

MFC r277359:
Fix a bug in libvmmapi 'vm_copy_setup()' where it would return success even
if the 'gpa' was in the guest MMIO region.

MFC r277360:
MOVS instruction emulation.

MFC r277626:
Add macro to identify AVIC capability (advanced virtual interrupt controller)
in AMD processors.

MFC r279220:
Don't close a block context if it couldn't be opened avoiding a null deref.

MFC r279225:
Add "-u" option to bhyve(8) to indicate that the RTC should maintain UTC time.

MFC r279227:
Emulate MSR 0xC0011024 when running on AMD processors.

MFC r279228:
Always emulate MSR_PAT on Intel processors and don't rely on PAT save/restore
capability of VT-x. This lets bhyve run nested in older VMware versions that
don't support the PAT save/restore capability.

MFC r279540:
Fix warnings/errors when building vmm.ko with gcc.

MFC r274720:

Fix improper .Fx macro usage.

Sponsored by: The FreeBSD Foundation

MFC r271711:
Update the bhyve(8) manual to reflect that it is no
longer considered 'experimental.'

Approved by: re (delphij)
Sponsored by: The FreeBSD Foundation

MFC r267921, r267934, r267949, r267959, r267966, r268202, r268276,
r268427, r268428, r268521, r268638, r268639, r268701, r268777,
r268889, r268922, r269008, r269042, r269043, r269080, r269094,
r269108, r269109, r269281, r269317, r269700, r269896, r269962,
r269989.

Catch bhyve up to CURRENT.

Lightly tested with FreeBSD i386/amd64, Linux i386/amd64, and
OpenBSD/amd64. Still resolving an issue with OpenBSD/i386.

Many thanks to jhb@ for all the hard work on the prior MFCs !

r267921 - support the "mov r/m8, imm8" instruction
r267934 - document options
r267949 - set DMI vers/date to fixed values
r267959 - doc: sort cmd flags
r267966 - EPT misconf post-mortem info
r268202 - use correct flag for event index
r268276 - 64-bit virtio capability api
r268427 - invalidate guest TLB when cr3 is updated, needed for TSS
r268428 - identify vcpu's operating mode
r268521 - use correct offset in guest logical-to-linear translation
r268638 - chs value
r268639 - chs fake values
r268701 - instr emul operand/address size override prefix support
r268777 - emulation for legacy x86 task switching
r268889 - nested exception support
r268922 - fix INVARIANTS build
r269008 - emulate instructions found in the OpenBSD/i386 5.5 kernel
r269042 - fix fault injection
r269043 - Reduce VMEXIT_RESTARTs in task_switch.c
r269080 - fix issues in PUSH emulation
r269094 - simplify return values from the inout handlers
r269108 - don't return -1 from the push emulation handler
r269109 - avoid permanent sleep in vm_handle_hlt()
r269281 - list VT-x features in base kernel dmesg
r269317 - Mark AHCI fatal errors as not completed
r269700 - Support PCI extended config space in bhyve
r269896 - Minor cleanup
r269962 - use max guest memory when creating IOMMU domain
r269989 - fix interrupt mode names

MFC 264353,264509,264768,264770,264825,264846,264988,265114,265165,265365,
265941,265951,266390,266550,266910:
Various bhyve fixes:
- Don't save host's return address in 'struct vmxctx'.
- Permit non-32-bit accesses to local APIC registers.
- Factor out common ioport handler code.
- Use calloc() in favor of malloc + memset.
- Change the vlapic timer frequency to be in the ballpark of contemporary
hardware.
- Allow the guest to read the TSC via MSR 0x10.
- A VMCS is always inactive when it exits the vmx_run() loop. Remove
redundant code and the misleading comment that suggest otherwise.
- Ignore writes to microcode update MSR. This MSR is accessed by RHEL7
guest.
Add KTR tracepoints to annotate wrmsr and rdmsr VM exits.
- Provide an alias for the userboot console and name it 'comconsole'.
- Use EV_ADD to create an mevent and EV_ENABLE to enable it.
- abort(3) the process in response to a VMEXIT_ABORT.
- Don't include the guest memory segments in the bhyve(8) process core dump.
- Make the vmx asm code dtrace-fbt-friendly.
- Allow vmx_getdesc() and vmx_setdesc() to be called for a vcpu that is in
the VCPU_RUNNING state.
- Enable VMX in the IA32_FEATURE_CONTROL MSR if it not enabled and the MSR
isn't locked.

MFC 260847,264055,264867:
- Add a very simple virtio_random(4) driver for FreeBSD guests to harvest
entropy from hypervisors.
- Add support to bhyve for the virtio RNG entropy-source device to provide
entry to bhyve guests.

MFC 263432,265366,265376:
Fixes for vcpu management in bhyve:
- Use 'cpuset_t' to represent the vcpus active in a virtual machine.
- Modify the "-p" option to be more flexible when associating a 'vcpu' with
a 'hostcpu'.

MFC 261904,261905,262143,262184,264921,265211,267169,267292,267294:
Various PCI fixes:
- Allow PCI devices to be configured on all valid bus numbers from 0 to 255.
- Tweak the handling of PCI capabilities in emulated devices to remove
the non-standard zero capability list terminator.
- Add a check to validate that memory BARs of passthru devices are 4KB
aligned.
- Respect and track the enable bit in the PCI configuration address word.
- Handle quad-word access to 32-bit register pairs.

MFC 262139,262140,262236,262281,262532:
Various x2APIC fixes and enhancements:
- Use spinlocks for the vioapic.
- Handle the SELF_IPI MSR.
- Simplify the APIC mode switching between MMIO and x2APIC. The guest is
no longer allowed to switch modes at runtime. Instead, the desired mode
is set when the virtual machine is created.
- Disallow MMIO access in x2APIC mode and MSR access in xAPIC mode.
- Add support for x2APIC virtualization assist in Intel VT-x.

MFC 261217:
Remove support for legacy PCI devices. These haven't been needed since
support for LPC uart devices was added and it conflicts with upcoming
patches to add PCI INTx support.

Approved by: grehan

MFC 259635:
Add an option to ignore accesses by the guest to unimplemented MSRs.

Also, ignore a couple of SandyBridge uncore PMC MSRs that Centos 6.4 writes
to during boot.

MFC r260446

virtio-block does not exist, the correct name is virtio-blk.

PR: 185573
Submitted by: Allan Jude
Facilitated by: Snow B.V.

MFC r259302,r259413

r259302
bhyve(8) man page

r259413
mdoc: sort SEE ALSO

MFC: r286409

Manpage cleanup.

- new sentence -> new line
- fix manpage references
- fix macro usage
- fix a typo

MFC r286621, r286622: Fix couple minor typos.

MFC r282209:
Emulate the 'bit test' instruction.

MFC r282259:
Re-implement RTC current time calculation to eliminate the possibility of
losing time.

MFC r282281:
Advertise the MTRR feature via CPUID and emulate the minimal set of MTRR MSRs.

MFC r282284:
When an instruction cannot be decoded just return to userspace so bhyve(8)
can dump the instruction bytes.

MFC r282287:
Don't require <sys/cpuset.h> to be always included before <machine/vmm.h>.

MFC r282296:
Emulate MSR_SYSCFG which is accessed by Linux on AMD cpus when MTRRs are
enabled.

MFC r282301:
Relax limits when transitioning a vector from the IRR to the ISR and also
when extinguishing it from the ISR in response to an EOI.

MFC r282335:
Advertise an additional memory BAR in the "dummy" device emulation.

MFC r282336:
Emulate machine check related MSRs to allow guest OSes like Windows to boot.

MFC r282351:
Don't advertise the Intel SMX capability to the guest.

MFC r282407:
Emulate the 'CMP r/m8, imm8' instruction.

MFC r282519:
Add macros for AMD-specific bits in MSR_EFER: LMSLE, FFXSR and TCE.

MFC r282520:
Emulate guest writes to EFER_MSR properly.

MFC r282558:
Deprecate the 3-way return values from vm_gla2gpa() and vm_copy_setup().

MFC r282571:
Check 'td_owepreempt' and yield the vcpu thread if it is set.

MFC r282595:
Allow byte reads of AHCI registers.

MFC r282784:
Handling indirect descriptors is a capability of the host and not one that
needs to be negotiated. Use the host capabilities field and not the negotiated
field when verifying that indirect descriptors are supported.

MFC r282788:
Allow configuration of the sector size advertised to the guest.

MFC r282865:
Set the subvendor field in config space to the vendor ID. This is required
by the Windows virtio drivers to correctly match a device.

MFC r282922:
Bump the size of the blockif scatter-gather list to 67.

MFC r283075:
Fix off-by-one in array index bounds check. bhyveload would allow you to
create 33 entries on an array that only has 32 slots

MFC r283168:
Temporarily revert r282922 which bumped the max descriptors.

MFC r283255:
Emulate the "CMP r/m, reg" instruction (opcode 39H).

MFC r283256:
Add an option "--get-vmcs-exit-inst-length" to display the instruction length
of the instruction that caused the VM-exit.

MFC r283264:
Change the header type of the emulated host-bridge from type 1 to type 0.

MFC r283293:
Don't rely on the 'VM-exit instruction length' field in the VMCS to always
have an accurate length on an EPT violation.

MFC r283299:
Remove bogus verification of instruction length after instruction decode.

MFC r283308:
Exceptions don't deliver an error code in real mode.

MFC r283657:
Fix non-deterministic delays when accessing a vcpu that was in "running" or
"sleeping" state.

MFC r283973:
Use tunable 'hw.vmm.svm.features' to disable specific SVM features even
though they might be available in hardware. Use tunable 'hw.vmm.svm.num_asids'
to limit the number of ASIDs used by the hypervisor.

MFC r284046:
Fix regression in 'verify_gla()' with the RIP-relative addressing mode.

MFC r284174:
Support guest writes to the TSC by enabling the "use TSC offsetting"
execution control.

MFC r276428:
Replace bhyve's minimal RTC emulation with a fully featured one in vmm.ko.

MFC r276432:
Initialize all fields of 'struct vm_exception exception' before passing it
to vm_inject_exception().

MFC r276763:
Clear blocking due to STI or MOV SS in the hypervisor when an instruction is
emulated or when the vcpu incurs an exception.

MFC r277149:
Clean up usage of 'struct vm_exception' to only to communicate information
from userspace to vmm.ko when injecting an exception.

MFC r277168:
Fix typo (missing comma).

MFC r277309:
Make the error message explicit instead of just printing the usage if the
virtual machine name is not specified.

MFC r277310:
Simplify instruction restart logic in bhyve.

MFC r277359:
Fix a bug in libvmmapi 'vm_copy_setup()' where it would return success even
if the 'gpa' was in the guest MMIO region.

MFC r277360:
MOVS instruction emulation.

MFC r277626:
Add macro to identify AVIC capability (advanced virtual interrupt controller)
in AMD processors.

MFC r279220:
Don't close a block context if it couldn't be opened avoiding a null deref.

MFC r279225:
Add "-u" option to bhyve(8) to indicate that the RTC should maintain UTC time.

MFC r279227:
Emulate MSR 0xC0011024 when running on AMD processors.

MFC r279228:
Always emulate MSR_PAT on Intel processors and don't rely on PAT save/restore
capability of VT-x. This lets bhyve run nested in older VMware versions that
don't support the PAT save/restore capability.

MFC r279540:
Fix warnings/errors when building vmm.ko with gcc.

MFC r274720:

Fix improper .Fx macro usage.

Sponsored by: The FreeBSD Foundation

MFC r271711:
Update the bhyve(8) manual to reflect that it is no
longer considered 'experimental.'

Approved by: re (delphij)
Sponsored by: The FreeBSD Foundation

MFC r267921, r267934, r267949, r267959, r267966, r268202, r268276,
r268427, r268428, r268521, r268638, r268639, r268701, r268777,
r268889, r268922, r269008, r269042, r269043, r269080, r269094,
r269108, r269109, r269281, r269317, r269700, r269896, r269962,
r269989.

Catch bhyve up to CURRENT.

Lightly tested with FreeBSD i386/amd64, Linux i386/amd64, and
OpenBSD/amd64. Still resolving an issue with OpenBSD/i386.

Many thanks to jhb@ for all the hard work on the prior MFCs !

r267921 - support the "mov r/m8, imm8" instruction
r267934 - document options
r267949 - set DMI vers/date to fixed values
r267959 - doc: sort cmd flags
r267966 - EPT misconf post-mortem info
r268202 - use correct flag for event index
r268276 - 64-bit virtio capability api
r268427 - invalidate guest TLB when cr3 is updated, needed for TSS
r268428 - identify vcpu's operating mode
r268521 - use correct offset in guest logical-to-linear translation
r268638 - chs value
r268639 - chs fake values
r268701 - instr emul operand/address size override prefix support
r268777 - emulation for legacy x86 task switching
r268889 - nested exception support
r268922 - fix INVARIANTS build
r269008 - emulate instructions found in the OpenBSD/i386 5.5 kernel
r269042 - fix fault injection
r269043 - Reduce VMEXIT_RESTARTs in task_switch.c
r269080 - fix issues in PUSH emulation
r269094 - simplify return values from the inout handlers
r269108 - don't return -1 from the push emulation handler
r269109 - avoid permanent sleep in vm_handle_hlt()
r269281 - list VT-x features in base kernel dmesg
r269317 - Mark AHCI fatal errors as not completed
r269700 - Support PCI extended config space in bhyve
r269896 - Minor cleanup
r269962 - use max guest memory when creating IOMMU domain
r269989 - fix interrupt mode names

MFC 264353,264509,264768,264770,264825,264846,264988,265114,265165,265365,
265941,265951,266390,266550,266910:
Various bhyve fixes:
- Don't save host's return address in 'struct vmxctx'.
- Permit non-32-bit accesses to local APIC registers.
- Factor out common ioport handler code.
- Use calloc() in favor of malloc + memset.
- Change the vlapic timer frequency to be in the ballpark of contemporary
hardware.
- Allow the guest to read the TSC via MSR 0x10.
- A VMCS is always inactive when it exits the vmx_run() loop. Remove
redundant code and the misleading comment that suggest otherwise.
- Ignore writes to microcode update MSR. This MSR is accessed by RHEL7
guest.
Add KTR tracepoints to annotate wrmsr and rdmsr VM exits.
- Provide an alias for the userboot console and name it 'comconsole'.
- Use EV_ADD to create an mevent and EV_ENABLE to enable it.
- abort(3) the process in response to a VMEXIT_ABORT.
- Don't include the guest memory segments in the bhyve(8) process core dump.
- Make the vmx asm code dtrace-fbt-friendly.
- Allow vmx_getdesc() and vmx_setdesc() to be called for a vcpu that is in
the VCPU_RUNNING state.
- Enable VMX in the IA32_FEATURE_CONTROL MSR if it not enabled and the MSR
isn't locked.

MFC 260847,264055,264867:
- Add a very simple virtio_random(4) driver for FreeBSD guests to harvest
entropy from hypervisors.
- Add support to bhyve for the virtio RNG entropy-source device to provide
entry to bhyve guests.

MFC 263432,265366,265376:
Fixes for vcpu management in bhyve:
- Use 'cpuset_t' to represent the vcpus active in a virtual machine.
- Modify the "-p" option to be more flexible when associating a 'vcpu' with
a 'hostcpu'.

MFC 261904,261905,262143,262184,264921,265211,267169,267292,267294:
Various PCI fixes:
- Allow PCI devices to be configured on all valid bus numbers from 0 to 255.
- Tweak the handling of PCI capabilities in emulated devices to remove
the non-standard zero capability list terminator.
- Add a check to validate that memory BARs of passthru devices are 4KB
aligned.
- Respect and track the enable bit in the PCI configuration address word.
- Handle quad-word access to 32-bit register pairs.

MFC 262139,262140,262236,262281,262532:
Various x2APIC fixes and enhancements:
- Use spinlocks for the vioapic.
- Handle the SELF_IPI MSR.
- Simplify the APIC mode switching between MMIO and x2APIC. The guest is
no longer allowed to switch modes at runtime. Instead, the desired mode
is set when the virtual machine is created.
- Disallow MMIO access in x2APIC mode and MSR access in xAPIC mode.
- Add support for x2APIC virtualization assist in Intel VT-x.

MFC 261217:
Remove support for legacy PCI devices. These haven't been needed since
support for LPC uart devices was added and it conflicts with upcoming
patches to add PCI INTx support.

Approved by: grehan

MFC 259635:
Add an option to ignore accesses by the guest to unimplemented MSRs.

Also, ignore a couple of SandyBridge uncore PMC MSRs that Centos 6.4 writes
to during boot.

MFC r260446

virtio-block does not exist, the correct name is virtio-blk.

PR: 185573
Submitted by: Allan Jude
Facilitated by: Snow B.V.

MFC r259302,r259413

r259302
bhyve(8) man page

r259413
mdoc: sort SEE ALSO