#
322094 |
|
05-Aug-2017 |
marius |
MFC: r306375
Add a WITHOUT_DIALOG src.conf(5) knob.
It also turns off dependencies (bsdinstall, bsdconfig, dpv, tzsetup).
|
#
313199 |
|
04-Feb-2017 |
ngie |
MFC r312937:
Fix typo in lib/Makefile
The SUBDIR_DEPEND variable should be for librpcsec_gss, not liblibrpc_gss
PR: 216409
|
#
295131 |
|
01-Feb-2016 |
jhb |
MFC 278320,278336,278830,285621: Add devctl(8): a utility for manipulating new-bus devices. Note that this version does not include the 'suspend' and 'resume' commands present in HEAD as those depend on larger changes to the suspend and resume code in the kernel.
278320: Add a new device control utility for new-bus devices called devctl. This allows the user to request administrative changes to individual devices such as attach or detaching drivers or disabling and re-enabling devices. - Add a new /dev/devctl2 character device which uses ioctls for device requests. The ioctls use a common 'struct devreq' which is somewhat similar to 'struct ifreq'. - The ioctls identify the device to operate on via a string. This string can either by the device's name, or it can be a bus-specific address. (For unattached devices, a bus address is the only way to locate a device.) Bus drivers register an eventhandler to claim unrecognized device names that the driver recognizes as a valid address. Two buses currently support addresses: ACPI recognizes any device in the ACPI namespace via its full path starting with "\" and the PCI bus driver recognizes an address specification of 'pci[<domain>:]<bus>:<slot>:<func>' (identical to the PCI selector strings supported by pciconf). - To make it easier to cut and paste, change the PnP location string in the PCI bus driver to output a full PCI selector string rather than 'slot=<slot> function=<func>'. - Add a devctl(3) interface in libdevctl which provides a wrapper around the ioctls and is the preferred interface for other userland code. - Add a devctl(8) program which is a simple wrapper around the requests supported by devctl(3). - Add a resource_unset_value() function that can be used to remove a hint from the kernel environment. This is used to clear a hint.<driver>.<unit>.disabled hint when re-enabling a boot-time disabled device.
278336: Unbreak the build (memchr is explicitly required by devctl(9) after r278320)
278830: install the man page...
285621: Fix formatting.
Approved by: re (marius)
|
#
294364 |
|
19-Jan-2016 |
ian |
MFC r292337, r292552, r292553:
Build mount_smbfs for arm. Also sort the subdirs.
Avoid unaligned memory accesses when encoding netbios names in libsmb.
The current code for encoding a netbios name converts each byte to a 16-bit value and stores the result by casting a char* to u_short*, resulting in alignment faults on strict-alignment platforms.
This change reimplements the encoding routine using only byte accesses to memory. There is no particular reason to work with 16-bit values just because the encoding process creates two bytes of output for every byte of input. Working a byte at at time also avoids endian problems for big-endian platforms.
Make the building of libsmb and mount_smbfs unconditional, now that r292552 has eliminated alignment and endian problems that were making it fail on some platforms.
PR: 180438 PR: 189415 Relnotes: Yes
|
#
292973 |
|
31-Dec-2015 |
ngie |
MFC nv(3) and part of nv(9) to stable/10
This includes the following revisions from head:
r258065,r258594,r259430,r260222,r261407,r261408,r263479,r264021,r266351, r269603,r271026,r271027,r271028,r271241,r271578,r271579,r271847,r272102, r272843,r273752,r277920,r277921,r277925,r277926,r277927,r279421,r279422, r279423,r279424,r279425,r279426,r279427,r279428,r279429,r279430,r279431, r279432,r279434,r279435,r279436,r279438,r279439,r279440,r279760,r282122, r282254,r282257,r282304,r282312,r285339,r288340
This change reverts stable/10@r282122 and stable/10@r288340, and re-MFCs the series again (r282122, r285339, and r288340).
More changes are pending to nv(9)/pci(4) after further review/work. Please see the Phabricator review for more details (both https://reviews.freebsd.org/D4232 and https://reviews.freebsd.org/D4249 ).
- Tested with: -- Booting VMware Fusion 8.1.0 running on a Haswell Apple Macbook Pro -- Booting a Haswell machine with zfs and running some stress workloads with VirtualBox guests -- make tinderbox -- kyua test -k /usr/tests/lib/libnv
Differential Revision: https://reviews.freebsd.org/D4249 (part of a larger diff) Relnotes: yes Reviewed by: oshogbo (implicit), sbruno (implicit) Submitted by: Kevin Bowling <kevin.bowling@kev009.com> Sponsored by: EMC / Isilon Storage Division
|
#
292588 |
|
22-Dec-2015 |
delphij |
MFC: xz 5.2.2.
Relnotes: yes
|
#
292339 |
|
16-Dec-2015 |
emaste |
MFC r282821: Remove redundant csu subdir logic
The appropriate subdirectories are handled by lib/csu/Makefile. There's no need to duplicate this logic in Makefile.inc1 and lib/Makefile.
Sponsored by: The FreeBSD Foundation
|
#
283847 |
|
31-May-2015 |
ngie |
MFC r283502:
Remove SUBDIR_DEPEND for non-existent libreadline directory
libreadline lives under gnu/lib
|
#
280438 |
|
24-Mar-2015 |
ken |
MFC sa(4) and mt(1) improvements.
This includes these changes: 279219, 279229, 279261, 279534, 279570, 280230, 280231.
In addition, bump __FreeBSD_version for the addition of the new mtio(4) / sa(4) ioctls.
Thanks to Dan Langille, Harald Schmalzbauer and Rudolf Cejka for spending a significant amount of time and effort testing these changes.
------------------------------------------------------------------------ r279219 | ken | 2015-02-23 14:59:30 -0700 (Mon, 23 Feb 2015) | 282 lines
Significant upgrades to sa(4) and mt(1).
The primary focus of these changes is to modernize FreeBSD's tape infrastructure so that we can take advantage of some of the features of modern tape drives and allow support for LTFS.
Significant changes and new features include:
o sa(4) driver status and parameter information is now exported via an XML structure. This will allow for changes and improvements later on that will not break userland applications. The old MTIOCGET status ioctl remains, so applications using the existing interface will not break.
o 'mt status' now reports drive-reported tape position information as well as the previously available calculated tape position information. These numbers will be different at times, because the drive-reported block numbers are relative to BOP (Beginning of Partition), but the block numbers calculated previously via sa(4) (and still provided) are relative to the last filemark. Both numbers are now provided. 'mt status' now also shows the drive INQUIRY information, serial number and any position flags (BOP, EOT, etc.) provided with the tape position information. 'mt status -v' adds information on the maximum possible I/O size, and the underlying values used to calculate it.
o The extra sa(4) /dev entries (/dev/saN.[0-3]) have been removed.
The extra devices were originally added as place holders for density-specific device nodes. Some OSes (NetBSD, NetApp's OnTap and Solaris) have had device nodes that, when you write to them, will automatically select a given density for particular tape drives.
This is a convenient way of switching densities, but it was never implemented in FreeBSD. Only the device nodes were there, and that sometimes confused users.
For modern tape devices, the density is generally not selectable (e.g. with LTO) or defaults to the highest availble density when the tape is rewritten from BOT (e.g. TS11X0). So, for most users, density selection won't be necessary. If they do need to select the density, it is easy enough to use 'mt density' to change it.
o Protection information is now supported. This is either a Reed-Solomon CRC or CRC32 that is included at the end of each block read and written. On write, the tape drive verifies the CRC, and on read, the tape drive provides a CRC for the userland application to verify.
o New, extensible tape driver parameter get/set interface.
o Density reporting information. For drives that support it, 'mt getdensity' will show detailed information on what formats the tape drive supports, and what formats the tape drive supports.
o Some mt(1) functionality moved into a new mt(3) library so that external applications can reuse the code.
o The new mt(3) library includes helper routines to aid in parsing the XML output of the sa(4) driver, and build a tree of driver metadata.
o Support for the MTLOAD (load a tape in the drive) and MTWEOFI (write filemark immediate) ioctls needed by IBM's LTFS implementation.
o Improve device departure behavior for the sa(4) driver. The previous implementation led to hangs when the device was open.
o This has been tested on the following types of drives: IBM TS1150 IBM TS1140 IBM LTO-6 IBM LTO-5 HP LTO-2 Seagate DDS-4 Quantum DLT-4000 Exabyte 8505 Sony DDS-2
contrib/groff/tmac/doc-syms, share/mk/bsd.libnames.mk, lib/Makefile, Add libmt.
lib/libmt/Makefile, lib/libmt/mt.3, lib/libmt/mtlib.c, lib/libmt/mtlib.h, New mt(3) library that contains functions moved from mt(1) and new functions needed to interact with the updated sa(4) driver.
This includes XML parser helper functions that application writers can use when writing code to query tape parameters.
rescue/rescue/Makefile: Add -lmt to CRUNCH_LIBS.
src/share/man/man4/mtio.4 Clarify this man page a bit, and since it contains what is essentially the mtio.h header file, add new ioctls and structure definitions from mtio.h.
src/share/man/man4/sa.4 Update BUGS and maintainer section.
sys/cam/scsi/scsi_all.c, sys/cam/scsi/scsi_all.h: Add SCSI SECURITY PROTOCOL IN/OUT CDB definitions and CDB building functions.
sys/cam/scsi/scsi_sa.c sys/cam/scsi/scsi_sa.h Many tape driver changes, largely outlined above.
Increase the sa(4) driver read/write timeout from 4 to 32 minutes. This is based on the recommended values for IBM LTO 5/6 drives. This may also avoid timeouts for other tape hardware that can take a long time to do retries and error recovery. Longer term, a better way to handle this is to ask the drive for recommended timeout values using the REPORT SUPPORTED OPCODES command. Modern IBM and Oracle tape drives at least support that command, and it would allow for more accurate timeout values.
Add XML status generation. This is done with a series of macros to eliminate as much duplicate code as possible. The new XML-based status values are reported through the new MTIOCEXTGET ioctl.
Add XML driver parameter reporting, using the new MTIOCPARAMGET ioctl.
Add a new driver parameter setting interface, using the new MTIOCPARAMSET and MTIOCSETLIST ioctls.
Add a new MTIOCRBLIM ioctl to get block limits information.
Add CCB/CDB building routines scsi_locate_16, scsi_locate_10, and scsi_read_position_10().
scsi_locate_10 implements the LOCATE command, as does the existing scsi_set_position() command. It just supports additional arguments and features. If/when we figure out a good way to provide backward compatibility for older applications using the old function API, we can just revamp scsi_set_position(). The same goes for scsi_read_position_10() and the existing scsi_read_position() function.
Revamp sasetpos() to take the new mtlocate structure as an argument. It now will use either scsi_locate_10() or scsi_locate_16(), depending upon the arguments the user supplies. As before, once we change position we don't have a clear idea of what the current logical position of the tape drive is.
For tape drives that support long form position data, we read the current position and store that for later reporting after changing the position. This should help applications like Bacula speed tape access under FreeBSD once they are modified to support the new ioctls.
Add a new quirk, SA_QUIRK_NO_LONG_POS, that is set for all drives that report SCSI-2 or older, as well as drives that report an Illegal Request type error for READ POSITION with the long format. So we should automatically detect drives that don't support the long form and stop asking for it after an initial try.
Add a partition number to the sa(4) softc.
Improve device departure handling. The previous implementation led to hangs when the device was open.
If an application had the sa(4) driver open, and attempted to close it after it went away, the cam_periph_release() call in saclose() would cause the periph to get destroyed because that was the last reference to it. Because destroy_dev() was called from the sa(4) driver's cleanup routine (sacleanup()), and would block waiting for the close to happen, a deadlock would result.
So instead of calling destroy_dev() from the cleanup routine, call destroy_dev_sched_cb() from saoninvalidate() and wait for the callback.
Acquire a reference for devfs in saregister(), and release it in the new sadevgonecb() routine when all devfs devices for the particular sa(4) driver instance are gone.
Add a new function, sasetupdev(), to centralize setting per-instance devfs device parameters instead of repeating the code in saregister().
Add an open count to the softc, so we know how many peripheral driver references are a result of open sessions.
Add the D_TRACKCLOSE flag to the cdevsw flags so that we get a 1:1 mapping of open to close calls instead of a N:1 mapping.
This should be a no-op for everything except the control device, since we don't allow more than one open on non-control devices.
However, since we do allow multiple opens on the control device, the combination of the open count and the D_TRACKCLOSE flag should result in an accurate peripheral driver reference count, and an accurate open count.
The accurate open count allows us to release all peripheral driver references that are the result of open contexts once we get the callback from devfs.
sys/sys/mtio.h: Add a number of new mt(4) ioctls and the requisite data structures. None of the existing interfaces been removed or changed.
This includes definitions for the following new ioctls:
MTIOCRBLIM /* get block limits */ MTIOCEXTLOCATE /* seek to position */ MTIOCEXTGET /* get tape status */ MTIOCPARAMGET /* get tape params */ MTIOCPARAMSET /* set tape params */ MTIOCSETLIST /* set N params */
usr.bin/mt/Makefile: mt(1) now depends on libmt, libsbuf and libbsdxml.
usr.bin/mt/mt.1: Document new mt(1) features and subcommands.
usr.bin/mt/mt.c: Implement support for mt(1) subcommands that need to use getopt(3) for their arguments.
Implement a new 'mt status' command to replace the old 'mt status' command. The old status command has been renamed 'ostatus'.
The new status function uses the MTIOCEXTGET ioctl, and therefore parses the XML data to determine drive status. The -x argument to 'mt status' allows the user to dump out the raw XML reported by the kernel.
The new status display is mostly the same as the old status display, except that it doesn't print the redundant density mode information, and it does print the current partition number and position flags.
Add a new command, 'mt locate', that will supersede the old 'mt setspos' and 'mt sethpos' commands. 'mt locate' implements all of the functionality of the MTIOCEXTLOCATE ioctl, and allows the user to change the logical position of the tape drive in a number of ways. (Partition, block number, file number, set mark number, end of data.) The immediate bit and the explicit address bits are implemented, but not documented in the man page.
Add a new 'mt weofi' command to use the new MTWEOFI ioctl. This allows the user to ask the drive to write a filemark without waiting around for the operation to complete.
Add a new 'mt getdensity' command that gets the XML-based tape drive density report from the sa(4) driver and displays it. This uses the SCSI REPORT DENSITY SUPPORT command to get comprehensive information from the tape drive about what formats it is able to read and write.
Add a new 'mt protect' command that allows getting and setting tape drive protection information. The protection information is a CRC tacked on to the end of every read/write from and to the tape drive.
Sponsored by: Spectra Logic MFC after: 1 month
------------------------------------------------------------------------ ------------------------------------------------------------------------ r279229 | ken | 2015-02-23 22:43:16 -0700 (Mon, 23 Feb 2015) | 5 lines
Fix printf format warnings on sparc64 and mips.
Sponsored by: Spectra Logic MFC after: 1 month
------------------------------------------------------------------------ ------------------------------------------------------------------------ r279261 | ken | 2015-02-24 21:30:23 -0700 (Tue, 24 Feb 2015) | 23 lines
Fix several problems found by Coverity.
lib/libmt/mtlib.c: In mt_start_element(), make sure we don't overflow the cur_sb array. CID 1271325
usr.bin/mt/mt.c: In main(), bzero the mt_com structure so that we aren't using any uninitialized stack variables. CID 1271319
In mt_param(), only allow one -s and one -p argument. This will prevent a memory leak caused by overwriting the param_name and/or param_value variables. CID 1271320 and CID 1271322
To make things simpler in mt_param(), make sure there there is only one exit path for the function. Make sure the arguments are explicitly freed.
Sponsored by: Spectra Logic Pointed out by: emaste MFC after: 1 month
------------------------------------------------------------------------ ------------------------------------------------------------------------ r279534 | ken | 2015-03-02 11:09:49 -0700 (Mon, 02 Mar 2015) | 18 lines
Change the sa(4) driver to check for long position support on SCSI-2 devices.
Some older tape devices claim to be SCSI-2, but actually do support long position information. (Long position information includes the current file mark.) For example, the COMPAQ SuperDLT1.
So we now only disable the check on SCSI-1 and older devices.
sys/cam/scsi/scsi_sa.c: In saregister(), only disable fetching long position information on SCSI-1 and older drives. Update the comment to explain why.
Confirmed by: dvl Sponsored by: Spectra Logic MFC after: 3 weeks
------------------------------------------------------------------------ ------------------------------------------------------------------------ r279570 | ken | 2015-03-03 15:49:07 -0700 (Tue, 03 Mar 2015) | 21 lines
Add density code for DAT-72, and notes on DAT-160.
As it turns out, the density code for DAT-160 (0x48) is the same as for SDLT220. Since the SDLT values are already in the table, we will leave them in place.
Thanks to Harald Schmalzbauer for confirming the DAT-72 density code.
lib/libmt/mtlib.c: Add DAT-72 density code, and commented out DAT-160 density code. Explain why DAT-160 is commented out. Add notes explaining where the bpi values for these formats came from.
usr.bin/mt/mt.1: Add DAT-72 density code, and add a note explaining that the SDLTTapeI(110) density code (0x48) is the same as DAT-160.
Sponsored by: Spectra Logic MFC after: 3 weeks
------------------------------------------------------------------------ ------------------------------------------------------------------------ r280230 | ken | 2015-03-18 14:52:34 -0600 (Wed, 18 Mar 2015) | 25 lines
Fix a couple of problems in the sa(4) media type reports.
The only drives I have discovered so far that support medium type reports are newer HP LTO (LTO-5 and LTO-6) drives. IBM drives only support the density reports.
sys/cam/scsi/scsi_sa.h: The number of possible density codes in the medium type report is 9, not 8. This caused problems parsing all of the medium type report after this point in the structure.
usr.bin/mt/mt.c: Run the density codes returned in the medium type report through denstostring(), just like the primary and secondary density codes in the density report. This will print the density code in hex, and give a text description if it is available.
Thanks to Rudolf Cejka for doing extensive testing with HP LTO drives and Bacula and discovering these problems.
Tested by: Rudolf Cejka <cejkar at fit.vutbr.cz> Sponsored by: Spectra Logic MFC after: 4 days
------------------------------------------------------------------------ ------------------------------------------------------------------------ r280231 | ken | 2015-03-18 14:54:54 -0600 (Wed, 18 Mar 2015) | 16 lines
Improve the mt(1) rblim display.
The granularity reported by READ BLOCK LIMITS is an exponent, not a byte value. So a granularity of 0 means 2^0, or 1 byte. A granularity of 1 means 2^1, or 2 bytes.
Print out the individual block limits on separate lines to improve readability and avoid exceeding 80 columns.
usr.bin/mt/mt.c: Fix and improve the 'mt rblim' output. Add a MT_PLURAL() macro so we can print "byte" or "bytes" as appropriate.
Sponsored by: Spectra Logic MFC after: 4 days
------------------------------------------------------------------------
Sponsored by: Spectra Logic
|
#
279506 |
|
01-Mar-2015 |
ngie |
MFC r278193:
Add MK_FILE to control whether or not to build file(1), libmagic(3), etc
Sponsored by: EMC / Isilon Storage Division
|
#
278569 |
|
11-Feb-2015 |
ngie |
MFC r278182:
r278182:
Conditionalize building radius support into libpam, ppp, etc via MK_RADIUS_SUPPORT
Sponsored by: EMC / Isilon Storage Division
|
#
278557 |
|
11-Feb-2015 |
ngie |
MFC r277727:
r277727:
Add MK_BHYVE knob for building and installing bhyve(4), et al
Sponsored by: EMC / Isilon Storage Division
|
#
275040 |
|
25-Nov-2014 |
dteske |
MFC r274116:
Add new libraries/utilities for data throughput visualization. dpv(3): dialog progress view library dpv(1): stream data from stdin or multiple paths with dialog progress view figpar(3): configuration file parsing library
MFC r274120, r274121, r274123, r274124, r274144, r274146, r274159, r274192, r274203, r274209, r274226, r274270, and r274851: Fixes following r274116
Reviews: D714 Relnotes: New libdpv/libfigpar and dpv(1) utility Reviewed by: jelischer, shurd Discussed at: MeetBSD California 2014 Vendor/Dev Summit Discussed on: -current Thanks to: ngie, ian, jelischer, shurd, bapt
|
#
270187 |
|
19-Aug-2014 |
ian |
MFC r266473,267331,267511:
Use an intermediate target to associate with _SUBDIR which is marked .MAKE this allows make -n to do tree walks as expected without doing anything else (as intended). Use prefix _sub. to help avoid conflict with any real target.
Put the test suite in its own tests.txz distribution file. Force all the contents of /usr/tests to go into a separate distribution file so that users of binary releases can easily choose to not install
Create a mechanism for providing fine-grained build order dependencies during SUBDIR_PARALLEL builds. This augments the coarse .WAIT mechanism, which is still useful if you've got a situation such as "almost everything depends on A and B".
|
#
264303 |
|
09-Apr-2014 |
dim |
MFC r263778:
Add a SUBDIR_PARALLEL option to bsd.subdir.mk, to allow make to process all the SUBDIR entries in parallel, instead of serially. Apply this option to a selected number of Makefiles, which can greatly speed up the build on multi-core machines, when using make -j.
This can be extended to more Makefiles later on, whenever they are verified to work correctly with parallel building.
I tested this on a 24-core machine, with make -j48 buildworld (N = 6):
before stddev after stddev ======= ====== ======= ====== real time 1741.1 16.5 959.8 2.7 user time 12468.7 16.4 14393.0 16.8 sys time 1825.0 54.8 2110.6 22.8
(user+sys)/real 8.2 17.1
E.g. the build was approximately 45% faster in real time. On machines with less cores, or with lower -j settings, the speedup will not be as impressive. But at least you can now almost max out a machine with buildworld!
Submitted by: jilles
MFC r263833:
Enable parallel building for gnu/usr.bin and usr.bin/clang too.
|
#
263019 |
|
11-Mar-2014 |
bapt |
MFC: r262398,r262399,r262410,r262975
Import libucl into head
UCL is heavily infused by nginx configuration as the example of a convenient configuration system. However, UCL is fully compatible with JSON format and is able to parse json files.
UCL is used by pkg(8) for its configuration file as well for the manifest format in packages, it will be used in base for the pkg boostrap (signature checking and configuration file parsing.)
libucl has been developped and is maintained by vsevolod@
|
#
260013 |
|
28-Dec-2013 |
jmmv |
Set up the /usr/tests hierarchy.
This is a MFC of the following into stable/10: - r257097 Set up the /usr/tests hierarchy. - r257098 Add missing WITHOUTTESTS file. - r257100 Add a tests(7) manual page. - r257105 Disable WITHTESTS= for now. - r257848 Fix buildworld when WITHTESTS is enabled. - r257850 Subsume the functionality of MKATF into MKTESTS. - r257851 Handle the removal of the test suite when WITHOUTTESTS=yes. - r257852 Install category Kyuafiles from their category directories. - r258232 Install BSD.tests.mtree when MKTESTS is yes.
Note that building with WITH_TESTS is still broken at this point (and hence why WITHOUT_TESTS is the set as the default). Subsequent pullups will fix the remaining issues.
|
#
258398 |
|
20-Nov-2013 |
peter |
MFC r258283 - move iconv* symbols out of libc.so.7 namespace with compatibility symbols to aid transition for existing 10.x installs.
Approved by: re (kib)
|
#
258230 |
|
16-Nov-2013 |
gjb |
MFC r257583, r258012, r258013:
r257583 (peter): Remove the WITH_LIBICONV_COMPAT hack that seems to do more harm than good. This caused libc to spoof the ports libiconv namespace and provide a colliding libiconv.so.3 to fool rtld. This should have been removed some time ago.
r258012: Remove WITH_LIBICONV_COMPAT file to chase after r257583.
r258013: Regenerate src.conf.5 after removal of WITH_LIBICONV_COMPAT.
Approved by: re (kib) Sponsored by: The FreeBSD Foundation
Sponsored by: The FreeBSD Foundation
|
#
283847 |
|
31-May-2015 |
ngie |
MFC r283502:
Remove SUBDIR_DEPEND for non-existent libreadline directory
libreadline lives under gnu/lib
|
#
280438 |
|
24-Mar-2015 |
ken |
MFC sa(4) and mt(1) improvements.
This includes these changes: 279219, 279229, 279261, 279534, 279570, 280230, 280231.
In addition, bump __FreeBSD_version for the addition of the new mtio(4) / sa(4) ioctls.
Thanks to Dan Langille, Harald Schmalzbauer and Rudolf Cejka for spending a significant amount of time and effort testing these changes.
------------------------------------------------------------------------ r279219 | ken | 2015-02-23 14:59:30 -0700 (Mon, 23 Feb 2015) | 282 lines
Significant upgrades to sa(4) and mt(1).
The primary focus of these changes is to modernize FreeBSD's tape infrastructure so that we can take advantage of some of the features of modern tape drives and allow support for LTFS.
Significant changes and new features include:
o sa(4) driver status and parameter information is now exported via an XML structure. This will allow for changes and improvements later on that will not break userland applications. The old MTIOCGET status ioctl remains, so applications using the existing interface will not break.
o 'mt status' now reports drive-reported tape position information as well as the previously available calculated tape position information. These numbers will be different at times, because the drive-reported block numbers are relative to BOP (Beginning of Partition), but the block numbers calculated previously via sa(4) (and still provided) are relative to the last filemark. Both numbers are now provided. 'mt status' now also shows the drive INQUIRY information, serial number and any position flags (BOP, EOT, etc.) provided with the tape position information. 'mt status -v' adds information on the maximum possible I/O size, and the underlying values used to calculate it.
o The extra sa(4) /dev entries (/dev/saN.[0-3]) have been removed.
The extra devices were originally added as place holders for density-specific device nodes. Some OSes (NetBSD, NetApp's OnTap and Solaris) have had device nodes that, when you write to them, will automatically select a given density for particular tape drives.
This is a convenient way of switching densities, but it was never implemented in FreeBSD. Only the device nodes were there, and that sometimes confused users.
For modern tape devices, the density is generally not selectable (e.g. with LTO) or defaults to the highest availble density when the tape is rewritten from BOT (e.g. TS11X0). So, for most users, density selection won't be necessary. If they do need to select the density, it is easy enough to use 'mt density' to change it.
o Protection information is now supported. This is either a Reed-Solomon CRC or CRC32 that is included at the end of each block read and written. On write, the tape drive verifies the CRC, and on read, the tape drive provides a CRC for the userland application to verify.
o New, extensible tape driver parameter get/set interface.
o Density reporting information. For drives that support it, 'mt getdensity' will show detailed information on what formats the tape drive supports, and what formats the tape drive supports.
o Some mt(1) functionality moved into a new mt(3) library so that external applications can reuse the code.
o The new mt(3) library includes helper routines to aid in parsing the XML output of the sa(4) driver, and build a tree of driver metadata.
o Support for the MTLOAD (load a tape in the drive) and MTWEOFI (write filemark immediate) ioctls needed by IBM's LTFS implementation.
o Improve device departure behavior for the sa(4) driver. The previous implementation led to hangs when the device was open.
o This has been tested on the following types of drives: IBM TS1150 IBM TS1140 IBM LTO-6 IBM LTO-5 HP LTO-2 Seagate DDS-4 Quantum DLT-4000 Exabyte 8505 Sony DDS-2
contrib/groff/tmac/doc-syms, share/mk/bsd.libnames.mk, lib/Makefile, Add libmt.
lib/libmt/Makefile, lib/libmt/mt.3, lib/libmt/mtlib.c, lib/libmt/mtlib.h, New mt(3) library that contains functions moved from mt(1) and new functions needed to interact with the updated sa(4) driver.
This includes XML parser helper functions that application writers can use when writing code to query tape parameters.
rescue/rescue/Makefile: Add -lmt to CRUNCH_LIBS.
src/share/man/man4/mtio.4 Clarify this man page a bit, and since it contains what is essentially the mtio.h header file, add new ioctls and structure definitions from mtio.h.
src/share/man/man4/sa.4 Update BUGS and maintainer section.
sys/cam/scsi/scsi_all.c, sys/cam/scsi/scsi_all.h: Add SCSI SECURITY PROTOCOL IN/OUT CDB definitions and CDB building functions.
sys/cam/scsi/scsi_sa.c sys/cam/scsi/scsi_sa.h Many tape driver changes, largely outlined above.
Increase the sa(4) driver read/write timeout from 4 to 32 minutes. This is based on the recommended values for IBM LTO 5/6 drives. This may also avoid timeouts for other tape hardware that can take a long time to do retries and error recovery. Longer term, a better way to handle this is to ask the drive for recommended timeout values using the REPORT SUPPORTED OPCODES command. Modern IBM and Oracle tape drives at least support that command, and it would allow for more accurate timeout values.
Add XML status generation. This is done with a series of macros to eliminate as much duplicate code as possible. The new XML-based status values are reported through the new MTIOCEXTGET ioctl.
Add XML driver parameter reporting, using the new MTIOCPARAMGET ioctl.
Add a new driver parameter setting interface, using the new MTIOCPARAMSET and MTIOCSETLIST ioctls.
Add a new MTIOCRBLIM ioctl to get block limits information.
Add CCB/CDB building routines scsi_locate_16, scsi_locate_10, and scsi_read_position_10().
scsi_locate_10 implements the LOCATE command, as does the existing scsi_set_position() command. It just supports additional arguments and features. If/when we figure out a good way to provide backward compatibility for older applications using the old function API, we can just revamp scsi_set_position(). The same goes for scsi_read_position_10() and the existing scsi_read_position() function.
Revamp sasetpos() to take the new mtlocate structure as an argument. It now will use either scsi_locate_10() or scsi_locate_16(), depending upon the arguments the user supplies. As before, once we change position we don't have a clear idea of what the current logical position of the tape drive is.
For tape drives that support long form position data, we read the current position and store that for later reporting after changing the position. This should help applications like Bacula speed tape access under FreeBSD once they are modified to support the new ioctls.
Add a new quirk, SA_QUIRK_NO_LONG_POS, that is set for all drives that report SCSI-2 or older, as well as drives that report an Illegal Request type error for READ POSITION with the long format. So we should automatically detect drives that don't support the long form and stop asking for it after an initial try.
Add a partition number to the sa(4) softc.
Improve device departure handling. The previous implementation led to hangs when the device was open.
If an application had the sa(4) driver open, and attempted to close it after it went away, the cam_periph_release() call in saclose() would cause the periph to get destroyed because that was the last reference to it. Because destroy_dev() was called from the sa(4) driver's cleanup routine (sacleanup()), and would block waiting for the close to happen, a deadlock would result.
So instead of calling destroy_dev() from the cleanup routine, call destroy_dev_sched_cb() from saoninvalidate() and wait for the callback.
Acquire a reference for devfs in saregister(), and release it in the new sadevgonecb() routine when all devfs devices for the particular sa(4) driver instance are gone.
Add a new function, sasetupdev(), to centralize setting per-instance devfs device parameters instead of repeating the code in saregister().
Add an open count to the softc, so we know how many peripheral driver references are a result of open sessions.
Add the D_TRACKCLOSE flag to the cdevsw flags so that we get a 1:1 mapping of open to close calls instead of a N:1 mapping.
This should be a no-op for everything except the control device, since we don't allow more than one open on non-control devices.
However, since we do allow multiple opens on the control device, the combination of the open count and the D_TRACKCLOSE flag should result in an accurate peripheral driver reference count, and an accurate open count.
The accurate open count allows us to release all peripheral driver references that are the result of open contexts once we get the callback from devfs.
sys/sys/mtio.h: Add a number of new mt(4) ioctls and the requisite data structures. None of the existing interfaces been removed or changed.
This includes definitions for the following new ioctls:
MTIOCRBLIM /* get block limits */ MTIOCEXTLOCATE /* seek to position */ MTIOCEXTGET /* get tape status */ MTIOCPARAMGET /* get tape params */ MTIOCPARAMSET /* set tape params */ MTIOCSETLIST /* set N params */
usr.bin/mt/Makefile: mt(1) now depends on libmt, libsbuf and libbsdxml.
usr.bin/mt/mt.1: Document new mt(1) features and subcommands.
usr.bin/mt/mt.c: Implement support for mt(1) subcommands that need to use getopt(3) for their arguments.
Implement a new 'mt status' command to replace the old 'mt status' command. The old status command has been renamed 'ostatus'.
The new status function uses the MTIOCEXTGET ioctl, and therefore parses the XML data to determine drive status. The -x argument to 'mt status' allows the user to dump out the raw XML reported by the kernel.
The new status display is mostly the same as the old status display, except that it doesn't print the redundant density mode information, and it does print the current partition number and position flags.
Add a new command, 'mt locate', that will supersede the old 'mt setspos' and 'mt sethpos' commands. 'mt locate' implements all of the functionality of the MTIOCEXTLOCATE ioctl, and allows the user to change the logical position of the tape drive in a number of ways. (Partition, block number, file number, set mark number, end of data.) The immediate bit and the explicit address bits are implemented, but not documented in the man page.
Add a new 'mt weofi' command to use the new MTWEOFI ioctl. This allows the user to ask the drive to write a filemark without waiting around for the operation to complete.
Add a new 'mt getdensity' command that gets the XML-based tape drive density report from the sa(4) driver and displays it. This uses the SCSI REPORT DENSITY SUPPORT command to get comprehensive information from the tape drive about what formats it is able to read and write.
Add a new 'mt protect' command that allows getting and setting tape drive protection information. The protection information is a CRC tacked on to the end of every read/write from and to the tape drive.
Sponsored by: Spectra Logic MFC after: 1 month
------------------------------------------------------------------------ ------------------------------------------------------------------------ r279229 | ken | 2015-02-23 22:43:16 -0700 (Mon, 23 Feb 2015) | 5 lines
Fix printf format warnings on sparc64 and mips.
Sponsored by: Spectra Logic MFC after: 1 month
------------------------------------------------------------------------ ------------------------------------------------------------------------ r279261 | ken | 2015-02-24 21:30:23 -0700 (Tue, 24 Feb 2015) | 23 lines
Fix several problems found by Coverity.
lib/libmt/mtlib.c: In mt_start_element(), make sure we don't overflow the cur_sb array. CID 1271325
usr.bin/mt/mt.c: In main(), bzero the mt_com structure so that we aren't using any uninitialized stack variables. CID 1271319
In mt_param(), only allow one -s and one -p argument. This will prevent a memory leak caused by overwriting the param_name and/or param_value variables. CID 1271320 and CID 1271322
To make things simpler in mt_param(), make sure there there is only one exit path for the function. Make sure the arguments are explicitly freed.
Sponsored by: Spectra Logic Pointed out by: emaste MFC after: 1 month
------------------------------------------------------------------------ ------------------------------------------------------------------------ r279534 | ken | 2015-03-02 11:09:49 -0700 (Mon, 02 Mar 2015) | 18 lines
Change the sa(4) driver to check for long position support on SCSI-2 devices.
Some older tape devices claim to be SCSI-2, but actually do support long position information. (Long position information includes the current file mark.) For example, the COMPAQ SuperDLT1.
So we now only disable the check on SCSI-1 and older devices.
sys/cam/scsi/scsi_sa.c: In saregister(), only disable fetching long position information on SCSI-1 and older drives. Update the comment to explain why.
Confirmed by: dvl Sponsored by: Spectra Logic MFC after: 3 weeks
------------------------------------------------------------------------ ------------------------------------------------------------------------ r279570 | ken | 2015-03-03 15:49:07 -0700 (Tue, 03 Mar 2015) | 21 lines
Add density code for DAT-72, and notes on DAT-160.
As it turns out, the density code for DAT-160 (0x48) is the same as for SDLT220. Since the SDLT values are already in the table, we will leave them in place.
Thanks to Harald Schmalzbauer for confirming the DAT-72 density code.
lib/libmt/mtlib.c: Add DAT-72 density code, and commented out DAT-160 density code. Explain why DAT-160 is commented out. Add notes explaining where the bpi values for these formats came from.
usr.bin/mt/mt.1: Add DAT-72 density code, and add a note explaining that the SDLTTapeI(110) density code (0x48) is the same as DAT-160.
Sponsored by: Spectra Logic MFC after: 3 weeks
------------------------------------------------------------------------ ------------------------------------------------------------------------ r280230 | ken | 2015-03-18 14:52:34 -0600 (Wed, 18 Mar 2015) | 25 lines
Fix a couple of problems in the sa(4) media type reports.
The only drives I have discovered so far that support medium type reports are newer HP LTO (LTO-5 and LTO-6) drives. IBM drives only support the density reports.
sys/cam/scsi/scsi_sa.h: The number of possible density codes in the medium type report is 9, not 8. This caused problems parsing all of the medium type report after this point in the structure.
usr.bin/mt/mt.c: Run the density codes returned in the medium type report through denstostring(), just like the primary and secondary density codes in the density report. This will print the density code in hex, and give a text description if it is available.
Thanks to Rudolf Cejka for doing extensive testing with HP LTO drives and Bacula and discovering these problems.
Tested by: Rudolf Cejka <cejkar at fit.vutbr.cz> Sponsored by: Spectra Logic MFC after: 4 days
------------------------------------------------------------------------ ------------------------------------------------------------------------ r280231 | ken | 2015-03-18 14:54:54 -0600 (Wed, 18 Mar 2015) | 16 lines
Improve the mt(1) rblim display.
The granularity reported by READ BLOCK LIMITS is an exponent, not a byte value. So a granularity of 0 means 2^0, or 1 byte. A granularity of 1 means 2^1, or 2 bytes.
Print out the individual block limits on separate lines to improve readability and avoid exceeding 80 columns.
usr.bin/mt/mt.c: Fix and improve the 'mt rblim' output. Add a MT_PLURAL() macro so we can print "byte" or "bytes" as appropriate.
Sponsored by: Spectra Logic MFC after: 4 days
------------------------------------------------------------------------
Sponsored by: Spectra Logic
|
#
279506 |
|
01-Mar-2015 |
ngie |
MFC r278193:
Add MK_FILE to control whether or not to build file(1), libmagic(3), etc
Sponsored by: EMC / Isilon Storage Division
|
#
278569 |
|
11-Feb-2015 |
ngie |
MFC r278182:
r278182:
Conditionalize building radius support into libpam, ppp, etc via MK_RADIUS_SUPPORT
Sponsored by: EMC / Isilon Storage Division
|
#
278557 |
|
11-Feb-2015 |
ngie |
MFC r277727:
r277727:
Add MK_BHYVE knob for building and installing bhyve(4), et al
Sponsored by: EMC / Isilon Storage Division
|
#
275040 |
|
25-Nov-2014 |
dteske |
MFC r274116:
Add new libraries/utilities for data throughput visualization. dpv(3): dialog progress view library dpv(1): stream data from stdin or multiple paths with dialog progress view figpar(3): configuration file parsing library
MFC r274120, r274121, r274123, r274124, r274144, r274146, r274159, r274192, r274203, r274209, r274226, r274270, and r274851: Fixes following r274116
Reviews: D714 Relnotes: New libdpv/libfigpar and dpv(1) utility Reviewed by: jelischer, shurd Discussed at: MeetBSD California 2014 Vendor/Dev Summit Discussed on: -current Thanks to: ngie, ian, jelischer, shurd, bapt
|
#
270187 |
|
19-Aug-2014 |
ian |
MFC r266473,267331,267511:
Use an intermediate target to associate with _SUBDIR which is marked .MAKE this allows make -n to do tree walks as expected without doing anything else (as intended). Use prefix _sub. to help avoid conflict with any real target.
Put the test suite in its own tests.txz distribution file. Force all the contents of /usr/tests to go into a separate distribution file so that users of binary releases can easily choose to not install
Create a mechanism for providing fine-grained build order dependencies during SUBDIR_PARALLEL builds. This augments the coarse .WAIT mechanism, which is still useful if you've got a situation such as "almost everything depends on A and B".
|
#
264303 |
|
09-Apr-2014 |
dim |
MFC r263778:
Add a SUBDIR_PARALLEL option to bsd.subdir.mk, to allow make to process all the SUBDIR entries in parallel, instead of serially. Apply this option to a selected number of Makefiles, which can greatly speed up the build on multi-core machines, when using make -j.
This can be extended to more Makefiles later on, whenever they are verified to work correctly with parallel building.
I tested this on a 24-core machine, with make -j48 buildworld (N = 6):
before stddev after stddev ======= ====== ======= ====== real time 1741.1 16.5 959.8 2.7 user time 12468.7 16.4 14393.0 16.8 sys time 1825.0 54.8 2110.6 22.8
(user+sys)/real 8.2 17.1
E.g. the build was approximately 45% faster in real time. On machines with less cores, or with lower -j settings, the speedup will not be as impressive. But at least you can now almost max out a machine with buildworld!
Submitted by: jilles
MFC r263833:
Enable parallel building for gnu/usr.bin and usr.bin/clang too.
|
#
263019 |
|
11-Mar-2014 |
bapt |
MFC: r262398,r262399,r262410,r262975
Import libucl into head
UCL is heavily infused by nginx configuration as the example of a convenient configuration system. However, UCL is fully compatible with JSON format and is able to parse json files.
UCL is used by pkg(8) for its configuration file as well for the manifest format in packages, it will be used in base for the pkg boostrap (signature checking and configuration file parsing.)
libucl has been developped and is maintained by vsevolod@
|
#
260013 |
|
28-Dec-2013 |
jmmv |
Set up the /usr/tests hierarchy.
This is a MFC of the following into stable/10: - r257097 Set up the /usr/tests hierarchy. - r257098 Add missing WITHOUTTESTS file. - r257100 Add a tests(7) manual page. - r257105 Disable WITHTESTS= for now. - r257848 Fix buildworld when WITHTESTS is enabled. - r257850 Subsume the functionality of MKATF into MKTESTS. - r257851 Handle the removal of the test suite when WITHOUTTESTS=yes. - r257852 Install category Kyuafiles from their category directories. - r258232 Install BSD.tests.mtree when MKTESTS is yes.
Note that building with WITH_TESTS is still broken at this point (and hence why WITHOUT_TESTS is the set as the default). Subsequent pullups will fix the remaining issues.
|
#
258398 |
|
20-Nov-2013 |
peter |
MFC r258283 - move iconv* symbols out of libc.so.7 namespace with compatibility symbols to aid transition for existing 10.x installs.
Approved by: re (kib)
|
#
258230 |
|
16-Nov-2013 |
gjb |
MFC r257583, r258012, r258013:
r257583 (peter): Remove the WITH_LIBICONV_COMPAT hack that seems to do more harm than good. This caused libc to spoof the ports libiconv namespace and provide a colliding libiconv.so.3 to fool rtld. This should have been removed some time ago.
r258012: Remove WITH_LIBICONV_COMPAT file to chase after r257583.
r258013: Regenerate src.conf.5 after removal of WITH_LIBICONV_COMPAT.
Approved by: re (kib) Sponsored by: The FreeBSD Foundation
Sponsored by: The FreeBSD Foundation
|