Copy stable/10@r272459 to releng/10.1 as part of
the 10.1-RELEASE process.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC revisions 268999, 269027, 269351-269352, 269354, 269460, 270283,
270505, 270954, and 270989:
r268999: Add new bsdconfig example scripts; remove obsolete ones
r269027: Update bsdconfig dot module; fixes and enhancements
r269351: Add setvar() for non-FreeBSD platforms using bash as /bin/sh
r269352: Fix syntax error when run under bash
r269354: Update setvar() function introduced in r269351
r269460: Update f_xdialog_info() in bsdconfig's dialog.subr include
r270283: Add `-A' flag to pkg-install(8) when installing dependencies
r270505: Optimize f_which() to be slightly faster still (common.subr)
r270954: Update f_dialog_init() for safer getopts usage (dialog.subr)
r270989: Fix for previously mentioned r270954

Approved by: re (gjb)

MFC r267680:
Fix a code typo that prevented mkdir from firing (unnoticed
usually because another part of the code succeeded in making
the same directory).

MFC r266297: Update example portion of comment to coincide with r264840

MFC r264840: Implement GEOM based media device classification.

MFC revisions 260894,260899,262895-262902,262904,262908-262910,262982,
262984,263133-263137,263139,263141,263144-263150, and (partially) 263249
(31 revisions; summarized below)...
r260894: Optimize f_expand_number(), improving performance
r260899: s/__number/__num/ in f_expand_number()
r262895: Allow dispatched reswords to carry arguments
r262896: Add missing local declaration
r262897: Fix a typo in a comment
r262898: Fix incorrect return status if var_to_set and var_to_get are same
r262899: Make f_show_err non-fatal
r262900: Centralize function name; Update a comment while here
r262901: s/__num/__number/ in f_expand_number()
r262902: Comment to go with NL global introduced by previous commit
r262904: Rewrite groupmgmt -- hooking it into the scripting system
r262908: Change dispatch words from group* to *Group for backward compat
r262909: Fix copy/paste error in a comment
r262910: Take a group name on the command-line if available
r262982: Whitespace
r262984: Remove vestigial global, no longer used since r262904
r263133: Remove indexfile from debug statement (already logged)
r263134: Add debug statement just before attempting to exec a module
r263135: Comments
r263136: Update copyright
r263137: Fix future namespace issues for functions taking $var_to_set
r263139: Remove useless NULL string in compound strings
r263141: Pointy hat! Fix a broken f_isinteger()
r263144: Fix a code-typo that prevented auto-sizing of a dialog
r263145: Fix comments and whitespace
r263146: Reduce the sleep cycle when using dialog(1) [infobox] to 1-second
r263147: Fix a bug preventing errors from pw(8) from appearing
r263148: For non-interactive scripts, forgot to check argument
r263149: Add protection against input containing single-quotes
r263150: Rewrite usermgmt
r263249: (partial) Add more obsolete files

MFC r259054:
Performance and debugging enhancements:
+ Remove UNAME_P=$(...) from startup/misc -- already supplied by common.subr
+ Use f_getvar instead of $(eval echo \$$var) -- f_getvar is sub-shell free
+ Add `-e' and `-k var' options to f_eval_catch -- increasing use-cases
+ Use f_eval_catch to display errors on failure -- reducing duplicated code
+ Use f_eval_catch when we need output from a command -- improving debugging
+ Optimize f_isinter of strings.subr for performance -- now sub-shell free
+ Improve error checking on pidfiles -- using f_eval_catch and f_isinteger
+ Use $var_to_set arg of f_ifconfig_{inet,netmask} -- eliminate sub-shells
+ Use f_sprintf instead of $(printf ...) -- consolidate sub-shells
+ Use $var_to_set arg of f_route_get_default -- eliminate sub-shells
+ Add f_count to replace $(set -- ...;echo $#) -- eliminate sub-shells
+ Add f_count_ifs to replace $(IFS=x;set -- ...;echo $#) -- no sub-shells
+ Replace var="$var${var:+ }..." in loops with var="$var ..." with a follow-
up var="${var# }" to trim leading whitespace -- optimize loops
+ Use $var_to_set arg of f_resolv_conf_nameservers -- eliminate sub-shells
+ Comments for the f_eval_catch function
+ Remove a duplicate `local ... desc ...' in f_device_get_all of device.subr
+ Use $var_to_set arg of f_device_capacity -- eliminate sub-shells
+ Whitespace fixes in f_dialog_init of dialog.subr
+ Optimize f_inet_atoi of media/tcpip.subr for performance -- sub-shell free
+ In several cases, send stderr to /dev/null -- clean up runtime execution
+ Change f_err of common.subr to go to program stderr not terminal stderr,
allowing redirection of output from functions that use f_err
+ Disable debugging when using f_getvar to get variable argument to
f_startup_rcconf_map_expand of startup/rcconf.subr
+ Use f_replace_all instead of $(echo ... | tr | sed) -- performance
+ Add a $var_to_set option to f_index_{file,menusel_{command,keyword}} of
common.subr -- centralize sub-shells

MFC r258458:
Improve network device scanning in the netdev module. First, make it use the
`device.subr' framework (improving performane and reducing sub-shells). Next
improve the `device.subr' framework itself. Make use of the `flags' device
struct member for network interfaces to indicate if an interface is Active,
Wired Ethernet, or 802.11 Wireless. Functions have been added to make checks
against the `flags' bit-field quick and efficient. Last, add function for
rescanning the network to update the device registers. Remove an unnecessary
local (ifn) while we're here (use already provided local `if').

MFC r258355,258360,258400-258401,258406-258407,258418,258430,258439,258589-
258590,258592,258727-258728,258785, with the following descriptions:

Revisions 258400-258401,258406-258407,258590,258727-258728,258785:
+ Add a new `includes' module (see: bsdconfig includes -h) for exploring
the bsdconfig(8) API

Remaining revisions:
258592: Sort all the messages.subr files
258355: Fix typo in a comment in networking/device.subr
258360: Whitespace, style, and other changes in networking/device.subr
258418: Quote interface names for good measure in networking/device.subr
258430: Remove an unused line in networking/device.subr
258439: Minor bugfix to f_die() function in common.subr
258589: Minor bugfix to syscons_ttys module

MFC r259426, r259427:
r259426:
Add a pkg(8) repository configuration file for cdrom-based package
installation.

As part of the 'pkg-stage' target, copy the configuration file
to the 'packages/repos/' directory on the DVD filesystem.

r259427:
Export 'REPOS_DIR' when the selected source medium for package
installation is cdrom. This enables bsdconfig(8) to make use
of the on-disc pkg(8) repository configuration, which fixes
package selection and installation from the dvd installer.

Sponsored by: The FreeBSD Foundation

MFC r259113, r259115, r259144, r259148:

r259113 (dteske):
Fix failed attempt to send pkg(8) stderr to /dev/null

r259115 (dteske):
Prevent truncating /tmp/bsdinstall_log each time we
exec a module.

r259144 (dteske):
Fix a regression after successfully installing to encrypted
ZFS root, the passphrase is not accepted and a message about
"incorrect key" is displayed.

r259148 (dteske):
Fix a regression resulting in mountroot prompt after attempting
to install to encrypted ZFS root (caused by a typo in a
variable name -- ZFSBOOT_BOOT_FSNAME -> ZFSBOOT_BOOTFS_NAME).

Sponsored by: The FreeBSD Foundation

MFC r258854:
Fix PKG_ABI detection in bsdconfig(8) after pkg-1.2 is released, since
the format of 'pkg -vv' output has changed.

Approved by: re (hrs)
Sponsored by: The FreeBSD Foundation

MFC r257755-257756,257780-257785,257787-257793, and
257795,257817,257819,257937-257938,258029,258263-258267:
257755: SRV records
257756: fix spurious error message
257780: Whitespace
257781: Comments and funny syntax
257782: Debug file truncation is optional
257783: f_show_err for debugging
257784: f_eval_catch for debugging
257785: fix size calculations bug
257787: fix broken HTTP "any" media type
257788: more debugging
257789: Comments
257790: fix printf usage bug
257791: f_[v]sprintf added
257792: Comments
257793: fix off-by-one error in size calcs
257795: Replace pkg-tools with pkgng
257817: fix cosmetic typos
257819: Use `pkg -vv' to get ABI
257937: Adjustment to last
257938: Adjustment to last
258029: Comments
258263: Shuffle code around
258264: Remove unused code
258265: Debugging. Use f_eval_catch with pkg
258266: Shutdown media on exit from packages
258267: Fix pkg install from DVD

Reviewed by: many
Discussed on: -current
Approved by: re (hrs)

MFC r256391:
Fix signed integer overflow detection in f_expand_number() of strings.subr.

Approved by: re (glebius)

MFC revisions 256321-256323,256331,256333,256335,256343:
Bring in a new zfsboot auto script for performing automatic setup of a boot
pool (optionally encrypted) with many other options, validations, features.
Originally submitted by Allan Jude; modified in collaboration.

MFC revisions 256325,256330,256345:
Rewrite the keymap selection menu to display keymaps and provide a test
mechanism. Test mechanism originally submitted by Warren Block; modified.

MFC r256347:
Prominently display "Wireless" for each wireless network interface. Part
of PR bin/161547; submitted by Warren Block; slightly modified.

MFC r256348:
Remove the dumpdev configuration dialog, merge it into the regular
services configuration and enable it by default. Originally submitted by
Allan Jude; slightly modified.

PR: bin/161547
Submitted by: Allan Jude, Warren Block <wblock@wonkity.com>
In collaboration with: Allan Jude <freebsd@allanjude.com>
Approved by: re (glebius)

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Introduce centralized [X]dialog(1) exit codes to `dialog.subr' and make-use
throughout the bsdconfig(8) code. While we're here, add an explicit argument
to lvalue-seeking invocations of "return" that previously had no argument.
Also, consolidate a single instance of double-newline and remove some
comments that are no longer required (given increased readability with new
exit codes).

Approved by: re (glebius)

Remove ftp5.se.f.o, as per request to -hubs@

Approved by: re (glebius)
MFC after: 3 days

- Remove the is (Iceland) mirror per mail from the admins.

Approved by: re
With hat: clusteradm@

Fix a bug in HTTP checking/fetching.
Fix a bug in HTTP checking/fetching. Add Main Site to HTTP menu. Add new
example script browse_packages_http.sh and move existing example script
browse_packages.sh -> browse_packages_ftp.sh

Reviewed by: gjb, brd
Approved by: re (gjb), clusteradm (brd)
MFC after: 3 days

Remove ftp2.tr.freebsd.org, it is out of date.

MFC after: 3 days

Sync-up with bsdinstall(8) FTP mirrorselect changes.

SVN r224656:
Add back ftp4, a new machine has taken its place. ftp2 and ftp8 are both
IPv6 capable mirrors. cvsup4/ftp4 have already been added to the handbook.

NOTE: Except put IPv6 entries in IPv6 section at the top of the list

SVN r225467:
Add ftp4.se.freebsd.org to the IPv4 and IPv6 mirror lists.

SVN r235228:
Add the IPv6 tag to ftp4.fr.freebsd.org.

NOTE: Except dup entry into IPv6 section.

SVN r243832:
- Remove snapshots.se.freebsd.org [1]
- Add ftp6.se.freebsd.org

SVN r243854:
Remove snapshots.jp.freebsd.org. It stopped working years ago.

SVN r253543:
Update mirror list to more closely match
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/mirrors-ftp.html

MFC after: 1 week

Re-implement $probe_only aspect of f_media_get_TYPE() (where TYPE is cdrom,
nfs, ftp, http, httpproxy, etc.) and f_device_get() (abstract method for
calling aforementioned f_media_get_TYPE()).

Previously, if $probe_only was present and non-NULL, the TYPE functions
would check for $file and exit with an appropriate error status (success if
the file exists and readable, failure otherwise).

While this has been retained, a pair of globals has been introduced:
$PROBE_EXIST and $PROBE_SIZE (see `/usr/share/bsdconfig/media/common.subr')

The $PROBE_EXIST global can be used where you need the functionality of
simply testing for existence (previously the _only_ functionality).

Meanwhile, the new $PROBE_SIZE global can be used to cause the TYPE function
to print the size of the file (in bytes) on standard-out (or -1) if not
found or an error occurs. NOTE: If an error occurs, it is logged with the
dprintf function, which is visible with `-d' flag or debug=1.

In many cases, where you need to get the size of a file _and_ check for its
existence, you can use the return status of a $PROBE_SIZE call.

Introduce a new [yet unused] function for [efficiently] getting the path to
an executable by-name without forking or using externals.

In a performance benchmark of 10,000 runs on circa 2006 hardware, f_which
out-performed `which' with an average completion time of ~2.5 seconds versus
~56 seconds.

This should be handy for future use (not that I make it a habit to call
`which' in a loop 10,000 times).

Adjust comments to fit within 80-columns.

Fix conditional (der should match the comment above it).

MFC after: 1 day

Do two things: First, don't obscure the backtitle. Second, read ~/.dialogrc
if it exists to determine if use_shadow is true (ON) or false (OFF).

The purpose of determining the value of use_shadow is to know how many lines
to subtract from the maximum height value in assuring that the backtitle is
not obscured.

The detriment of obscuring the backtitle is that it provides information
that is not easily obtained elsewhere. That is the command-line shortcut
used to access the current menu. As you navigate from one dialog to the
next, invariably transparently corssing module boundaries, the backtitle
represents the command-line argument used to get there. Obscuring this
information with a widget that is too-tall and/or too-wide would see that
data go unnoticed (leaving few other ways to get that information in the
same helpful context).

So despite the fact that this change reduces the standard maximum height for
all widgets, there is a trap-door to prevent this calculation. If you want
to utilize the full screen height on the terminal (remember, this adjustment
is not made for Xdialog(1)) you can set $NO_BACKTITLE to 1 (or any non-NULL
value for that matter) and this calculation will be skipped. You will be
able to draw a widget that partially obscures the backtitle if-necessary.

MFC after: 1 day

Fix typos in the BSD License.

Be consistent with other usr.sbin programs w/respect to the copyright
wording ("All rights reserved."); I had the casing wrong on many instances.

Be consistent with other usr.sbin programs w/respect to the copyright
wording ("All rights reserved."); I had the casing wrong on many instances.

Add the necessary code to reinstall packages. Both scripted access
(packageReinstall) and UI access have been tested successfully with a
variation of different situations including:
+ Reinstall a package for which no other packages depend
+ Purposefully do thinks like reinstall a package that is not installed
+ Try to reinstall a package which other installed packages still depend

NOTE: There is no "force" used; if a package is required by other packages,
it will not be uninstalled (and therefore no reinstall is done).

Remove superfluous continue at end of loop. (pointy-hat)

Add support for processing add-on modules from /usr/local/libexec/bsdconfig
(this is designed to allow new modules to be installed via ports/packages).

To prevent conflict with itself (sysutils/bsdconfig) as a port (which
installs its base modules to the above directory, it was long-ago decided
that so-called `base' modules would look different than now-defined `add-on'
modules. The structure of the contents for each is the same, but the naming
convention for the module directory must be different.

Base modules are named `[0-9][0-9][0-9].*' to allow SysV-style organization
while add-on modules must avoid this naming style and are simply listed in
alphabetical order by their module directory.

For example, a hypothetical port named `bsdconfig-jails' could install
/usr/local/libexec/bsdconfig/jails and provide `bsdconfig jails' as well as
a new menu entry in the main-menu.

Add-on modules are listed in the main-menu (when bsdconfig is executed with-
out arguments) below a separator after the last base-module.

In `bsdconfig -h' output, add-on modules are listed right alongside base
modules (sorted alphabetically in columnar fashion; left-to-right).

If a base module declares a keyword used by an add-on module, the base
module will always win when given `bsdconfig keyword' syntax.

Add-on modules should avoid declaring any keyword found in `script.subr' as
a reserved-word (`Resword') since bsdconfig also supports `bsdconfig resword'
as a fall-back if no keyword is found to be declared by any module.

Oops, r252833 was not supposed to touch this file. Back-out and recommit
this file with the rest of the files it was supposed to go with.

Don't calculate the tag until we know that we're going to make a new menu
item entry. Also join simple NULL assignments into a single line.

Use f_show_msg() instead of f_dialog_msgbox() where appropriate. The main
difference between these two functions:

Usage: f_show_msg() $format_string [ $format_args ... ]
Usage: f_dialog_msgbox() $text [ $hline ]

The former lends itself well to displaying the $msg_* i18n text, prompts,
etc. While the latter is better for text you do not control (error strings
captured as a response from external commands) -- or if you have to control
the hline.

Add the necessary code to uninstall packages (re-install still pending).
Both scripted access (packageDelete) and UI access have been tested
successfully with a variation of different situations including:
+ Uninstall a package which no other installed package depends
+ Uninstall multiple packages which no other installed packages depend
+ Uninstall multiple packages which depend on each other
+ Similar to above but when ordered removal requires tracing dependencies
+ Purposefully do things like uninstall a package that is not installed
+ Try to uninstall a package which other installed packages still depend
+ Try to uninstall multiple packages which other installed packages depend
+ And many more.

Add minor debugging.

Prevent f_package_add() from re-performing f_package_detect_installed()
even though f_package_config() may have already detected the installed
packages.

Don't deselect a package right after installing it (this went unnoticed
because [currently] we exit after performing requests actions -- if we
ever change this to returning to the package selection screen, this would
have made an impact).

Minor debugging changes/additions.

Fix a typo in a comment.

Add the beginnings of the old networking services functionality of
sysinstall(8). The UI for this will come later while this exposes the
functionality to the scripted interface. Tested successfully to first
install the pcnfsd package from the `Latest' repository, and second to
configure mountd_flags="-n" in rc.conf(5).

Add the necessary code to install packages (uninstall and re-install still
pending). Both scripted access (packageAdd) and UI access have been tested
successfully with a variation of different situations including:
+ Install a package with no dependencies
+ Install a package with dependencies that are already installed
+ Install a package with dependencies where some are already installed
+ Repeat each of the above from FTP and local Directory
+ Purposefully do things like try to install a package that does not exist
+ Try to install a package for which a dependency could not be loaded
+ Try to install a package that is in the INDEX bot not on the media
+ And many more.

Add debugging for each of the initialized default variables.

Add mustberoot.subr to the list of includes for scripts, exposing the helper
function f_mustberoot_init() to make it easy to write scripts that can more
easily transition to the super-user via sudo(8).

Remove separate processing of dependent packages from the "Review" screen
since f_package_add (packageAdd resword) processes dependencies itself.
NOTE: This is in preparation for incoming package management code.

Get rid of place-holder in preparation for incoming package mgmt code.

Comments.

Allow global default as general fall-back for package management routines.

Set name of device to directory path (displayed during package operations).

Implement $probe_only for the media access modules. sysinstall(8) was
allowed to ignore the probe_only argument of its member functions solely
because in the C language, the file accessor methods open and return a file
descriptor and reading of the data is optional. In shell, the file accessor
methods return data on stdout and that data should not be ignored (large
files could block execution).

So, we must adhere to the probe_only flags and in some cases (in the case of
FTP, for example) change the `get' strategy to simply test existence and
return an appropriate status.

This was required because the up-coming package management stuff makes heavy
use of the probe_only argument to try different package suffixes. Every
media access module must implement $probe_only for the `get' accessor.

More whitespace.

Merge r248313 from stable/9 sysinstall(8) to head bsdconfig(8):
Add support for installation directly via HTTP.

While we're here, remove the menu-item for Passive FTP (since moving to
ftp(1) and switching FTPMODE to `auto' by default -- see r251613 -- the
single remaining FTP menu-item works for both ftp.f.o and ftp-archive.f.o;
previously each requiring separately active versus passive both work with
the `auto' setting). In scripting you still have mediaSetFTPActive and
mediaSetFTPPassive but the remaining FTP menu-item uses mediaSetFTP which
defaults to `auto' (aforementioned SVN r251613).

Implement the $probe_only parameter (previously unimplemented).

Fine-tune the parsing of the URL. Re-order, comment, and add debugging to
each case of unique URL format.

Fix a code typo that prevented the probing of the current defaultrouter
in-order to pre-populate the editor.

Oops, previous revision (r252084) broke non-full-path resolution (e.g.,
ftp://ftp.freebsd.org or ftp://ftp.freebsd.org/). Now both full-path and
non-full resolution is working regardless of trailing-slash.

Trim the trailing `/' from the ftp_dir when validating that the directory
exists on the FTP server. This now means that when you specify a full-path
to the repository on the FTP server that it doesn't matter whether you have
a trailing `/' or not.

Take advantage of the fact that f_getvar evaluates the $var_to_get parameter
within braces, allowing all manner of parameter expansions (properly quoted
or escaped, of course).

Add missing local declaration.

Remove unused library include.

Alphabetize library includes.

Style.

Typo and grammar fixes to comments.

Fix a code typo in a case-statement match expression that prevented IPv6
URLs with port designator from working properly (e.g. http://[::1]:80/).

Alphabetize reserved-word (resword) registration.

Alphabetize includes.

Change the f_dialog_{yesno,noyes}() function-arguments in `dialog.subr' to
accomodate an $hline value for overriding the default. This change does
not effect any current modules as it turns out that not one single usage of
either f_dialog_yesno() or f_dialog_noyes() relied on accepting more than a
first argument (read: all occurrences quoted the first parameter; so $* was
never depended upon).

This will allow some custom invocations of --yesno and --noyes to roll over
to these functions (for example, in `timezone/timezone').

Change the f_dialog_msgbox() arguments in `dialog.subr' to accomodate an
$hline argument for setting the --hline parameter value. This change does
not effect any current modules as it turns out that not one single usage of
f_dialog_msgbox() relied on accepting more than a first argument (read: all
occurrences quoted the first parameter; so $* was never depended upon).

This will allow some custom invocations of --msgbox to roll over to this
function (for example, in `mouse/disable').

Add skeleton processing of selected packages in the order in which they
should be processed. De-select each package as we "process" (actual
processing still pending) each package so that if we get an error, we can
return to an accurate state of selected packages.

Make the default choice for FTP media access use "auto".

Use f_shell_escape() instead of forking to awk. In this case, the
replacement comes with a great performance increase (as f_shell_escape()
uses the built-in based f_replaceall() which out-performs forking to
awk(1)). This should also improve readability slightly.

Similar to r251236, r251242, and r251244. Introduce new function-pair to
dialog(1) API in dialog.subr responsible for retrieving menu-choice data
(for the --menu widget).

Add f_dialog_menuitem_store()/f_dialog_menuitem_fetch() for storing and
retrieving the menuitem (not the tag) from a --menu widget result.

The dialog --menu widget returns the `tag' from the tag/item pair of choices
for a menu list. Because the menu list is often local to the function that
is displaying it, the ability to dereference this into the item is not
possible unless you use a global. This changes things so the function (upon
successful return of dialog) dereferences the tag into the item and stores
it for later retrieval using these functions.

NOTE: Tags are dereferenced into items using f_dialog_menutag2item().

Fix a code typo (missing `eval') which lead to improperly-sized menu when
displaying packages (menu was always maximum height). The problem was
introduced by SVN r251232.

Fix menu accenting for the package category list. The single-space indent
for duplicate initial-characters was removed for making comparisons but was
not put back.

Whitespace and comments.

Improve debugging for f_dialog_*_constrain() to print constrained rather
than unconstrained values.

Like r250701, introduce another handy function for truncating variables to a
specific byte-length. Works like vsnprintf(3).

Fix a bug in which the user's choice to Cancel was not recognized. This was
caused by the standard (and correct) behavior of the shell to discard the
return status of lvalue-operands in a pipe-chain.

The solution is to not pipe the file-acquisition directly into sort(1) but
instead store the output (allowing immediate testing of the return status)
and later sort it.

Fix a bug that would cause a spurious warning to appear to appear in an
edge-case. The case was that you have been through the FTP setup once before
and on the second time through, you cancel at the re-selection of a new FTP
server.

The spurious warning was "device_media: not found" and was caused because
the underlying call to f_device_network_down() did not check to see if the
network device existed before attempting to shut it down.

Add checks to make sure we don't forge ahead unless the device exists.

Fix a bug in which the user's choice to Cancel was not properly recognized.

Fix a copy/paste error within a comment.

Add some additional debugging to the common media library.

Standardize NULL initialization to be consistent.

Use f_isset() (introduced by r250317) instead of `f_quietly f_getvar ...'.

Make proper use of locals in f_dialog_init().

Improvements to whitespace and comments.

Fix option processing from the library layer to address unexpected
behavior(s); e.g., `-Xd' versus `-dX' did not produce the same results.

The libraries common.subr and dialog.subr automatically process the
arguments passed to the program and enable/disable functionality without the
need to process the arguments within your program. For example, if "$@"
contains `-d', common.subr will see this and enable debugging regardless of
whether you process "$@" yourself or not (this automatic processing can
easily be disabled for custom scripts that don't want it; see the afore-
mentioned scripts for additional details).

NOTE: common.subr stores a copy of "$@" in $ARGV for convenient (and
repeated) processing by libraries such as dialog.subr which provide such
transparent functionality for the consuming script(s).

However, the libraries don't know if a program wants to accept `extra'
options. Flags are not really a problem, because the library can be
programmed to silently ignore unknown flags. The trouble comes into play
when the program wants to define an option that takes an argument.

For example:

bsdconfig -D logfile -X

In the above example, the library uses getopts to process $ARGV and if it
doesn't know that `-D' takes an argument, the option processing will
prematurely terminate on `logfile' (this is standard/correct behavior for
getopts but is undesired in our situation where we have partially off-loaded
main argument processing).

The problem is solved by allowing the program to define an extra set of
options to be included in each library's handling of $ARGV. Only options
that require arguments are truly necessary to be pre-specified in this new
manner.

Standardize the way functions build their arguments leading up to a dialog
invocation. Specifically, "top-load" your arguments and in the order in-
which they will be displayed. For example, many [if not all] widgets display
information in the following order, top-to-bottom (visually):

+ backtitle (displayed behind the widget at top-left)
+ title (at the top of the `window')
+ prompt text (just below the title and above whatever widget you choose)
+ Depending on widget, _one_ of the following:
- menu list
- radio list
- check list
- text input box with initial text
- [Xdialog(1)] 2x or 3x text input boxes
- [dialog(1)] a multi-part form
- progress bar
- etc. (many more widget choices)
+ buttons (right below the selected widget)
+ [dialog(1)] the hline (displayed at bottom of `window')

NOTE: Xdialog(1) accepts and silently ignores --hline

When building local arguments for your dialog invocation, if the value can't
be cleanly loaded into a local, add "# Calculated below" to the end of the
local declaration while retaining the block order of argument declarations.

Move other local declarations that are not associated with this top-loading
the dialog arguments to right-above where they are first-used.

Also, standardize on the names of the arguments. For example, always use
$prompt (instead of sometimes $msg and sometimes $prompt); use $menu_list
or $shell_list or $radio_list for those respective widgets; ad nauseum.

While we're doing this, flush-out full arguments for many invocations that
were passing NULL strings (making it unapparent if you were staring at this
one invocation what argument that NULL string was supposed to represent).

Last, while we're in startup/rcconf let's remove the unnecessary use of a
GLOBAL (RCCONF_MENU_LIST) for the menu_list.

Improve the dialog(1) API in dialog.subr by adding f_dialog_default_store()
and f_dialog_default_fetch(). Operating similar to functions introduced by
SVN r251236 and r251242, these functions operate as a pair for helping track
the default-item data (for the --menu, --checklist, and --radiolist
widgets).

This replaces the direct usage of a global to store the data with an
abstract method for readability and to centralize the code.

Similar to r251236, improve the portion of dialog(1) API in dialog.subr
responsible for retrieving stored input (for the --inputbox and --password
widgets).

When we (Ron McDowell and I) developed the first version of bsdconfig, it
used temporary files to store responses from dialog(1). That hasn't been
true for a very long time, so the need to always execute some clean-up
function is long-deprecated. The function that used to perform these clean-
up routines for these widgets was f_dialog_inputstr().

We really don't need f_dialog_inputstr() for its originally designed purpose
as all dialog invocations no longer require temporary files.

Just as in r251236, redesign f_dialog_inputstr() in the following four ways:

1. Rename f_dialog_inputstr() to f_dialog_inputstr_fetch()
2. Introduce the new first-argument of $var_to_set to reduce forking
3. Create a corresponding f_dialog_inputstr_store() to abstract storage
4. Offload the sanitization to a new function, f_dialog_line_sanitize()

It should be noted that f_dialog_line_sanitize() -- unlike its cousin from
SVN r251236, f_dialog_data_sanitize() -- trims leading/trailing whitespace
from the user's input. This helps prevent errors and common mistakes caused
by the fact that the new cdialog implementation allows the right-arrow
cursor key to go beyond the last byte of realtime input (adding whitespace
at the end of the typed value).

While we're centralizing the sanitization, let's rewrite f_dialog_input()
while we're here to likewise reduce forking. The f_dialog_input() function
now expects the first argument of $var_to_set instead of producing results
on standard-out.

These changes greatly improve readability and also improve performance.

Improve portion of the dialog(1) API in dialog.subr responsible for
retrieving stored data (for the --menu, --calendar, --timebox, --checklist,
and --radiolist widgets).

When we (Ron McDowell and I) developed the first version of bsdconfig, it
used temporary files to store responses from dialog(1). That hasn't been
true for some very long time, so the need to always store the return status
of dialog(1) and then call some function to clean-up is long-deprecated. The
function that used to do the clean-up was f_dialog_menutag().

We really don't need f_dialog_menutag() for its originally designed purpose,
as all dialog invocations (even when in a sub-shell) do not use temporary
files anymore.

However, we do need to keep f_dialog_menutag() around because it still fills
the need of being able to abstract the procedure for fetching stored data
provided by functions that display the aforementioned widgets.

In re-designing f_dialog_menutag(), four important changes are made:

1. Rename f_dialog_menutag() to f_dialog_menutag_fetch()
2. Introduce the new first-argument of $var_to_set to reduce number of forks
3. Create a corresponding f_dialog_menutag_store() to abstract the storage
4. Offload the sanitization to a new function, f_dialog_data_sanitize()

NOTE: That last one is important. Not all functions need to store their data
for later fetching, meanwhile every invocation of dialog should be sanitized
(as we learned early-on in the i18n-effort -- underlying libraries will spit
warnings to stderr for bad values of $LANG and since dialog outputs its
responses to stderr, we need to sanitize every response of these warnings).

These changes greatly improve readbaility and also improve performance by
reducing unnecessary forking.

Fix a regression in the packages module introduced by recent r251190.
I somehow neglected this module in merging that API change.

Improve portion of the dialog(1) API in dialog.subr responsible for
calculating widget sizes. Instead of forking a sub-shell to calculate the
optimum size for a widget, use a byRef style call-out to set variables in
the parent namespace. For example, instead of:

size=$( f_dialog_buttonbox_size title btitle msg )
$DIALOG --title title --backtitle btitle --msgbox msg $size

The new API replaces the above with the following:

f_dialog_buttonbox_size height width title btitle msg
$DIALOG --title title --backtitle btitle --msgbox msg $height $width

This reduces the number of forks, improves performance, and makes the code
more readable by revealing the argument-order for widget sizing. It also
makes performing minor adjustments to the calculated values easier as
you no longer have to split-out the response (which required knowledge of
ordering so was counter-intuitive).

Fix a typo in a comment.

Add a handy function for truncating variables to a specific byte-length. It
should be noted that newlines are both preserved and included in said byte-
count. If you want to truncate single-line values without regard to line
termination, there's always f_substr() which already exists herein.

Centralize standard getopts arguments, both for convenience and to correct
a bug in which certain combinations of arguments produced unexpected results
such as `-dX' (now properly produces debugging and X11), `-XS' (now properly
produces X11 in secure mode), `-df-' (enables debugging when reading a
script from standard-input, etc. Multi-word variations such as `-d -X',
`-X -S', `-d -f-', `-d -f -', etc. also work as expected. Also tested were
variations in argument order, which are now working as expected.

Comment.

Fix i18n violations in the package management module. A few words like
`packages', `installed', and `selected' were not internationalized.

Remove duplicated string.

Comments.

Whitespace.

Fix a bug that would cause the category menu to display the wrong value
for number of packages available in the "All" category. Problem caused by
re-using a variable that was still needed; fixed by variable name change.

Commit first portion of package module -- this includes the ability to view
categories, view packages, mark packages for installation, de-installation,
or re-installation, calculate and track dependencies, as well as ability to
review selections.

Still to come is the actual processing of selections (performing the
various actions associated with the user's selections, such as installing
dependencies first, then selections, etc.).

Re-organize and add missing installVarDefaults and mediaSetDirectory.

Add f_isset() utility subroutine for checking if a variable is set but in
a more readable fashion.

Properly sanitize --menu results (guards against Gtk library warnings from
X11 side of things from bleeding into Xdialog(1) stderr output). It should
be duely noted that such errors are not a by-product of anything in the
Xdialog(1) utility or API, but optional libraries that it can link against
(such as Gtk1 versus Gtk2; if you compile xdialog from ports against Gtk2
AND misconfigure your fonts or generally make Gtk2 unhappy, these warning
messages can bleed into the captured stderr -- that is we we sanitize!).

Comments.

Add missing include.

Preserve debugFile preference across the exec boundary.

Partially uncommit r249779. The changes to share/common.subr were good
while the remaining changes were part of a much larger ``secret sauce''
involved in an up-coming commit that I'm still laboring on.

Fix "-D file" to automagically enable debugging if not explicitly disabled.

New helper functions for common widgets.

Proper fix for copy/paste error (first attempt r249756).

Fix a copy/paste error.

Update comment for accuracy.

Style nit (to be consistent across project).

UI improvements. First, implement --default-item whenever and wherever
possible to save keystrokes. Second, overhaul startup/rcdelete for much
improved performance. Last, but not least, kill-off useage of --clear
and implement --keep-tite in harmony to minimize jarring transitions.
Also, fix local variable names where necessary while we're here with
other minor comment-enhancements/typo-corrections.

Add new flags `-d' (sets debug=1) and `-D file' (sets debugFile) and
improve debugging initialization. Also fixup USAGE statements while we're
here. Also, change initialization of main program to _not_ change working
directory, allowing the debugFile to be relative without confusion.

Remove some more references to legacy ATA.

Submitted by: Dmitry Luhtionov <dmitryluhtionov@gmail.com>

Import media selection/preparation framework (sysinstall inspired). Makes
accessing files from various types of media nice and abstracted away from
the wet-work involved in preparing, validating, and initializing those
types of media. This will be used for the package management system module
and other modules that need access to files and want to allow the user to
decide where those files come from (either in a scripted fashion, prompted
fashion, or any combination thereof).

Heavily inspired by sysinstall and even uses the same reserved words so
that scripts are portable. Coded over months, tested continuously through-
out, and reviewed several times.

Some notes about the changes:
- Move network-setting acquisition/validation routines to media/tcpip.subr
- The options screen from sysinstall has been converted to a dialog menu
- The "UFS" media choice is renamed to "Directory" to reflect how sysinstall
treats the choice and a new [true] "UFS" media choice has been added that
acts on real UFS partitions (such as external disks with disklabels).
- Many more help files have been resurrected from sysinstall (I noticed that
some of the content seems a bit dated; I gave them a once-over but they
could really use an update).
- A total of 10 media choices are presented (via mediaGetType) including:
CD/DVD, FTP, FTP Passive, HTTP Proxy, Directory, NFS, DOS, UFS, Floppy, USB
- Novel struct/device management layer for managing the issue of passing
more information than can comfortably fit in an argument list.

Backward compatibility fix: treat cmds loaded as a script as nonInteractive

Don't use f_show_msg() unless printf(1) syntax is required (this reduces the
number of unnecessary forks).

Add new f_yesno/f_noyes wrapper functions (which take printf(1) syntax).

Update copyright following last commit.

Add nonInteractive support to f_dialog_yesno/noyes().

Add support for scripting (sysinstall style).

Reviewed by: jilles

Comments.

Comments.

Add missing enforcement of height restriction after minor adjustment.

Add support for running without a controlling terminal (for example, when
running as an rvalue to a pipe).

Allow debug output to be logged to a file (set $debugFile to target pathname)
or both stdout and a file (precede $debugFile pathname with a plus-sign, `+').

Add more debugging to help with diagnosis of program-flow when needed.

Change axiom for initialization. Including script can disable (default)
automatic initialization by setting appropriate variable to particular value.

New f_show_info() function for dialog(1) --infobox using printf(1) syntax.

Use f_show_help() where printf(1) syntax is desired.

Improve the debugging abilities and clean up debug messages. In most cases,
all one has to do is set the environment variable DEBUGGING to get the debug
messages to appear on the console.

Add much-needed dialog(1) --inputbox function for simplifying the process of
requesting input from the user.

Allow setting of NULL titles with f_dialog_[back]title().

Approved by: adrian (co-mentor) (implicit)

Change self-initialization to occur when loaded versus the previous behavior
which was to self-initialize during the first function-call. This didn't work
so well because the first call was may or may-not be within a sub-shell
(which prevented proper setup of the pass-thru file descriptor, resulting in
dialogs that would not display).

Approved by: adrian (co-mentor) (implicit)

Whitespace.

Approved by: adrian (co-mentor) (implicit)

Replicate a feature from sysinstall documented in
stable/9/usr.sbin/sysinstall/help/shortcuts.hlp (reproduced below):

If /usr/sbin/sysinstall is linked to another filename, say
`/usr/local/bin/configPackages', then the basename will be used
as an implicit command name.

To get a list of modules you can link to, see `bsdconfig -h' output.

Approved by: adrian (co-mentor) (implicit)

Tighten-up displays that use a NULL prompt (e.g., networking and usermgmt).

Approved by: adrian (co-mentor) (implicit)

Fix copy/paste error.

Approved by: adrian (co-mentor) (implicit)

Switch from using a msgbox to display help files to a textbox instead. The
problem with using a msgbox was one of truncation in the case of Xdialog(1)
and lack of screen real-estate (since the msgbox is not scrollable in X11
while a textbox is).

The textbox renders the text much better and is more appropriate for this
type of data display.

Approved by: adrian (co-mentor) (implicit)

Resurrect usage.hlp and securelevel.hlp from sysinstall(8) and integrate them
into bsdconfig(8).

Approved by: adrian (co-mentor) (implicit)

More-accurately replicate the help system from sysinstall(8). However, also
improve upon the system by giving the user a "Help" button instead of requiring
the user to press F1.

NOTE: In FreeBSD-9 and higher, dialog(1) does not support the F1 hook, so the
mechanism for providing help to the user had to be changed to a button.

This now means we can resurrect *.hlp files from usr.sbin/sysinstall/help/ as-
is and reusing them as-needed in bsdconfig (holding to the goal of losing as
little functionality from sysinstall as possible).

Reviewed by: adrian (co-mentor)
Approved by: adrian (co-mentor)

Fix a bug where blank lines were not being counted (because awk's split()
function returns zero when $0 is the NULL string).

Reviewed by: adrian (co-mentor)
Approved by: adrian (co-mentor)

Fix typo.

Reviewed by: adrian (co-mentor)
Approved by: adrian (co-mentor)

Ask to become root via sudo(8) and give user ability to save this preference.

Reviewed by: adrian (co-mentor)
Approved by: adrian (co-mentor)

Import sysutils/sysrc from the ports tree (current version 5.1). Importing
disconnected under the WITH_BSDCONFIG flag (a good idea since this version of
sysrc(8) indeed requires the `sysrc.subr' module installed by bsdconfig(8)).

Multiple reasons sysrc should not simply continue to live in ports. The most
important being that it is tightly coupled with the base.

Approved by: adrian (co-mentor)

Allow deferred word-splitting via f_sysrc_get() by allowing $IFS in the
"clean-room" environment used to query rc.conf(5) parameters.

This brings bsdconfig(8)'s sysrc.subr in-line with both the sysrc(8) manual
[provided by sysutils/sysrc] and sysrc(8)'s own sysrc.subr (now identical to
bsdconfig(8)'s sysrc.subr as of this patch).

Finally, this will allow a clean import of sysutils/sysrc (sans sysrc.subr,
already provided here).

Reviewed by: jilles
Approved by: adrian (co-mentor)

Sanitize varname argument in f_sysrc_find. This is as much for security as it
is for sanity.

Reviewed by: jilles
Approved by: adrian (co-mentor)

Always use RC_CONFS when set, even if NULL. Previously only used if non-NULL.

Reviewed by: jilles, adrian (co-mentor)
Approved by: adrian (co-mentor)

Replace "( : ${var?} )" syntax with better "[ ${var+set} ]" syntax.

Reviewed by: jilles, adrian (co-mentor)
Approved by: jilles, adrian (co-mentor)

Spelling and whitespace corrections.

Reviewed by: adrian (co-mentor)
Approved by: adrian (co-mentor)

Reverse SVN r240770 -- jilles@ made a suggestion that allowed us to redesign
our embedded rescue environment to support /dev/null making r240770 obsolete.

Reviewed by: jilles, adrian (co-mentor)
Approved by: jilles, adrian (co-mentor)

Replace redirections to /dev/null with "close file-descriptor" syntax (>&-).

Reviewed by: adrian (co-mentor)
Approved by: adrian (co-mentor)

Change all invocations of dialog(1) to no-longer require temporary files.
This allows bsdconfig to -- like bsdinstall -- operate from read-only media.

Reviewed by: adrian (co-mentor)
Approved by: adrian (co-mentor)

Move major includes into /usr/share/bsdconfig for easy external access.

Reviewed by: adrian (co-mentor)
Approved by: adrian (co-mentor)