Copy stable/10@r272459 to releng/10.1 as part of
the 10.1-RELEASE process.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC revisions 260894,260899,262895-262902,262904,262908-262910,262982,
262984,263133-263137,263139,263141,263144-263150, and (partially) 263249
(31 revisions; summarized below)...
r260894: Optimize f_expand_number(), improving performance
r260899: s/__number/__num/ in f_expand_number()
r262895: Allow dispatched reswords to carry arguments
r262896: Add missing local declaration
r262897: Fix a typo in a comment
r262898: Fix incorrect return status if var_to_set and var_to_get are same
r262899: Make f_show_err non-fatal
r262900: Centralize function name; Update a comment while here
r262901: s/__num/__number/ in f_expand_number()
r262902: Comment to go with NL global introduced by previous commit
r262904: Rewrite groupmgmt -- hooking it into the scripting system
r262908: Change dispatch words from group* to *Group for backward compat
r262909: Fix copy/paste error in a comment
r262910: Take a group name on the command-line if available
r262982: Whitespace
r262984: Remove vestigial global, no longer used since r262904
r263133: Remove indexfile from debug statement (already logged)
r263134: Add debug statement just before attempting to exec a module
r263135: Comments
r263136: Update copyright
r263137: Fix future namespace issues for functions taking $var_to_set
r263139: Remove useless NULL string in compound strings
r263141: Pointy hat! Fix a broken f_isinteger()
r263144: Fix a code-typo that prevented auto-sizing of a dialog
r263145: Fix comments and whitespace
r263146: Reduce the sleep cycle when using dialog(1) [infobox] to 1-second
r263147: Fix a bug preventing errors from pw(8) from appearing
r263148: For non-interactive scripts, forgot to check argument
r263149: Add protection against input containing single-quotes
r263150: Rewrite usermgmt
r263249: (partial) Add more obsolete files

MFC r257755-257756,257780-257785,257787-257793, and
257795,257817,257819,257937-257938,258029,258263-258267:
257755: SRV records
257756: fix spurious error message
257780: Whitespace
257781: Comments and funny syntax
257782: Debug file truncation is optional
257783: f_show_err for debugging
257784: f_eval_catch for debugging
257785: fix size calculations bug
257787: fix broken HTTP "any" media type
257788: more debugging
257789: Comments
257790: fix printf usage bug
257791: f_[v]sprintf added
257792: Comments
257793: fix off-by-one error in size calcs
257795: Replace pkg-tools with pkgng
257817: fix cosmetic typos
257819: Use `pkg -vv' to get ABI
257937: Adjustment to last
257938: Adjustment to last
258029: Comments
258263: Shuffle code around
258264: Remove unused code
258265: Debugging. Use f_eval_catch with pkg
258266: Shutdown media on exit from packages
258267: Fix pkg install from DVD

Reviewed by: many
Discussed on: -current
Approved by: re (hrs)

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Introduce centralized [X]dialog(1) exit codes to `dialog.subr' and make-use
throughout the bsdconfig(8) code. While we're here, add an explicit argument
to lvalue-seeking invocations of "return" that previously had no argument.
Also, consolidate a single instance of double-newline and remove some
comments that are no longer required (given increased readability with new
exit codes).

Approved by: re (glebius)

Check for INDEX file first before anything else when processing modules.

Check menuitem before calculating tag.

Add support for processing add-on modules from /usr/local/libexec/bsdconfig
(this is designed to allow new modules to be installed via ports/packages).

To prevent conflict with itself (sysutils/bsdconfig) as a port (which
installs its base modules to the above directory, it was long-ago decided
that so-called `base' modules would look different than now-defined `add-on'
modules. The structure of the contents for each is the same, but the naming
convention for the module directory must be different.

Base modules are named `[0-9][0-9][0-9].*' to allow SysV-style organization
while add-on modules must avoid this naming style and are simply listed in
alphabetical order by their module directory.

For example, a hypothetical port named `bsdconfig-jails' could install
/usr/local/libexec/bsdconfig/jails and provide `bsdconfig jails' as well as
a new menu entry in the main-menu.

Add-on modules are listed in the main-menu (when bsdconfig is executed with-
out arguments) below a separator after the last base-module.

In `bsdconfig -h' output, add-on modules are listed right alongside base
modules (sorted alphabetically in columnar fashion; left-to-right).

If a base module declares a keyword used by an add-on module, the base
module will always win when given `bsdconfig keyword' syntax.

Add-on modules should avoid declaring any keyword found in `script.subr' as
a reserved-word (`Resword') since bsdconfig also supports `bsdconfig resword'
as a fall-back if no keyword is found to be declared by any module.

Don't calculate the tag until we know that we're going to make a new menu
item entry. Also join simple NULL assignments into a single line.

More whitespace.

Whitespace.

When the fall-back of a case-statement is the last thing executed in a
while-loop _and_ all prior matches in the same case-statement either break
or continue, we can safely break the fall-back out of the case-statement.
This should improve readability and allow for longer-lines by reducing the
level of indentation by-one for the fall-back case.

Whitespace and comments.

Style/comments.

Use f_shell_escape() instead of forking to awk. In this case, the
replacement comes with a great performance increase (as f_shell_escape()
uses the built-in based f_replaceall() which out-performs forking to
awk(1)). This should also improve readability slightly.

Make some obvious consolidations in local declaration for readability.

Improvements to whitespace and comments.

Fix option processing from the library layer to address unexpected
behavior(s); e.g., `-Xd' versus `-dX' did not produce the same results.

The libraries common.subr and dialog.subr automatically process the
arguments passed to the program and enable/disable functionality without the
need to process the arguments within your program. For example, if "$@"
contains `-d', common.subr will see this and enable debugging regardless of
whether you process "$@" yourself or not (this automatic processing can
easily be disabled for custom scripts that don't want it; see the afore-
mentioned scripts for additional details).

NOTE: common.subr stores a copy of "$@" in $ARGV for convenient (and
repeated) processing by libraries such as dialog.subr which provide such
transparent functionality for the consuming script(s).

However, the libraries don't know if a program wants to accept `extra'
options. Flags are not really a problem, because the library can be
programmed to silently ignore unknown flags. The trouble comes into play
when the program wants to define an option that takes an argument.

For example:

bsdconfig -D logfile -X

In the above example, the library uses getopts to process $ARGV and if it
doesn't know that `-D' takes an argument, the option processing will
prematurely terminate on `logfile' (this is standard/correct behavior for
getopts but is undesired in our situation where we have partially off-loaded
main argument processing).

The problem is solved by allowing the program to define an extra set of
options to be included in each library's handling of $ARGV. Only options
that require arguments are truly necessary to be pre-specified in this new
manner.

Standardize the way functions build their arguments leading up to a dialog
invocation. Specifically, "top-load" your arguments and in the order in-
which they will be displayed. For example, many [if not all] widgets display
information in the following order, top-to-bottom (visually):

+ backtitle (displayed behind the widget at top-left)
+ title (at the top of the `window')
+ prompt text (just below the title and above whatever widget you choose)
+ Depending on widget, _one_ of the following:
- menu list
- radio list
- check list
- text input box with initial text
- [Xdialog(1)] 2x or 3x text input boxes
- [dialog(1)] a multi-part form
- progress bar
- etc. (many more widget choices)
+ buttons (right below the selected widget)
+ [dialog(1)] the hline (displayed at bottom of `window')

NOTE: Xdialog(1) accepts and silently ignores --hline

When building local arguments for your dialog invocation, if the value can't
be cleanly loaded into a local, add "# Calculated below" to the end of the
local declaration while retaining the block order of argument declarations.

Move other local declarations that are not associated with this top-loading
the dialog arguments to right-above where they are first-used.

Also, standardize on the names of the arguments. For example, always use
$prompt (instead of sometimes $msg and sometimes $prompt); use $menu_list
or $shell_list or $radio_list for those respective widgets; ad nauseum.

While we're doing this, flush-out full arguments for many invocations that
were passing NULL strings (making it unapparent if you were staring at this
one invocation what argument that NULL string was supposed to represent).

Last, while we're in startup/rcconf let's remove the unnecessary use of a
GLOBAL (RCCONF_MENU_LIST) for the menu_list.

Improve the dialog(1) API in dialog.subr by adding f_dialog_default_store()
and f_dialog_default_fetch(). Operating similar to functions introduced by
SVN r251236 and r251242, these functions operate as a pair for helping track
the default-item data (for the --menu, --checklist, and --radiolist
widgets).

This replaces the direct usage of a global to store the data with an
abstract method for readability and to centralize the code.

Improve portion of the dialog(1) API in dialog.subr responsible for
retrieving stored data (for the --menu, --calendar, --timebox, --checklist,
and --radiolist widgets).

When we (Ron McDowell and I) developed the first version of bsdconfig, it
used temporary files to store responses from dialog(1). That hasn't been
true for some very long time, so the need to always store the return status
of dialog(1) and then call some function to clean-up is long-deprecated. The
function that used to do the clean-up was f_dialog_menutag().

We really don't need f_dialog_menutag() for its originally designed purpose,
as all dialog invocations (even when in a sub-shell) do not use temporary
files anymore.

However, we do need to keep f_dialog_menutag() around because it still fills
the need of being able to abstract the procedure for fetching stored data
provided by functions that display the aforementioned widgets.

In re-designing f_dialog_menutag(), four important changes are made:

1. Rename f_dialog_menutag() to f_dialog_menutag_fetch()
2. Introduce the new first-argument of $var_to_set to reduce number of forks
3. Create a corresponding f_dialog_menutag_store() to abstract the storage
4. Offload the sanitization to a new function, f_dialog_data_sanitize()

NOTE: That last one is important. Not all functions need to store their data
for later fetching, meanwhile every invocation of dialog should be sanitized
(as we learned early-on in the i18n-effort -- underlying libraries will spit
warnings to stderr for bad values of $LANG and since dialog outputs its
responses to stderr, we need to sanitize every response of these warnings).

These changes greatly improve readbaility and also improve performance by
reducing unnecessary forking.

Improve portion of the dialog(1) API in dialog.subr responsible for
calculating widget sizes. Instead of forking a sub-shell to calculate the
optimum size for a widget, use a byRef style call-out to set variables in
the parent namespace. For example, instead of:

size=$( f_dialog_buttonbox_size title btitle msg )
$DIALOG --title title --backtitle btitle --msgbox msg $size

The new API replaces the above with the following:

f_dialog_buttonbox_size height width title btitle msg
$DIALOG --title title --backtitle btitle --msgbox msg $height $width

This reduces the number of forks, improves performance, and makes the code
more readable by revealing the argument-order for widget sizing. It also
makes performing minor adjustments to the calculated values easier as
you no longer have to split-out the response (which required knowledge of
ordering so was counter-intuitive).

Centralize standard getopts arguments, both for convenience and to correct
a bug in which certain combinations of arguments produced unexpected results
such as `-dX' (now properly produces debugging and X11), `-XS' (now properly
produces X11 in secure mode), `-df-' (enables debugging when reading a
script from standard-input, etc. Multi-word variations such as `-d -X',
`-X -S', `-d -f-', `-d -f -', etc. also work as expected. Also tested were
variations in argument order, which are now working as expected.

UI improvements. First, implement --default-item whenever and wherever
possible to save keystrokes. Second, overhaul startup/rcdelete for much
improved performance. Last, but not least, kill-off useage of --clear
and implement --keep-tite in harmony to minimize jarring transitions.
Also, fix local variable names where necessary while we're here with
other minor comment-enhancements/typo-corrections.

Use new f_getvar for clarity and general code follow-up to r249746.

Add new flags `-d' (sets debug=1) and `-D file' (sets debugFile) and
improve debugging initialization. Also fixup USAGE statements while we're
here. Also, change initialization of main program to _not_ change working
directory, allowing the debugFile to be relative without confusion.

Add missing dialog(1) arguments to internationalize "Help"-button label.

Update copyrights and dates following last commit.

Add support for scripting (sysinstall style).

Reviewed by: jilles

Add more debugging to help with diagnosis of program-flow when needed.

Remove unnecessary duplicate initialization of the dialog(1) API (automatically
bootstrapped on-include unless DIALOG_SELF_INITIALIZE is set to NO before-hand)

Improve the debugging abilities and clean up debug messages. In most cases,
all one has to do is set the environment variable DEBUGGING to get the debug
messages to appear on the console.

Replicate a feature from sysinstall documented in
stable/9/usr.sbin/sysinstall/help/shortcuts.hlp (reproduced below):

If /usr/sbin/sysinstall is linked to another filename, say
`/usr/local/bin/configPackages', then the basename will be used
as an implicit command name.

To get a list of modules you can link to, see `bsdconfig -h' output.

Approved by: adrian (co-mentor) (implicit)

Switch from using a msgbox to display help files to a textbox instead. The
problem with using a msgbox was one of truncation in the case of Xdialog(1)
and lack of screen real-estate (since the msgbox is not scrollable in X11
while a textbox is).

The textbox renders the text much better and is more appropriate for this
type of data display.

Approved by: adrian (co-mentor) (implicit)

When Xdialog(1) is passed a NULL argument to its `--help' option, Xdialog(1)
acts like dialog(1) in that it returns exit status 2 when the help button is
chosen.

Approved by: adri (co-mentor) (implicit)

Resurrect usage.hlp and securelevel.hlp from sysinstall(8) and integrate them
into bsdconfig(8).

Approved by: adrian (co-mentor) (implicit)

More-accurately replicate the help system from sysinstall(8). However, also
improve upon the system by giving the user a "Help" button instead of requiring
the user to press F1.

NOTE: In FreeBSD-9 and higher, dialog(1) does not support the F1 hook, so the
mechanism for providing help to the user had to be changed to a button.

This now means we can resurrect *.hlp files from usr.sbin/sysinstall/help/ as-
is and reusing them as-needed in bsdconfig (holding to the goal of losing as
little functionality from sysinstall as possible).

Reviewed by: adrian (co-mentor)
Approved by: adrian (co-mentor)

Reverse SVN r240770 -- jilles@ made a suggestion that allowed us to redesign
our embedded rescue environment to support /dev/null making r240770 obsolete.

Reviewed by: jilles, adrian (co-mentor)
Approved by: jilles, adrian (co-mentor)

Replace redirections to /dev/null with "close file-descriptor" syntax (>&-).

Reviewed by: adrian (co-mentor)
Approved by: adrian (co-mentor)

Change all invocations of dialog(1) to no-longer require temporary files.
This allows bsdconfig to -- like bsdinstall -- operate from read-only media.

Reviewed by: adrian (co-mentor)
Approved by: adrian (co-mentor)

Move major includes into /usr/share/bsdconfig for easy external access.

Reviewed by: adrian (co-mentor)
Approved by: adrian (co-mentor)

Import bsdconfig(8) as a replacement for the post-install abilities of
deprecated sysinstall(8). NOTE: WITH_BSDCONFIG is currently required.

Submitted by: Devin Teske (dteske), Ron McDowell <rcm@fuzzwad.org>
Reviewed by: Ron McDowell <rcm@fuzzwad.org>
Approved by: Ed Maste (emaste)