MFC r316677: Do not register in CTL portal groups without portals.

From config synthax point of view such portal groups are not incorrect,
but they are useless since can not receive any connection. And since
CTL port resource is very limited, it is good to save it.

MFC r316050,r316051:

r316050:

ctld: sort #includes per style(9)

- Only include sys/types.h or sys/param.h, not both.
- Sort alphabetically.

r316051:

ctldadm: sort #includes per style(9)

- Only include sys/types.h or sys/param.h, not both.
- Sort alphabetically.

MFC r316049:

Allow WARNS to be overridden by the end-user

MFC r310633: Add MAX_LUNS overflow safety checks.

While this MAX_LUNS limitation is too synthetic and should be removed,
it is better to enforce it while it is here.

MFC r309241,309243:

Plug memory leaks.

MFC r308133, r308134: Fix wrong copy/paste in error message.

MFC r299894

pdu_delete(request) frees request, so move the call after
login_new_response(request) to avoid a use-after-free error

Reported by: Coverity
CID: 1331219, 1331220

MFC r295562: Don't check CmdSN for initial login request.

This fixes connection errors for some initiators not starting CmdSN
from zero.

While there, fix wrong status details reported for couple errors.

MFC r289677:

Fix a ton of speelling errors

MFC r290615: Introduce portal group options in ctl.conf.

While CTL has concept of port options, used at least for iSCSI ports now,
before this change it was impossible to set them manually. There still
no user-configurable port options now, but I am planning to change that.

MFC r288486, r288488: Set default block size for CD to expected 2048 bytes.

MFC r288310: Add to CTL initial support for CDROMs and removable devices.

Relnotes: yes

MFC r288259: Remove some duplicate, legacy, dead and questionable code.

MFC r288061: Make cltd ignore HA ports.

MFC r287823: Add ctl-lun config option for consistency in HA setups.

MFC r287766: Add negotiation of iSCSIProtocolLevel to 2 (RFC7144).

We may need to pass negotiated value to kernel level, but so far it is
not necessary, since it does not use any new features without request.

MFC r287757: When updating port, apply only change of LUN map, not whole.

MFC r287534: Add two new portal group options "tag" and "foreign".

They are going to be useful in clustered setups.

MFC r287500: Allow LUN options modification via CTL_LUNREQ_MODIFY.

Not all changes take effect, but that is a different question.

MFC r287455, r287458, r287459: Remove some dead code.

MFC r286806: Drop "internal" CTL frontend.

Its idea was to be a simple initiator and execute several commands from
kernel level, but FreeBSD never had consumer for that functionality,
while its implementation polluted many unrelated places.

MFC r281532 (by delphij): Eliminate unused headers.

MFC r279314 (by trasz): Add missing error check.

MFC r278622 (by trasz): Remove unused code.

MFC 288208

Fix typo.

Sponsored by: iXsystems

MFC r286462: Refactor early stages of security negotiation.

MFC r286456: Remove some code duplication.

MFC r282086:

Make setproctitle(3) work in Capsicum capability mode. This makes
ctld(8) child processes to indicate initiator address and name in
their titles, similar to what iscsid(8) child processes do.

PR: 181352
Sponsored by: The FreeBSD Foundation

MFC r284542:

Allow '@' in unquoted strings, such as with the "path" statement. Note
that one can use any character they like by using double quotes.

PR: 200895
Sponsored by: The FreeBDS Foundation

MFC r285086:

Remove OpenSSL dependency from iscsid(8) and ctld(8).

Sponsored by: The FreeBSD Foundation

MFC r284765, r284875:
Teach ctld about CTL's physical_port and virtual_port fields.

This allows ctld to work with isp(4) virtual ports, specifying them as
isp0/1, isp0/2, etc. There are still problems on isp(4) layer with
disabling those ports after enabling, but hopefully they can be fixed.

MFC r283285:

Advertise ctlstat(8) a little better.

Sponsored by: The FreeBSD Foundation

MFC r281820:

Add hint about "volmode=dev" to ctl.conf(5).

Sponsored by: The FreeBSD Foundation

MFC r281163: Make ctld to not exit on ECONNABORTED on accept().

That is not really an error for the main process.

MFC 281084

Fix thinko/copypaste error.

When checking the length of the mutual secret password the variable for
the secret password was used by mistake. This resulted in ctld never
warning about the length of the mutual secret being wrong even if it was.

Sponsored by: iXsystems

Merge an applicable subset of r263234 from HEAD to stable/10:

Update most userspace consumers of capability.h to use capsicum.h instead.

auditdistd is not updated as I will make the change upstream and then do a
vendor import sometime in the next week or two.

Note that a significant fraction does not apply, as FreeBSD 10 doesn't
contain a Capsicumised ping, casperd, libcasper, etc. When these features
are merged, the capsicum.h change will need to be merged with them.

Sponsored by: Google, Inc.

MFC r279591: Add example configuration for FibreChannel ports.

MFC r279590: If target name starts with "naa.", set it as WWNN for CTL port.

MFC r279589: Fix handling of queued text and logout requests.

While it may have little sense, text and logout requests can be queued.
If they are, they consume cmdsn, so we should increment our conn_cmdsn.

MFC r279276: Add checks for malloc() failures.

Submitted by: Dmitry Luhtionov <dmitryluhtionov@gmail.com>

MFC r279277: Fix memory leak on incorrect initiator portal.

Submitted by: Dmitry Luhtionov <dmitryluhtionov@gmail.com>

MFC r278594: Add missing asprintf() status checks.

MFC r278354: Teach ctld(8) to control non-iSCSI CTL ports.

This change introduces new target option "port", that assigns current target
to specified CTL port. On config application ctld(8) will apply LUN mapping
according to target configuration to specified port and bring the port up.
On shutdown cltd(8) will remove the mapping and put the port down.

This change allows to configure both iSCSI and FibreChannel targets in the
same configuration file in alike way.

Kernel side support was added earlier at r278037.

Relnotes: yes
Sponsored by: iXsystems, Inc.

MFC r278322: Add support for multiple portal groups per target.

This change allows multiple "portal-group" options to be specified per
target. Each of them may include new optional auth-group name parameter
to override per-target auth parameters for specific portal group.

Kernel side support was added earlier at r278161.

Sponsored by: iXsystems, Inc.

MFC r278161: Bring some more order into iSCSI portal group tags support.

While ctld(8) still does not allow multiple portal groups per target
to be configured, kernel should now be able to handle it.

Sponsored by: iXsystems, Inc.

MFC r278037: CTL LUN mapping rewrite.

Replace iSCSI-specific LUN mapping mechanism with new one, working for any
ports. By default all ports are created without LUN mapping, exposing all
CTL LUNs as before. But, if needed, LUN mapping can be manually set on
per-port basis via ctladm. For its iSCSI ports ctld does it via ioctl(2).
The next step will be to teach ctld to work with FibreChannel ports also.

Respecting additional flexibility of the new mechanism, ctl.conf now allows
alternative syntax for LUN definition. LUNs can now be defined in global
context, and then referenced from targets by unique name, as needed. It
allows same LUN to be exposed several times via multiple targets.

While there, increase limit for LUNs per target in ctld from 256 to 1024.
Some initiators do not support LUNs above 255, but that is not our problem.

Relnotes: yes
Sponsored by: iXsystems, Inc.

MFC r274853:
For both iSCSI initiator and target increase socket buffer sizes before
establishing connection.

This is a workaround for Chelsio TOE driver, that does not update socket
buffer size in hardware after connection established, and unless that is
done beforehand, kernel code will stuck, attempting to send/receive full
PDU at once.

MFC r274804:
In conf_apply() remove iSCSI ports from kernel before removing LUNs.

Previous order confused initiators with messages about "removed" LUNs
during simple ctld restart without any real config change. After this
commit initiators only reestablish lost connection, receive "Power on
occurred" UNIT ATTENTION status and continue normal operation.

MFC r274791:

Add missing error checking for kernel_port_{add,remove}(). Both can fail
for reasons yet unknown; don't make it increment cumulated_error as a kind
of temporary workaround.

MFC r275399:

Fix null pointer dereference.

Sponsored by: The FreeBSD Foundation

MFC r275864: Make sequence numbers checks more strict.

While we don't support MCS, hole in received sequence numbers may mean
only PDU loss. While we don't support lost PDU recovery, terminate the
connection to avoid stuck commands.

While there, improve handling of sequence numbers wrap after 2^32 PDUs.

MFC r274954:

Remove the pidfile clause from the ctl.conf example. Why do everyone
just copy/paste this line into their configs, it's beyond me.

Sponsored by: The FreeBSD Foundation

MFC r275925: Slightly polish iSCSI parameters negotiation.

MFC r275452: Do not corrupt the listen string when parsing it.

This fixes problem with ctld reload when it is configured to listen on two
portals with same IP, but different ports.

MFC r274310:

Add HISTORY section to ctld(8).

Sponsored by: The FreeBSD Foundation

MFC r274308:

Add support for sending redirections to iSCSI target.

MFC r274309:

Fix several nits in redirection handling - don't use wrong CSG,
and avoid use-after-free.

Sponsored by: The FreeBSD Foundation

MFC r274277:

ctld(8) doesn't require -lcam or -lssl; remove those from DPADD and LDADD.

Sponsored by: The FreeBSD Foundation

MFC r274248: Remove unused assignments, noticed by Clang analyzer.

MFC r274278:

Improve wording in ctl.conf(5).

Sponsored by: The FreeBSD Foundation

MFC r273822:

Fix iscsictl(8) and ctld(8) to correctly handle Windows newlines
(CRLF) in iscsi.conf and ctl.conf.

Sponsored by: The FreeBSD Foundation

MFC r273821:

Keep the token list sorted.

Sponsored by: The FreeBSD Foundation

MFC r273820:

Make it possible to optionally use semicolon to separate statements.
This makes it possible to format stuff like this:

target xxx {
lun 0 { path /foo/bar; size 4G; }
}

Sponsored by: The FreeBSD Foundation

MFC r273816:

Simplify code; no functional changes.

Sponsored by: The FreeBSD Foundation

MFC r273813:

Add discovery-filter. This makes it possible to restrict which targets
are returned during discovery based on initiator portal, name, and CHAP
credentials.

MFC r273770:

Fix build after previous commit. While here, improve error messages.

Sponsored by: The FreeBSD Foundation

MFC r273768:

Remove the distinction between strings and numbers from ctld(8) yacc parser.
This fixes problems with passing strings that look like numbers to clauses
that expect strings; previously it caused syntax errors and had to be worked
by user, using quotes. The workaround introduced in r267833 is no longer
neccessary.

Sponsored by: The FreeBSD Foundation

MFC r273585:

Improve ctld.conf example.

Sponsored by: The FreeBSD Foundation

MFC r273470:

Fix ctl.conf example to use proper paths to ZVOLs.

Sponsored by: The FreeBSD Foundation

MFC r273467:

Comment out parts about iSER; it's not implemented.

Sponsored by: The FreeBSD Foundation

MFC r273584:

Make the initiator-name and initiator-portal checks a little nicer.

Sponsored by: The FreeBSD Foundation

MFC 273583:

Tidy up the login code; no functional changes.

Sponsored by: The FreeBSD Foundation

MFC r273635, r273793, r274797: Add basic iSNS client to the iSCSI target.

This makes ctld(8) register its iSCSI targets and portals on configured
iSNS servers to allow initiators find them without active discovery.

Fetching of allowed initiators from iSNS is not implemented now, so target
ACLs still should be configured manually.

Relnotes: Yes
Sponsored by: iXsystems, Inc.

MFC r274328:
Make both iSCSI initiator and target support base64 encoded CHAP data.

While all tested initiators and targets use hex-encoded CHAP data, RFC also
allows base64 encoding there, and Microsoft certificaition tool uses it.

MFC r273543:

Clean up (refactor) discovery a little; no functional changes.

Sponsored by: The FreeBSD Foundation

MFC r273468:

Remove misleading statement. Bump date.

Sponsored by: The FreeBSD Foundation

MFC r273466:

Remove spurious empty line.

Sponsored by: The FreeBSD Foundation

MFC r273465:

Fix ctld(8) to not show the "auth-group <name> not assigned to any target"
warning for auth-groups assigned to a portal-group.

Sponsored by: The FreeBSD Foundation

MFC r273464:

Whitespace fixes.

Sponsored by: The FreeBSD Foundation

MFC r271320:

Use keys_add_int() where appropriate. No functional changes.

Sponsored by: The FreeBSD Foundation

MFC r273459:

Untangle iSCSI authentication code by splitting off the CHAP
implementation.

Sponsored by: The FreeBSD Foundation

MFC r272911:
Make ctld start even if some LUNs are unable to open backing storage.

Such LUNs will be visible to initiators, but return "not ready" status
on media access commands. If backing storage become available later,
`ctladm modify ...` or `service ctld reload` can trigger its reopen.

MFC r271797: Make kernel to update LUN size from the backing storage
on configuration reload also if that size was not specified in the new
configuration.

Previously it happened only if size was explicitly changed in config.

Approved by: re (delphij)

MFC r271571:

Fix two small nits in ctl.conf(5).

Approved by: re (glebius)
Sponsored by: The FreeBSD Foundation

MFC r271445,r271446,r271560:
Improve markup and language throughout the ctl.conf man page

MFC r271543:
Add the new iscsi(4) man page
Cross reference it from iscsid(8) and iscsictl(8)

Approved by: re (gjb), bcr (mentor)

MFC r271319:

Fix ctld(8) to not forget to send TargetPortalGroupTag and TargetAlias
when the initiator skips security negotiation. This fixes interoperability
with Xtend SAN initiator.

PR: 193021
Approved by: re (marius)
Sponsored by: The FreeBSD Foundation

MFC r271187:

Avoid ctld(8) crash on getaddrinfo(3) failure.

Approved by: re (gjb)
Sponsored by: The FreeBSD Foundation

MFC r271169:

Turn two errors, which are possible to trigger only by bugs,
into assertions.

Approved by: re (gjb)
Sponsored by: The FreeBSD Foundation

MFC r271157:

Fix typo.

Approved by: re (marius)
Sponsored by: The FreeBSD Foundation

MFC r270179:

Add missing libraries to DPADD; sort DPADD so DPADD and LDADD match up

This fixes "make checkdpadd"

Approved by: jmmv (mentor)
Phabric: D630
PR: 192765

MFC r270279:

Make the iSCSI stack use __FBSDID() properly.

Sponsored by: The FreeBSD Foundation

MFC r269183, r269191:
Add netmasks support to initiator-portal option.

MFC 268931:

Correct spelling errors in ctld.8. While here, correct similar errors
in ctl.conf.5 and fix a couple of contractions.

MFC r268795:
Fix ctld crash on startup if target alias is not set.

MFC r268328:
Close race in r268291 between port destruction, delayed by sessions
teardown, and new port creation during `service ctld restart`.

Close it by returning iSCSI port internal state, that allows to identify
dying ports, which should not be counted as existing, from really alive.

MFC r268302:
Pass through iSCSI session ISID from LOGIN request to the CTL frontend.

ISID is an important part of initiator transport ID for iSCSI. It is not
used now, but should be to properly implement persistent reservation.

MFC r268293:
Burry devid port method, which was a gross hack.

Instead make ports provide wanted port and target IDs, and LUNs provide
wanted LUN IDs. After that core Device ID VPD code only had to link all
of them together and add relative port and port group numbers.

LUN ID for iSCSI LUNs no longer created by CTL, but by ctld, and passed
to CTL as "scsiname" LUN option. This makes LUNs to report the same set
of IDs, independently from the port through which it is accessed, as
required by SCSI specifications.

MFC r268291:
Create separate CTL port for every iSCSI target (and maybe portal group).

Having single port for all iSCSI connections makes problematic implementing
some more advanced SCSI functionality in CTL, that require proper ports
enumeration and identification.

This change extends CTL iSCSI API, making ctld daemon to control list of
iSCSI ports in CTL. When new target is defined in config fine, ctld will
create respective port in CTL. When target is removed -- port will be
also removed after all active commands through that port properly aborted.
This change require ctld to be rebuilt to match the kernel.

As a minor side effect, this allows to have iSCSI targets without LUNs.
While that may look odd and not very useful, that is not incorrect.

MFC r267648:
serial_num and device_id fields are not necessarily null-terminated.

Before this it was impossible to use all 16 bytes of serial number, and
client always got serial number NULL-terminated, that is not required.

MFC r267606:
On discovery stage add set of TargetAddress keys to reply, reporting to
the client all the portal groups addresses and ports.

MFC: 267833

Fix issues in config parser relating to lun serial numbers.

Without this fix some serial numbers needed to be quoted
to avoid the config parser bailing out.

Submitted by: delphij
Sponsored by: iXsystems

MFC r264877:

Style fixes.

Sponsored by: The FreeBSD Foundation

MFC r264535:

English.

Sponsored by: The FreeBSD Foundation

MFC r264534:

If we fail to create LUN, try again on next configuration reload.

Sponsored by: The FreeBSD Foundation

MFC r264533:

Use consistent punctuation.

Sponsored by: The FreeBSD Foundation

MFC r264532:

Use proper terminology in debug messages.

Sponsored by: The FreeBSD Foundation

MFC r264531:

Constify.

Sponsored by: The FreeBSD Foundation

MFC r264530:

Properly pass the initiator address when running in proxy mode.

Sponsored by: The FreeBSD Foundation

MFC r264529:

Use socket address from accept(2) instead of retrieving it via
getpeername(2).

Sponsored by: The FreeBSD Foundation

MFC r264528:

Rework the way we enable CTL iSCSI port. Previously conf_apply()
needed it to be already enabled, because listening in proxy mode
requires it; however, it's conf_apply() that opens pidfiles,
so it resulted in port being enabled before pidfile was opened.
This was not so bad, but it was also disabled when pidfile couldn't
be opened due to ctld already running; this means that starting
second ctld instance screwed up the first.

Sponsored by: The FreeBSD Foundation

MFC r264526:

Properly identify target portal when running in proxy mode. While here,
remove CTL_ISCSI_CLOSE, it wasn't used or implemented anyway.

Sponsored by: The FreeBSD Foundation

MFC r264524:

Make it possible for the iSCSI target side to operate in both normal
and ICL_KERNEL_PROXY mode, and fix some bit rot so the latter actually
works again.

Sponsored by: The FreeBSD Foundation

MFC r264500:

Stop treating LUN 0 as mandatory. There is no reason to do that.

Sponsored by: The FreeBSD Foundation

MFC r263356:

Make the error message more clear.

Sponsored by: The FreeBSD Foundation

MFC r261752:

Empty data segment during Login Phase is rather unlikely, but it's
not, by itself, a protocol error. This fixes interoperability with
OpenBSD initiator.

Sponsored by: The FreeBSD Foundation

MFC r261751:

The log_whatever() routines don't accept NULL for format strings,
so mark them as __printflike instead of__printf0like.

Sponsored by: The FreeBSD Foundation

MFC r261764:

So, it turns out SIGCHLD is discarded by default, so we have to set up
a dummy handler to make it interrupt an ioctl(2) or select(2).

This makes those short-lived ctld(8) zombies disappear.

Sponsored by: The FreeBSD Foundation

MFC r261763:

Use new auth-type "deny" instead of using "chap" with no chap entries;
it's cleaner this way, and gives better feedback to the user.

Sponsored by: The FreeBSD Foundation

MFC r261762:

Use "default" as default discovery-auth-group, instead of "no-access".
It doesn't change visible behaviour, as previously auth-group "default"
wasn't redefinable, so by default access was always denied.

Sponsored by: The FreeBSD Foundation

MFC r261761:

Update manual page to match recent auth-group "default" changes.

Sponsored by: The FreeBSD Foundation

MFC r261760:

Add a new auth-group "default", defaulting to deny, and make it possible
to redefine it. From now on, assigning auth-group to a target is no longer
mandatory.

Sponsored by: The FreeBSD Foundation

MFC r261759:

Make it possible to redefine portal-group "default".

Sponsored by: The FreeBSD Foundation

MFC r261758:

Add auth-type.

Sponsored by: The FreeBSD Foundation

MFC r261757:

Rename a variable, no functional changes.

Sponsored by: The FreeBSD Foundation

MFC r261756:

Yacc cleanup; no functional changes.

Sponsored by: The FreeBSD Foundation

MFC r261755:

Make function ordering slightly more logical; no functional changes.

Sponsored by: The FreeBSD Foundation

MFC r261754:

Implement initiator-name and initiator-portal restrictions.

Sponsored by: The FreeBSD Foundation

MFC r261753:

Daemonize after applying configuration, not before, so that
any problems - including "daemon already running" - are visible
on the terminal and not just in logs.

Sponsored by: The FreeBSD Foundation

MFC r261750:

Improve check for duplicated paths. It shows the warning twice for every
path (once for each duplicate found), but it should do for now.

Sponsored by: The FreeBSD Foundation

MFC r261749:

Refactor.

Sponsored by: The FreeBSD Foundation

MFC r261768:

Make the debug messages during ctld(8) shutdown less confusing.

Sponsored by: The FreeBSD Foundation

MFC r261767:

Show proper configuration file line numbers in error messages.

Sponsored by: The FreeBSD Foundation

MFC r261766:

Make ctld error out on invalid characters in ctl.conf.

Sponsored by: The FreeBSD Foundation

MFC r261765:

Improve error reporting.

Sponsored by: The FreeBSD Foundation

MFC r259182:

Fix handling for empty auth-groups. Without it, ctld child process
would either exit on assertion, or, if assertions are not enabled,
fail to authenticate the target.

Sponsored by: The FreeBSD Foundation

Hoist all the mergeinfo up to the root in preparation for enforcing merges
to the root only. All MFC's were rerecorded to the root.

Going forward, if an MFC includes mergeinfo, it will need to be made to
the root and committed from the root. Merges with --ignore-ancestry
or diff | patch can go anywhere.

The mergeinfo in HEAD is in a bad state from years of neglect and manual
tampering and this was branched into 10.x. This confuses the coalescing
code and prevents it from doing its job.

Approved by: re (gjb, implicit)

MFC r258842:

Grammar fix.

Approved by: re (gjb)
Sponsored by: The FreeBSD Foundation

MFC r258841:

Fix typos.

Approved by: re (gjb)
Sponsored by: The FreeBSD Foundation

Fix typo.

Approved by: re (gjb)
Sponsored by: The FreeBSD Foundation

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Plug memory leaks.

Coverity CID: 1087993, 1087992
Approved by: re (marius)
Sponsored by: FreeBSD Foundation

Fix off-by-one.

Coverity CID: 1011375
Approved by: re (glebius)
Sponsored by: FreeBSD Foundation

Fix several problems in the new iSCSI stack; this includes interoperability
fix for LIO (Linux target), removing possibility for the target to avoid mutual
CHAP by choosing to skip authentication altogether, and fixing truncated error
messages in iscsictl(8) output. This also fixes several of the problems found
with Coverity.

Note that this change requires world rebuild.

Coverity CID: 1088038, 1087998, 1087990, 1088004, 1088044, 1088041, 1088040
Approved by: re (blanket)
Sponsored by: FreeBSD Foundation

Make iscsictl(8) automatically try to load the iscsi module. While here,
improve module loading in iscsid(8) and ctld(8).

Approved by: re (delphij)

Minor mdoc fixes.

Approved by: re (blanket)

Bring in the new iSCSI target and initiator.

Reviewed by: ken (parts)
Approved by: re (delphij)
Sponsored by: FreeBSD Foundation