Copy stable/10@r272459 to releng/10.1 as part of
the 10.1-RELEASE process.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC: r267131, r267132, r267133, r268493, r268671

Use NULL instead of 0 (Patch by Sascha Wildner <saw at online.de> for Dragonfly)
Remove unnecessary semicolons (Patch by Sascha Wildner <saw at online.de> for Dragonfly)
Add support for arbitrary http requests [1]
Support EAGAIN in fetch_writev

Submitted by: Alex Hornung <alex at alexhornung.com> [1]
Reviewed by: des

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

latin1 -> utf8

Update copyright dates and strip my middle name.

Add support for HTTP 1.1 If-Modified-Since behavior.

fetch(1) accepts a new argument -i <file> that if specified will cause
the file to be downloaded only if it is more recent than the mtime of
<file>.

libfetch(3) accepts the mtime in the url structure and a flag to
indicate when this behavior is desired.

PR: bin/87841
Submitted by: Jukka A. Ukkonen <jau@iki.fi> (partially)
Reviewed by: des, ru
MFC after: 3 weeks

Update copyright years.

Surround prototypes with __BEGIN_DECLS / __END_DECLS.

PR: misc/40399
MFC after: 3 days

Reindent, and add parentheses to return statements. Some functions in
ftp.c and http.c now have exceedingly long lines due to deep nesting;
this will be corrected by reorganizing the code in a later revision.

Introduce a fetchDebug global. Change the DEBUG macro so it only runs the
debugging code if fetchDebug is set.

PR: bin/32615
MFC after: 1 week

Add rudimentary support for an authentication callback function.

Apply 'const' liberally.

Fix some other minor glitches.

Add SCHEME_HTTPS.

Add macros for the names of the URL schemes we support.

Introduce fetchXGet*(), which combine the functionalities of fetchGet*() and
fetchStat*(). In most cases, either fetchGet*() or fetchXGet*() is a wrapper
around the other; in all cases, calling fetchGet*() is identical to calling
fetchXGet*() with the second argument set to NULL.

Don't forget to declare fetchRestartCalls.

Remove fetchContentType(), which disappeared in the HTTP rewrite.

Add (and document) fetchMakeURL()

Dump com_err, it's a stinking crock of shit.

Reorganize some of the http code and split it into more functions.
Implement fetchStatHTTP().
Unbungle struct url, and add fetchFreeURL().
Document it.

While we're at it, add a length field too in case we want to fully implement
partial fetches later.

Add offset field to struct url

More old uncommitted patches: implement timeouts at the protocol level.
Currently only supported for ftp connections.

$Id$ -> $FreeBSD$

Implement and document file list retrieval.

Add verbose flag, and support functions.
Brucify the Makefile.
Differentiate atime and mtime in fetch*Stat().
Fix a few pointer bugs.
Tweak some error messages.
Don't #include sys/param.h and stdio.h in fetch.h.
Document that sys/param.h and stdio.h must be #included before fetch.h.

Second of a series of cleanups to libfetch.

This commit introduces the following features:

a) the fetchStat*() functions, which return meta-information for a
document, such as size, modification time, etc.

b) the use of the com_err(3) facilities to report errors.

It also fixes a bunch of style bugs and a few logic bugs and somewhat
improves the man page.

Changed files, in alphabetical order:

Makefile:
Don't generate macros in {ftp,http}err.c.

Generate category fields for the error message lists.

Compile the error table.

Install fetch_err.h along with fetch.h.

common.c:
Remove the _netdb_errstring() macro, and add FETCH_ERR_NETDB to the
error code in the _netdb_seterr() macro.

Add categories to the _netdb_errlist table.

Report errors through the Common Error library.

common.h:
Add the DEBUG macros.

Add prototype for fetchConnect().

Remove the prototype for _fetch_errstring(), which is local to common.c

Add a categroy field to struct fetcherr, and define constants for
error categories.

Define macros for _{url,netdb,ftp,http}_seterr().

errors.et: (new file)
List error categories.

fetch.3:
Document the fetchStat*() functions.

Move the "unimplemented functionality" comments from NOTES to BUGS.

Document that applications which use libfetch must also use
libcom_err, and list existing error codes.

Undocument fetchLastErr{Code,String}.

Remove the (empty) DIAGNOSTICS section.

Mention Eugene Skepner in the AUTHORS section.

fetch.c:
Move the DEBUG macros to common.c

Add fetchStat() and fetchStatURL().

Generate error messages for URL parser errors, and fix a minor bug
in the parser.

Use 'struct url' instead of 'url_t'.

Remove fetchLastErr{Code,String}.

fetch.h:
Use 'struct url' instead of 'url_t', and remove the typedef.

Define struct url_stat (used by fetchStat()).

Add prototypes for fetchStat*().

Remove the declarations for fetchLastErr{Code,String}.

Include fetch_err.h.

fetch_err.et: (new file)
Error table for libfetch.

file.c:
Add fetchStatFile().

Use 'struct url' instead of 'url_t'.

ftp.c:
Add fetchStatFTP().

Use 'struct url' instead of 'url_t'.

Don't use fetchLastErrCode.

ftp.errors:
Add categories to all error messages.

http.c:
Add fetchStatHTTP().

Use 'struct url' instead of 'url_t'.

Don't use fetchLastErr{Code,Text}.

http.errors:
Add categories to all error messages.

Prompted by: jkh and Eugene Skepner
Numerous sugestions from: Garett Wollman and Eugene Skepner

First of a series of cleanups to libfetch. Changed files, in
alphabetical order:

Makefile:
Add common.c to SRCS.

Make debugging easier by making 'CFLAGS += -DNDEBUG' conditional on DEBUG

Don't declare struct {ftp,http}err in {ftp,http}err.c; use struct fetcherr
instead.

README:
Remove the todo list, which is out of date anyway.

common.c: (new file)
Gather utility functions in this file.

Merge the error reporting functions intp _fetch_errstring(),
_fetch_seterr() and _fetch_syserr().

Set fetchLastErrCode and fetchLastErrText appropriately when fetchConnect
fails.

common.h: (new file)
Gather internal prototypes and structures in this files.

fetch.3:
Undocument fetchFreeURL().

Document a few more known bugs.

Document fetchLastErrCode and fetchLastErrText.

fetch.c:
Add descriptive comments to all functions that lacked them.

Move fetchConnect() to common.c.

Obviate the need for fetchFreeURL(), and remove it.

fetch.h:
Modify struct url_t so the document part is at the end.

ftp.c:
Remove code that is duplicated elsewhere.

http.c:
Remove code that is duplicated elsewhere.

Prompted by: jkh

Commit a bunch of patches that have been accumulating:

- Fix the README to reflect the new status of the ftp code.
- Change tons of 'if (xxx < 0)' to 'if (xxx == -1)'
- Add two new interface functions
- Fix the Makefile so it actually works (yay!)

Now the manpage is lagging even further behind... :( Next on the todo
list is to clean up the http code.

FTP retrieve works.

Too many changes to list. Basically, FTP is nearly there and error
reporting is kinda sorted out. Now HTTP needs to catch up...

This commit was generated by cvs2svn to compensate for changes in r37535,
which included commits to RCS files with non-trunk default branches.

Imported libfetch into the tree. It compiles, but there's still some
work to do. I especially need help with the man page.