#
1d386b48 |
|
16-Aug-2023 |
Warner Losh <imp@FreeBSD.org> |
Remove $FreeBSD$: one-line .c pattern Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/
|
#
57fbafb8 |
|
15-Nov-2022 |
John Baldwin <jhb@FreeBSD.org> |
libfetch: Pass a zeroed digest to DigestCalcResponse. GCC 12 warns that passing "" (a constant of char[1]) to a parameter of type char[33] could potentially overread. It is not clear from the context that c->qops can never be "auth-int" (and if it can't, then the "auth-int" handling in DigestCalcResponse is dead code that should be removed since this is the only place the function is called). Reviewed by: emaste Differential Revision: https://reviews.freebsd.org/D36825
|
#
ce700f78 |
|
20-Apr-2022 |
Stefan Eßer <se@FreeBSD.org> |
libfetch: remove a set-but-not-uswed variable
|
#
635eb7ac |
|
09-Sep-2021 |
Baptiste Daroussin <bapt@FreeBSD.org> |
fetch: do not confuse capacity and length The patch converting fetch to getline (ee3ca711a898cf41330c320826ea1e0e6e451f1d), did confuse the capacity of the line buffer with the actual len of the read line confusing fetch -v.
|
#
ee3ca711 |
|
17-Aug-2021 |
Daniel Kolesa <daniel@octaforge.org> |
libfetch: use more portable getline() interface this is for better portability in order to avoid using a function which is BSD-only or available via libbsd MFC after: 3 weeks
|
#
345c30a9 |
|
01-Apr-2021 |
Renato Botelho <garga@FreeBSD.org> |
libfetch: Retry with proxy auth when server returns 407 PR: 220468 Submitted by: Egil Hasting <egil.hasting@higen.org> (based on) Reviewed by: kevans, kp Approved by: kp MFC after: 2 weeks Sponsored by: Rubicon Communications, LLC ("Netgate") Differential Revision: https://reviews.freebsd.org/D29533
|
#
f4e05cc5 |
|
28-Aug-2019 |
Mark Johnston <markj@FreeBSD.org> |
Document fetchReqHTTP(). Submitted by: Farhan Khan <khanzf@gmail.com> Reviewed by: 0mp MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D18788
|
#
a768df3e |
|
27-Nov-2018 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
When deciding whether to send the complete URL or just the document part, we were looking at the original URL rather than the one we were currently processing. This meant that if we were trying to retrieve an HTTP URL but were redirected to an HTTPS URL, and HTTPS proxying was enabled, we would send an invalid request and most likely get garbage back. MFC after: 3 days
|
#
ceedec4b |
|
27-Nov-2018 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
A few more cases where strcasecmp() is no longer required. MFC after: 1 week
|
#
f2eac202 |
|
29-May-2018 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Fix a few (but far from all) style issues. MFC after: 3 weeks
|
#
c5712d6d |
|
29-May-2018 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Use __VA_ARGS__ to simplify the DEBUG macro. MFC after: 3 weeks
|
#
b847b083 |
|
12-May-2018 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Preserve if-modified-since timestamps across redirects. PR: 224426 MFC after: 1 week
|
#
5e53a4f9 |
|
25-Nov-2017 |
Pedro F. Giffuni <pfg@FreeBSD.org> |
lib: further adoption of SPDX licensing ID tags. Mainly focus on files that use BSD 2-Clause license, however the tool I was using mis-identified many licenses so this was mostly a manual - error prone - task. The Software Package Data Exchange (SPDX) group provides a specification to make it easier for automated tools to detect and summarize well known opensource licenses. We are gradually adopting the specification, noting that the tags are considered only advisory and do not, in any way, superceed or replace the license texts.
|
#
08a49957 |
|
17-Mar-2017 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
r308996 broke IP literals by assuming that a colon could only occur as a separator between host and port, and using strchr() to search for it. Rewrite fetch_resolve() so it handles bracketed literals correctly, and remove similar code elsewhere to avoid passing unbracketed literals to fetch_resolve(). Remove #ifdef INET6 so we still parse IP literals correctly even if we do not have the ability to connect to them. While there, fix an off-by-one error which caused HTTP 400 errors to be misinterpreted as redirects. PR: 217723 MFC after: 1 week Reported by: bapt, bz, cem, ngie
|
#
c8453e5b |
|
04-Mar-2017 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Fix partial requests (used by fetch -r) when the requested file is already complete. Since 416 is an error code, any Content-Range header in the response would refer to the error message, not the requested document, so relying on the value of size when we know we got a 416 is wrong. Instead, just verify that offset == 0 and assume that we've reached the end of the document (if offset > 0, we did not request a range, and the server is screwing with us). Note that we cannot distinguish between reaching the end and going past it, but that is a flaw in the protocol, not in the code, so we just have to assume that the caller knows what it's doing. A smart caller would request an offset slightly before what it believes is the end and compare the result to what is already in the file. PR: 212065 Reported by: mandree MFC after: 3 weeks
|
#
21ca0912 |
|
30-Dec-2016 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Fix inverted loop condition which broke multi-line responses to CONNECT. PR: 194483 Submitted by: Miłosz Kaniewski <milosz.kaniewski@gmail.com> MFC after: 1 week
|
#
a5fc9a29 |
|
31-May-2016 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
r169386 (PR 112515) was incomplete: it treated 307 as an error except in verbose mode, and did not handle 308 at all. r241840 (PR 172451) added support for 308, but with the same bug. Correctly handle both by recognizing them as redirects in all places where we check the HTTP result code. PR: 112515 173451 209546 Submitted by: novel@ MFC after: 1 week
|
#
77b822db |
|
12-May-2016 |
Don Lewis <truckman@FreeBSD.org> |
Use strlcpy() instead of strncpy() to copy the string returned by setlocale() so that static analyzers know that the string is NUL terminated. This was causing a false positive in Coverity even though the longest string returned by setlocale() is ENCODING_LEN (31) and we are copying into a 64 byte buffer. This change is also a bit of an optimization since we don't need the strncpy() feature of padding the rest of the destination buffer with NUL characters. Reported by: Coverity CID: 974654
|
#
a982c4c7 |
|
11-Feb-2016 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Fix double-free error: r289419 moved all error handling in http_connect() to the end of the function, but did not remove a fetch_close() call which was made redundant by the one in the shared error-handling code. PR: 206774 Submitted by: Christian Heckendorf <heckendorfc@gmail.com> MFC after: 3 days
|
#
adc1aa7a |
|
16-Dec-2015 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
As a followup to r292330, standardize on size_t and add a few comments.
|
#
a568844c |
|
16-Dec-2015 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Reset bufpos to 0 immediately after refilling the buffer. Otherwise, we risk leaving the connection in an indeterminate state if the server fails to send a chunk delimiter. Depending on the application and on the sizes of the preceding chunks, the result can be anything from missing data to a segfault. With this patch, it will be reported as a protocol error. PR: 204771 MFC after: 1 week
|
#
a1b9b174 |
|
29-Nov-2015 |
Dimitry Andric <dim@FreeBSD.org> |
Fix buildworld after r291453, similar to r284346: url->user and url->pwd are arrays, so they can never be NULL. Reported by: many Pointy hat to: des
|
#
4d8b056e |
|
29-Nov-2015 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Use .netrc for HTTP sites and proxies, not just FTP. PR: 193740 Submitted by: TEUBEL György <tgyurci@gmail.com> MFC after: 1 week
|
#
c3f9b93b |
|
15-Oct-2015 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Fix two bugs in HTTPS tunnelling: - If the proxy returns a non-200 result, set the error code accordingly so the caller / user gets a somewhat meaningful error message. - Consume and discard any HTTP response header following the result line. PR: 194483 Tested by: Fabian Keil <fk@fabiankeil.de> MFC after: 1 week
|
#
ddcc2ecb |
|
04-Jul-2015 |
Marcelo Araujo <araujo@FreeBSD.org> |
Remove unused variable to silence clang warning. Differential Revision: D2683 Reviewed by: rodrigc, bapt
|
#
f2c41c55 |
|
13-Jun-2015 |
Dimitry Andric <dim@FreeBSD.org> |
Fix the following clang 3.7.0 warnings in lib/libfetch/http.c: lib/libfetch/http.c:1628:26: error: address of array 'purl->user' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion] aparams.user = purl->user ? ~~~~~~^~~~ ~ lib/libfetch/http.c:1630:30: error: address of array 'purl->pwd' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion] aparams.password = purl->pwd? ~~~~~~^~~~ lib/libfetch/http.c:1657:25: error: address of array 'url->user' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion] aparams.user = url->user ? ~~~~~^~~~ ~ lib/libfetch/http.c:1659:29: error: address of array 'url->pwd' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion] aparams.password = url->pwd ? ~~~~~^~~ ~ lib/libfetch/http.c:1669:25: error: address of array 'url->user' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion] aparams.user = url->user ? ~~~~~^~~~ ~ lib/libfetch/http.c:1671:29: error: address of array 'url->pwd' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion] aparams.password = url->pwd ? ~~~~~^~~ ~ Since url->user and url->pwd are arrays, they can never be NULL, so the checks can be removed. Reviewed by: bapt MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D2673
|
#
ef12c36e |
|
30-May-2015 |
Dimitry Andric <dim@FreeBSD.org> |
Tentatively fix a few libfetch warnings, see this review for the details: https://reviews.freebsd.org/D2673
|
#
c4199130 |
|
05-Jun-2014 |
Baptiste Daroussin <bapt@FreeBSD.org> |
Add support for arbitrary http requests Submitted by: Alex Hornung <alex@alexhornung.com> Reviewed by: des Obtained from: Dragonfly MFC after: 3 week
|
#
4bd8c06c |
|
05-Jun-2014 |
Baptiste Daroussin <bapt@FreeBSD.org> |
Remove unnecessary semicolons Patch by Sascha Wildner <saw@online.de> for Dragonfly Reviewed by: des Obtained from: Dragonfly MFC after: 1 week
|
#
6064928d |
|
05-Jun-2014 |
Baptiste Daroussin <bapt@FreeBSD.org> |
Use NULL instead of 0 Patch by Sascha Wildner <saw@online.de> for Dragonfly Reviewed by: des Obtained from: Dragonfly MFC after: 1 week
|
#
c257f99e |
|
05-Jun-2014 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
If HTTP_USER_AGENT is defined but empty, don't send User-Agent at all. PR: 184507 Submitted by: jbeich@tormail.org (with modifications) MFC after: 1 week
|
#
b36853ca |
|
11-Mar-2014 |
Bryan Drewery <bdrewery@FreeBSD.org> |
Support Last-Modified behind proxies which return UTC instead of GMT. The standard states that GMT must be used, but that UTC is equivalent. Still parse UTC as otherwise this causes problems for pkg(8). It will refetch the repository every time 'pkg update' or other remote operations are used behind these proxies. RFC2616: "All HTTP date/time stamps MUST be represented in Greenwich Mean Time (GMT), without exception. For the purposes of HTTP, GMT is exactly equal to UTC (Coordinated Universal Time)."" Approved by: bapt (mentor) Reviewed by: des, peter Sponsored by: EMC / Isilon Storage Division MFC after: 1 week
|
#
4524013c |
|
30-Jan-2014 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Bump copyright dates
|
#
9c1ca3a1 |
|
28-Jan-2014 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
r261230 broke the cases where the amount of data to be read is not known in advance, or where the caller doesn't care and just keeps reading until it hits EOF. In fetch_read(): the socket is non-blocking, so read() will return 0 on EOF, and -1 (errno == EAGAIN) when the connection is still open but there is no data waiting. In the first case, we should immediately return 0. The EINTR case was also broken, although not in a way that matters. In fetch_writev(): use timersub() and timercmp() as in fetch_read(). In http_fillbuf(): set errno to a sensible value when an invalid chunk header is encountered. In http_readfn(): as in fetch_read(), a zero return from down the stack indicates EOF, not an error. Furthermore, when io->error is EINTR, clear it (but no errno) before returning so the caller can retry after dealing with the interrupt. MFC after: 3 days
|
#
215a27f1 |
|
27-Jan-2014 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Solve http buffering issues and hangs once and for all (hopefully!) by simply not trying to return exactly what the caller asked for - just return whatever we got and let the caller be the judge of whether it was enough. If an error occurs or the connection times out after we already received some data, return a short read, under the assumption that the next call will fail or time out before we read anything. As it turns out, none of the code that calls fetch_read() assumes an all-or-nothing result anyway, except for a couple of lines where we read the CR LF at the end of a hunk in HTTP hunked encoding, so the changes outside of fetch_read() and http_readfn() are minimal. While there, replace select(2) with poll(2). MFC after: 3 days
|
#
615c5740 |
|
22-Aug-2013 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Even though it doesn't really make sense in the context of a CONNECT request, RFC 2616 14.23 mandates the presence of the Host: header in all HTTP 1.1 requests. PR: kern/181445 Submitted by: Kimo <kimor79@yahoo.com> MFC after: 3 days
|
#
1453595f |
|
30-Jul-2013 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Include an Accept header in requests. PR: kern/180917 MFC after: 1 week
|
#
dcd47379 |
|
26-Jul-2013 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Implement certificate verification, and many other SSL-related imrovements; complete details in the PR. PR: kern/175514 Submitted by: Michael Gmelin <freebsd@grem.de> MFC after: 1 week
|
#
ba7c6aec |
|
21-Jul-2013 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Use the correct request syntax for proxied (tunneled) HTTPS requests. PR: bin/180666 MFC after: 3 days
|
#
4056bae9 |
|
12-Apr-2013 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Use the CONNECT method to proxy HTTPS connections through HTTP proxies. PR: bin/80176 Submitted by: Yuichiro NAITO <naito.yuichiro@gmail.com>
|
#
eab7a548 |
|
15-Nov-2012 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Fix weird indentation.
|
#
8d049fb2 |
|
21-Oct-2012 |
Eitan Adler <eadler@FreeBSD.org> |
Implement HTTP 305 redirect handling. PR: 172452 Submitted by: gcooper Reviewed by: des Approved by: cperciva MFC after: 1 week
|
#
c4fa1489 |
|
21-Oct-2012 |
Eitan Adler <eadler@FreeBSD.org> |
Don't deny non-temporary redirects if the -A option is set (per the man page) [0] While here add support for draft-reschke-http-status-308-07 PR: 172451 [0] Submitted by: gcooper [0] Reviewed by: des Approved by: cperciva MFC after: 1 week
|
#
e6c0e200 |
|
21-Oct-2012 |
Eitan Adler <eadler@FreeBSD.org> |
Be a bit more lenient in the maximum number of redirects allowed. Chrome and Firefox have a limit of 20. IE has a limit of 8. Reviewed by: des Approved by: cperciva MFC after: 3 days
|
#
0e50a833 |
|
14-Sep-2012 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Use libmd if and only if OpenSSL is not available. PR: bin/171402 MFC after: 3 days
|
#
f51b84bc |
|
29-Apr-2012 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Don't reuse credentials if redirected to a different host. Submitted by: Niels Heinen <heinenn@google.com> MFC after: 3 weeks
|
#
2a7daafe |
|
18-Jan-2012 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Fix two issues related to the use of SIGINFO in fetch(1) to display progress information. The first is that fetch_read() (used in the HTTP code but not the FTP code) can enter an infinite loop if it has previously been interrupted by a signal. The second is that when it is interrupted, fetch_read() will discard any data it may have read up to that point. Luckily, both bugs are extremely timing-sensitive and therefore difficult to trigger. PR: bin/153240 Submitted by: Mark <markjdb@gmail.com> MFC after: 3 weeks
|
#
578153f1 |
|
19-Oct-2011 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
latin1 -> utf8
|
#
6337341d |
|
27-Sep-2011 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Update copyright dates and strip my middle name.
|
#
eb9b80c3 |
|
12-May-2011 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Increase WARNS to 4.
|
#
c12c6e3c |
|
12-May-2011 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Mechanical whitespace cleanup.
|
#
a42eecde |
|
12-May-2011 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Increase WARNS to 3.
|
#
a7d5f7eb |
|
19-Oct-2010 |
Jamie Gritton <jamie@FreeBSD.org> |
A new jail(8) with a configuration file, to replace the work currently done by /etc/rc.d/jail.
|
#
c954ded2 |
|
28-Jul-2010 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Fix a couple of embarrassing mistakes in the previous commit. Submitted by: Dimitry Andric <dimitry@andric.com>
|
#
962cf295 |
|
01-Jul-2010 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
If the A flag is supplied, http_request() will attempt the request only once, even if authentication is required, instead of retrying with the proper credentials. Fix this by bumping the countdown if the origin or proxy server requests authentication so that the initial unauthenticated request does not count as an attempt. PR: 148087 Submitted by: Tom Evans <tevans.uk@googlemail.com> MFC after: 2 weeks
|
#
fe0506d7 |
|
09-Mar-2010 |
Marcel Moolenaar <marcel@FreeBSD.org> |
Create the altix project branch. The altix project will add support for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting is a two-module system, consisting of a base compute module and a CPU expansion module. SGI's NUMAFlex architecture can be an excellent platform to test CPU affinity and NUMA-aware features in FreeBSD.
|
#
da4af7b6 |
|
26-Jan-2010 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
MFH (r202613, r202623): HTTP digest authentication support.
|
#
79ad329d |
|
19-Jan-2010 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Add HTTP digest authentication. Submitted by: Jean-Francois Dockes <jf@dockes.org> Forgotten by: des (repeatedly)
|
#
7f92799f |
|
15-Dec-2008 |
Murray Stokely <murray@FreeBSD.org> |
Add support for HTTP 1.1 If-Modified-Since behavior. fetch(1) accepts a new argument -i <file> that if specified will cause the file to be downloaded only if it is more recent than the mtime of <file>. libfetch(3) accepts the mtime in the url structure and a flag to indicate when this behavior is desired. PR: bin/87841 Submitted by: Jukka A. Ukkonen <jau@iki.fi> (partially) Reviewed by: des, ru MFC after: 3 weeks
|
#
e374393a |
|
24-Oct-2008 |
Ruslan Ermilov <ru@FreeBSD.org> |
Don't fail mistakenly with -r when we already have the whole file. Reviewed by: des
|
#
d7f03759 |
|
19-Oct-2008 |
Ulf Lilleengen <lulf@FreeBSD.org> |
- Import the HEAD csup code which is the basis for the cvsmode work.
|
#
340b079b |
|
08-Feb-2008 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Use memcpy(3) instead of the BSD-specific bcopy(3). Submitted by: Joerg Sonnenberger <joerg@britannica.bec.de> MFC after: 2 weeks
|
#
79257dd7 |
|
06-Feb-2008 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Add necessary cast for tolower() argument. Submitted by: Joerg Sonnenberger <joerg@britannica.bec.de> MFC after: 1 week
|
#
facd9827 |
|
18-Dec-2007 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
As several people pointed out, I did all the ctype casts the wrong way (not for the first time...) Noticed by: bde, ru ++ MFC after: 1 week
|
#
62a2681c |
|
18-Dec-2007 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Add support for the NO_PROXY / no_proxy environment variable as used by lynx, curl etc. Note that this patch differs significantly from that in the PR, as the submitter refined it after submitting the PR. PR: 110388 Submitted by: Alexander Pohoyda <alexander.pohoyda@gmx.net> MFC after: 3 weeks
|
#
836e34ee |
|
18-Dec-2007 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Old patch I had lying around: correctly cast the argument to is*(). IWBNI gcc could warn about this the way it warns about printf() abuse. MFC after: 1 week
|
#
a1b37df2 |
|
14-Dec-2007 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Clean up namespace violations. MFC after: 1 week
|
#
cd842257 |
|
08-May-2007 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Handle temporary redirects (HTTP status code 307) PR: kern/112515 Submitted by: Ryan C. Gordon <icculus@icculus.org> MFC after: 3 weeks
|
#
2738229e |
|
23-Aug-2005 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Ignore HTTP_PROXY if it is defined but empty. This was already handled correctly in the case of FTP_PROXY, because an empty FTP_PROXY has a specific meaning ("don't use any proxy at all for ftp, even if HTTP_PROXY is defined"), while an empty HTTP_PROXY has no meaning at all. PR: bin/85185 Submitted by: Conall O'Brien <conallob=freebsd@maths.tcd.ie> MFC after: 2 weeks
|
#
448980e7 |
|
02-Mar-2005 |
Kelly Yancey <kbyanc@FreeBSD.org> |
Set the TCP_NODELAY socket option and clear TCP_NOPUSH in order to flush any pending HTTP request rather than calling shutdown(2) with SHUT_WR. This makes libfetch (and thus fetch(1)) work again with Squid proxies configured to not allow half-closed connections. Reported by: Pawel Worach (pawel.worach AT telia DOT com)
|
#
20a2a51b |
|
15-Feb-2005 |
Kelly Yancey <kbyanc@FreeBSD.org> |
Set TCP_NOPUSH on HTTP requests, reducing the number of round-trips necessary to establish each connection. MFC after: 2 weeks
|
#
2cbbf9da |
|
21-Sep-2004 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Update copyright years.
|
#
264a4c03 |
|
07-Aug-2004 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Don't forget to allocate space for the terminating NUL when converting to base 64. PR: misc/70022 Submitted by: Herve Masson <herve-bsdbt@mindstep.com> MFC after: 3 days
|
#
4d3d90ad |
|
11-Feb-2004 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Slight tweak to previous commit: don't forget to call _http_seterr() if we get a genuine 416 reply.
|
#
b5e12145 |
|
11-Feb-2004 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
When restarting a transfer that has already completed, the server will reply with a 416 error code (requested range not satisfiable) because we ask it to start at the end of the file. Handle this gracefully by considering a 416 reply a success if the requested offset exactly matches the length of the file and the requested length is zero.
|
#
2576e459 |
|
11-Feb-2004 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Re-wrap some comments.
|
#
762892c0 |
|
29-Mar-2003 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Don't dereference flags if NULL (see http.c rev 1.87)
|
#
b91d6074 |
|
11-Mar-2003 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Don't parse the proxy URL unless we're actually going to use it. No real functional difference, but debugging output will be less confusing.
|
#
930105c1 |
|
28-Jan-2003 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
style(9): add parentheses to sizeof even when not strictly required. MFC after: 3 days
|
#
591c0199 |
|
22-Jan-2003 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
A negative offset means "get it all".
|
#
d5216a4f |
|
27-Nov-2002 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Implement and document support for an HTTP_REFERER environment variable. PR: 28171 Submitted by: Andre Albsmeier <andre.albsmeier@mchp.siemens.de> Approved by: re (bmah) MFC after: 1 week
|
#
a49b853d |
|
30-Oct-2002 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Use __func__ and break a long line.
|
#
cab84950 |
|
29-Oct-2002 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Don't forget to set the error flag when _fetch_read() returns -1.
|
#
9f788e9c |
|
27-Oct-2002 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Eliminate two cases of undefined behaviour: total in _fetch_write() was not initialized before use, and _http_growbuf() did not return a value on success. Reported by: Peter Edwards <pmedwards@eircom.net> MFC after: 2 weeks
|
#
721d559c |
|
03-Oct-2002 |
Ruslan Ermilov <ru@FreeBSD.org> |
HTTP authentication got broken in rev. 1.49.
|
#
f8bd33a0 |
|
19-Jun-2002 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Don't try to dereference conn when we know it's NULL.
|
#
3070f6cb |
|
05-Jun-2002 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Make SSL support conditional on NOCRYPT.
|
#
111e2510 |
|
04-Jun-2002 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Add SSL support + slight cleanup. Submitted by: Henry Whincup <henry@techiebod.com> (in principle)
|
#
9601e333 |
|
04-Jun-2002 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Wrap everything in struct connection, and enforce timeouts everywhere (except for DNS operations). Always use funopen() for HTTP, to support both timeouts and SSL.
|
#
ccdd94bd |
|
05-Jun-2002 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Rename struct cookie to struct httpio to avoid confusion (it's not an HTTP cookie) and increase symmetry with equivalent FTP code.
|
#
d3b03a90 |
|
05-Jun-2002 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Add comments to struct cookie.
|
#
3f4823c5 |
|
05-Jun-2002 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Fix a bug I introduced in the chunk decoder in the previous commit..
|
#
dea29ca1 |
|
05-Jun-2002 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
First step towards SSL support: wrap connections in a 'struct connection' which contains the socket descriptor, the input buffer and (yet unused) SSL state variables. This has the neat side effect of greatly improving reentrance (though we're not *quite* there yet) and opening the door to HTTP connection caching. This commit is inspired by email conversations with and patches from Henry Whincup <henry@techiebod.com> last fall.
|
#
4cd01193 |
|
29-Mar-2002 |
Mark Murray <markm@FreeBSD.org> |
Do not use __progname directly (except in [gs]etprogname(3)). Also, make an internal _getprogname() that is used only inside libc. For libc, getprogname(3) is a weak symbol in case a function of the same name is defined in userland.
|
#
e19e6098 |
|
05-Feb-2002 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Reindent, and add parentheses to return statements. Some functions in ftp.c and http.c now have exceedingly long lines due to deep nesting; this will be corrected by reorganizing the code in a later revision.
|
#
e6f0a33e |
|
20-Jan-2002 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Check the return value from read() when reading the CR/LF at the end of a chunk. PR: bin/33608 MFC after: 2 weeks
|
#
a8e9bd87 |
|
01-Jan-2002 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
In verbose mode, display the full error message from the server, stripping it of HTML tags and comments. PR: bin/32989 MFC after: 1 week
|
#
f67efa37 |
|
01-Jan-2002 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Remove VT100 escapes from debugging messages now that they're enabled by default. PR: 32988 MFC after: 3 days
|
#
7f807cb8 |
|
09-Dec-2001 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Conditionalize some debugging code that didn't use the DEBUG macro. MFC after: 1 week
|
#
65986545 |
|
03-Dec-2001 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Properly handle being redirected to an FTP URL. Also fix a couple of not-so-minor bugs: - missing case for HTTP_SEE_OTHER - incorrect definition of HTTP_NEED_PROXY_AUTH PR: 20259 MFC after: 2 weeks
|
#
f573a5fc |
|
18-Oct-2001 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Tons of type, style and warning fixes that have been rotting in my tree for ages - some of which wouldn't be necessary if gcc wasn't broken or TPTB were willing to do something (-fno-builtin) about it.
|
#
cecb889f |
|
30-Sep-2001 |
Matthew Dillon <dillon@FreeBSD.org> |
Add __FBSDID()s to libfetch
|
#
6a0cf64b |
|
27-May-2001 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Implement a HTTP_USER_AGENT environment variable. PR: 27669 Submitted by: Eoin Lawless <eoin@maths.tcd.ie>
|
#
6490b215 |
|
26-May-2001 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Add rudimentary support for an authentication callback function.
|
#
38c7e4a6 |
|
23-Apr-2001 |
Archie Cobbs <archie@FreeBSD.org> |
Apply 'const' liberally. Fix some other minor glitches.
|
#
4cee73c8 |
|
06-Mar-2001 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Support lower-case versions of the proxy environment variables. PR: bin/25494
|
#
e828ada7 |
|
10-Nov-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Use the documented (and historical) defaults. Centralize the decision logic in order to avoid this bug in the future. Submitted by: se
|
#
d74a913b |
|
29-Oct-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Use CHECK_FLAG
|
#
1a16ed4c |
|
12-Oct-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Rework proxy handling so that proxies can be specified as URLs. As a side effect, remove a lot of duplicate and now redundant code.
|
#
a898bb8d |
|
23-Sep-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Work around buggy servers such as NCSA httpd which send an incomplete HTTP-Version on the Status-Line (see RFC2616 sections 3.1 and 6.1).
|
#
be6aff99 |
|
02-Aug-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Fix signedness bogon.
|
#
10851dc4 |
|
25-Jul-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Centralize the default port finding code. Work around YA Apache bug: don't send port in Host: header if it's the default port.
|
#
c78f1cc9 |
|
21-Jul-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Ignore environment variables that are set but empty. Sort out the size / length confusion. Always try to report the *real* file size in the url_stat structure, no matter how much of it is actually being sent, and try to detect inconsistencies between sizes. Rearrange the request loop to avoid having to add meaningless code just to silence compiler warnings. Switch to a more sensible and consistent interface for the _http_parse*() functions.
|
#
87b41116 |
|
19-Jul-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Don't try to skip to the requested offset if the server returns more data than requested. Instead, inform the caller of the real offset by modifying the offset field in the original struct url, and let him decide how to handle the situation.
|
#
1a5faa10 |
|
17-Jul-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Introduce fetchXGet*(), which combine the functionalities of fetchGet*() and fetchStat*(). In most cases, either fetchGet*() or fetchXGet*() is a wrapper around the other; in all cases, calling fetchGet*() is identical to calling fetchXGet*() with the second argument set to NULL.
|
#
c7897277 |
|
17-Jul-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Don't forget to set file descriptor to -1 after closing it, since the code outside the loop inspects it to determine whether or not we succeeded in retrieving the requested document. This fixes a bug where fetchGetHTTP() would return a FILE with an invalid file descriptor if it hit the redirect limit without locating the requested document.
|
#
e66b3802 |
|
16-Jul-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
#ifdef DEBUG -> #ifndef NDEBUG Pointed out by: ache
|
#
6290ee73 |
|
15-Jul-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Reinsert the MIT copyright, which applies to the base64 code and was left out by accident during the rewrite.
|
#
f4683775 |
|
13-Jul-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Fix a bug (misplaced continue) that caused redirects to fail. Lots of code moved around, but the acutal functional changes are small. Add support for site-internal redirects (where the Location: header gives a path instead of an absolute URI) Pointed out by: kuriyama
|
#
e4878e39 |
|
12-Jul-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Extensive rewrite of the HTTP code. The new code is significantly cleaner and more robust, and somewhat more efficient. It also handles authorization and redirects properly, and supports timeouts like the FTP code. Many thanks to Umemoto-san for his assistance with IPv6 support, both here and in other parts of libfetch.
|
#
35f723db |
|
11-Jul-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Rework the authorization code. Only send absolute URI if connected to a proxy, since Apache doesn't always understand absolute URIs. Clean up some of the debugging output.
|
#
4d029f13 |
|
08-Jul-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Fix basic authentication, and add proxy authentication. Submitted by: se
|
#
cf5af79c |
|
21-Jun-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Don't incorrectly report a protocol error when we get a 302 and redirects are disabled.
|
#
6f9febf9 |
|
26-May-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Hackish support for 302 redirect.
|
#
28c645cf |
|
20-May-2000 |
Hajimu UMEMOTO <ume@FreeBSD.org> |
IPv6 support.
|
#
6efb30c8 |
|
19-May-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Better handling of some boundary conditions. Submitted by: ume
|
#
c97925ad |
|
15-May-2000 |
Hajimu UMEMOTO <ume@FreeBSD.org> |
Make HTTP_PROXY work for FTP. Reported by: Ben Smithurst <ben@scientia.demon.co.uk> Reviewed by: des
|
#
f8f4130b |
|
15-May-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Remove unused 'verbose'. Initialize the struct url_stat at the beginning of fetchStatHTTP().
|
#
60245e42 |
|
11-May-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Reorganize some of the http code and split it into more functions. Implement fetchStatHTTP(). Unbungle struct url, and add fetchFreeURL(). Document it.
|
#
3d2a8471 |
|
07-May-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Implement restart
|
#
e6182307 |
|
07-May-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Use sizeof more consistently. Outline sanity checking of server port spec.
|
#
f5f109a0 |
|
07-Jan-2000 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Patches I've had lying around for several months: * Add the 'h' ftp flag (allocate local port in high range) * Add the 'd' flag (use direct connection even if proxy is defined) * Make sure flags != NULL before calling strchr(). * Minor changes to some comments.
|
#
7f3dea24 |
|
27-Aug-1999 |
Peter Wemm <peter@FreeBSD.org> |
$Id$ -> $FreeBSD$
|
#
ce71b736 |
|
21-Dec-1998 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Implement and document file list retrieval.
|
#
d941fd2d |
|
18-Dec-1998 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Don't specify a language to compile_et. Rename fetchConnect to _fetch_connect since it's internal.
|
#
3b7a6740 |
|
16-Dec-1998 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Remove unused header files. Fix a few shadowing bugs. Improve error reporting.
|
#
0fba3a00 |
|
16-Dec-1998 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Add verbose flag, and support functions. Brucify the Makefile. Differentiate atime and mtime in fetch*Stat(). Fix a few pointer bugs. Tweak some error messages. Don't #include sys/param.h and stdio.h in fetch.h. Document that sys/param.h and stdio.h must be #included before fetch.h.
|
#
d8acd8dc |
|
06-Nov-1998 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Second of a series of cleanups to libfetch. This commit introduces the following features: a) the fetchStat*() functions, which return meta-information for a document, such as size, modification time, etc. b) the use of the com_err(3) facilities to report errors. It also fixes a bunch of style bugs and a few logic bugs and somewhat improves the man page. Changed files, in alphabetical order: Makefile: Don't generate macros in {ftp,http}err.c. Generate category fields for the error message lists. Compile the error table. Install fetch_err.h along with fetch.h. common.c: Remove the _netdb_errstring() macro, and add FETCH_ERR_NETDB to the error code in the _netdb_seterr() macro. Add categories to the _netdb_errlist table. Report errors through the Common Error library. common.h: Add the DEBUG macros. Add prototype for fetchConnect(). Remove the prototype for _fetch_errstring(), which is local to common.c Add a categroy field to struct fetcherr, and define constants for error categories. Define macros for _{url,netdb,ftp,http}_seterr(). errors.et: (new file) List error categories. fetch.3: Document the fetchStat*() functions. Move the "unimplemented functionality" comments from NOTES to BUGS. Document that applications which use libfetch must also use libcom_err, and list existing error codes. Undocument fetchLastErr{Code,String}. Remove the (empty) DIAGNOSTICS section. Mention Eugene Skepner in the AUTHORS section. fetch.c: Move the DEBUG macros to common.c Add fetchStat() and fetchStatURL(). Generate error messages for URL parser errors, and fix a minor bug in the parser. Use 'struct url' instead of 'url_t'. Remove fetchLastErr{Code,String}. fetch.h: Use 'struct url' instead of 'url_t', and remove the typedef. Define struct url_stat (used by fetchStat()). Add prototypes for fetchStat*(). Remove the declarations for fetchLastErr{Code,String}. Include fetch_err.h. fetch_err.et: (new file) Error table for libfetch. file.c: Add fetchStatFile(). Use 'struct url' instead of 'url_t'. ftp.c: Add fetchStatFTP(). Use 'struct url' instead of 'url_t'. Don't use fetchLastErrCode. ftp.errors: Add categories to all error messages. http.c: Add fetchStatHTTP(). Use 'struct url' instead of 'url_t'. Don't use fetchLastErr{Code,Text}. http.errors: Add categories to all error messages. Prompted by: jkh and Eugene Skepner Numerous sugestions from: Garett Wollman and Eugene Skepner
|
#
842a95cc |
|
05-Nov-1998 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
First of a series of cleanups to libfetch. Changed files, in alphabetical order: Makefile: Add common.c to SRCS. Make debugging easier by making 'CFLAGS += -DNDEBUG' conditional on DEBUG Don't declare struct {ftp,http}err in {ftp,http}err.c; use struct fetcherr instead. README: Remove the todo list, which is out of date anyway. common.c: (new file) Gather utility functions in this file. Merge the error reporting functions intp _fetch_errstring(), _fetch_seterr() and _fetch_syserr(). Set fetchLastErrCode and fetchLastErrText appropriately when fetchConnect fails. common.h: (new file) Gather internal prototypes and structures in this files. fetch.3: Undocument fetchFreeURL(). Document a few more known bugs. Document fetchLastErrCode and fetchLastErrText. fetch.c: Add descriptive comments to all functions that lacked them. Move fetchConnect() to common.c. Obviate the need for fetchFreeURL(), and remove it. fetch.h: Modify struct url_t so the document part is at the end. ftp.c: Remove code that is duplicated elsewhere. http.c: Remove code that is duplicated elsewhere. Prompted by: jkh
|
#
ecc91352 |
|
17-Aug-1998 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Commit a bunch of patches that have been accumulating: - Fix the README to reflect the new status of the ftp code. - Change tons of 'if (xxx < 0)' to 'if (xxx == -1)' - Add two new interface functions - Fix the Makefile so it actually works (yay!) Now the manpage is lagging even further behind... :( Next on the todo list is to clean up the http code.
|
#
f62e5228 |
|
12-Jul-1998 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Base64 code (and the MIT copyright) moved to http.c FTP STORe and APPEnd added. FTP proxy support added (untested).
|
#
8e3986ea |
|
11-Jul-1998 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Too many changes to list. Basically, FTP is nearly there and error reporting is kinda sorted out. Now HTTP needs to catch up...
|
#
4ca1ab94 |
|
09-Jul-1998 |
Dag-Erling Smørgrav <des@FreeBSD.org> |
Imported libfetch into the tree. It compiles, but there's still some work to do. I especially need help with the man page.
|