Import statements sorted with isort and fixed MockBuilder.py (#266)

* Fixed missing closing bracket in the MockBuilder.py

* Import statements sorted with isort

haikuporter/SourceFetcher: guard mkdir for source package fetcher

* While this code worked for single-download packages, multi-download
packages fail due to the already-existing source directory.
* Just guard the mkdir with a check

haikuporter/SourceFetcher: Create target directory before extraction

* extract won't extract to a missing directory. This code isn't
used very often, so went unnoticed for a looong time.

Make sure sourceDir is created before asking tar to change to it

getting git:// sources didn't work anymore.

python3: Finish python3 change. python2 breaking

* Based on mmlr's 2017 work to use Python3 with
changes and adjustments for the current codebase

python3: begin python3 fixes. print -> print()

SourceFetcher: Make progress dots 64KB instead of 1KB.

Most files we download are pretty large and connection speeds are pretty fast,
so this makes more sense.

SourceFetcher: create directory only if missing

Fixes #163.

Fix extraction of #noarchive sources

The code assumed that the directory already existed, but it doesn't
anymore since the changes to detect a mis-set SOURCE_DIR. As a result
the file would be copied in the place where a source directory was
expected.

Solution: just create the dir and then copy the file inside it.

SourceFetcher: Re-add wget tries argument and set to 1.

Retries were in-sourced, but this caused wget to use its default of 20
retires for some types of errors.

Cleanup: Add missing raw string prefix to regex patterns.

Cleanup: Whitespace and line length cleanup.

Add support for checkout on a specific revision in Mercurial. (#144)

SourceFetcher: Internalize download retry handling.

The wget command does not retry for some error cases like DNS resoltion
failures. The tries are moved internally and depend on the reported
error code (taken from the wget man page). Non-final errors are now
retried with a 3 second delay.

Fix git source fetch

Was missing setting the output variable, making the
following info(output) fail (when using verbose mode).

Add some big, nasty warnings when using unsafe sources.

Revert "Permanently disable downloading from unsafe sources."

This reverts commit a2a800093d927d4372bc4aaada66597d11c8a5ef.

Permanently disable downloading from unsafe sources.

We've slowly been phasing these out over the past few years,
at this point we might as well disable them (if there are no major
reasons to keep them, I'll remove them totally in a few months.)

Implement local builders.

* provide info(), warn() to be used instead of print. By default outputs to stdout.
For builders, outputs to files.
* replace most calls to check_call by check_output, and log with info(). Also outputs
the outputs an eventual CalledProcessError output attribute.
* add option --local-builders to be used with --build-master: indicates how many
local builders to use instead of remote builders. If paramiko isn't available,
defaults to 1, otherwise 0.
* rename Builder to RemoteBuilder.
* remove changes of current directory: this doesn't work in a
multithreaded environnement. Policy.py is the most affected as it
expected to run in the packaging directory.

Use Unicode strings for warn() & sysExit().

Add option to select Sourceforge mirror.

* sourceforge mirror selector often returns a bad mirror (for me at least).

SourceFetcher: use "file://" as prefix for local file...

because we now check that they are actual URI.

SourceFetcher: Check certificates.

Remove local check_output, we're now using Python 2.7.

Rename SRC_URI to SOURCE_URI for consistency's sake.

Add ISO date format for CVS source fetching

New versions of CVS no longer support the DD/MM/YYYY format and now use
YYYY-MM-DD

Cleanup imports.

Let wget use a 10 second network timeout.

Revert to tab spacing for Pe's sake.

Try again with whitespace.

Second attempt to fix whitespace. Options are different on OS X.

Convert tabs to spaces.

Add configuration option to enable/disable unsafe sources

Fix bugs in handling of multiple sources.

* make sure to overrule the SRC_URIs for all sources when a port is
based on a source package, as otherwise we'd try to download those
other sources, which isn't needed and might not work (e.g. when
bootstrapping)
* use correct paths for each of multiple sources when populating and
extracting source packages

Fix base path when extracting from rigged source packages.

Drop debug leftover.

Fix automatic fetching of non-local revisions in git.

Fix handling of sources with empty SOURCE_DIR

unpackFile() didn't have enough information to discriminate this case
from the single-component case and thus try to extract a subdir from the
archive.

Whitespace cleanup

Add source fetcher for source packages (pkg: protocol)

Clean up trailing whitespace

Add automatic repository update for changed source revisions

* implement support for automatic fetching from remote repository when
a change in the used source revision has been detected (currently
only implemented for git)

Fix warnings about use of reserved internal symbol 'type'.

SourceFetcherForGit: use a bare repository

... instead of one without a checkout. This seems less weird and makes
it easier to manually update (at least for me). Of course, ideally
haikuporter would automatically recognize that it can just update the
repository instead of checking everything out again.

Actually make use of a specified rev when fetching via git.

Improve robustness of foldSubdirIntoSourceDir():

* rename the subdir to a (hopefully) unique name to avoid problems
with contained files/folder with an identical name (as is the case
for binutils)

SourceFetcherFor{Download,LocalFile}: fix unpack() with subdir

The assumption that the first subdir component is the source dir name
was incorrect as subdir is relative to the source directory.

Add license header (MIT)

* additionally: uppercase copyright notices and separate them from the
top by a single line

Squash a TODO that no longer applies.

SourceFetcherForCvs: quote module name

... in case it contains weird characters (like for keymapswitcher).

Generalize support for SOURCE_EXPORT_SUBDIR:

* add support for exporting only a subdir to unpackArchive()
* add automatic folding of subdir-contents into source directory,
after unpacking from archive and using tar for "unpacking" a checkout

Fix SourceFetcherForLocalFile:

* adjusting self.fetchTarget doesn't cut it, as Source has its own
version of it, which it is accessing - so now we simply symlink the
local file into the download directory

Refactor handling of checkouts (and somewhat downloads, too):

* Pull protocol specific SourceFetcher-classes out of Source.
* Do checkouts into download directory and then export the relevant
source revision into the source dir. This way, removing the work
dir no longer drops the precious checkouts and checkouts can be
shared between different port versions.
* Always create an implicit git repo in the source dir, no more need
for two code paths.
* Handle SOURCE_EXPORT_SUBDIR a bit more intelligently, such that only
the stuff living underneath the export subdir is put into the source
dir (i.e. without the export subdir itself as relative path)
* After a source has been fetched, write the URI that has been used
into a marker file, such that during following runs, HaikuPorter knows
that URI and can create an appropriate SourceFetcher object for it.

Cleanup imports.

Let wget use a 10 second network timeout.

Revert to tab spacing for Pe's sake.

Try again with whitespace.

Second attempt to fix whitespace. Options are different on OS X.

Convert tabs to spaces.

Add configuration option to enable/disable unsafe sources

Fix bugs in handling of multiple sources.

* make sure to overrule the SRC_URIs for all sources when a port is
based on a source package, as otherwise we'd try to download those
other sources, which isn't needed and might not work (e.g. when
bootstrapping)
* use correct paths for each of multiple sources when populating and
extracting source packages

Fix base path when extracting from rigged source packages.

Drop debug leftover.

Fix automatic fetching of non-local revisions in git.

Fix handling of sources with empty SOURCE_DIR

unpackFile() didn't have enough information to discriminate this case
from the single-component case and thus try to extract a subdir from the
archive.

Whitespace cleanup

Add source fetcher for source packages (pkg: protocol)

Clean up trailing whitespace

Add automatic repository update for changed source revisions

* implement support for automatic fetching from remote repository when
a change in the used source revision has been detected (currently
only implemented for git)

Fix warnings about use of reserved internal symbol 'type'.

SourceFetcherForGit: use a bare repository

... instead of one without a checkout. This seems less weird and makes
it easier to manually update (at least for me). Of course, ideally
haikuporter would automatically recognize that it can just update the
repository instead of checking everything out again.

Actually make use of a specified rev when fetching via git.

Improve robustness of foldSubdirIntoSourceDir():

* rename the subdir to a (hopefully) unique name to avoid problems
with contained files/folder with an identical name (as is the case
for binutils)

SourceFetcherFor{Download,LocalFile}: fix unpack() with subdir

The assumption that the first subdir component is the source dir name
was incorrect as subdir is relative to the source directory.

Add license header (MIT)

* additionally: uppercase copyright notices and separate them from the
top by a single line

Squash a TODO that no longer applies.

SourceFetcherForCvs: quote module name

... in case it contains weird characters (like for keymapswitcher).

Generalize support for SOURCE_EXPORT_SUBDIR:

* add support for exporting only a subdir to unpackArchive()
* add automatic folding of subdir-contents into source directory,
after unpacking from archive and using tar for "unpacking" a checkout

Fix SourceFetcherForLocalFile:

* adjusting self.fetchTarget doesn't cut it, as Source has its own
version of it, which it is accessing - so now we simply symlink the
local file into the download directory

Refactor handling of checkouts (and somewhat downloads, too):

* Pull protocol specific SourceFetcher-classes out of Source.
* Do checkouts into download directory and then export the relevant
source revision into the source dir. This way, removing the work
dir no longer drops the precious checkouts and checkouts can be
shared between different port versions.
* Always create an implicit git repo in the source dir, no more need
for two code paths.
* Handle SOURCE_EXPORT_SUBDIR a bit more intelligently, such that only
the stuff living underneath the export subdir is put into the source
dir (i.e. without the export subdir itself as relative path)
* After a source has been fetched, write the URI that has been used
into a marker file, such that during following runs, HaikuPorter knows
that URI and can create an appropriate SourceFetcher object for it.