Import statements sorted with isort and fixed MockBuilder.py (#266)

* Fixed missing closing bracket in the MockBuilder.py

* Import statements sorted with isort

respect --quiet flag in repository commands (#246)

python3: Use input instead of raw_input

- Fixes https://github.com/haikuports/haikuporter/issues/215
- raw_input() has been deprecated and replaced with input()

python3: Fix missed itervalues -> values

python3: Finish python3 change. python2 breaking

* Based on mmlr's 2017 work to use Python3 with
changes and adjustments for the current codebase

python3: begin python3 fixes. print -> print()

Add a --check-ports-releases option to check for new upstream releases

This should help monitoring releases from upstream projects,
and make sure to test and update the recipes accordingly.

For now it only checks releases/tags at GitHub, maybe we should check
at repology.org instead.

Also it doesn't attempt to sort the tags, but only checks for the first one,
so in some cases it doesn't work (like with cronie-crond/cronie).

At least it works for qtkeychains which are a bit reluctant to merge Haiku
support in fear it might bitrot in the future.

Rename --list-build-dependencies option to --list-dependencies.

Print out test dependencies when combined with --test, otherwise print
build dependencies as before.

Main: Build unbuilt ports before testing.

Build the port automatically instead of just instructing the user to
do so before testing.

Introduce TEST_REQUIRES for test-only requires.

This lets recipes specify dependencies that are only needed for the test
phase. Separating them out from BUILD_[PRE]REQUIRES allows for
situations that could otherwise result in cyclic dependencies.

Note that this bumps the repository format version, causing a repository
rebuild (re-parsing of all recipes, flushing of all caches). This is
required because the dependency info files and various caches need to be
regenerated to cope with the added test requires keys.

Fixes #151.

Repository: Add --check-repository-consistency.

It dependency solves all active ports and prints out the problems found.

Cleanup: Remove unused imports.

Cleanup: Make some constructs more idiomatic.

* Change 'not x in y' to 'x not in y'.
* Use 'is' to compare None singleton.
* Use 'instanceof(x, y)' instead of 'type(x) == y'.
* Replace map builtin with list comprehension.
* Rename self argument to cls for static method.

Cleanup: Remove trailing semicolons and a unneeded pass.

Cleanup: Whitespace and line length cleanup.

Main: BuildMaster mode implies --all-dependencies.

Main: Only import DisplayContext when needed.

When used in a bootstrap environment, ncurses may not be available.

PackageRepository: Implement repository consistency checking.

It runs dependency resolution for each package and prints the problems
found.

Repository/Port: Force allow unstable if explicitly requested.

When an unstable port is explicitly requested, a prompt allows to
continue. Since the port is unstable, the actual generation of the
dependency info was still skipped however.

Force allowing unstable ports when an explicit version was requested.
Since the port packages are filtered by their status as well, also
force reparsing (and therefore recreating the package list) of the
port recipe.

Repository/Main: Rework handling of obsolete port building.

Repository: The method supportBackwardsCompatibility was a near 100%
duplicate of the original _updateRepository (with all of it's
shortcomings) except that it skipped everything that didn't match the
port/version parameters. Remove it completely and port over the explicit
port version selection into an optional parameter of _updateRepository.

Main: Factor out a _setupForPossiblyObsoletePort method and use it from
_buildMainPort and _buildPort. This ensures that the package redirection
happens in all cases and make the warning message reflect that
redirection.

This has the nice side effect that built dependencies that are actually
the latest version will not be put into the obsolete directory anymore
and therefore fixes the initial part of #22.

Also only call supportBackwardsCompatibility if the version actually
isn't the active one, since the operation is somewhat expensive and
produces otherwise confusing output.

BuildPlatformUnix: Add B_SYSTEM_PACKAGES_DIRECTORY override.

This allows --analyze-dependencies to work also on this platform when
given a --system-packages-directory.

Shallow init is also enough for --analyze-dependencies.

Main: Shallow init is enough for --why.

Port: Rework configuration of repository dir.

Use a class variable and a setter to make the repository dir known to
Port and add a context manager for temporarily adjusting the repository
dir of specific ports.

This removes the need to hand in the repository dir to various methods,
which makes things more consistent. Some methods were previously using
the repository dir given to the constructor while others needed it as
argument.

The Repository now makes itself known to Port and can change the current
repository dir for the purpose of populating a new repository at a
different path.

With this, the ugly manual adjustment of the recipe cache of the Port by
the Repository can be removed. It was directly changing internals of
Port but used slightly different logic than within Port which lead to
two different filenames being used for the recipe cache file. This
caused recipes to be parsed once unnecessarily.

Main: Fix typo: The printed list are build deps, not built ones.

Main: Fix wrong port being recorded as skipped in buildmaster.

Main: Exit buildmaster when instructed to build obsolete ports.

This can happen when dependencies require specific versions that are
not active.

For example this may happen when already present packages were built
against a version of a port that is later deactivated. In such a case,
these packages need to be rebuilt to not depend on the unavailable
port anymore. They would otherwise be uninstallable in any case.

Main: Shallow init is enough for --create-package-repository.

BuildMaster: Make skip reason include unresolved dependencies.

Also drop the verbose debug output of used repositories.

BuildMaster: Collect and expose skipped builds.

Builds that cannot be run due to some error condition are now displayed
in the frontend with the respective reason.

Main: Sort the package list output of --list-build-dependencies.

This makes it easier to read.

PackageRepository: Implement repository creation.

PackageRepository: Add class to manage the packages directory.

Initially it allows pruning packages, which does:

* Obsolete packages that are newer than the active port version, with
the assumption that the active version was downgraded.
* Obsolete packages that have newer version packages available.

This is slightly different from normal package obsoletion in that it
does keep older versions of packages as long as no newer ones are
available.

The use case for this is to prepare a packages directory to be usable
for repo creation and continuous updates.

Main: Fix explicit repository update to not expect search arg.

The repository update would still happen, but an error about the
expected search string was printed.

Add option --ports-for-packages to find port of resulting HPKG.

Add --purge option to clean the work dir and remove downloads.

Main: Print reason for build dependency resolution error.

BuildMaster: Fix TODO about hardcoded ports tree URL.

Determine the local ports tree origin and use that.

Main: Use both packagesPath and systemPackagesDirectory if set.

This allows to consider a separate set of packages in dependency
resolution and therefore avoids the need to put such dependencies into
the packagesPath.

BuildMaster: Use paths, not names, of present packages.

This allows to source them from multiple package directories.

Main: --list now also supports --print-filenames.

Easier to find the active recipe for a port.

Fix initial ports tree checkout

haikuporter: no-dependencies now default behaviour

* Inverse --all-dependencies added which will
build all outdated dependencies of a port.
* Your fingers will thank me.

Configuration: new optional REPOSITORY_PATH parameter (#115)

to allow relocation/specification of the repository directory.

While at it, drop the suffix ('-' + $targetArchitecture) that
was added to the default path for the repository directory in
the previous commit. The default path is therefore restored:
$OUTPUT_DIRECTORY/repository/

Reminder: the default OUTPUT_DIRECTORY is $TREE_PATH.

Add a console mode.

Implement local builders.

* provide info(), warn() to be used instead of print. By default outputs to stdout.
For builders, outputs to files.
* replace most calls to check_call by check_output, and log with info(). Also outputs
the outputs an eventual CalledProcessError output attribute.
* add option --local-builders to be used with --build-master: indicates how many
local builders to use instead of remote builders. If paramiko isn't available,
defaults to 1, otherwise 0.
* rename Builder to RemoteBuilder.
* remove changes of current directory: this doesn't work in a
multithreaded environnement. Policy.py is the most affected as it
expected to run in the packaging directory.

Use Unicode strings for warn() & sysExit().

Avoid using git when noGitRepo option is given. (#100)

Main: Replace another instance of str.lower with unicode.lower.

Another instance of #82. I didn't test this very well, I'm mostly going off of @ed-dismantle's comment that this is the right thing to do.

Fix error where --get doesn't work

Fix support for building older versions of packages

Add option to explicitly request repository update only.

Add option to print ports affected by file list.

It checks all [active] ports and prints those that reference at least
one of the supplied files.

Source: Rename and generalize additional file extraction.

This is now used to populate the work dir with referenced additional
files in additon to populating them in the source package.

Additional files are now populated into the chroot at
/additional-files[-<index>] analogous to source directories.

Main: Allow non-fatal validation of ports in build master mode.

Return a result instead of prompting/exiting when building obsolete
packages. Ports that do not validate are then skipped.

Add option --ignore-messages to avoid message prompts.

This option is also implied by build master mode.

Add --print-filenames option to resolve package names.

Package listings and searches then resolve and return the package file
names instead of the package names. Note that this is horribly
inefficient due to the way the current version of packages is currently
determined.

Add option to suppress obsoletion of packages.

While the dependency info should be purged when a new version becomes
available, the packages should stay available in buildmaster mode and
on builders where no system dependencies may be used.

Main: Make build failure more expressive.

Also catch some of the failures in buildmaster mode and make them
non-fatal. As there may be a whole list of other ports to build it
doesn't make sense to abort early.

Main: Make non-buildability non-fatal in buildmaster mode.

Add --print-raw option to make some outputs machine readable.

Main: Apply search to all supplied args.

BuildMaster: Sync builder portstree with the one of the master.

Add BuildMaster mode and class.

Instead of building itself it distributes builds to builders via SSH
and coordinates pushing the dependencies around as needed.

Add option --list-build-dependencies to list all dependencies.

This includes the list of already present packages as well as the list
of packages that need to be built and their corresponding port.

Allow shallow init for the about option.

Add option --no-system-packages to constrain used packages.

If supplied the system packages directory will not be added to the
list of package repositories used to resolve dependencies.

This means that only local packages from the packages path will be used
for the search of all dependency packages, including the search for the
main Haiku package itself. It therefore requires one to provide all
needed packages as part of the local packages but allows for more
control over what exact packages are visible.

Update documentation to refer to git instead of svn.

Allow specifying multiple ports to operate on.

All remaining arguments are treated as port specifications.

Add options to search/list packages in addition to ports.

Configuration: Allow relocation/specification of packages path.

Main: Fix printing output of SystemExit when "code" is not an "int".

When linting recipes, exit with 1 if one or more of the lints failed.

This also fixes the main exception handler to not print the exception
if the exception is a "SystemExit".

Add --get-dependencies option to haikuporter.

This will automatically search for packages with pkgman,
which get installed to your system. Use with caution!

Using --get-dependencies automatically sets --no-dependencies,
so you'll still need to manually build packages which aren't
available using pkgman.

Fix #56: --lint not working on non-Haiku platform.

* Suppress platform check for all options that only require a shallow
initialization.

Drop support for $haikuVersion.

* Referring the current haiku version explicitly is not needed, since
the RequiresUpdater takes care of setting the version of Haiku used
for building a package.

Fix #43: output for --location is missing the category.

* Additionally, improve output for --list (sorted) and --search (show
category), too.

Minor optimization: use set instead of list.

Cleanup imports.

Refactoring: use properties where it seems fit.

* Trying to be somewhat more Pythonic.

Revert to tab spacing for Pe's sake.

Try again with whitespace.

Second attempt to fix whitespace. Options are different on OS X.

Convert tabs to spaces.

Minor improvements of output, suggested by Diver.

* Show full recipe path in output header for a specific port.
* Show full path of resulting packages in output footer.

Automatic whitespace cleanup - no functional change intended.

Simplify conversion from package-name/ID to port-name/ID.

* adjust getPortIdForPackageId() and getPortNameForPackageName()
to check if the given parameter is already valid, simplifying
the loop, too
* drop explicit checks before invocation of getPortIdForPackageId()

Minor cleanup: avoid use of os._exit().

* os._exit() doesn't invoked cleanup handlers, so it should be avoided.
* Additionally, replace invocations of sys.exit() by return as the
former doesn't seem to yield a exit code of 0.

Only require LICENSES_DIRECTORY when it is needed.

* The only informational command that needs it is --lint.

lint: Accept full recipe name / version, or just name

* If user specifies the full recipe name / version, we
return a proper return code to detect issues.

Allow informational commands (e.g. --lint) to run on Linux.

* Introduce support for shallow initialization to Main and
BuildPlatform classes and make use of that for the informational
commands.

Introduce explicit recipe format version.

* Added support for reading the recipe format version used by a ports
tree from the file 'FormatVersions'.
* Added check that ensures that the recipe format version of a given
ports tree matches the one supported by haikuporter.

lint: Language cleanup

lint: Allow users to scan port names

* You can now specify a port name with --lint
to lint scan a single port.
* You have to pass the basename for this to work
(you can't pass recipe versions yet)
* Example: ./haikuporter --lint llvm
* --lint with no package name results in a full
tree scan as before.
* Improvements welcome

Revert "Allow fixing of the port versions put into the repository."

* due to to way the build system works, this is an incomplete
solution to the problem, so it makes no sense

This reverts commit 939d630b79c96c00fe3adb5791ff2be252ec5d22.

Allow fixing of the port versions put into the repository.

* explicit port versions specified via a portsfile are now respected
when the repository is being populated

Add Verbose option

Adjust behaviour of --clean to only clean, but not build.

* --clean now only removes the work-dir of the port(s) given via cmdline,
none of them will be built

Refactor setup of chroot and reactivate port testing

* refactor setup of chroot such that it can be used for more than building
* re-add support for testing a port (--test)

Add --debug option for showing Python stacktraces on errors.

* by default, Python stack traces are no longer printed, use --debug when
you want to see them

Fix use of non-existing function in implementation of --why

Print summary for policy violations when done.

During bootstrap, skip all existing packages.

* before, only existing packages with cyclic dependencies were
being skipped, which doesn't make much sense

Extend --do-bootstrap to build all available ports.

* after building ports with circular dependencies, --do-bootstrap will
trigger a build of all available ports, too
* additionally: take notice of the ports built in a haikuporter session
and avoid building them again

Fix error when using --lint.

Improve implementation of --why option.

* no longer require full port specs (name+version) as argument to --why,
but use the active version if only a name is given
* instead of showing a (confusing) error message, just show the output
from pkgman as to what problems are caused when the given port is
not available

Add option --create-source-packages

It builds (only) the regular source packages.

In --do-bootstrap mode skip already built ports

Add option --no-dependencies.

* --no-dependencies will tell haikuporter to ignore any dependencies
and just build the port(s) given

Add option --do-bootstrap

* --do-bootstrap takes care of building the ports with cyclic
dependencies in a working order

Restructure DependencyAnalyzer a bit.

* let __init__ only compute the different groups of nodes and print
them in printDependencies()
* keep full set of ports with cyclic dependencies (i.e. do not leave
out ports required by HaikuPorter directly)

Adjust creation and handling of source packages
* rename option 'only-source-packages' to
'create-source-packages-for-bootstrap', as it does a bit more than
just limit to creation of source packages
* rename option 'nosrcpackage' to 'no-source-packages' and adjust the
behaviour to never create source packages in this case (instead of
only avoiding default source packages)
* source packages that are being created for the bootstrap image now
contain the source with all patches applied (as git isn't available
in the bootstrap image), standard source packages contain pristine
sources

Don't try to build source package for ports that don't have one

Also allow specifying a package name/ID on the command line

Implement strict separation of buildhost and target packages.

* REQUIRES and BUILD_REQUIRES refer to target architecture packages,
while BUILD_PREREQUIRES refers to buildhost architecture packages
* added support for building prerequired packages when needed
* add a separate set of implicit target packages to BuildPlatformUnix,
which only contains the 'haiku' package

Minor cleanup.

Whitespace cleanup

Add support for meta-ports, specifically 'meta_portsfile'

* a meta-port is a port that just builds other (non-meta) ports, but it
doesn't provide any sources or build actions of its own
* when building a list of ports specified via --portsfile, the meta-
port 'meta_portsfile' is now being used to group all the required
ports (pulling in any runtime requirements, too)
* when building only source packages, consider all packages stable, as
otherwise sources for not-yet-stable packages could not be collected
(one would have to mark them as stable manually)
* add more missing implicit build provides to Unix build platform

Cleanup: reduce code duplication a bit.

* add Port.isBuildableOnTargetArchitecture() as well as
Package.isBuildableOnArchitecture() and make use of them
* introduce Repository.getActiveVersionOf() for determining
the currently active version of a port in the repository

More work on building only source packages.

* building source packages for a list of ports is now basically
possible, but the current limits of dependency resolution have
the effect that not all required ports are being pulled in

Fix order when searching for a buildable version.

Fix import-related warnings and errors.

Add support for separate output directory

* Add configuration file value OUTPUT_DIRECTORY specifying the directory
where haikuporter shall write files. It mirrors the ports repository
directory layout. The default value is the repository path.
* Add configuration file value DOWNLOAD_IN_PORT_DIRECTORY. Only
meaningful, if OUTPUT_DIRECTORY is specified. In that case a value of
"yes" will still cause the download directory to be placed in the port
directory. This may be useful when building for different target
architectures in separate output directories -- the downloads can thus
be shared.

Unifiy the Options and GlobalConfig somewhat

* Module GlobalConfig is replaced with new module Configuration. The
global variable globalConfiguration is gone. Instead Configuration
provides getter methods.
* For several of the (newer) command line options there are now optional
configuration file entries. If given the command line values override
the configuration file values. Configuration also provides getter
methods for these options, which should be used instead of
getOption().

Clean up trailing whitespace

Fix crossSysrootDir for non-Haiku build platforms

* BuildPlatform: Determine target architecture and cross sysroot dir.
Add respective getTargetArchitecture() and getCrossSysrootDirectory()
method.
* BuildPlatformUnix: Make getInstallDestDir() synonymous with
getCrossSysrootDirectory().
* Set the crossSysrootDir shell variable in
Port._updateShellVariables() instead of in Main. This way it gets
adjusted depending on whether we're in a chroot or not.

BuildPlatform*: move more initialization to init()

Start with a build platform abstraction

* Add '--cross-devel-package' option to specify the path to the cross
development package. Can be used for cross-compiling.
* Add module BuildPlatform which exports a buildPlatform variable, which
is initialized to an instance of BuildPlatformHaiku or
BuildPlatformUnix as appropriate.
* Move findDirectory() to BuildPlatformHaiku. BuildPlatformUnix has a
version with some hard-coded mappings.
* Add PackageInfo class. Very simple ATM. It only extracts the version
and the architecture from a package file.
* Rework Main._initGlobalShellVariables() somewhat: Extracting the Haiku
version from the build platform's system haiku package is obviously
not correct when cross-building. Also add support for non-Haiku build
platforms.
* Remove no longer needed MachineArchitecture.getBuildTripleFor().

Improve handling of multiple available versions of a port:

* instead of always using the port with the highest version, now we
explicitly search for the newest version of a port that is buildable
on the requested target architecture

Minor cleanup:

* rename Port.getStatusOnCurrentArchitecture to
Port.getStatusOnTargetArchitecture() for clarification

Fix --location, --search and --about.

* additionally: make use of the repository info when searching/locating
ports

Add license header (MIT)

* additionally: uppercase copyright notices and separate them from the
top by a single line

Fix contents of source packages.

* instead of putting the patched sources into the source packages, we
now export the unchanged sources
* instead of adding only the patches that were used for the build, we
now export every patch that could be relevant, i.e. patches for other
architectures, too
* Re-add lost ReadMe and extend it to include the haikuports-URL, the
haikuports-revision that was used and the haikuporter-URL
* move haikuportsRepoUrl from Main to Utils
* add haikuporterRepoUrl to Utils

Always use faked build machine if build-arch matches target-arch.

* we need to always use e.g. x86_gcc2_build as build machine triple
when building for x86_gcc2, as otherwise it wouldn't be a cross build.

Cleanup: unused imports, let classes inherit from object by default.

Drop HOST_ARCHITECTURE from general settings.

* Building for a host that's different from the build machine doesn't
seem to work reliably after all. Since it wouldn't be helping us much
anyway, we drop that idea.

Rename cross-build sysroot folder and fix contents.

Add support for cross-sysroot in /boot/cross/<target-arch>

* pass environment variables isCrossRepository and targetArchitecture
to scriptlets that set up / tear down the chroot
* extend chroot-setup scriptlet to symlink the sysroot package into
a separate packages folder under /boot/cross/<target-arch> and
mount that folder
* extend chroot-tear-down scriptlet to unmount /boot/cross/<target-arch>

Define explicit support for build, host and target architecture.

* added global configuration option HOST_ARCHITECTURE which defines the
architecture the resulting packages will be running on
* instead of TARGET_ARCHITECTURE, HOST_ARCHITECTURE has to be specified
for the cross-build repsository, TARGET_ARCHITECTURE needs only to be
set when it is different (e.g. when building gcc4 packages for a
x86_gcc2 Haiku)
* drop option '--target-arch', it doesn't make sense to switch the
target architecture on-the-fly, as the repository needs to be
repopulated after switching the architecture
* added boolean recipe type (YesNo)
* added global configuration option ALLOW_UNTESTED
* implement support for automatically accepting allowing untested ports
(via ALLOW_UNTESTED)

Add option -D for analyzing port dependencies

Currently it only prints the required system packages, the
self-depending ports, and the ports with cyclic dependencies.

Main -> Repository: Move shutil import

Cleanup imports

* drop unneded imports
* add missing import to Repository

Move repository code from Main to new class Repository

For cross-build repo, try to find target-arch specific port

* instead of having to give the target architecture explicitly as part
of the port name, one can now just give the base name (e.g.
'gcc_cross') and haikuporter will automatically append the current
target architecture and try to find the corresponding port this way.

Beginning of packaging policy checking

* Add class Policy which contains the checks. Currently only a few.
* Add command line option -S/--strict-policy to enable strict policy
checking. Instead of printing warnings, the packaging will be
aborted.

Implement improved patch handling via implicit git repository

* unless the port already uses a git repository, automatically create
an empty git repository and import the sources of a port into it
* when applying patches, support both application of individual patches
(old style) and git patchsets
* automatically apply changes introduced by the patch function as
separate commit
* add new option --extract-patchset which can be used to (re-)create
a patchset (git format-patch) for a port from its git repo, taking
care to temporarily filter out the changes done by the patch function
* add new option --init-git that can be used to force importing of the
source into a new git repository (that's only done automatically when
there are already patches available for a port)

When no version has been given for a port, automatically pick the
most recent one in the repository.

Pass on target architecture as environment variable

Work on support for cross-building

* add detection of cross-build repository (via file .cross in root)
* add support for specifying target architecture via --target-arch
or via TARGET_ARCHITECTURE in haikuports.conf
* pull MachineArchitecture out of Architectures.

Fix problems when a port depends on a sub-package of another port.

Add new option --why

* this can be used to ask haikuporter which port pulls in the port
given via --why as a dependency of the main port

Print category of dependent ports, too.

Implement support for building a list of ports read from a file.

Implement support for more than one source per port.

* pull new class Source out of Port, which encapsulates download,
checksum-validation, checkout, unpacking and patching of a source
* adjust (and simplify) handling of source

Add shell variable jobArgs

There's already jobs, but that isn't that handy to use. jobArgs is
undefined in case of one job and defined to "-j<jobs>" otherwise.

Switch checkout of haikuports-tree to git

Make sure that touching a file really bumps the modification date.

Fix cleanup of broken recipes.

* when a recipe used to work, but is broken now, remove corresponding
package-info and obsolete any corresponding packages, too

Fix "-b" option

It wouldn't do anything.

Automatically unset the build flag when recipe is newer

Whitespace-related cleanup

Treat building of obsolete packages more explicitly

* give warning when an obsolete package is being built and put the
resulting package(s) into the '.obsolete'-subfolder of the packages
directory

Improve repository management.

* consistently maintain only a single version of a port in the
repository (the one with the highest version number that can be
parsed and is stable)
* automatically adjust the port version in the repository to any
recipe changes/additions/removals

Unbreak automatic detection of dependencies to update first

* the last commit changed the folder where dependent packages will
be found, so the corresponding filter has to be adjusted, too
* drop debug output

Implement automatic selection of newest version of each port

* this may have to be changed later, but it should for now avoid the
need to declare recipes as 'untested' on all architectures to avoid
them being added to the repository

Update regexps for port name and version

Also change error output in _getAllPorts() from "Error:..." to
"Warning:...". There are quite a few recipe names that don't match
anymore.

Avoid show duplicate name during repository update

* as there can be more than a single package per port now, we better
show the packageID instead of the portID

* improved indication of progress when populating the repository

Switch name-internal separator from '-' (dash) to '_' (underline), as otherwise the package naming scheme exhibits potential collisions for parsers

* pulled 'Package' out of 'Port' and adjusted everything to support creating multiple
packages from a single recipe file
Not many new features have been implemented yet, but building packages seems to work again.

* remove reference to no longer existing option

Fix the fix: restore missing svnPath, too

* restore implementation for --get (fetching/updating of haikuports-tree)
* drop --archive option, as we will provide source packages for that purpose

Refactoring: rip haikuporter apart into separate modules

Add --get-dependencies option to haikuporter.

This will automatically search for packages with pkgman,
which get installed to your system. Use with caution!

Using --get-dependencies automatically sets --no-dependencies,
so you'll still need to manually build packages which aren't
available using pkgman.

Fix #56: --lint not working on non-Haiku platform.

* Suppress platform check for all options that only require a shallow
initialization.

Drop support for $haikuVersion.

* Referring the current haiku version explicitly is not needed, since
the RequiresUpdater takes care of setting the version of Haiku used
for building a package.

Fix #43: output for --location is missing the category.

* Additionally, improve output for --list (sorted) and --search (show
category), too.

Minor optimization: use set instead of list.

Cleanup imports.

Refactoring: use properties where it seems fit.

* Trying to be somewhat more Pythonic.

Revert to tab spacing for Pe's sake.

Try again with whitespace.

Second attempt to fix whitespace. Options are different on OS X.

Convert tabs to spaces.

Minor improvements of output, suggested by Diver.

* Show full recipe path in output header for a specific port.
* Show full path of resulting packages in output footer.

Automatic whitespace cleanup - no functional change intended.

Simplify conversion from package-name/ID to port-name/ID.

* adjust getPortIdForPackageId() and getPortNameForPackageName()
to check if the given parameter is already valid, simplifying
the loop, too
* drop explicit checks before invocation of getPortIdForPackageId()

Minor cleanup: avoid use of os._exit().

* os._exit() doesn't invoked cleanup handlers, so it should be avoided.
* Additionally, replace invocations of sys.exit() by return as the
former doesn't seem to yield a exit code of 0.

Only require LICENSES_DIRECTORY when it is needed.

* The only informational command that needs it is --lint.

lint: Accept full recipe name / version, or just name

* If user specifies the full recipe name / version, we
return a proper return code to detect issues.

Allow informational commands (e.g. --lint) to run on Linux.

* Introduce support for shallow initialization to Main and
BuildPlatform classes and make use of that for the informational
commands.

Introduce explicit recipe format version.

* Added support for reading the recipe format version used by a ports
tree from the file 'FormatVersions'.
* Added check that ensures that the recipe format version of a given
ports tree matches the one supported by haikuporter.

lint: Language cleanup

lint: Allow users to scan port names

* You can now specify a port name with --lint
to lint scan a single port.
* You have to pass the basename for this to work
(you can't pass recipe versions yet)
* Example: ./haikuporter --lint llvm
* --lint with no package name results in a full
tree scan as before.
* Improvements welcome

Revert "Allow fixing of the port versions put into the repository."

* due to to way the build system works, this is an incomplete
solution to the problem, so it makes no sense

This reverts commit 939d630b79c96c00fe3adb5791ff2be252ec5d22.

Allow fixing of the port versions put into the repository.

* explicit port versions specified via a portsfile are now respected
when the repository is being populated

Add Verbose option

Adjust behaviour of --clean to only clean, but not build.

* --clean now only removes the work-dir of the port(s) given via cmdline,
none of them will be built

Refactor setup of chroot and reactivate port testing

* refactor setup of chroot such that it can be used for more than building
* re-add support for testing a port (--test)

Add --debug option for showing Python stacktraces on errors.

* by default, Python stack traces are no longer printed, use --debug when
you want to see them

Fix use of non-existing function in implementation of --why

Print summary for policy violations when done.

During bootstrap, skip all existing packages.

* before, only existing packages with cyclic dependencies were
being skipped, which doesn't make much sense

Extend --do-bootstrap to build all available ports.

* after building ports with circular dependencies, --do-bootstrap will
trigger a build of all available ports, too
* additionally: take notice of the ports built in a haikuporter session
and avoid building them again

Fix error when using --lint.

Improve implementation of --why option.

* no longer require full port specs (name+version) as argument to --why,
but use the active version if only a name is given
* instead of showing a (confusing) error message, just show the output
from pkgman as to what problems are caused when the given port is
not available

Add option --create-source-packages

It builds (only) the regular source packages.

In --do-bootstrap mode skip already built ports

Add option --no-dependencies.

* --no-dependencies will tell haikuporter to ignore any dependencies
and just build the port(s) given

Add option --do-bootstrap

* --do-bootstrap takes care of building the ports with cyclic
dependencies in a working order

Restructure DependencyAnalyzer a bit.

* let __init__ only compute the different groups of nodes and print
them in printDependencies()
* keep full set of ports with cyclic dependencies (i.e. do not leave
out ports required by HaikuPorter directly)

Adjust creation and handling of source packages
* rename option 'only-source-packages' to
'create-source-packages-for-bootstrap', as it does a bit more than
just limit to creation of source packages
* rename option 'nosrcpackage' to 'no-source-packages' and adjust the
behaviour to never create source packages in this case (instead of
only avoiding default source packages)
* source packages that are being created for the bootstrap image now
contain the source with all patches applied (as git isn't available
in the bootstrap image), standard source packages contain pristine
sources

Don't try to build source package for ports that don't have one

Also allow specifying a package name/ID on the command line

Implement strict separation of buildhost and target packages.

* REQUIRES and BUILD_REQUIRES refer to target architecture packages,
while BUILD_PREREQUIRES refers to buildhost architecture packages
* added support for building prerequired packages when needed
* add a separate set of implicit target packages to BuildPlatformUnix,
which only contains the 'haiku' package

Minor cleanup.

Whitespace cleanup

Add support for meta-ports, specifically 'meta_portsfile'

* a meta-port is a port that just builds other (non-meta) ports, but it
doesn't provide any sources or build actions of its own
* when building a list of ports specified via --portsfile, the meta-
port 'meta_portsfile' is now being used to group all the required
ports (pulling in any runtime requirements, too)
* when building only source packages, consider all packages stable, as
otherwise sources for not-yet-stable packages could not be collected
(one would have to mark them as stable manually)
* add more missing implicit build provides to Unix build platform

Cleanup: reduce code duplication a bit.

* add Port.isBuildableOnTargetArchitecture() as well as
Package.isBuildableOnArchitecture() and make use of them
* introduce Repository.getActiveVersionOf() for determining
the currently active version of a port in the repository

More work on building only source packages.

* building source packages for a list of ports is now basically
possible, but the current limits of dependency resolution have
the effect that not all required ports are being pulled in

Fix order when searching for a buildable version.

Fix import-related warnings and errors.

Add support for separate output directory

* Add configuration file value OUTPUT_DIRECTORY specifying the directory
where haikuporter shall write files. It mirrors the ports repository
directory layout. The default value is the repository path.
* Add configuration file value DOWNLOAD_IN_PORT_DIRECTORY. Only
meaningful, if OUTPUT_DIRECTORY is specified. In that case a value of
"yes" will still cause the download directory to be placed in the port
directory. This may be useful when building for different target
architectures in separate output directories -- the downloads can thus
be shared.

Unifiy the Options and GlobalConfig somewhat

* Module GlobalConfig is replaced with new module Configuration. The
global variable globalConfiguration is gone. Instead Configuration
provides getter methods.
* For several of the (newer) command line options there are now optional
configuration file entries. If given the command line values override
the configuration file values. Configuration also provides getter
methods for these options, which should be used instead of
getOption().

Clean up trailing whitespace

Fix crossSysrootDir for non-Haiku build platforms

* BuildPlatform: Determine target architecture and cross sysroot dir.
Add respective getTargetArchitecture() and getCrossSysrootDirectory()
method.
* BuildPlatformUnix: Make getInstallDestDir() synonymous with
getCrossSysrootDirectory().
* Set the crossSysrootDir shell variable in
Port._updateShellVariables() instead of in Main. This way it gets
adjusted depending on whether we're in a chroot or not.

BuildPlatform*: move more initialization to init()

Start with a build platform abstraction

* Add '--cross-devel-package' option to specify the path to the cross
development package. Can be used for cross-compiling.
* Add module BuildPlatform which exports a buildPlatform variable, which
is initialized to an instance of BuildPlatformHaiku or
BuildPlatformUnix as appropriate.
* Move findDirectory() to BuildPlatformHaiku. BuildPlatformUnix has a
version with some hard-coded mappings.
* Add PackageInfo class. Very simple ATM. It only extracts the version
and the architecture from a package file.
* Rework Main._initGlobalShellVariables() somewhat: Extracting the Haiku
version from the build platform's system haiku package is obviously
not correct when cross-building. Also add support for non-Haiku build
platforms.
* Remove no longer needed MachineArchitecture.getBuildTripleFor().

Improve handling of multiple available versions of a port:

* instead of always using the port with the highest version, now we
explicitly search for the newest version of a port that is buildable
on the requested target architecture

Minor cleanup:

* rename Port.getStatusOnCurrentArchitecture to
Port.getStatusOnTargetArchitecture() for clarification

Fix --location, --search and --about.

* additionally: make use of the repository info when searching/locating
ports

Add license header (MIT)

* additionally: uppercase copyright notices and separate them from the
top by a single line

Fix contents of source packages.

* instead of putting the patched sources into the source packages, we
now export the unchanged sources
* instead of adding only the patches that were used for the build, we
now export every patch that could be relevant, i.e. patches for other
architectures, too
* Re-add lost ReadMe and extend it to include the haikuports-URL, the
haikuports-revision that was used and the haikuporter-URL
* move haikuportsRepoUrl from Main to Utils
* add haikuporterRepoUrl to Utils

Always use faked build machine if build-arch matches target-arch.

* we need to always use e.g. x86_gcc2_build as build machine triple
when building for x86_gcc2, as otherwise it wouldn't be a cross build.

Cleanup: unused imports, let classes inherit from object by default.

Drop HOST_ARCHITECTURE from general settings.

* Building for a host that's different from the build machine doesn't
seem to work reliably after all. Since it wouldn't be helping us much
anyway, we drop that idea.

Rename cross-build sysroot folder and fix contents.

Add support for cross-sysroot in /boot/cross/<target-arch>

* pass environment variables isCrossRepository and targetArchitecture
to scriptlets that set up / tear down the chroot
* extend chroot-setup scriptlet to symlink the sysroot package into
a separate packages folder under /boot/cross/<target-arch> and
mount that folder
* extend chroot-tear-down scriptlet to unmount /boot/cross/<target-arch>

Define explicit support for build, host and target architecture.

* added global configuration option HOST_ARCHITECTURE which defines the
architecture the resulting packages will be running on
* instead of TARGET_ARCHITECTURE, HOST_ARCHITECTURE has to be specified
for the cross-build repsository, TARGET_ARCHITECTURE needs only to be
set when it is different (e.g. when building gcc4 packages for a
x86_gcc2 Haiku)
* drop option '--target-arch', it doesn't make sense to switch the
target architecture on-the-fly, as the repository needs to be
repopulated after switching the architecture
* added boolean recipe type (YesNo)
* added global configuration option ALLOW_UNTESTED
* implement support for automatically accepting allowing untested ports
(via ALLOW_UNTESTED)

Add option -D for analyzing port dependencies

Currently it only prints the required system packages, the
self-depending ports, and the ports with cyclic dependencies.

Main -> Repository: Move shutil import

Cleanup imports

* drop unneded imports
* add missing import to Repository

Move repository code from Main to new class Repository

For cross-build repo, try to find target-arch specific port

* instead of having to give the target architecture explicitly as part
of the port name, one can now just give the base name (e.g.
'gcc_cross') and haikuporter will automatically append the current
target architecture and try to find the corresponding port this way.

Beginning of packaging policy checking

* Add class Policy which contains the checks. Currently only a few.
* Add command line option -S/--strict-policy to enable strict policy
checking. Instead of printing warnings, the packaging will be
aborted.

Implement improved patch handling via implicit git repository

* unless the port already uses a git repository, automatically create
an empty git repository and import the sources of a port into it
* when applying patches, support both application of individual patches
(old style) and git patchsets
* automatically apply changes introduced by the patch function as
separate commit
* add new option --extract-patchset which can be used to (re-)create
a patchset (git format-patch) for a port from its git repo, taking
care to temporarily filter out the changes done by the patch function
* add new option --init-git that can be used to force importing of the
source into a new git repository (that's only done automatically when
there are already patches available for a port)

When no version has been given for a port, automatically pick the
most recent one in the repository.

Pass on target architecture as environment variable

Work on support for cross-building

* add detection of cross-build repository (via file .cross in root)
* add support for specifying target architecture via --target-arch
or via TARGET_ARCHITECTURE in haikuports.conf
* pull MachineArchitecture out of Architectures.

Fix problems when a port depends on a sub-package of another port.

Add new option --why

* this can be used to ask haikuporter which port pulls in the port
given via --why as a dependency of the main port

Print category of dependent ports, too.

Implement support for building a list of ports read from a file.

Implement support for more than one source per port.

* pull new class Source out of Port, which encapsulates download,
checksum-validation, checkout, unpacking and patching of a source
* adjust (and simplify) handling of source

Add shell variable jobArgs

There's already jobs, but that isn't that handy to use. jobArgs is
undefined in case of one job and defined to "-j<jobs>" otherwise.

Switch checkout of haikuports-tree to git

Make sure that touching a file really bumps the modification date.

Fix cleanup of broken recipes.

* when a recipe used to work, but is broken now, remove corresponding
package-info and obsolete any corresponding packages, too

Fix "-b" option

It wouldn't do anything.

Automatically unset the build flag when recipe is newer

Whitespace-related cleanup

Treat building of obsolete packages more explicitly

* give warning when an obsolete package is being built and put the
resulting package(s) into the '.obsolete'-subfolder of the packages
directory

Improve repository management.

* consistently maintain only a single version of a port in the
repository (the one with the highest version number that can be
parsed and is stable)
* automatically adjust the port version in the repository to any
recipe changes/additions/removals

Unbreak automatic detection of dependencies to update first

* the last commit changed the folder where dependent packages will
be found, so the corresponding filter has to be adjusted, too
* drop debug output

Implement automatic selection of newest version of each port

* this may have to be changed later, but it should for now avoid the
need to declare recipes as 'untested' on all architectures to avoid
them being added to the repository

Update regexps for port name and version

Also change error output in _getAllPorts() from "Error:..." to
"Warning:...". There are quite a few recipe names that don't match
anymore.

Avoid show duplicate name during repository update

* as there can be more than a single package per port now, we better
show the packageID instead of the portID

* improved indication of progress when populating the repository

Switch name-internal separator from '-' (dash) to '_' (underline), as otherwise the package naming scheme exhibits potential collisions for parsers

* pulled 'Package' out of 'Port' and adjusted everything to support creating multiple
packages from a single recipe file
Not many new features have been implemented yet, but building packages seems to work again.

* remove reference to no longer existing option

Fix the fix: restore missing svnPath, too

* restore implementation for --get (fetching/updating of haikuports-tree)
* drop --archive option, as we will provide source packages for that purpose

Refactoring: rip haikuporter apart into separate modules