Fix #256. (#275)

Import statements sorted with isort and fixed MockBuilder.py (#266)

* Fixed missing closing bracket in the MockBuilder.py

* Import statements sorted with isort

python3: Finish python3 change. python2 breaking

* Based on mmlr's 2017 work to use Python3 with
changes and adjustments for the current codebase

DependencyResolver: remove whitespace before passing to pkgman (#191)

pkgman doesn't seem to be able to process this: 'cmd:lrelease >= 5', but
it can if you remove all the spaces: 'cmd:lrelease>=5'.

This appears to be a bug in our outdated `libsolv`, which probably won't
be updated anytime soon.

Fixes #188

DependencyResolver: Use stopAtHpkgs for --update-dependencies.

The existing mechanism to stop propagation across HPKGs works for the
--update-dependencies case as well and removes the need for the custom
logic in the DependencyResolver.

Introduce TEST_REQUIRES for test-only requires.

This lets recipes specify dependencies that are only needed for the test
phase. Separating them out from BUILD_[PRE]REQUIRES allows for
situations that could otherwise result in cyclic dependencies.

Note that this bumps the repository format version, causing a repository
rebuild (re-parsing of all recipes, flushing of all caches). This is
required because the dependency info files and various caches need to be
regenerated to cope with the added test requires keys.

Fixes #151.

DependencyResolver: Avoid needless creation of PackageNodes.

These duplicate PackageNodes were just created to be converted to a
string. Use the list of pending nodes instead, which contains the same
info.

DependencyResolver: Cache successful dependency resolution.

Dramatically cuts down the overhead when doing multiple dependency
resolutions as common dependencies don't need to be fully resolved each
time. This impacts consistency checks (package repository and
repository) that now run in a fraction of the time.

DependencyResolver: Move updateDependencies evaluation to init.

DependencyResolver: Fix missing superclass init in restart.

Cleanup: Make some constructs more idiomatic.

* Change 'not x in y' to 'x not in y'.
* Use 'is' to compare None singleton.
* Use 'instanceof(x, y)' instead of 'type(x) == y'.
* Replace map builtin with list comprehension.
* Rename self argument to cls for static method.

DependencyResolver: Add quiet keyword arg to prevent printing.

DependencyResolver: Make build platform optional.

It is only used to get the implicit provides of the platform.

DependencyResolver: Factor out population of ProvidesManager.

This will allow reuse of the DependencyResolver with less overhead.

DependencyResolver: Try harder to resolve dependencies.

When a dependency of a package can't be met, remove the provides of that
package from the ProvidesManager and restart the dependency resolution.

This allows to still find valid solutions where dependencies can be met
by multiple sources of which earlier ones fail to have their requires
fulfilled.

For example:

program.recipe requires lib:first
libfirst-1.0.hpkg provides lib:first requires lib:second >= 1
libfirst-1.1.recipe provides lib:first requires lib:second >= 2
libsecond-2.0.hpkg provides lib:second == 2 compat >= 2

In the case above, the dependency lib:second of libfirst-1.0.hpkg
can't be resolved, because libsecond-2.0.hpkg provides an incompatible
version. However, the recipe for libfirst-1.1 would build a compatible
package and should therefore be used instead. Before this change,
dependency resolution would fail at the former. Now it will remove
the source of the provides that can't have its dependencies met,
libfirst-1.0.hpkg in the example above, and restart the dependency
resolution so that libfirst-1.1.recipe will be picked up and built.

The dependency resolution failures are collected and a combined message
including all of them is presented if the dependency resolution
ultimately still fails. This has the nice effect of showing the whole
dependency tree that failed, not just the last leave, which should make
it more obvious why a certain, possibly indirect, unresolved dependency
was pulled in in the first place.

BuildMaster: Make skip reason include unresolved dependencies.

Also drop the verbose debug output of used repositories.

DependencyResolver: Make HPKG parse errors non-fatal.

These come from various repositories and may contain broken package
files. Warn instead of exiting in such cases. Still treat errors in
other types of PackageInfo sources as fatal as these are under direct
management of HaikuPorter and errors there should be highlighted.

Use a more precise regexp to parse pkgman output.

Matching anything that ends with ".hpkg" is a bad idea when the URL has
"eu.hpkg" in it. Now we use greedy matching and a capturing group to
make sure only the last ".hpkg" in an URL is found.

Add option --update-dependencies

Use Unicode strings for warn() & sysExit().

DependencySolver: Add way to track present dependency packages.

When a list for tracking present dependency packages is supplied, all
packages that are directly or indirectly required and already present
will be collected into that list.

DependencyResolver: Cache all package infos.

The package infos from HPKGs were already cached at a deeper level, but
the DependencyInfo based ones were not. This lead to a lot of
unnecessary file reading/parsing. Instead of caching them at the
PackageInfo level, do that directly in the DependencyResolver.

ShellScriptlets: Internalize parsing of scriptlet prerequires.

Reduces duplication and knowledge of the original format.

Add --get-dependencies option to haikuporter.

This will automatically search for packages with pkgman,
which get installed to your system. Use with caution!

Using --get-dependencies automatically sets --no-dependencies,
so you'll still need to manually build packages which aren't
available using pkgman.

Handle scriptlet-prerequires when resolving dependencies, too.

* That part apparently has been lost during the rewrite of the resolver.

Add new DependencyResolver.

* Instead of delegating the dependency resolution to pkgman, this class
implements a manual, graph-based dependency resolution algorithm:
+ Starting from a set of given dependency info files, a graph of
package nodes is built, taking into account the different types
of requires (requires, build-requires and build-prerequires).
+ For each package node, all the dependencies are processed and
(depending on the type of file that the package node represents)
added to the graph, too, until all dependencies have been handled.
+ The resulting graph is sorted topologically, yielding a list of
packages that can be built one after the other in that order.

Add --get-dependencies option to haikuporter.

This will automatically search for packages with pkgman,
which get installed to your system. Use with caution!

Using --get-dependencies automatically sets --no-dependencies,
so you'll still need to manually build packages which aren't
available using pkgman.

Handle scriptlet-prerequires when resolving dependencies, too.

* That part apparently has been lost during the rewrite of the resolver.

Add new DependencyResolver.

* Instead of delegating the dependency resolution to pkgman, this class
implements a manual, graph-based dependency resolution algorithm:
+ Starting from a set of given dependency info files, a graph of
package nodes is built, taking into account the different types
of requires (requires, build-requires and build-prerequires).
+ For each package node, all the dependencies are processed and
(depending on the type of file that the package node represents)
added to the graph, too, until all dependencies have been handled.
+ The resulting graph is sorted topologically, yielding a list of
packages that can be built one after the other in that order.