Remove wqy_microhei font from Haiku release images

It has been replaced by Noto as the default font for CJK characters and
is not referenced anywhere else anymore.

Change-Id: I9eb706531deb44395d71e0a04b0ec5e2f2dea230
Reviewed-on: https://review.haiku-os.org/c/haiku/+/7499
Reviewed-by: nephele nephele <nep-git@packageloss.eu>
Tested-by: Commit checker robot <no-reply+buildbot@haiku-os.org>

icu: bump OS (and all dependencies) to ICU 74

* Non-x86 will likely break until they get an updated icu74
build-package

Change-Id: I5762d14f848e1c4e9b837e1dea9c1a54efa8401d
Reviewed-on: https://review.haiku-os.org/c/haiku/+/7346
Reviewed-by: Alex von Gluck IV <kallisti5@unixzen.com>

build-packages: update to the latest packages

The main motivator is the update to GCC 13. As part of this, some of the other
dependencies have been updated as well.

Newly introduced:
- gawk on x68_64 (new dependency of texinfo)
- libjx (new dependency of haikuwebkit)
- openexr30 (new indirect dependency of haikuwebkit)
- brotli (new indirect dependency of haikuwebkit)

On x86_gcc2, some packages have switched to the modern GCC version:
- diffutils
- findutils
- libpsl
- tcpdump

Change-Id: Ic617b5b4af9eb34c0d28259a3c0ddbcc33f98a5d
Reviewed-on: https://review.haiku-os.org/c/haiku/+/6772
Reviewed-by: Adrien Destugues <pulkomandy@pulkomandy.tk>

build/profile: Add nightly-mmc for riscv64 (and arm later)

Change-Id: I6aeddabedb8cf3e1505519f4393e4d0f2af5ea7f

release: Bump source release to 1.4GiB

* The if statements now "make sense"
* 1.4 GiB release images easily fit a "mini single layer DVD"
* Our source build is around 1.1GiB
* 200 MiB of "space" in our read-only image allows
a little room for live-media shenanigans
* Empty space is compressed anyway in release zip files

Change-Id: I5ad017e3f80d5977ca9604f5bf200684796fc7b9
Reviewed-on: https://review.haiku-os.org/c/haiku/+/5818
Reviewed-by: Alex von Gluck IV <kallisti5@unixzen.com>
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

release: Bump image size to 800MiB. Sorry CD-R

Change-Id: Ib21a3fb8c26a25d84b0b3dd298c22618a53cb7d1
Reviewed-on: https://review.haiku-os.org/c/haiku/+/5817
Reviewed-by: Alex von Gluck IV <kallisti5@unixzen.com>
Reviewed-by: Niels Sascha Reedijk <niels.reedijk@gmail.com>

HaikuPorts/x86_64: Synchronize.

* Userguide now provided from HaikuPorts.
* Python 3.7 -> 3.9.
* Lots of other version bumps.

HaikuPorts/x86_gcc2: Update.

build: Small bump to release image size

* Haiku now consumes right at 700 MiB on x86_gcc2h
after the webkit update.

Change-Id: Ia41a3e6252190e73feaa49ebe7c58ef4c80df4dc
Reviewed-on: https://review.haiku-os.org/c/haiku/+/4161
Reviewed-by: Alex von Gluck IV <kallisti5@unixzen.com>

build/image: Slightly bump release image size

* Solves out of space issue building release images

Change-Id: I888836774732f65fde3574ef498b272d8285356f
Reviewed-on: https://review.haiku-os.org/c/haiku/+/4107
Reviewed-by: Alex von Gluck IV <kallisti5@unixzen.com>

Update build packages repository

- Switch to Python3
- Switch to mawk instead of gawk
- Update ffmpeg (there are several new dependencies)
- Update ICU
- Fix various pre-existing problems with packages not being properly
declared for gcc2 (but somehow it ended up working)
- remove curl, subversion, mercurial

Confirmed that both nightly and release images are building fine on both
32 and 64bit. However, non-x86 architecture may require re-bootstrapping
to do a complete nightly or release image build (the minimal profile
should be fine)

Fixes #16751.

Change-Id: Iacac92923c4113b3e0a49a64b0b4cc1b8e2f5e2e
Reviewed-on: https://review.haiku-os.org/c/haiku/+/3871
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>

Remove deprecated build profiles @image and @vmware-image.

The @image build is not used anymore and doesn't work. Use @nightly-raw
or @nightly-anyboot instead. The new method of release type/image type
configuration of profiles is more flexible and allows to manage the
various configurations a lot more easily for us. But the old profiles
were never formally deprecated.

Fixes #15963

Change-Id: I1c72f0e77bfbcda10502476e595f120280e8c7a4
Reviewed-on: https://review.haiku-os.org/c/haiku/+/2625
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

DefaultBuildProfiles: Remove "gcc" from default source packages.

OptionalPackages will add its source package when shipping GCC itself.

HaikuPorts: Synchronize build-packages set.

Notable changes:
* jpeg replaced with libjpeg_turbo across the board.
* man replaced with mandoc.
* ilmbase merged into openexr.
* gnutls no longer needed in this set.

A few other packages unused in the local scripts were
also removed. Build fixes are in following commits.

Move source and development packages out of /system/packages on release images

Activating all these packages on live boot (especially live DVD where we
also have the write overlay eating up RAM) significantly increases
memory requirements for running Haiku, to the point that Installer may
fail to run. Move them outside the packages/ directory so they are not
activated.

Installer already skips the _sources_ directory when installing, and
lists the contents of _packages_ with checkbox to decide what to include
in the install (they are put in system/packages on the target disk).

Fix from previous version: the Haiku package was installed to /system
instead of /system/packages.

Fixes #15621.

Revert "Move source and development packages out of /system/packages on release images"

This reverts commit 0ab9c3c53849f10a3b7aef1e03714e69c7cb0c15.

It broke booting.

Move source and development packages out of /system/packages on release images

Activating all these packages on live boot (especially live DVD where we
also have the write overlay eating up RAM) significantly increases
memory requirements for running Haiku, to the point that Installer may
fail to run. Move them outside the packages/ directory so they are not
activated.

Installer already skips the _sources_ directory when installing, and
lists the contents of _packages_ with checkbox to decide what to include
in the install (they are put in system/packages on the target disk).

Fixes #15621.

Change-Id: Ibfac14c43dbe708316a0d82ca1c9d49b5cba2aef
Reviewed-on: https://review.haiku-os.org/c/haiku/+/2157
Reviewed-by: Sergei Reznikov <diver@gelios.net>
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

Remove mercurial sources from release images

We don't ship the binaries, so I don't see why we would ship the
sources.

DefaultBuildProfiles: Add OpenSSL to minimum profile

This makes pkgman functional in minimum-profile images.

Change-Id: I3578886743fecbf76779ac1b4b76762d8ce1be3d
Reviewed-on: https://review.haiku-os.org/c/haiku/+/1715
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

ImageRules: Ensure nightly builds will have a stable volume-id

Let's make the nightly builds consistent in a way that they would always
be recognized by libosinfo (and, consequently, by GNOME Boxes,
virt-manager, virt-install, ...) by explictly using "nightly" instead
of the $(HAIKU_VERSION) when generating nightly builds.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>

Change-Id: I6fc4f15be1e8e9244abf14e75308fc825f37b2e8
Reviewed-on: https://review.haiku-os.org/c/1053
Reviewed-by: Alex von Gluck IV <kallisti5@unixzen.com>

DefaultBuildProfiles: Make sure the final CD image is <= 694MB.

DefaultBuildProfiles: Without sources, release images fit in 700MB.

We don't want to ship these images as the ones for main distribution,
but at least we can make them available as downloads for those who do want
to use CDs.

profiles: Bump release to 1100/1300 MiB

* This gives us plenty of space for source packages.
* A Mini-DVD is 1.4 GiB, and USB sticks of 2 GiB are at
the sweet spot of low-price vs size.
* Unused space will be compressed in release zip.
* We blew by 700 MiB long ago. Sorry CD-R folks.

Change-Id: I3bbe4508777027f6fe7c0ee2992637541feeb88f

build/profiles: Add @minimum-cd profile for things like ppc

Change-Id: Id50d6ff6eaca598c00043d5668ad836a48a74262

build: Rework InstallSourceArchive into AddHaikuImageSourcePackages.

Also add AddHaikuImageSourcePackages invocations for all relevant
(i.e. [L]GPL'ed) packages.

Doesn't truly fix #9990 as PackageKit cannot be used to install source
packages, but this is enough for creating a redistributable image,
so that ticket can be moved out of the beta1 milestone now.

Set the image size for debug releases to 1GB.

Set the image size for debug releases to 900MB.

build: Fix release build profile.

Set the image size for debug nightlies to 850MB.

Set the image size for the debug minimum profile to 450MB.

Add pdfwriter package to the release profile.

noto_cjk_jp doesn't exist, replace with noto_sans_cjk_jp.

Complete the transition to Noto as the default font.

- Both Noto and Noto Sans CJK JP are now used as font fallbacks,
allowing to cover a rather large range of characters. This also makes
it possible to mix the two fonts easily.
- Remove VL-Gothic from packages and from AboutSystem
- Add Noto fonts to the dependencies of the Haiku package.

This provides a similar look for all languages as discussed on the
mailing list.

Revert "Revert "Switch from DejaVu to Noto font""

This reverts commit 2423ba84701f064cb78a40010113a8a45cc02a92.

Revert "Switch from DejaVu to Noto font"

This reverts commit 945566ff43583e4f8102b4440c88f53dae775cb4.

As discussed on the mailing lists and with Humdinger off-list:
* The general design concensus tends slightly towards DejaVu, as metrics
of DejaVu look much better (DejaVu 12 and Noto 13 are roughly the same size,
but Noto has much wider margins with that)
* While Noto does have a wider set of fonts with support for lots of
different languages, DejaVu actually has built-in support for more
Unicode languages (the default Noto has, as far as I can tell, only
Latin/Greek/Cyrillic [2416 glyphs], while DejaVu also has Armenian, Georgian,
and a few other scripts too [5119 glyphs].)
* The worse rendering of DejaVu appears to have been somewhat rectified by
disabling the average-based subpixel filter in app_server.

build: Change anyboot output from image to iso

* Re ML discussions, this should make a lot more
sense to users as it is inline with what most
linux distros ship.
* This will require a tweak to the buildbot.
* First step to shipping anyboot instead of plain iso.

Switch from DejaVu to Noto font

As discussed, Noto renders much better with freetype 2.7 [1].
The DejaVu package is still in the HaikuPorts repo and can be installed
manually just like all the other available fonts.

[1] http://www.freelists.org/post/haiku-development/freetype-update-from-265-to-27

Set the image size for debug nightlies to 700MB.

Add profile bootstrap-anyboot.

* update version for freetype_bootstrap package.

HaikuBootstrap: also include launch scripts.

* fix ncurses package name for bootstrap build profile.

build: Convert to external libedit build package

* Drop lib/edit and matching bsd header
* Convert Debugger to libedit build package
* Should solve problems with libedit consumers
not defining _BSD_SOURCE
* Progress on #10267

added expat packages to bootstrap profiles.

Add the DejaVu fonts to the nightly profile.

Outsource DejaVu fonts

Signed-off-by: Timothy Gu <timothygu99@gmail.com>

Outsource VL-Gothic font.

Fixes #11576.

Drop autotools and libtool from haikuports.cross.

* None of these are required to be cross-built, as they all can be built on
the bootstrap image just fine.
* Additionally, update bash_boostrap from 4.27 to 4.30, as 4.27 causes the
build of gcc on the bootstrap image to fail.

Add less to bootstrap image

Define the minimum-mmc profile for the ARM build.

Now that the fake packages are in place, it is much easier to build the
MMC image for ARM without the need for a bootstrap build.

This image still does not manage to access the tarfs and load the kernel
modules, but it gets to KDL, at least.

Add pseudo target for bootstrap_stage0 platform.

Patch by dnivra (GSoC 2014, ARM suppport)

* Cleanup the SD card image building to allow jam -q @bootstrap-mmc to
work.

There are a few remaining tricks before you can safely build an image:
* This uses a non-POSIX du option, and is only tested with Linux du
only (Linux is the only supported system to run bootstrap builds,
anyway)
* The Python recipe in haikuports.cross is known to not build on
Debian/Ubuntu, but work fine on OpenSuse. There is a patch available in
haikuports bugtracker to allow the reverse.
* You need to populate the haikuports repo package list with some
packages (which don't exist yet) to make the build system happy. But our
git hook to generate the repositories is preventnig me to share this
hack.

Once built, the image currently crashes early in the kernel execution.
On to debug that!

Create packages_build folders per build type.

* The content of the preprocessed package-info files and the package
contents depend on the build type, so we use a different folder for
each build type.

Whitespace cleanup in root and 'build' folder.

* Removed trailing whitespace and empty lines at top of files.

Replace HAIKU_{BOOTSTRAP,MINIMUM}_BUILD with HAIKU_BUILD_TYPE.

* Instead of separate variables, HAIKU_BUILD_TYPE is set to one of
'bootstrap', 'minimum' or 'regular'.
* Adjust uses of HAIKU_BOOTSTRAP_BUILD accordingly.

Make set of source packages for bootstrap-image configurable.

* Add new build profile 'minimum', which defines a minimum set of
packages.
* Introduce HAIKU_BOOTSTRAP_SOURCES_PROFILE and let it default to
'@minimum-raw'. This can be overruled in UserBuildConfig, setting
it to '@release-raw' will cause all source packages required for a
full release to be put onto the bootstrap-image.
* Add new image HaikuImageMinimum, which is meant to define the minimum
useful Haiku image (yeah, I know that's vague).
* Add HAIKU_MINIMUM_BUILD, which indicates that HaikuImageMinimum should
be used (it would be better to merge this with HAIKU_BOOTSTRAP_BUILD
into something like HAIKU_BUILD_TYPE)
* Cleanup duplicate references to basic packages - those are now added
by the topmost Jamfile (no longer referenced by the build profiles).

Add missing texinfo to bootstrap package list.

gawk: use the HaikuPorts package.

* left the in-tree version as its libintl is used by gdb.
* added as bootstrap package.

findutils: use the HaikuPorts package.

* removed in-tree version.
* added as bootstrap package.
* note: HaikuPorts package has no rdef for binaries.

package kit: Disable curl requirement on bootstrap build

* As per the mailing list.
* Introduce HAIKU_BOOTSTRAP_BUILD define to sources
to let them know they are taking part in a bootstrap

Enable WebPositive for x86_64 in DefaultBuildProfiles.

Thanks waddlesplash for the note!

build: Only build WebPositive on x86, x86_gcc2

* Should fix the non-x86 builds

DefineDefaultBuildProfiles: Various fixes

* Fix adding WebPositive optional package.
* Fix adding wonderbrush package. Add it only for gcc2 to avoid a
warning by AddHaikuImagePackages.
* Simplify adding xz_utils package.

build: Disable WebPositive on x86_64 to fix build

* This commit is temporary and should be reverted
when the x86_64 build is fixed.

Replace alpha-* profiles with release-*. Remove beta-*.

Make some values of default build profiles overridable.

* Defining things like HAIKU_ROOT_USER_NAME in UserBuildConfig now
has an effect on the image again.

Initial changes to remove /boot/common

* find_directory() and hard-coded paths use /boot/system instead of
/boot/common.
* The build system creates the writable directories in /boot/system
instead of /boot/common.
* The build system no longer installs any packages in /boot/common.

Add libcurl build feature.

Increase size of bootstrap image to 20 GB.

Don't try to build xz_utils for a mere gcc2 system.

Increase size of bootstrap image to 4 GB.

* 1 GB isn't enough to build the basic set

DefineDefaultBuildProfiles: simplify things a bit

Build secondary architecture Haiku package for regular image

Also add secondary architecture packages to image.

Add the missing secondary arch bootstrap packages

The Haiku bootstrap image is now built with (hopefully) all required
primary and secondary architecture packages. The runtime loader is still
resisting our wish to run secondary architecture programs, though.

Remove no longer needed HAIKU_STRIP_DEBUG_FROM_OPTIONAL_PACKAGES

Add all bootstrap packages in DefaultBuildProfiles

* Don't handle the bootstrap case in the main Jamfile. Add all bootstrap
packages to the image in the profile definition. That's less
confusing, hopefully.
* Add the missing bootstrap packages (ncurses*, libtool_libltdl). The
would normally be added in build_haiku_image when the package
dependencies are resolved, but we don't do that for the bootstrap
image, since we intentionally leave some dependencies out (e.g. perl)
because they can be built.

Add haikuporter and python to Haiku bootstrap image

Include DefaultBuildProfiles earlier

DefaultBuildProfiles is now included earlier (right after BuildSetup).
This allows us to set HAIKU_BOOTSTRAP_BUILD earlier, so it can be used
for the repository selection. The actual build profile definitions,
which cannot be done that early, live in the rule
DefineDefaultBuildProfiles, which is invoked where the file was
previously included.

ReleaseBuildProfiles -> Default*, add bootstrap profiles

build: Convert to external libedit build package

* Drop lib/edit and matching bsd header
* Convert Debugger to libedit build package
* Should solve problems with libedit consumers
not defining _BSD_SOURCE
* Progress on #10267

added expat packages to bootstrap profiles.

Add the DejaVu fonts to the nightly profile.

Outsource DejaVu fonts

Signed-off-by: Timothy Gu <timothygu99@gmail.com>

Outsource VL-Gothic font.

Fixes #11576.

Drop autotools and libtool from haikuports.cross.

* None of these are required to be cross-built, as they all can be built on
the bootstrap image just fine.
* Additionally, update bash_boostrap from 4.27 to 4.30, as 4.27 causes the
build of gcc on the bootstrap image to fail.

Add less to bootstrap image

Define the minimum-mmc profile for the ARM build.

Now that the fake packages are in place, it is much easier to build the
MMC image for ARM without the need for a bootstrap build.

This image still does not manage to access the tarfs and load the kernel
modules, but it gets to KDL, at least.

Add pseudo target for bootstrap_stage0 platform.

Patch by dnivra (GSoC 2014, ARM suppport)

* Cleanup the SD card image building to allow jam -q @bootstrap-mmc to
work.

There are a few remaining tricks before you can safely build an image:
* This uses a non-POSIX du option, and is only tested with Linux du
only (Linux is the only supported system to run bootstrap builds,
anyway)
* The Python recipe in haikuports.cross is known to not build on
Debian/Ubuntu, but work fine on OpenSuse. There is a patch available in
haikuports bugtracker to allow the reverse.
* You need to populate the haikuports repo package list with some
packages (which don't exist yet) to make the build system happy. But our
git hook to generate the repositories is preventnig me to share this
hack.

Once built, the image currently crashes early in the kernel execution.
On to debug that!

Create packages_build folders per build type.

* The content of the preprocessed package-info files and the package
contents depend on the build type, so we use a different folder for
each build type.

Whitespace cleanup in root and 'build' folder.

* Removed trailing whitespace and empty lines at top of files.

Replace HAIKU_{BOOTSTRAP,MINIMUM}_BUILD with HAIKU_BUILD_TYPE.

* Instead of separate variables, HAIKU_BUILD_TYPE is set to one of
'bootstrap', 'minimum' or 'regular'.
* Adjust uses of HAIKU_BOOTSTRAP_BUILD accordingly.

Make set of source packages for bootstrap-image configurable.

* Add new build profile 'minimum', which defines a minimum set of
packages.
* Introduce HAIKU_BOOTSTRAP_SOURCES_PROFILE and let it default to
'@minimum-raw'. This can be overruled in UserBuildConfig, setting
it to '@release-raw' will cause all source packages required for a
full release to be put onto the bootstrap-image.
* Add new image HaikuImageMinimum, which is meant to define the minimum
useful Haiku image (yeah, I know that's vague).
* Add HAIKU_MINIMUM_BUILD, which indicates that HaikuImageMinimum should
be used (it would be better to merge this with HAIKU_BOOTSTRAP_BUILD
into something like HAIKU_BUILD_TYPE)
* Cleanup duplicate references to basic packages - those are now added
by the topmost Jamfile (no longer referenced by the build profiles).

Add missing texinfo to bootstrap package list.

gawk: use the HaikuPorts package.

* left the in-tree version as its libintl is used by gdb.
* added as bootstrap package.

findutils: use the HaikuPorts package.

* removed in-tree version.
* added as bootstrap package.
* note: HaikuPorts package has no rdef for binaries.

package kit: Disable curl requirement on bootstrap build

* As per the mailing list.
* Introduce HAIKU_BOOTSTRAP_BUILD define to sources
to let them know they are taking part in a bootstrap

Enable WebPositive for x86_64 in DefaultBuildProfiles.

Thanks waddlesplash for the note!

build: Only build WebPositive on x86, x86_gcc2

* Should fix the non-x86 builds

DefineDefaultBuildProfiles: Various fixes

* Fix adding WebPositive optional package.
* Fix adding wonderbrush package. Add it only for gcc2 to avoid a
warning by AddHaikuImagePackages.
* Simplify adding xz_utils package.

build: Disable WebPositive on x86_64 to fix build

* This commit is temporary and should be reverted
when the x86_64 build is fixed.

Replace alpha-* profiles with release-*. Remove beta-*.

Make some values of default build profiles overridable.

* Defining things like HAIKU_ROOT_USER_NAME in UserBuildConfig now
has an effect on the image again.

Initial changes to remove /boot/common

* find_directory() and hard-coded paths use /boot/system instead of
/boot/common.
* The build system creates the writable directories in /boot/system
instead of /boot/common.
* The build system no longer installs any packages in /boot/common.

Add libcurl build feature.

Increase size of bootstrap image to 20 GB.

Don't try to build xz_utils for a mere gcc2 system.

Increase size of bootstrap image to 4 GB.

* 1 GB isn't enough to build the basic set

DefineDefaultBuildProfiles: simplify things a bit

Build secondary architecture Haiku package for regular image

Also add secondary architecture packages to image.

Add the missing secondary arch bootstrap packages

The Haiku bootstrap image is now built with (hopefully) all required
primary and secondary architecture packages. The runtime loader is still
resisting our wish to run secondary architecture programs, though.

Remove no longer needed HAIKU_STRIP_DEBUG_FROM_OPTIONAL_PACKAGES

Add all bootstrap packages in DefaultBuildProfiles

* Don't handle the bootstrap case in the main Jamfile. Add all bootstrap
packages to the image in the profile definition. That's less
confusing, hopefully.
* Add the missing bootstrap packages (ncurses*, libtool_libltdl). The
would normally be added in build_haiku_image when the package
dependencies are resolved, but we don't do that for the bootstrap
image, since we intentionally leave some dependencies out (e.g. perl)
because they can be built.

Add haikuporter and python to Haiku bootstrap image

Include DefaultBuildProfiles earlier

DefaultBuildProfiles is now included earlier (right after BuildSetup).
This allows us to set HAIKU_BOOTSTRAP_BUILD earlier, so it can be used
for the repository selection. The actual build profile definitions,
which cannot be done that early, live in the rule
DefineDefaultBuildProfiles, which is invoked where the file was
previously included.

ReleaseBuildProfiles -> Default*, add bootstrap profiles