Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

localedef: unbreak WITHOUT_LOCALES

After 0fa5403d493b ("pkgbase: move locales into their own package") we
need usr.bin/localedef as a bootstrap tool independent on where
WITHOUT_LOCALE was specified as we ALWAYS process C.UTF-8.

At the same time LOCALES= in the local Makefile is empty but
C.UTF-8 with WITHOUT_LOCALES. C.UTF-8 is excluded from FILES, and thus
after the replacement FILES= is set to only .LC_CTYPE which results in
a build failure not knowing how to build that. Tweak the substitution to
replace only non-empty words so that FILES remains harmlessly empty.

Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D31589

pkgbase: locales: Also tag the files dir

Otherwise bsd.dirs.mk will create the directory with the default
package (utilities) and we end up with a bunch of empty dirs managed
by this package while it shouldn't be the case.

ctypedef: fix installation of C.UTF-8

The appropriate directory and name were assigned to the FILESDIR
grouping, but not the ALWAYS grouping where C.UTF-8 is actually
assigned. Add the appropriate bits for ALWAYSDIR, and remove an
obsolete *PACKAGE= assignment since C.UTF-8 is explicitly not included
in FILES.

Prior to this change, C.UTF-8 was being installed as
/usr/share/C.UTF-8.LC_CTYPE.

Reviewed by: manu
Fixes: 0fa5403d493b ("pkgbase: move locales into their own package")
Differential Revision: https://reviews.freebsd.org/D31429

pkgbase: move locales into their own package

The only exception here being C.UTF-8 as this is the default
locales so it needs to always be installed

Reviewed by: pkgbase (emaste)
Differential Revision: https://reviews.freebsd.org/D31397

locales: fix abuse of bsd.dirs.mk

the way SAMEDIRS was defined was an abuse of bsd.dirs.mk resulting in
all the directory to be created in one single command, but DESTDIR is
only prepend once on the first element of the list

Switch to the properway to use bsd.dirs.mk

locales: stop hardcoding the directories in the mtree

The framework knows how to create directories and tag them properly
for a the creation of a mtree, not need to hardcode all the locales
entries in bsd.usr.mk

This simplifies addition of new locales but also allow people building
with WITHOUT_LOCALES to end up with a directory full of empty files

locale: Add Farsi/Persian locales

MFC after: 3 weeks
Reviewed by: farrokhi
Differential Revision: https://reviews.freebsd.org/D24359

locales: now that C.UTF-8 is the default locale, always install it

MFC after: 3 weeks

locales: separate unicode from other locales

Unicode locales are maintained and to evolved in cldr.
Generating the other locales from the unicode gets more and more painful
over the time.

Let's freeze the other locales to cldr 34.0.

Add ga_IE.UTF-8 locale.

PR: 228587
Submitted by: Micil <micheal.maclochlainn@oegaillimh.ie> (LC_TIME)
Reviewed by: bapt
Approved by: kib (mentor, implicit)
MFC after: 3 days
Differential revision: https://reviews.freebsd.org/D17997

Add hybrid C.UTF-8 locale being identical to default C locale except
that it uses the same ctype maps and functions as other UTF-8 locales.

Reviewed by: bapt, cem, eadler
Approved by: kib (mentor, implicit)
Differential Revision: https://reviews.freebsd.org/D17833

Update to CLDR 34 and UNICODE 11.

Discussed with: bapt
Approved by: kib (mentor)
MFC after: 1 month

Add -b/-l options to localedef(1) to specify output endianness and use
it appropriately when building share/ctypedef and share/colldef.

This makes the resulting locale data in EL->EB (amd64->powerpc64) cross
build and in the native EB build match. Revert the changes done to libc
in r308170 as they are no longer needed.

PR: 231965
Reviewed by: bapt, emaste, sbruno, 0mp
Approved by: kib (mentor)
Differential Revision: https://reviews.freebsd.org/D17603

Update to CLDR 32 and Unicode 10

MFC after: 2 weeks
Relnotes: yes

Readd Big5: some large databases setup are still requiring it.

Reported by: "張君天(Chun-Tien Chang)" <tcs@kitty.2y.idv.tw>

Remove zh_TW.Big5 locale

After discussion with many Taiwanese, in IT or not. Big5 is not used anymore.
It is not able to represent lots of the characters used in the language.

share: normalize paths using SRCTOP-relative paths or :H when possible

This simplifies make logic/output

MFC after: 1 month
Sponsored by: Dell EMC Isilon

Update locales cldr to v30.0.3 and unicode to 9.0.0

MFC after: 1 month
Relnotes: yes

Posixify the locales name for variants

For all locales with variants:
- if no ambiguity on the locale (only one variant) just use the regular name
- if ambiguity, pick one as default and append @<variant> to the others
respecting POSIX

As a result:
- All the 3 components locales added recently are renamed to the usual 2
components version for all but sr_RS.UTF-8
- Set sr_RS.UTF-8 to the cyrillic variant
- Add sr_RS.UTF-8@latin
- Remove the symlinks aliases they were created to represent the 2 components
version as aliasas and are now useless
- Update the OptionalObsoleteFiles.inc and ObsoleteFiles.inc to reflect those
changes

Discussed with: ache@
Approved by: re@ (gjb)

Regenerates locales Makefiles and locales

Use :E instead of regex in ctypedef Makefiles

Simplify the locale generated Makefiles by using bmake multi variables for loops

Bring back a couple of ISO8859-15 locales:
- af_ZA.ISO8859-15
- en_AU.ISO8859-15
- en_CA.ISO8859-15
- en_NZ.ISO8859-15
- en_US.ISO8859-15
- fr_CA.ISO8859-15

Regenerate locales after readding ISO8859-1 for locales that have ISO8859-15

Requested by: arche

Use explicit filename when creating locale symlinks to avoid creating a
directory symlink when the target directory does not exist. This will
cause an error instead of a broken setup.

Sponsored by: EMC / Isilon Storage Division

Regenerate locales

Add newly generated locales from CLDR 27.0.1
only skip timedef until I figure out how to workaround svn painful mime-type handling