Builds what native gettext serves for the migrated modules (see TranslationGettextRepository) from every shipped .po of every contributed LanguageFileDirectory; run by Setup's build (ShippedLanguageFilesCompiledObjective) only, never by the web server (it does not write artifacts/; "merge" only writes the shipped .po, see ShippedPoMerger). More...

Collaboration diagram for ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild:

Public Member Functions
	__construct (private readonly LanguageFileDirectoryManager $language_file_directory_manager, private readonly string $ilias_absolute_path, private readonly ShippedTranslations $shipped_translations=new ShippedTranslations(), private readonly ?string $locale_source=null)

	run (callable $inform)

Static Public Member Functions
static	readPointer (string $artifact_directory)
	The pointer of this PHP's locale kind in $artifact_directory, `null` if it is missing or not valid. More...

static	readPointerOf (string $artifact_directory, bool $system_locale)

static	servesShippedPo (string $artifact_directory, string $module, string $lang_key, string $shipped_po)
	Whether the build of $artifact_directory serves exactly the shipped `.po` $shipped_po of $module for $lang_key (its content hash, see the manifest) - `false` without a build or after a change of the `.po` since (e.g. More...

static	platformProblem (string $build_directory)
	Why a build made with a system locale (macOS/BSD) cannot be activated by a PHP needing the shipped one. More...

Private Member Functions
	runLocked (string $artifact_directory, callable $inform)

	write (string $artifact_directory, string $build_directory, string $build_id, array $modules, ?string $locale_source, array $former_builds)
	Writes the build $build_id into a temporary directory and renames that to $build_directory. More...

	formerCatalog (string $artifact_directory, array $former_builds, string $module, string $lang_key)
	The catalog of $module/$lang_key in the first former build that has it intact and compiled the same way (see catalogCompatibility()). More...

	templateKeys (?string $template, string $module, array &$warnings)
	The identifiers of $module in its template (`.pot`), `[]` if there is none or it cannot be read (reported). More...

	findCollisions (array $values_per_language)
	The identifiers with different values in several modules, per language. More...

	assertNativeGettextServes (string $build_directory, array $sample)
	Looks $sample up through native gettext in the build $build_directory. More...

	assertActivatedNativeGettextServes (string $build_directory, array $sample)

	findShippedFiles (callable $inform)
	Every shipped `.po` of every contributed directory with a module prefix (the main `lang/` directory has none and holds no `.po`), per module. More...

	buildId (array $source_parts, ?string $locale_source)

	sourceParts (array $modules)

	codeFingerprint ()
	Over the code that determines the build, see FORMAT_VERSION. More...

	directoryFingerprint (string $directory)
	Over the content of every file below $directory - a locale copied from another version of the C library may not be loadable by the one in use. More...

	listFiles (string $directory)

	findLocaleSource ()

	copyLocale (string $source, string $target, string $build_directory, array &$files)

	isIntact (string $build_directory, array $manifest)
	Whether every file of $manifest is in $build_directory as it was written - without fsync(), a crash can leave empty or truncated files behind. More...

	writePointer (string $artifact_directory, array $pointer)

	removeObsolete (string $artifact_directory, array $keep)
	Removes everything of a former build of this locale kind except the builds $keep: other builds, temporary directories and files of aborted builds, and the former layout (`<lang>/<module>.mo`, `index.json`). More...

	holdsOnlyCatalogs (string $directory)
	Whether $directory (of the former layout `<lang>/<module>.mo`) holds nothing but catalogs and temporary files of them. More...

	removeTree (string $artifact_directory, string $path)
	Removes $path recursively - only below $artifact_directory, and without following a symbolic link (a link itself is removed). More...

	writeFile (string $file, string $content)

	ensureDirectory (string $directory)

	relativePath (string $file)

Static Private Member Functions
static	catalogCompatibility ()
	Fingerprint of the compiling and cleaning code: formerCatalog() only takes over catalogs with the same one. More...

static	classesFingerprint (array $classes)

static	readManifest (string $build_directory)

static	hashes (mixed $value)

static	stringList (mixed $value)

Private Attributes
const int	FORMAT_VERSION = 4
	Raise to rebuild when the output changes for a reason the fingerprint of the involved classes does not capture, e.g. More...

const array	CATALOG_CLASSES
	The classes that determine the bytes of a compiled catalog (MigratedLanguageFileSync: moduleEntries()). More...

const string	MANIFEST_FILE = 'manifest.json'

const string	MARKUP_WARNING = 'Markup not allowed in a language value'

const string	LOCK_FILE = 'build.lock'

const array	LOCALE_SOURCES
	Where the C library's "C.utf8" locale is looked for (Debian/Ubuntu, Fedora/RHEL, Arch, ...). More...

Static Private Attributes
static array	$manifests = []

static array	$system_locale_platforms = []

Detailed Description

Builds what native gettext serves for the migrated modules (see TranslationGettextRepository) from every shipped .po of every contributed LanguageFileDirectory; run by Setup's build (ShippedLanguageFilesCompiledObjective) only, never by the web server (it does not write artifacts/; "merge" only writes the shipped .po, see ShippedPoMerger).

Requirement: on glibc the locale is copied from the C library of the build host and checked in that (CLI) process only. The web server must use the same C library (glibc with C.utf8) and have the PHP extension "gettext"; with separate CLI/web images or a musl based system (which ignores LOCPATH) the runtime serves -identifier- and reports why (see MigratedTranslations::getProblem()). macOS/BSD ship no locale.

Layout below artifacts/language/ (see MigratedLanguageFilePaths):

<build>/messages/LC_MESSAGES/<module>.<lang>.mo: the shipped state of a module, see ShippedTranslations::compile();
<build>/keys/<module>.php: all identifiers of the module's .po files and template, the same for every language, as a PHP file for the opcache;
<build>/locale/: the locale native gettext is activated with (see NativeGettext), a copy of "C.utf8" (glibc only);
<build>/manifest.json: size and hash of every file, to tell an intact build, the hash of each built .po (see servesShippedPo()), the warnings and collisions, whether it ships a locale and the building platform (see platformProblem());
current.json (glibc) and current-system.json (system locale, see NativeGettext::usesSystemLocale()): the build the runtime of that locale kind serves, its languages per module, every contributed module ("shipped") and a fingerprint of the built files ("sources") - read on every request, so nothing else. A build writes only the pointer of its kind and keeps the builds of the other one, so a build on the host (e.g. macOS) does not take a build of the web server (e.g. a glibc container) away.

The build id fingerprints everything that determines the build: the shipped files, the compiling code (see FORMAT_VERSION) and the locale (or the platform using a system locale). An intact build with the same id (its manifest also keeps its warnings) is reused, otherwise a new one is written into a temporary directory and renamed to its id. Either way it is checked with a real native gettext lookup before its pointer is switched atomically; running requests keep their build. The previous build is kept, older ones (and the files of the former layout <lang>/<module>.mo, index.json) are removed. Builds are serialized by build.lock.

A .po that cannot be compiled is reported and served with the catalog of the served (or else the previous) build, so a broken file never turns a working language into -identifier-; without such a catalog the module is not served for that language. Values with disallowed markup are compiled cleaned and reported; identifiers with different values in several modules (txt() serves the one loaded last) are listed, grouped by modules (collision_summary) when the build is written. All of that stays in the manifest until fixed.

Files are written without fsync(): a build can be repeated any time, and the manifest reveals a build damaged by a crash.

@phpstan-type Manifest array{files: array<string, array{0: int, 1: string}>, sample: array{module: string, lang: string, key: string, value: string}, languages: array<string, list<string>>, hashes: array<string, array<string, string>>, warnings: list<string>, collisions: list<string>, compatibility: string, locale: 'shipped'|'system', platform: string} @phpstan-type Pointer array{build: ?string, previous: ?string, modules: array<string, list<string>>, shipped: ?list<string>, sources: ?string}

Definition at line 73 of file ShippedTranslationsBuild.php.

Constructor & Destructor Documentation

◆ __construct()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::__construct	(	private readonly LanguageFileDirectoryManager	$language_file_directory_manager,
		private readonly string	$ilias_absolute_path,
		private readonly ShippedTranslations	$shipped_translations = `new ShippedTranslations()`,
		private readonly ?string	$locale_source = `null`
	)

Parameters

string | null $locale_source the locale directory to ship; default: found in LOCALE_SOURCES

Definition at line 126 of file ShippedTranslationsBuild.php.

131 {

132 }

Member Function Documentation

◆ assertActivatedNativeGettextServes()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::assertActivatedNativeGettextServes	(	string	$build_directory,
		array	$sample
	)

private

Parameters

array{module string, lang: string, key: string, value: string} $sample

Definition at line 592 of file ShippedTranslationsBuild.php.

                                                                                               : void
    {
        if (!NativeGettext::activate(MigratedLanguageFilePaths::buildLocaleDirectory($build_directory))) {
            throw new RuntimeException(sprintf('Native gettext is not available: %s.', (string) NativeGettext::getProblem()));
        }
        $domain = MigratedLanguageFilePaths::shippedDomain($sample['module'], $sample['lang']);
        if (!NativeGettext::bind($domain, $build_directory)) {
            throw new RuntimeException(sprintf('Native gettext cannot bind the domain "%s" to "%s".', $domain, $build_directory));
        }
        $value = NativeGettext::translate($domain, $sample['key']);
        if ($value !== $sample['value']) {
            throw new RuntimeException(sprintf(
                'Native gettext does not translate: "%s" in the domain "%s" gave "%s" instead of "%s".',
                $sample['key'],
                $domain,
                $value,
                $sample['value']
            ));
        }
    }

References Vendor\Package\$sample.

◆ assertNativeGettextServes()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::assertNativeGettextServes	(	string	$build_directory,
		array	$sample
	)

private

Looks $sample up through native gettext in the build $build_directory.

Parameters

array{module string, lang: string, key: string, value: string} $sample

Exceptions

RuntimeException if native gettext does not serve the expected value

Definition at line 575 of file ShippedTranslationsBuild.php.

                                                                                      : void
    {
        $messages = setlocale(LC_MESSAGES, '0');
        // with "C" activate() loads the shipped locale instead of keeping a real one, so the check covers it
        setlocale(LC_MESSAGES, 'C');
        try {
            $this->assertActivatedNativeGettextServes($build_directory, $sample);
        } finally {
            if ($messages !== false) {
                setlocale(LC_MESSAGES, $messages);
            }
        }
    }

References Vendor\Package\$sample.

Referenced by ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild\runLocked().

Here is the caller graph for this function:

◆ buildId()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::buildId	(	array	$source_parts,
		?string	$locale_source
	)

private

Parameters

	list<string>	$source_parts see sourceParts()
string \| null	$locale_source	`null`: no locale is shipped, see NativeGettext::usesSystemLocale()

Definition at line 685 of file ShippedTranslationsBuild.php.

                                                                         : string
    {
        $parts = [
            (string) self::FORMAT_VERSION,
            $this->codeFingerprint(),
            $locale_source === null ? 'system-locale|' . PHP_OS_FAMILY : $this->directoryFingerprint($locale_source),
            ...$source_parts,
        ];
 
        return 'b-' . substr(hash('sha256', implode("\n", $parts)), 0, 32);
    }

Referenced by ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild\runLocked().

Here is the caller graph for this function:

◆ catalogCompatibility()

static ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::catalogCompatibility ( )

staticprivate

Fingerprint of the compiling and cleaning code: formerCatalog() only takes over catalogs with the same one.

Definition at line 498 of file ShippedTranslationsBuild.php.

                                                  : string
    {
        return hash('sha256', self::FORMAT_VERSION . '|' . self::classesFingerprint(self::CATALOG_CLASSES));
    }

◆ classesFingerprint()

static ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::classesFingerprint ( array $classes )

staticprivate

Parameters

list<class-string> $classes

Definition at line 734 of file ShippedTranslationsBuild.php.

                                                              : string
    {
        $parts = [];
        foreach ($classes as $class) {
            $file = (new \ReflectionClass($class))->getFileName();
            $parts[] = is_string($file) ? (string) @hash_file('sha256', $file) : '';
        }
 
        return hash('sha256', implode('|', $parts));
    }

◆ codeFingerprint()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::codeFingerprint ( )

private

Over the code that determines the build, see FORMAT_VERSION.

Definition at line 721 of file ShippedTranslationsBuild.php.

                                      : string
    {
        return self::classesFingerprint([
            self::class,
            MigratedLanguageFilePaths::class,
            NativeGettext::class,
            ...self::CATALOG_CLASSES,
        ]);
    }

◆ copyLocale()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::copyLocale	(	string	$source,
		string	$target,
		string	$build_directory,
		array &	$files
	)

private

Parameters

array<string,array{0 int, 1: string}> $files the manifest, extended by the copied files

Definition at line 803 of file ShippedTranslationsBuild.php.

                                                                                                       : void
    {
        foreach ($this->listFiles($source) as $relative => $file) {
            $content = @file_get_contents($file);
            if ($content === false) {
                throw new RuntimeException(sprintf('Could not read "%s".', $file));
            }
            $this->ensureDirectory(dirname($target . '/' . $relative));
            $this->writeFile($target . '/' . $relative, $content);
            $files[substr($target . '/' . $relative, strlen($build_directory) + 1)] = [strlen($content), hash('sha256', $content)];
        }
    }

References $files, and ILIAS\Scripts\PHPStan\$target.

◆ directoryFingerprint()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::directoryFingerprint ( string $directory )

private

Over the content of every file below $directory - a locale copied from another version of the C library may not be loadable by the one in use.

Definition at line 749 of file ShippedTranslationsBuild.php.

                                                            : string
    {
        $parts = [];
        foreach ($this->listFiles($directory) as $relative => $file) {
            $parts[] = $relative . '|' . (string) @hash_file('sha256', $file);
        }
 
        return hash('sha256', implode("\n", $parts));
    }

◆ ensureDirectory()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::ensureDirectory ( string $directory )

private

Definition at line 989 of file ShippedTranslationsBuild.php.

                                                       : void
    {
        if (!is_dir($directory) && !@mkdir($directory, 0755, true) && !is_dir($directory)) {
            throw new RuntimeException(sprintf('Could not create the directory "%s".', $directory));
        }
    }

Referenced by ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild\run().

Here is the caller graph for this function:

◆ findCollisions()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::findCollisions ( array $values_per_language )

private

The identifiers with different values in several modules, per language.

Parameters

array<string,array<string,array<string,string>>> $values_per_language lang => module => identifier => value

Returns: array{0: list<string>, 1: list<string>} one line per identifier (for the manifest), and one per set of modules naming its identifiers (for the output)

Definition at line 533 of file ShippedTranslationsBuild.php.

                                                               : array
    {
        $collisions = [];
        foreach ($values_per_language as $lang_key => $modules) {
            $values_of = [];
            foreach ($modules as $module => $values) {
                foreach ($values as $identifier => $value) {
                    $values_of[(string) $identifier][$module] = $value;
                }
            }
            foreach ($values_of as $identifier => $values) {
                if (count(array_unique($values)) > 1) {
                    ksort($values, SORT_STRING);
                    $collisions[$identifier . "\0" . implode(', ', array_keys($values))][] = (string) $lang_key;
                }
            }
        }
        ksort($collisions, SORT_STRING);
 
        $list = [];
        $identifiers_of = [];
        foreach ($collisions as $key => $lang_keys) {
            [$identifier, $modules] = explode("\0", (string) $key, 2);
            sort($lang_keys, SORT_STRING);
            $list[] = sprintf('Key "%s" has different values in the modules %s (languages: %s) - txt() serves the one of the module loaded last.', $identifier, $modules, implode(', ', $lang_keys));
            $identifiers_of[$modules][] = $identifier;
        }
        ksort($identifiers_of, SORT_STRING);
        $summary = [];
        foreach ($identifiers_of as $modules => $identifiers) {
            $summary[] = sprintf('%d key(s) with different values in the modules %s - txt() serves the one of the module loaded last: %s', count($identifiers), $modules, implode(', ', $identifiers));
        }
 
        return [$list, $summary];
    }

References $module, and ILIAS\UI\examples\Symbol\Glyph\Sort\sort().

Here is the call graph for this function:

◆ findLocaleSource()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::findLocaleSource ( )

private

Exceptions

RuntimeException if no locale to ship is found

Definition at line 786 of file ShippedTranslationsBuild.php.

                                       : string
    {
        foreach ($this->locale_source !== null ? [$this->locale_source] : self::LOCALE_SOURCES as $candidate) {
            if (is_file($candidate . '/LC_MESSAGES/SYS_LC_MESSAGES') || is_file($candidate . '/LC_CTYPE')) {
                return $candidate;
            }
        }
 
        throw new RuntimeException(sprintf(
            'The locale "C.utf8" of the C library was not found (looked for %s) - it is needed to serve the language files with native gettext.',
            implode(', ', $this->locale_source !== null ? [$this->locale_source] : self::LOCALE_SOURCES)
        ));
    }

Referenced by ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild\runLocked().

Here is the caller graph for this function:

◆ findShippedFiles()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::findShippedFiles ( callable $inform )

private

Every shipped .po of every contributed directory with a module prefix (the main lang/ directory has none and holds no .po), per module.

Parameters

callable(string) void $inform

Returns: array{0: array<string, array{directory: LanguageFileDirectory, languages: array<string, string>, template: ?string}>, 1: list<string>} the modules with .po files, sorted by module (languages: lang => shipped .po, sorted by language), and every contributed module, sorted

Definition at line 622 of file ShippedTranslationsBuild.php.

                                                       : array
    {
        $modules = [];
        $contributed = [];
        $matching_files = [];
        $all_shipped_files = [];
        foreach ($this->language_file_directory_manager->getDirectories() as $directory) {
            $module = $directory->getPrefix();
            if ($module === '') {
                continue;
            }
            if (!MigratedLanguageFilePaths::isValidModule($module)) {
                $inform(sprintf('WARNING: The language files of module "%s" are not compiled: the module name cannot be part of a file name.', $module));
                continue;
            }
            if (isset($modules[$module])) {
                $inform(sprintf('WARNING: Module "%s" is contributed twice - only "%s" is compiled.', $module, $modules[$module]['directory']->getPath()));
                continue;
            }
            // also with an invalid file name pattern
            $contributed[$module] = $module;
            try {
                $pattern = MigratedLanguageFilePaths::shippedFileNamePattern($directory);
                $shipped_files = MigratedLanguageFilePaths::findShippedPoFiles($this->ilias_absolute_path, $directory);
                $template = MigratedLanguageFilePaths::shippedTemplatePath($this->ilias_absolute_path, $directory);
            } catch (\InvalidArgumentException $e) {
                $inform(sprintf('WARNING: The language files of module "%s" are not compiled: %s', $module, $e->getMessage()));
                continue;
            }
            if (!MigratedLanguageFilePaths::isFoundWithoutGraph($this->ilias_absolute_path, $directory)) {
                $inform(sprintf(
                    'WARNING: Module "%s" has no template "components/<Vendor>/<Component>/lang/%s.pot" - without a build Setup and CLI do not recognise it as shipped.',
                    $module,
                    $module
                ));
            }
            $base = rtrim($this->ilias_absolute_path, '/') . '/' . ltrim($directory->getPath(), '/');
            $matching_files = [...$matching_files, ...(glob($base . sprintf($pattern, '*') . '.po') ?: [])];
            $all_shipped_files = [...$all_shipped_files, ...array_values($shipped_files)];
            if ($shipped_files === []) {
                continue;
            }
            $modules[$module] = [
                'directory' => $directory,
                'languages' => $shipped_files,
                'template' => is_file($template) ? $template : null,
            ];
        }
        // files matching a pattern that yield no language key, e.g. "tos_pt_BR.po"; "chatroom_*.po" also
        // matches "chatroom_adm_de.po" of another module in the same directory
        foreach (array_unique(array_diff($matching_files, $all_shipped_files)) as $unknown) {
            $inform(sprintf('WARNING: "%s" is not compiled: its name contains no valid language key.', $this->relativePath($unknown)));
        }
        ksort($modules, SORT_STRING);
        sort($contributed, SORT_STRING);
 
        return [$modules, $contributed];
    }

References Vendor\Package\$e, $module, $pattern, and ILIAS\UI\examples\Symbol\Glyph\Sort\sort().

Referenced by ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild\runLocked().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ formerCatalog()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::formerCatalog	(	string	$artifact_directory,
		array	$former_builds,
		string	$module,
		string	$lang_key
	)

private

The catalog of $module/$lang_key in the first former build that has it intact and compiled the same way (see catalogCompatibility()).

The bytes are taken over, not recompiled: reading a .mo back would shift plural forms (see TranslationCatalog::fromMoString()). Values are checked against the markup policy again.

Parameters

list<string> $former_builds build ids

Returns: array{mo: string, values: array<string, string>, build: string, hash: string}|null values: see ShippedTranslations::compileCatalog(); hash: of the .po it was compiled from

Definition at line 445 of file ShippedTranslationsBuild.php.

                                                                                                                      : ?array
    {
        $policy = new TranslationMarkupPolicy();
        foreach ($former_builds as $former_build) {
            $build_directory = MigratedLanguageFilePaths::buildDirectory($artifact_directory, $former_build);
            $manifest = self::readManifest($build_directory);
            if ($manifest === null || $manifest['compatibility'] !== self::catalogCompatibility()) {
                continue;
            }
            $file = MigratedLanguageFilePaths::buildMoFile($build_directory, $module, $lang_key);
            [$size, $hash] = $manifest['files'][substr($file, strlen($build_directory) + 1)] ?? [null, null];
            $resolved_file = realpath($file);
            $resolved_directory = realpath($build_directory);
            $mo = $size === null
                || $resolved_file === false
                || $resolved_directory === false
                || !str_starts_with($resolved_file, $resolved_directory . '/')
                ? false
                : @file_get_contents($resolved_file);
            if ($mo === false || strlen($mo) !== $size || hash('sha256', $mo) !== $hash) {
                continue;
            }
            try {
                $catalog = TranslationCatalog::fromMoString($mo);
            } catch (RuntimeException) {
                continue;
            }
            $values = [];
            foreach ($catalog->getEntries() as $entry) {
                foreach ([$entry->getTranslation(), ...$entry->getPluralTranslations()] as $value) {
                    if (!$policy->isAllowed($value)) {
                        continue 3;
                    }
                }
                if ($entry->getContext() === null && !$entry->isPlural() && $entry->getTranslation() !== '') {
                    $values[$entry->getId()] = $entry->getTranslation();
                }
            }
 
            return [
                'mo' => $mo,
                'values' => $values,
                'build' => $former_build,
                'hash' => $manifest['hashes'][$module][$lang_key] ?? '',
            ];
        }
 
        return null;
    }

References $module, and $policy.

◆ hashes()

static ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::hashes ( mixed $value )

staticprivate

Returns: array<string, array<string, string>> module => language => hash

Definition at line 1006 of file ShippedTranslationsBuild.php.

                                                : array
    {
        $hashes = [];
        foreach (is_array($value) ? $value : [] as $module => $languages) {
            foreach (is_array($languages) ? $languages : [] as $lang_key => $hash) {
                if (is_string($module) && is_string($lang_key) && is_string($hash)) {
                    $hashes[$module][$lang_key] = $hash;
                }
            }
        }
 
        return $hashes;
    }

References $module.

◆ holdsOnlyCatalogs()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::holdsOnlyCatalogs ( string $directory )

private

Whether $directory (of the former layout <lang>/<module>.mo) holds nothing but catalogs and temporary files of them.

Definition at line 947 of file ShippedTranslationsBuild.php.

                                                         : bool
    {
        foreach (scandir($directory) ?: [] as $name) {
            if ($name !== '.' && $name !== '..' && (!str_contains($name, '.mo') || !is_file($directory . '/' . $name))) {
                return false;
            }
        }
 
        return true;
    }

◆ isIntact()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::isIntact	(	string	$build_directory,
		array	$manifest
	)

private

Whether every file of $manifest is in $build_directory as it was written - without fsync(), a crash can leave empty or truncated files behind.

Parameters

array{files array<string, array{0: int, 1: string}>} $manifest

Definition at line 874 of file ShippedTranslationsBuild.php.

                                                                       : bool
    {
        foreach ($manifest['files'] as $relative => [$size, $hash]) {
            $file = $build_directory . '/' . $relative;
            clearstatcache(true, $file);
            if (
                str_contains($relative, '..')
                || is_link($file)
                || !is_file($file)
                || @filesize($file) !== $size
                || @hash_file('sha256', $file) !== $hash
            ) {
                return false;
            }
        }
 
        return $manifest['files'] !== [];
    }

Referenced by ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild\runLocked().

Here is the caller graph for this function:

◆ listFiles()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::listFiles ( string $directory )

private

Returns: array<string, string> path relative to $directory => absolute path, sorted

Definition at line 762 of file ShippedTranslationsBuild.php.

                                                 : array
    {
        $files = [];
        foreach (scandir($directory) ?: [] as $name) {
            if ($name === '.' || $name === '..') {
                continue;
            }
            $path = $directory . '/' . $name;
            if (is_dir($path)) {
                foreach ($this->listFiles($path) as $relative => $file) {
                    $files[$name . '/' . $relative] = $file;
                }
            } elseif (is_file($path)) {
                $files[$name] = $path;
            }
        }
        ksort($files, SORT_STRING);
 
        return $files;
    }

References $files.

◆ platformProblem()

static ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::platformProblem ( string $build_directory )

static

Why a build made with a system locale (macOS/BSD) cannot be activated by a PHP needing the shipped one.

Definition at line 1045 of file ShippedTranslationsBuild.php.

                                                                   : ?string
    {
        if (!array_key_exists($build_directory, self::$system_locale_platforms)) {
            $manifest = self::$manifests[$build_directory] ?? self::readManifest($build_directory);
            self::$system_locale_platforms[$build_directory] = $manifest !== null && $manifest['locale'] === 'system'
                ? ($manifest['platform'] === '' ? 'another platform' : $manifest['platform'])
                : null;
        }
        $platform = self::$system_locale_platforms[$build_directory];
        if ($platform === null || NativeGettext::usesSystemLocale()) {
            return null;
        }
 
        return sprintf(
            'the build was made on %s with a system locale, run "php cli/setup.php build" on the platform of the web server',
            $platform
        );
    }

Referenced by ILIAS\Language\ComponentTranslation\TranslationGettextRepository\state(), ILIAS\Language\ComponentTranslation\ShippedTranslationsBuildTest\testPlatformProblemIsNullForAShippedLocaleBuildAndForAManifestOfAFormerVersion(), and ILIAS\Language\ComponentTranslation\ShippedTranslationsBuildTest\testPlatformProblemTellsASystemLocaleBuildOnlyWhereTheShippedLocaleIsNeeded().

Here is the caller graph for this function:

◆ readManifest()

static ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::readManifest ( string $build_directory )

staticprivate

Returns: Manifest|null

Definition at line 819 of file ShippedTranslationsBuild.php.

                                                                 : ?array
    {
        $content = is_link($build_directory) ? false : @file_get_contents($build_directory . '/' . self::MANIFEST_FILE);
        $data = $content === false ? null : json_decode($content, true);
        $sample = $data['sample'] ?? null;
        if (
            !is_array($data)
            || !is_array($data['files'] ?? null)
            || !is_array($sample)
            || !is_string($sample['module'] ?? null)
            || !is_string($sample['lang'] ?? null)
            || !is_string($sample['key'] ?? null)
            || !is_string($sample['value'] ?? null)
            || !MigratedLanguageFilePaths::isValidModule($sample['module'])
            || !MigratedLanguageFilePaths::isValidLanguageKey($sample['lang'])
        ) {
            return null;
        }
        $files = [];
        foreach ($data['files'] as $file => $details) {
            if (!is_string($file) || !is_array($details) || !is_int($details[0] ?? null) || !is_string($details[1] ?? null)) {
                return null;
            }
            $files[$file] = [$details[0], $details[1]];
        }
        $languages = [];
        foreach (is_array($data['languages'] ?? null) ? $data['languages'] : [] as $module => $lang_keys) {
            if (!is_string($module) || !MigratedLanguageFilePaths::isValidModule($module) || !is_array($lang_keys)) {
                return null;
            }
            $languages[$module] = array_values(array_filter(
                $lang_keys,
                static fn($lang_key): bool => is_string($lang_key) && MigratedLanguageFilePaths::isValidLanguageKey($lang_key)
            ));
        }
 
        return [
            'files' => $files,
            'sample' => $sample,
            'languages' => $languages,
            'warnings' => self::stringList($data['warnings'] ?? []),
            'hashes' => self::hashes($data['hashes'] ?? []),
            'collisions' => self::stringList($data['collisions'] ?? []),
            'compatibility' => is_string($data['compatibility'] ?? null) ? $data['compatibility'] : '',
            'locale' => ($data['locale'] ?? null) === 'system' ? 'system' : 'shipped',
            'platform' => is_string($data['platform'] ?? null) ? $data['platform'] : '',
        ];
    }

References $data, $files, $module, and Vendor\Package\$sample.

Referenced by ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild\runLocked().

Here is the caller graph for this function:

◆ readPointer()

static ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::readPointer ( string $artifact_directory )

static

The pointer of this PHP's locale kind in $artifact_directory, null if it is missing or not valid.

Returns: Pointer|null modules: module => the languages it is built for; shipped: null in a pointer written before it was recorded

Definition at line 254 of file ShippedTranslationsBuild.php.

                                                                  : ?array
    {
        return self::readPointerOf($artifact_directory, NativeGettext::usesSystemLocale());
    }

References ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild\readPointerOf(), and ILIAS\Language\ComponentTranslation\NativeGettext\usesSystemLocale().

Referenced by ILIAS\Language\ComponentTranslation\MigratedTranslations\pointer(), ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild\runLocked(), ILIAS\Language\ComponentTranslation\ShippedTranslationsBuildTest\testEveryContributedModuleIsRecordedAlsoWithoutAnyPoAndWithoutABuild(), ILIAS\Language\ComponentTranslation\ShippedTranslationsBuildTest\testOnlyTheCurrentAndThePreviousBuildStayAndUnrelatedFilesAreLeftAlone(), ILIAS\Language\ComponentTranslation\ShippedTranslationsBuildTest\testReadPointerTakesOnlyValidModulesOfTheShippedList(), and ILIAS\Language\ComponentTranslation\ShippedTranslationsBuildTest\testRebuildingAnEarlierStateSwitchesBackWithoutLosingThePreviousBuild().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ readPointerOf()

static ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::readPointerOf	(	string	$artifact_directory,
		bool	$system_locale
	)

static

Returns: Pointer|null sources: fingerprint of the shipped files built, null in a pointer written before it was recorded or without a build

Definition at line 263 of file ShippedTranslationsBuild.php.

                                                                                         : ?array
    {
        $content = @file_get_contents(MigratedLanguageFilePaths::buildPointerFileOf($artifact_directory, $system_locale));
        $data = $content === false ? null : json_decode($content, true);
        if (!is_array($data) || !is_array($data['modules'] ?? null)) {
            return null;
        }
        $build = $data['build'] ?? null;
        $previous = $data['previous'] ?? null;
        if (
            ($build !== null && (!is_string($build) || !MigratedLanguageFilePaths::isBuildId($build)))
            || ($previous !== null && (!is_string($previous) || !MigratedLanguageFilePaths::isBuildId($previous)))
        ) {
            return null;
        }
 
        $modules = [];
        foreach ($data['modules'] as $module => $languages) {
            if (!is_string($module) || !MigratedLanguageFilePaths::isValidModule($module) || !is_array($languages)) {
                return null;
            }
            $modules[$module] = array_values(array_filter(
                $languages,
                static fn($lang_key): bool => is_string($lang_key) && MigratedLanguageFilePaths::isValidLanguageKey($lang_key)
            ));
        }
 
        // an invalid list only loses itself: the runtime falls back to the templates
        $shipped = is_array($data['shipped'] ?? null) && array_is_list($data['shipped']) ? $data['shipped'] : null;
        if ($shipped !== null) {
            $shipped = array_values(array_filter(
                $shipped,
                static fn($module): bool => is_string($module) && MigratedLanguageFilePaths::isValidModule($module)
            ));
        }
 
        return [
            'build' => $build,
            'previous' => $previous,
            'modules' => $modules,
            'shipped' => $shipped,
            'sources' => is_string($data['sources'] ?? null) ? $data['sources'] : null,
        ];
    }

References $data, $module, ILIAS\Language\ComponentTranslation\MigratedLanguageFilePaths\buildPointerFileOf(), ILIAS\Language\ComponentTranslation\MigratedLanguageFilePaths\isBuildId(), ILIAS\Language\ComponentTranslation\MigratedLanguageFilePaths\isValidLanguageKey(), and ILIAS\Language\ComponentTranslation\MigratedLanguageFilePaths\isValidModule().

Referenced by ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild\readPointer(), ILIAS\Language\ComponentTranslation\MigratedTranslations\reportNewerOtherBuild(), and ILIAS\Language\ComponentTranslation\ShippedTranslationsBuildTest\testBuildsOfTheOtherLocaleKindAndTheirPointerSurviveTheCleanup().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ relativePath()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::relativePath ( string $file )

private

Definition at line 996 of file ShippedTranslationsBuild.php.

                                               : string
    {
        $root = rtrim($this->ilias_absolute_path, '/') . '/';
 
        return str_starts_with($file, $root) ? substr($file, strlen($root)) : $file;
    }

Referenced by ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild\runLocked().

Here is the caller graph for this function:

◆ removeObsolete()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::removeObsolete	(	string	$artifact_directory,
		array	$keep
	)

private

Removes everything of a former build of this locale kind except the builds $keep: other builds, temporary directories and files of aborted builds, and the former layout (<lang>/<module>.mo, index.json).

Anything else stays, also everything of the other kind (whose build may run at the same time where the lock does not work, e.g. across a Docker Desktop mount).

Parameters

list<string> $keep build ids

Returns: int the number of removed entries

Definition at line 917 of file ShippedTranslationsBuild.php.

                                                                            : int
    {
        $removed = 0;
        $system_locale = NativeGettext::usesSystemLocale();
        $pointer_name = basename(MigratedLanguageFilePaths::buildPointerFile($artifact_directory));
        foreach (scandir($artifact_directory) ?: [] as $name) {
            $path = $artifact_directory . '/' . $name;
            $obsolete = match (true) {
                $name === '.' || $name === '..' || in_array($name, $keep, true) => false,
                MigratedLanguageFilePaths::isBuildId($name) => is_dir(MigratedLanguageFilePaths::buildLocaleDirectory($path)) !== $system_locale,
                preg_match($system_locale ? '/\A\.b-[0-9a-f]+\.system\.[0-9a-f]+\z/' : '/\A\.b-[0-9a-f]+\.[0-9a-f]+\z/', $name) === 1,
                str_starts_with($name, '.' . $pointer_name . '.'),
                $name === 'index.json',
                str_starts_with($name, '.index.json.') => true,
                preg_match('/\A[a-z]{2}\z/', $name) === 1 => is_dir($path) && !is_link($path) && $this->holdsOnlyCatalogs($path),
                default => false,
            };
            if ($obsolete) {
                $this->removeTree($artifact_directory, $path);
                $removed++;
            }
        }
 
        return $removed;
    }

Referenced by ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild\runLocked().

Here is the caller graph for this function:

◆ removeTree()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::removeTree	(	string	$artifact_directory,
		string	$path
	)

private

Removes $path recursively - only below $artifact_directory, and without following a symbolic link (a link itself is removed).

Definition at line 962 of file ShippedTranslationsBuild.php.

                                                                         : void
    {
        if (!str_starts_with($path, rtrim($artifact_directory, '/') . '/') || str_contains($path, '/../')) {
            return;
        }
        if (is_link($path) || is_file($path)) {
            @unlink($path);
            return;
        }
        if (!is_dir($path)) {
            return;
        }
        foreach (scandir($path) ?: [] as $name) {
            if ($name !== '.' && $name !== '..') {
                $this->removeTree($artifact_directory, $path . '/' . $name);
            }
        }
        @rmdir($path);
    }

Referenced by ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild\runLocked().

Here is the caller graph for this function:

◆ run()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::run ( callable $inform )

Parameters

callable(string $message): void $inform progress and warnings

Returns: array{build: ?string, compiled: int, unchanged: bool, rebuilt: bool, removed: int, warnings: list<string>, collisions: list<string>, collision_summary: list<string>, manifest: ?string, markup_warnings: int, other_warnings: list<string>} build: the id now served (null without any migrated module); compiled: .po files compiled (0 for an unchanged build); rebuilt: the build was written now, not reused; removed: files and directories of former builds; collision_summary: the collisions grouped by modules, only if rebuilt; manifest: the manifest of the served build, relative to the ILIAS directory; markup_warnings: how many of the warnings are about disallowed markup; other_warnings: the rest

Exceptions

RuntimeException if the build cannot be written, the locale is missing, or native gettext does not serve it - the pointer is left unchanged then

Definition at line 145 of file ShippedTranslationsBuild.php.

                                         : array
    {
        $artifact_directory = MigratedLanguageFilePaths::shippedArtifactDirectory($this->ilias_absolute_path);
        $this->ensureDirectory($artifact_directory);
 
        // one build at a time (e.g. two Setup runs)
        $lock = @fopen($artifact_directory . '/' . self::LOCK_FILE, 'c');
        if ($lock === false || !flock($lock, LOCK_EX)) {
            throw new RuntimeException(sprintf('Could not lock "%s".', $artifact_directory . '/' . self::LOCK_FILE));
        }
        try {
            return $this->runLocked($artifact_directory, $inform);
        } finally {
            flock($lock, LOCK_UN);
            fclose($lock);
        }
    }

References ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild\ensureDirectory(), ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild\runLocked(), and ILIAS\Language\ComponentTranslation\MigratedLanguageFilePaths\shippedArtifactDirectory().

Here is the call graph for this function:

◆ runLocked()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::runLocked	(	string	$artifact_directory,
		callable	$inform
	)

private

Parameters

callable(string $message): void $inform

Returns: array{build: ?string, compiled: int, unchanged: bool, rebuilt: bool, removed: int, warnings: list<string>, collisions: list<string>, collision_summary: list<string>, manifest: ?string, markup_warnings: int, other_warnings: list<string>}

Definition at line 167 of file ShippedTranslationsBuild.php.

                                                                            : array
    {
        $pointer = self::readPointer($artifact_directory);
        [$modules, $shipped] = $this->findShippedFiles($inform);
 
        if ($modules === []) {
            $this->writePointer($artifact_directory, ['build' => null, 'previous' => null, 'modules' => [], 'shipped' => $shipped, 'sources' => null]);
            return [
                'build' => null,
                'compiled' => 0,
                'unchanged' => ($pointer['build'] ?? null) === null,
                'rebuilt' => false,
                'removed' => $this->removeObsolete($artifact_directory, []),
                'warnings' => [],
                'collisions' => [],
                'collision_summary' => [],
                'manifest' => null,
                'markup_warnings' => 0,
                'other_warnings' => [],
            ];
        }
 
        $locale_source = NativeGettext::usesSystemLocale() ? null : $this->findLocaleSource();
        $source_parts = $this->sourceParts($modules);
        $build_id = $this->buildId($source_parts, $locale_source);
        $build_directory = MigratedLanguageFilePaths::buildDirectory($artifact_directory, $build_id);
        $manifest = self::readManifest($build_directory);
        $compiled = 0;
        $collision_summary = null;
        if ($manifest === null || !$this->isIntact($build_directory, $manifest)) {
            $former_builds = array_values(array_diff(
                array_filter([$pointer['build'] ?? null, $pointer['previous'] ?? null], 'is_string'),
                [$build_id]
            ));
            [$manifest, $compiled, $collision_summary] = $this->write($artifact_directory, $build_directory, $build_id, $modules, $locale_source, $former_builds);
        }
 
        try {
            $this->assertNativeGettextServes($build_directory, $manifest['sample']);
        } catch (RuntimeException $e) {
            if (!in_array($build_id, [$pointer['build'] ?? null, $pointer['previous'] ?? null], true)) {
                $this->removeTree($artifact_directory, $build_directory);
            }
            throw $e;
        }
 
        $unchanged = ($pointer['build'] ?? null) === $build_id;
        $previous = ($pointer['build'] ?? null) !== null && !$unchanged
            ? $pointer['build']
            : ($pointer['previous'] ?? null);
        $new_pointer = [
            'build' => $build_id,
            'previous' => $previous,
            'modules' => $manifest['languages'],
            'shipped' => $shipped,
            'sources' => hash('sha256', implode("\n", $source_parts)),
        ];
        if ($new_pointer !== $pointer) {
            $this->writePointer($artifact_directory, $new_pointer);
        }
 
        $other_warnings = array_values(array_filter(
            $manifest['warnings'],
            static fn(string $warning): bool => !str_starts_with($warning, self::MARKUP_WARNING)
        ));
 
        return [
            'build' => $build_id,
            'compiled' => $compiled,
            'unchanged' => $unchanged && $compiled === 0,
            'rebuilt' => $collision_summary !== null,
            'removed' => $this->removeObsolete($artifact_directory, array_values(array_filter([$build_id, $previous]))),
            'warnings' => $manifest['warnings'],
            'collisions' => $manifest['collisions'],
            'collision_summary' => $collision_summary ?? [],
            'manifest' => $this->relativePath($build_directory . '/' . self::MANIFEST_FILE),
            'markup_warnings' => count($manifest['warnings']) - count($other_warnings),
            'other_warnings' => $other_warnings,
        ];
    }

Referenced by ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild\run().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ servesShippedPo()

static ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::servesShippedPo	(	string	$artifact_directory,
		string	$module,
		string	$lang_key,
		string	$shipped_po
	)

static

Whether the build of $artifact_directory serves exactly the shipped .po $shipped_po of $module for $lang_key (its content hash, see the manifest) - false without a build or after a change of the .po since (e.g.

by "merge", see ShippedPoMerger).

Definition at line 1025 of file ShippedTranslationsBuild.php.

                                                                                                                            : bool
    {
        $build = self::readPointer($artifact_directory)['build'] ?? null;
        if ($build === null) {
            return false;
        }
        $build_directory = MigratedLanguageFilePaths::buildDirectory($artifact_directory, $build);
        if (!isset(self::$manifests[$build_directory])) {
            $manifest = self::readManifest($build_directory);
            if ($manifest === null) {
                return false;
            }
            self::$manifests[$build_directory] = $manifest;
        }
        $hash = self::$manifests[$build_directory]['hashes'][$module][$lang_key] ?? null;
 
        return $hash !== null && $hash === @hash_file('sha256', $shipped_po);
    }

References $module.

Referenced by ILIAS\Language\ComponentTranslation\ShippedTranslationsBuildTest\testServesShippedPoIsFalseWhenTheManifestOfTheCurrentBuildHasNoHashes(), ILIAS\Language\ComponentTranslation\ShippedTranslationsBuildTest\testServesShippedPoIsFalseWhenTheManifestOfTheCurrentBuildIsGone(), and ILIAS\Language\ComponentTranslation\ShippedTranslationsBuildTest\testServesShippedPoIsTrueOnlyForThePoTheBuildWasMadeFrom().

Here is the caller graph for this function:

◆ sourceParts()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::sourceParts ( array $modules )

private

Parameters

array<string,array{directory LanguageFileDirectory, languages: array<string, string>, template: ?string}> $modules

Returns: list<string>

Definition at line 701 of file ShippedTranslationsBuild.php.

                                                : array
    {
        $parts = [];
        foreach ($modules as $module => $shipped) {
            $files = $shipped['languages'];
            if ($shipped['template'] !== null) {
                $files['pot'] = $shipped['template'];
            }
            foreach ($files as $lang_key => $file) {
                $hash = @hash_file('sha256', $file);
                $parts[] = $module . '|' . $lang_key . '|' . $this->relativePath($file) . '|' . ($hash === false ? 'unreadable' : $hash);
            }
        }
 
        return $parts;
    }

References $files, and $module.

Referenced by ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild\runLocked().

Here is the caller graph for this function:

◆ stringList()

static ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::stringList ( mixed $value )

staticprivate

Returns: list<string>

Definition at line 1067 of file ShippedTranslationsBuild.php.

                                                    : array
    {
        return is_array($value) ? array_values(array_filter($value, 'is_string')) : [];
    }

◆ templateKeys()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::templateKeys	(	?string	$template,
		string	$module,
		array &	$warnings
	)

private

The identifiers of $module in its template (.pot), [] if there is none or it cannot be read (reported).

Parameters

list<string> $warnings

Returns: array<string, true>

Definition at line 510 of file ShippedTranslationsBuild.php.

                                                                                      : array
    {
        if ($template === null) {
            return [];
        }
        try {
            return array_fill_keys(
                array_map('strval', array_keys(MigratedLanguageFileSync::moduleEntries(TranslationCatalog::fromPoFile($template), $module))),
                true
            );
        } catch (RuntimeException $e) {
            $warnings[] = sprintf('The template "%s" cannot be read: %s', $this->relativePath($template), $e->getMessage());
            return [];
        }
    }

References Vendor\Package\$e, and $module.

◆ write()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::write	(	string	$artifact_directory,
		string	$build_directory,
		string	$build_id,
		array	$modules,
		?string	$locale_source,
		array	$former_builds
	)

private

Writes the build $build_id into a temporary directory and renames that to $build_directory.

Parameters

array<string,array{directory	LanguageFileDirectory, languages: array<string, string>, template: ?string}> $modules
list<string>	$former_builds where the catalog of a `.po` that cannot be compiled is taken from

Returns: array{0: Manifest, 1: int, 2: list<string>} the manifest of the build, the number of compiled .po files and the collisions grouped by modules (see findCollisions())

Definition at line 316 of file ShippedTranslationsBuild.php.

     : array {
        $temporary = $artifact_directory . '/.' . $build_id . ($locale_source === null ? '.system.' : '.') . bin2hex(random_bytes(6));
        $this->ensureDirectory($temporary . '/' . NativeGettext::CATALOG_DIRECTORY);
        $this->ensureDirectory($temporary . '/keys');
        try {
            $files = [];
            $warnings = [];
            $languages = [];
            $values_per_language = [];
            $sample = null;
            $hashes = [];
            $compiled = 0;
            foreach ($modules as $module => $shipped) {
                $keys = $this->templateKeys($shipped['template'], $module, $warnings);
                foreach ($shipped['languages'] as $lang_key => $shipped_po) {
                    $source = $this->relativePath($shipped_po);
                    $po_hash = (string) @hash_file('sha256', $shipped_po);
                    try {
                        $catalog = $this->shipped_translations->compile(
                            $shipped_po,
                            $module,
                            static function (string $identifier, array $violations) use (&$warnings, $source, $module): void {
                                $warnings[] = sprintf('%s, %s (module "%s", key "%s"): removed %s', self::MARKUP_WARNING, $source, $module, $identifier, implode(', ', $violations));
                            },
                            static function (string $message) use (&$warnings, $source, $module): void {
                                $warnings[] = sprintf('%s (module "%s"): %s', $source, $module, $message);
                            }
                        );
                        $compiled++;
                    } catch (\Throwable $t) {
                        $former = $this->formerCatalog($artifact_directory, $former_builds, $module, $lang_key);
                        if ($former === null) {
                            $warnings[] = sprintf(
                                'Could not compile "%s", module "%s" is not served for "%s" until the file is fixed (no former build has it): %s',
                                $source,
                                $module,
                                $lang_key,
                                $t->getMessage()
                            );
                            continue;
                        }
                        $warnings[] = sprintf(
                            'Could not compile "%s", module "%s" is served for "%s" as in build %s until the file is fixed: %s',
                            $source,
                            $module,
                            $lang_key,
                            $former['build'],
                            $t->getMessage()
                        );
                        $catalog = $former;
                        // the build does not serve this .po, see servesShippedPo()
                        $po_hash = $former['hash'];
                    }
                    $file = MigratedLanguageFilePaths::buildMoFile($temporary, $module, $lang_key);
                    $this->writeFile($file, $catalog['mo']);
                    $files[substr($file, strlen($temporary) + 1)] = [strlen($catalog['mo']), hash('sha256', $catalog['mo'])];
                    $languages[$module][] = $lang_key;
                    $hashes[$module][$lang_key] = $po_hash;
                    $keys += array_fill_keys(array_map('strval', array_keys($catalog['values'])), true);
                    $values_per_language[$lang_key][$module] = $catalog['values'];
                    foreach ($catalog['values'] as $identifier => $value) {
                        if ($sample === null && $value !== (string) $identifier) {
                            $sample = ['module' => $module, 'lang' => $lang_key, 'key' => (string) $identifier, 'value' => $value];
                        }
                    }
                }
                if (!isset($languages[$module])) {
                    continue;
                }
                ksort($keys, SORT_STRING);
                $content = "<?php\n\n// Generated by the ILIAS build (ShippedTranslationsBuild) - do not edit.\n\nreturn "
                    . var_export(array_fill_keys(array_map('strval', array_keys($keys)), $module), true) . ";\n";
                $file = MigratedLanguageFilePaths::buildKeysFile($temporary, $module);
                $this->writeFile($file, $content);
                $files[substr($file, strlen($temporary) + 1)] = [strlen($content), hash('sha256', $content)];
            }
            if ($sample === null) {
                throw new RuntimeException('No migrated module has a single translated value - nothing to check native gettext with.');
            }
 
            if ($locale_source !== null) {
                $this->copyLocale($locale_source, MigratedLanguageFilePaths::buildLocaleDirectory($temporary) . '/' . NativeGettext::LOCALE_NAME, $temporary, $files);
            }
            ksort($files, SORT_STRING);
            [$collisions, $collision_summary] = $this->findCollisions($values_per_language);
            $manifest = [
                'files' => $files,
                'sample' => $sample,
                'languages' => $languages,
                'hashes' => $hashes,
                'warnings' => $warnings,
                'collisions' => $collisions,
                'compatibility' => self::catalogCompatibility(),
                'locale' => $locale_source === null ? 'system' : 'shipped',
                'platform' => PHP_OS_FAMILY,
            ];
            $this->writeFile($temporary . '/' . self::MANIFEST_FILE, json_encode($manifest, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE | JSON_THROW_ON_ERROR) . "\n");
 
            // a damaged build of the same id (see isIntact()) is replaced
            $this->removeTree($artifact_directory, $build_directory);
            if (!@rename($temporary, $build_directory)) {
                throw new RuntimeException(sprintf('Could not rename "%s" to "%s".', $temporary, $build_directory));
            }
        } catch (\Throwable $t) {
            $this->removeTree($artifact_directory, $temporary);
            throw $t instanceof RuntimeException ? $t : new RuntimeException($t->getMessage(), 0, $t);
        }
 
        return [$manifest, $compiled, $collision_summary];
    }

Referenced by ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild\runLocked().

Here is the caller graph for this function:

◆ writeFile()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::writeFile	(	string	$file,
		string	$content
	)

private

Definition at line 982 of file ShippedTranslationsBuild.php.

                                                             : void
    {
        if (@file_put_contents($file, $content) !== strlen($content)) {
            throw new RuntimeException(sprintf('Could not write "%s".', $file));
        }
    }

◆ writePointer()

ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::writePointer	(	string	$artifact_directory,
		array	$pointer
	)

private

Parameters

Pointer $pointer

Definition at line 896 of file ShippedTranslationsBuild.php.

                                                                             : void
    {
        AtomicFileWriter::write(
            MigratedLanguageFilePaths::buildPointerFile($artifact_directory),
            json_encode(
                $pointer,
                JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE | JSON_INVALID_UTF8_SUBSTITUTE | JSON_THROW_ON_ERROR
            ) . "\n",
            $artifact_directory
        );
    }

Referenced by ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild\runLocked().

Here is the caller graph for this function:

Field Documentation

◆ $manifests

array ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::$manifests = []

staticprivate

Definition at line 104 of file ShippedTranslationsBuild.php.

◆ $system_locale_platforms

array ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::$system_locale_platforms = []

staticprivate

Definition at line 111 of file ShippedTranslationsBuild.php.

◆ CATALOG_CLASSES

const array ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::CATALOG_CLASSES

private

Initial value:

= [
        ShippedTranslations::class,
        TranslationMarkupPolicy::class,
        TranslationCatalog::class,
        TranslationEntry::class,
        PluralForms::class,
        PluralFormKey::class,
        MigratedLanguageFileSync::class,
    ]

The classes that determine the bytes of a compiled catalog (MigratedLanguageFileSync: moduleEntries()).

Definition at line 84 of file ShippedTranslationsBuild.php.

◆ FORMAT_VERSION

const int ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::FORMAT_VERSION = 4

private

Raise to rebuild when the output changes for a reason the fingerprint of the involved classes does not capture, e.g.

an update of the gettext library.

Definition at line 79 of file ShippedTranslationsBuild.php.

◆ LOCALE_SOURCES

const array ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::LOCALE_SOURCES

private

Initial value:

= [
        '/usr/lib/locale/C.utf8',
        '/usr/lib/locale/C.UTF-8',
        '/usr/lib64/locale/C.utf8',
        '/usr/lib64/locale/C.UTF-8',
    ]

Where the C library's "C.utf8" locale is looked for (Debian/Ubuntu, Fedora/RHEL, Arch, ...).

Definition at line 116 of file ShippedTranslationsBuild.php.

◆ LOCK_FILE

const string ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::LOCK_FILE = 'build.lock'

private

Definition at line 97 of file ShippedTranslationsBuild.php.

◆ MANIFEST_FILE

const string ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::MANIFEST_FILE = 'manifest.json'

private

Definition at line 94 of file ShippedTranslationsBuild.php.

◆ MARKUP_WARNING

const string ILIAS\Language\ComponentTranslation\ShippedTranslationsBuild::MARKUP_WARNING = 'Markup not allowed in a language value'

private

Definition at line 95 of file ShippedTranslationsBuild.php.

The documentation for this class was generated from the following file:

components/ILIAS/Language/src/ComponentTranslation/ShippedTranslationsBuild.php

Public Member Functions

Static Public Member Functions

Private Member Functions

Static Private Member Functions

Private Attributes

Static Private Attributes

Detailed Description

Constructor & Destructor Documentation

◆ __construct()

Member Function Documentation

◆ assertActivatedNativeGettextServes()

◆ assertNativeGettextServes()

◆ buildId()

◆ catalogCompatibility()

◆ classesFingerprint()

◆ codeFingerprint()

◆ copyLocale()

◆ directoryFingerprint()

◆ ensureDirectory()

◆ findCollisions()

◆ findLocaleSource()

◆ findShippedFiles()

◆ formerCatalog()

◆ hashes()

◆ holdsOnlyCatalogs()

◆ isIntact()

◆ listFiles()

◆ platformProblem()

◆ readManifest()

◆ readPointer()

◆ readPointerOf()

◆ relativePath()

◆ removeObsolete()

◆ removeTree()

◆ run()

◆ runLocked()

◆ servesShippedPo()

◆ sourceParts()

◆ stringList()

◆ templateKeys()

◆ write()

◆ writeFile()

◆ writePointer()

Field Documentation

◆ $manifests

◆ $system_locale_platforms

◆ CATALOG_CLASSES

◆ FORMAT_VERSION

◆ LOCALE_SOURCES

◆ LOCK_FILE

◆ MANIFEST_FILE

◆ MARKUP_WARNING