The files which are really used for building Debian Installer packages are hosted
in the git repository used for the development of Debian Installer
packages. Each Debian Installer package sources is hosted in a subdirectory of
the packages/ directory, with a very classical
Debian package source tree organization, including the
debian/po directory where the package's
translations are hosted.
These files need to be synchronised with the so-called master files from the packages/po directory, since translators only work on these master files. This synchronization must work both ways with changes on the English original strings going from individual packages to the master files and translation changes/updates from the master files going to individual packages:
When Debian Installer packages templates are changed, the individual
debian/po/templates.pot file will
change. These changes have to be moved to files in
packages/po;
When translations are updated in the
packages/po directory, these changes have
to be moved back into individual
debian/po/*.po files for each package. This
does not need to be done at each translation update, but must be
done before building/uploading packages.
The task of keeping track of the correct synchronisation is one of the most important tasks of the Debian Installer i18n coordinators. It is handled by the scripts/l10n/l10n-sync script.
The l10n-sync script is run periodically under the "D-I role" account by a cron job on dillon.debian.org:
0 20 * * * cd $DI; mr -q up ; nice -n 19 $DI/scripts/l10n/l10n-sync --levels=5 --online --commit $DI
Its runs are monitored by the Debian Installer i18n coordinators.
It will update the general
template.pot and all PO files from individual
templates.pot files from all Debian Installer packages.
While running, this script will also run debconf-updatepo for each Debian Installer package and will commit back the regenerated templates.pot files (package maintainers often forget running debconf-udpatepo when commiting changes to English templates).
During normal development, this synchronization process is run once a day. During release preparations, the frequency might be increased to speed up the full cycle.
The general process is:
Initial step: synchronize the whole repository;
Individual packages update step. For each Debian Installer package:
synchronize the local copy with the Debian Installer GIT;
run debconf-updatepo;
commit back the files in debian/po.
Master templates file update step:
merge all templates.pot files to
packages/po/template.pot.
Master PO files update step. For each PO
file in packages/po:
synchronize with Debian Installer GIT;
if the script is run with the --merge option,
merge translations from the reference PO file (the reference file is
given priority for identical strings);
update with
packages/po/template.pot. This step
uses the msgmerge command. The resulting PO file
formatting details will depend on the version of this
utility. For that reasons, the synchronization script should
always be run from machines using the same Debian release
version (these days it permanently runs on dillon anyway);
commit back the changed files to Debian Installer GIT.
Individual packages PO files update step. For each Debian Installer package:
synchronize the local copy with the Debian Installer GIT (in case some update occurred in the meantime);
update debian/po/*.po files with master
files;
commit back the changes to Debian Installer GIT;
This system minimizes race conditions which could trigger conflicts.
The conflict windows have been minimised as much as possible in the l10n-sync script. However, in order to limit the number of commits made by the script, files are not commited as soon as they are modified. So, the script is sometimes likely to trigger conflicts. For that reason, it will stop working on a GIT copy where GIT conflict files are present. This explains why this script must always be monitored even when it is scheduled to be executed periodically.
The l10n synchronisation script is kept in Debian Installer repository in the
scripts/l10n/ directory and is named
l10n-sync.
It is a shell script (which may contain some bashisms until a skilled shell programmer cleans it out).
The script uses some command-line switches which may affect its behaviour. Some of these switches are mostly present for historical reasons and are kept because they may have some new use in the future:
--debug will trigger more output by the
script. Otherwise, the script reports about its actions but
commands output (such as git,
debconf-updatepo,
msgmerge) will be redirected to
/dev/null;
--online triggers git
pull commands before working on files. Except for
testing, there is no reason for not using
this switch.
--commit allows the script to
commit files back to the Debian Installer
repository. Otherwise, the modified files are kept in the
local GIT copy. Of course, this switch should be used in
production. Omitting it is useful only for testing.
the --debconf-updatepo defines the
command for calling the debconf-updatepo
utility. It allows using a specially crafted
debconf-updatepo switch, often the
--skip-merge switch which only updates
templates.pot files and not PO files
(using this generally speeds up the synchronization script);
the --merge switch allows merging
master files from another branch. It is used, for instance, on
the sarge branch for merging translations coming from trunk.
The script needs the location of the local copy of the Debian Installer repository as an argument. It makes some simple checks about the copy.
When using this script with commits, the Debian Installer copy must be as clean as possible. It should not be used for development tasks. The script checks for possible GIT conflict files and aborts if it finds some.
Prospective languages (see Section 6.3, “ Prospective languages ”) are handled in a special way: for all such languages, the translations are not copied in the individual packages directories.
When languages are on early stages of translation after they have been added
through the new language process (Chapter 3, The New Language Process: adding a new language to Debian Installer), they are
temporarily listed in a file named packages/po/PROSPECTIVE.
Languages listed in that file are excluded from synchronization, and,
therefore, PO files are not created in individual packages. As a consequence,
activating a language means that the language code is
removed from the PROSPECTUVE file.
A special file, named packages/po/packages_list,
lists the Debian Installer packages which are handled by the
l10n-sync script and are therefore
included in the master files.
The Debian Installer packages maintainers must request for their packages to be included in this file, when they estimate that their package is ready for translation. Only Debian Installer i18n coordinators can integrate packages there, after checking that the strings have been reviewed.
Some mechanisms have been implemented to add more safety and remote control possibilities to the synchronisation script.
First of all, before doing any work, the script opens a special file
kept in the GIT repository and named
packages/po/run-l10n-sync. The "run=" line in
this file mentions whether synchronisation should happen or not. If
the file contains "run=0", then the script will exit without taking
any action.
This mechanism gives all Debian Installer developers with commit access to the GIT repository a very simple way to disable the synchronization script actions. For this, developers just need to change the file and commit the new version.
While the script is disabled, runs can still be enforced by using the
-force swith, which will ignore the
packages/po/run-l10n-sync file.
While it is running, the script also creates a file named
.l10n-sync.lock at the root of the local copy of
Debian Installer repository. The file is removed only after successful runs of
the script. When this file already exists, the script does not run and
exits with an error message.
This mechanism prevents running the synchronization script twice on the same Debian Installer repository checkout copy.