Internationalization for GitLab

Introduced in GitLab 9.2.

For working with internationalization (i18n) we use GNU gettext given it's the most used tool for this task and we have a lot of applications that will help us to work with it.

Setting up GitLab Development Kit (GDK)

In order to be able to work on the GitLab Community Edition project we must download and configure it through GDK, we can do it by following this guide.

Once we have the GitLab project ready we can start working on the translation of the project.

Tools

We use a couple of gems:

gettext_i18n_rails: this gem allow us to translate content from models, views and controllers. Also it gives us access to the following raketasks:
- rake gettext:find: Parses almost all the files from the Rails application looking for content that has been marked for translation. Finally, it updates the PO files with the new content that it has found.
- rake gettext:pack: Processes the PO files and generates the MO files that are binary and are finally used by the application.
gettext_i18n_rails_js: this gem is useful to make the translations available in JavaScript. It provides the following raketask:
- rake gettext:po_to_json: Reads the contents from the PO files and generates JSON files containing all the available translations.
PO editor: there are multiple applications that can help us to work with PO files, a good option is Poedit which is available for macOS, GNU/Linux and Windows.

Preparing a page for translation

We basically have 4 types of files:

Ruby files: basically Models and Controllers.
HAML files: these are the view files.
ERB files: used for email templates.
JavaScript files: we mostly need to work with VUE JS templates.

Ruby files

If there is a method or variable that works with a raw string, for instance:

def hello
  "Hello world!"
end

Or:

hello = "Hello world!"

You can easily mark that content for translation with:

def hello
  _("Hello world!")
end

Or:

hello = _("Hello world!")

HAML files

Given the following content in HAML:

%h1 Hello world!

You can mark that content for translation with:

%h1= _("Hello world!")

ERB files

Given the following content in ERB:

<h1>Hello world!</h1>

You can mark that content for translation with:

<h1><%= _("Hello world!") %></h1>

JavaScript files

In JavaScript we added the __() (double underscore parenthesis) function for translations.

Updating the PO files with the new content

Now that the new content is marked for translation, we need to update the PO files with the following command:

bundle exec rake gettext:find

This command will update the locale/**/gitlab.edit.po file with the new content that the parser has found.

New translations will be added with their default content and will be marked fuzzy. To use the translation, look for the #, fuzzy mention in gitlab.edit.po and remove it.

We need to make sure we remove the fuzzy translations before generating the locale/**/gitlab.po file. When they aren't removed, the resulting .po will be treated as a binary file which could overwrite translations that were merged before the new translations.

When we are just preparing a page to be translated, but not actually adding any translations. There's no need to generate .po files.

Translations that aren't used in the source code anymore will be marked with ~#; these can be removed to keep our translation files clutter-free.

Validating PO files

To make sure we keep our translation files up to date, there's a linter that is running on CI as part of the static-analysis job.

To lint the adjustments in PO files locally you can run rake gettext:lint.

The linter will take the following into account:

Valid PO-file syntax
Variable usage
- Only one unnamed (%d) variable, since the order of variables might change in different languages
- All variables used in the message-id are used in the translation
- There should be no variables used in a translation that aren't in the message-id
Errors during translation.

The errors are grouped per file, and per message ID:

Errors in `locale/zh_HK/gitlab.po`:
  PO-syntax errors
    SimplePoParser::ParserErrorSyntax error in lines
    Syntax error in msgctxt
    Syntax error in msgid
    Syntax error in msgstr
    Syntax error in message_line
    There should be only whitespace until the end of line after the double quote character of a message text.
    Parseing result before error: '{:msgid=>["", "You are going to remove %{project_name_with_namespace}.\\n", "Removed project CANNOT be restored!\\n", "Are you ABSOLUTELY sure?"]}'
    SimplePoParser filtered backtrace: SimplePoParser::ParserError
Errors in `locale/zh_TW/gitlab.po`:
  1 pipeline
    <%d 條流水線> is using unknown variables: [%d]
    Failure translating to zh_TW with []: too few arguments

In this output the locale/zh_HK/gitlab.po has syntax errors. The locale/zh_TW/gitlab.po has variables that are used in the translation that aren't in the message with id 1 pipeline.

Working with special content

Interpolation

In Ruby/HAML:
```
_("Hello %{name}") % { name: 'Joe' }
```
In JavaScript: Not supported at this moment.

Plurals

In Ruby/HAML:

n_('Apple', 'Apples', 3) => 'Apples'

Using interpolation:

n_("There is a mouse.", "There are %d mice.", size) % size

In JavaScript:

n__('Apple', 'Apples', 3) => 'Apples'

Using interpolation:

n__('Last day', 'Last %d days', 30) => 'Last 30 days'

Namespaces

Sometimes you need to add some context to the text that you want to translate (if the word occurs in a sentence and/or the word is ambiguous).

In Ruby/HAML:
```
s_('OpenedNDaysAgo|Opened')
```
In case the translation is not found it will return Opened.
In JavaScript:
```
s__('OpenedNDaysAgo|Opened')
```

Just marking content for parsing

Sometimes there are some dynamic translations that can't be found by the parser when running bundle exec rake gettext:find. For these scenarios you can use the _N method.

There is also and alternative method to translate messages from validation errors.

Adding a new language

Let's suppose you want to add translations for a new language, let's say French.

The first step is to register the new language in lib/gitlab/i18n.rb:

...
AVAILABLE_LANGUAGES = {
  ...,
  'fr' => 'Français'
}.freeze
...

Next, you need to add the language:
```
bundle exec rake gettext:add_language[fr]
```
If you want to add a new language for a specific region, the command is similar, you just need to separate the region with an underscore (_). For example:
```
bundle exec rake gettext:add_language[en_GB]
```
Please note that you need to specify the region part in capitals.
Now that the language is added, a new directory has been created under the path: locale/fr/. You can now start using your PO editor to edit the PO file located in: locale/fr/gitlab.edit.po.
After you're done updating the translations, you need to process the PO files in order to generate the binary MO files and finally update the JSON files containing the translations:
```
bundle exec rake gettext:compile
```
In order to see the translated content we need to change our preferred language which can be found under the user's Settings (/profile).

After checking that the changes are ok, you can proceed to commit the new files. For example:

git add locale/fr/ app/assets/javascripts/locale/fr/
git commit -m "Add French translations for Cycle Analytics page"