GNU gettext in PHP: a tutorial with PO file examples

GNU gettext is the GNU Project’s set of tools for translating the messages a program shows its users. You wrap each user-facing string in a call such as gettext("Hello"), or its short form _("Hello"). Then xgettext extracts those strings into a template (POT) file, you create one PO file per language for the translations, and msgfmt compiles each PO file into an MO file. At runtime, gettext looks up the current locale’s MO file and returns the translation, or the original string when there is none.

PHP has a gettext extension, so this tutorial walks through the whole loop in PHP: setup, POT and PO files, plural forms, compiling, the PHP code, and keeping translations current as your code changes.

PHP Internationalization with gettext

This tutorial documents how PHP internationalization with gettext works, including setup, usage, and best practices.

Installing gettext

The gettext library must be installed on both development and production systems.

• Ubuntu/Debian / Fedora / CentOS / RHEL: install via apt-get or yum

• Other Unix-like systems: download from
  http://www.gnu.org/s/gettext

• Windows: download from
  http://gnuwin32.sourceforge.net/packages/gettext.htm

• Some Unix systems already include gettext pre-installed.

Creating a Portable Object Template File (POT)

xgettext is a command-line tool that extracts translatable strings from source files.

It supports many languages, including:

• C / C++

• PHP

• Python

• Java

• Perl

• Shell scripts

• C#

• and many others

Extract strings from PHP files

Run this in your project directory:

xgettext --from-code=UTF-8 -o messages.pot *.php

Limitations of xgettext

• Cannot detect file encoding (defaults to ASCII unless overridden)

• Expects ASCII unless --from-code is used

• Outputs everything into a single domain/file

• Does not automatically split translations into multiple domains

To explore more options:

man xgettext

The result is a POT (Portable Object Template) file, used as a base for translation files or synchronization.

Portable Object Files (PO)

PO files are plain text files that contain translations.

They can be:

• Generated from a POT file

• Created manually

• Edited using a PO editor

Create a PO file for a new language with msginit

msginit creates a PO file from your template and fills in the header for the language you name, including Plural-Forms where gettext knows the rules:

msginit --input=messages.pot \
  --locale=fr_CA \
  --output-file=Locale/fr_CA/LC_MESSAGES/example.po

PO file structure

A PO file contains:

• Optional header (metadata)

• Translation entries

Header metadata may include:

• Language

• Content-Type

• Encoding

• Plural-Forms

PO entry structure

white-space
# translator-comments
#. extracted-comments
#: reference
#, flag
#| previous-msgid
msgctxt context
msgid untranslated-string
msgstr translated-string

Explanation

• msgid → original string in code

• msgstr → translated string

• Comments:

  • # translator comments

  • #. extracted comments from code

  • #: references to source code

  • #, flags (e.g., fuzzy)

  • #| previous msgid

Plural forms

Some languages require different forms depending on number.

Plural entry structure

msgid "singular form"
msgid_plural "plural form"
msgstr[0] "form 0"
msgstr[1] "form 1"
msgstr[N] "form N"

Header requirement

PO files using plurals must define:

Plural-Forms: nplurals=X; plural=EXPRESSION;

Example (Serbian plural rules)

msgid "I wrote a line of code"
msgid_plural "I wrote %d lines of code"
msgstr[0] "Napisao sam %d liniju koda"
msgstr[1] "Napisao sam %d linije koda"
msgstr[2] "Napisao sam %d linija koda"

How it works

• msgstr[0] → numbers ending in 1 (except 11)

• msgstr[1] → numbers ending in 2–4 (except 12–14)

• msgstr[2] → all other numbers

PO file example

Link to code

Directory structure

Once you have a PO file, you should place it into the proper directory structure.

The parent directory that contains all locale directories can be named arbitrarily. Locale subdirectory names should consist of:

• A two-letter lowercase language code according to the ISO 639-1 specification.

• An underscore (_).

• A two-letter uppercase country code according to the ISO 3166-1 alpha-2 specification.

Each locale subdirectory should contain its own LC_MESSAGES directory.

Example directory structure

MyPHPProject/
└── Locale/
    ├── en_UK/
    │   └── LC_MESSAGES/
    ├── de/
    │   └── LC_MESSAGES/
    └── fr_CA/
        └── LC_MESSAGES/

Machine Object Files

Translations are made available to the web server through Machine Object (MO) files.

To compile PO files, a gettext library must be installed.

Installing Gettext

• Ubuntu/Debian: use apt-get

• Fedora/CentOS/RHEL: use yum

• Other Unix-like systems: download from http://www.gnu.org/s/gettext

• Windows: download from http://gnuwin32.sourceforge.net/packages/gettext.htm

Some Unix-like systems already include gettext by default.

Compiling a PO file

Navigate to the directory containing the PO file (for example Locale/de/LC_MESSAGES) and run:

msgfmt example.po -o example.mo

Repeat this process for all PO files in all locale directories.

You are now ready to use the translations to localize your website.

How do you update PO files when the code changes?

Run msgmerge every time you extract a new template. As your code changes, strings are added, removed and reworded. xgettext writes a fresh POT file, but it doesn’t touch the PO files your translators already filled in. msgmerge brings each PO file in line with the new template and keeps every translation that still matches:

xgettext --from-code=UTF-8 -o messages.pot *.php
msgmerge --update --previous \
  Locale/fr_CA/LC_MESSAGES/example.po \
  messages.pot

After the merge, three kinds of entries need a look:

• New strings come in as untranslated entries, with an empty msgstr.

• Reworded strings are marked fuzzy when msgmerge pairs a slightly changed msgid with an old translation. With --previous, the old msgid stays above the entry in a #| comment, so the translator sees what changed.

• Removed strings are commented out as obsolete instead of deleted, so their translations stay in the file.

Fuzzy entries matter when you compile. msgfmt leaves them out of the MO file unless you pass --use-fuzzy, and the gettext manual calls that option “usually wrong” because nobody has checked those translations yet. Until a translator reviews a reworded string, your users see the msgid instead. Before you ship, run msgfmt --check --statistics on each PO file: --check catches broken format strings, and --statistics prints how many strings are translated, fuzzy and untranslated.

With several languages and translators outside your team, this is the step where translations go missing: someone translates an old copy of a PO file, or two copies get merged by hand. The last section shows how LingoHub handles this step.

Note: This article is brought to you by LingoHub, a translation management system that helps you localize apps, games, and websites.

Gettext caching problems

If PHP is run as a module (mod_php), the first time a domain (MO file) is initialized, it is cached by the server.

After updating an MO file, it is often necessary to restart the web server before the new translations become available. This can be problematic in shared hosting environments where developers typically do not have server access.

A common workaround is to create new domains dynamically whenever an MO file is updated. This forces the server to cache the new content without requiring a restart.

References:

• Mastering gettext for localizing your PHP projects

• Workaround for gettext caching issue in PHP

Setting Up PHP for Internationalization with Gettext

After installing the gettext library, update your php.ini file.

Windows

extension=php_gettext.dll

Linux / Unix

extension=gettext.so

The PHP gettext extension does not define any configuration directives, resource types, or constants in php.ini.

Verify Gettext Support

Use the following code snippet to verify that gettext is available:

if (!function_exists("gettext")) {
    echo "gettext is not installed\n";
} else {
    echo "gettext is supported\n";
}

PHP gettext example: a file that uses example.mo

Link to code

• This example expects that locale switching is being done through a GET request and that a locale is being stored in the session; for new sessions a default locale is provided.

• The $locale identifier should correspond and be constructed using the same rules as the locale subdirectory, as was explained earlier.

• putenv() sets the LANG environment variable and instructs gettext which locale it will be using for this session.

• setlocale() specifies the locale used in the application and affects how PHP sorts strings, understands date and time formatting, and formats numeric values.

• domain refers to the catalog file used to store the translation.

• bindtextdomain() tells gettext where to find the domain to use; the first parameter is the catalog name without the .mo extension, and the second parameter is the path to the parent directory in which the de/LC_MESSAGES sub-path resides.

• bind_textdomain_codeset() sets the encoding in which messages from a domain will be returned by gettext() and similar functions. All domains that are called from the code have to be previously bound.

• textdomain() sets the domain to search within when calls are made to gettext().

• _() is an alias of gettext(); it looks up and returns the translation.

• sprintf() can be used to replace any placeholders that might occur in the string.

• dgettext() overrides the current domain for a single message lookup. This can be useful for large projects when it is convenient to split strings into multiple domains (files), for example: emails, countries, languages, etc.

• ngettext() is used when the plural form of the message depends on the count.

What form of msgids should be used?

As previously mentioned, msgid is used by gettext to identify the msgstr that should be displayed in its place. If multiple equal msgids are present, msgctxt is used to tell them apart.

There are several more important roles that msgid plays:

• msgid is what the programmer sees in the code.

• msgid is what is displayed by default if the proper msgstr in the current locale cannot be found (fallback).

• In traditional gettext usage, the translator translates from msgid to the target locale instead of from source msgstr to target msgstr; therefore msgid provides context for translation.

Therefore it is important to choose the form of the msgid carefully.

The value of the msgid can be any string. The gettext manual suggests that the original string in the source (or English) language should be used. However, this can pose problems if the original string changes, because every occurrence in code and all PO files would need to be updated. Long msgids can also reduce code readability.

Some developers propose that msgids should be constructed in a more structured way (5, 6), where the identifier describes its role in the application rather than its content. This can improve organization for developers but may make translation harder because translators no longer see the source string directly.

Others propose selecting a master locale at the start of the project and storing master locale strings only in the msgids (7). In this approach, no resource files exist for the master locale. The application naturally falls back to the msgid whenever a translation is unavailable.

This approach is especially convenient for smaller projects where developers also manage translations. However, if reviewers, marketers, or translators need access to the master locale strings, they would need to work directly with the code.

Internationalization with Gettext in PHP projects using LingoHub

Gettext is a powerful solution for PHP internationalization, but it introduces some inconveniences for both developers and translators. LingoHub, which works with gettext PO files, can simplify the workflow.

1. Single domain, master locale strings in msgid

This approach is straightforward:

• Add new strings to the code as needed.

• Extract them into a POT file using xgettext.

• Upload the POT file to LingoHub as the master locale.

Although a POT file contains only msgids, LingoHub merges it with existing translations and adds only new strings without affecting existing translations.

If a master locale resource file is used, translators or reviewers can populate master locale strings using the msgids as a basis. Translators for other locales will have access to both the master locale string and the msgid.

You can also include translator comments and LingoChecks directly in the source code. These comments can be extracted with xgettext and imported into LingoHub.

Be careful when changing existing msgids, as all existing translations are identified by them.

2. Multiple domains

If your project uses multiple domains, the simplest approach is to manually add new entries to the appropriate master locale template files and upload them as described above.

LingoHub supports multiple resource files (domains) and allows them to be downloaded separately once translations are complete.

3. Structured msgids

If your project uses structured msgids instead of source-language strings, manually add entries to the appropriate PO files and populate both the msgid and the initial master locale msgstr.

After uploading the files to LingoHub, translators can continue the translation process.

One advantage of LingoHub is that translators do not need to understand gettext, PO files, or install any special software. They can translate directly through a web browser, while developers receive translated files in the correct format without manual merging.

I hope this tutorial was helpful. Future articles will explore additional aspects of internationalization.

References

• GNU gettext utilities manual

• Localizing PHP Applications “The Right Way”, Part 3

• Mastering gettext for localizing your PHP projects

• Workaround for gettext caching issue in PHP

• Stack Overflow discussion on msgid naming conventions

• Another Stack Overflow discussion on msgid naming conventions

• xgettext documentation

Further reading

• Localizing PHP Applications “The Right Way”

• Localizing PHP Web Sites Using Gettext

• Localize your web app or website with LingoHub

Ready to localize?

Try LingoHub for your localization project!

Ready to Localize? Try LingoHub for Free

This article is provided by LingoHub, a translation management software used by development and product teams around the world to manage software translations more productively.

Sign up for a free 14-day trial