Upgrading the Localization System

In this post, I will upgrade the localization system I’ve been using for my projects. Check the system itself if you haven’t already. Download the project from Github if you don’t care about the particulars of implementation.

As I mentioned in the original post, I didn’t expect this localization system to scale well. As it turns out, I was correct! The more translations we got, the harder it became to maintain the CSV file. Duplicate strings with slightly different ids start to pop up, and referring to the file each time just to make sure there are no typos in the code became tedious. Also comparing lots of strings is not the best thing you could do from the optimization point of view.

What we needed was to store all those ids as enums. Let’s begin by doing just that. So far we have only one translation string in the Localization.csv file: ui_hello. A corresponding enum would look like this:

public enum Translation
{
    ui_hello = 1,
}

Now we need to change the definition of the Translations dictionary from

Dictionary<string, TranslationString> Translations;

Dictionary<Translation, TranslationString> Translations;

The next steps are more or less obvious. Both Get(in string id) and Get(in string language, in string id) should use the Translation enum as an id instead of a string.

public static string Get(in Translation id)
{
  return Get(CurrentLanguage, id);
}

public static string Get(in string language, Translation id)
{
  if (Translations.ContainsKey(id))
    return Translations[id].Get(language);
  
  Debug.LogErrorFormat("Translations does not contain a string with id '&lt;color=red&gt;{0}&lt;/color&gt;'!", id);
  return string.Format("{0} ({1})", id, language);
}

Now we need to translate a string id to the enum when reading the CSV file. Change the for loop at the end from

for (int i = 1; i < stringCount; i++)
{
    stringTranslations = lines[i].Split(';');//Split the line to individual translated strings
    temp = new TranslationString(langCount);
    for (int j = 0; j < langCount; j++)
    {
        if (stringTranslations.Length >j + 1)//Add the translated string only if it exists
            temp.translationDict.Add(Languages[j], stringTranslations[j + 1].Trim('"'));
        else//add the ID (or any other default string) otherwise
            temp.translationDict.Add(Languages[j], stringTranslations[0]);
    }

	translations.Add(id, temp);
}

for (int i = 1; i < stringCount; i++)
{
    stringTranslations = lines[i].Split(';'); //Split the line to individual translated strings

    if (!Enum.TryParse(stringTranslations[0], out Translation id)) //Check if there is a corresponding enum value
    {
        Debug.LogError(
            $"Translation enum does not contain a value of '{stringTranslations[0]}'! Did you forgot to update the Translations enum?");
        continue;
    }

    TranslationString temp = new TranslationString(langCount);
    for (int j = 0; j < langCount; j++)
    {
        temp.translationDict.Add(Languages[j],
            stringTranslations.Length > j + 1 ? stringTranslations[j + 1].Trim('"') : stringTranslations[0]);
    }

    translations.Add(id, temp);
}

Next, we need to update the StringLocalizer class. Change the type of defaultString from string to the Translation enum. We don’t need the null check in the UpdateLanguage() method, since the enum will never be null. The GetDefaultString() method was there for convenience, and we don’t really need it for the system to function. But I still like it, so I rewrote it as follows:

[ButtonMethod]
private void GetValueFromText()
{
    if(!Enum.TryParse(textField.text, out Translation id))
    {
        Debug.LogError($"The Translation enum does not have a value of '&lt;color=red&gt;{textField.text}&lt;/color&gt;'! Did you forgot to update it?");
        return;
    }

    defaultString = id;
}

Finally in the Reset() method remove the defaultString = textField.text; line since it doesn’t work anymore. You can replace it with the new GetValueFromText(); method if you want.

And that is pretty much it. From a technical point of view, it should work. The only thing you will have to remember is to update the Translation enum after adding a new translation string to the CSV file. A minor inconvenience, but a huge improvement over what we had before. Only if there was a way to make it more error-proof… Oh, wait there is! We can create a script that would read the CSV file, and write the contents to a new file containing the enum! Actually, a single method marked as “[MenuItem(“Localization/Generate Translation Constants”)]” will do just fine.

private static string savePath = Application.dataPath;

private static void Generate()
{
    // Get a path to the csv file with translations
    string translationFile = EditorUtility.OpenFilePanel("Get dictionary", Application.streamingAssetsPath, "csv");
    if (string.IsNullOrEmpty(translationFile)) return;

    // Get a path for the new file
    savePath = EditorUtility.OpenFolderPanel("Save to", savePath, "Translations.cs");
    if (string.IsNullOrEmpty(savePath)) return;

    // Read the csv file with translations
    string[] lines = File.ReadAllLines(translationFile);

    if (lines.Length == 0)
    {
        Debug.LogError($"File '{translationFile}' does not contain any text, or the format is wrong!");
        return;
    }
    
    // Initialize new static class we will be saving the translations into
    string newClass = "public enum Translation\n{\n";
    
    for (int i = 0; i < lines.Length; i++)
        lines[i] = lines[i].Split(';')[0];


    // Add the ID part for each translation to the new Translations class
    for (int i = 0; i < lines.Length; i++)
    {
        // Show the progress bar
        EditorUtility.DisplayProgressBar("Writing strings", "Converting the csv file to a cs class",
            (float) i / lines.Length);

        newClass += $"    {lines[i]} = {i.ToString()},\n";
    }

    // Add the final bracket
    newClass += "}\n";

    // Close the progress bar
    EditorUtility.ClearProgressBar();
    
    // Save the class to the path you choose in the beginning, with tne name Translations.cs
    File.WriteAllText($"{savePath}/Translations.cs", newClass);

    // Force Unity to recompile the scripts
    AssetDatabase.Refresh();

    // Show the success message
    EditorUtility.DisplayDialog("Translations.cs generated!", $"New class Translations.cs is saved to {savePath}!",
        "AWESOME!");
}

You can add this method to any class, but make sure it is compiled only in the editor. You can use platform-dependent compilation, or create a new class and put it in an Editor folder somewhere in the project.

Now, all we have to do is to select the CSV file, a folder to save the new Translations.cs file, and we are done! Of course, there is still some space for improvement. The most obvious one is to have an editor window which would take care of adding new strings both to the CSV file, and the Translation enum. A script to detect if the CSV file and the enum are out of sync would be nice. And depending on your need the CurrentLanguage variable can be an enum. I didn’t prefer that, since it will make it impossible to add new languages without the Unity project. But if you are 100% sure that you won’t be adding any new languages (or allow someone else to add one) after the release of your app, then you can make it so.

As always, you can pull the project from Github. And let me know if you have any feedback!