On the multilingual normalization of the Web

Summary

I will try to organize a meeting during the Third WWW Conference on this subject. If I can find facilities, I will post a time and place in this booch.

1. Definitions

1.1. Language Engineering

• Terminology
• Translator's Memory
• Multilingual documentary databases
• Aligned Text
• Translator's Workbench
• Author's Workbench
• Machine Translation
• Publishing (in particular, multilingual synchronized publishing)

1.2. Parallel Texts

1.3. Alignness

1.4. Level of Alignness

• Document level the trivial case; i.e., Parallel Texts
• Paragraph level not too hard
• Sentence level desirable and possible
• Term level it needs tagging (see below)

2. Character set

A Multilingual Web must be able to process Unicode and the present character set. There should be a way to indicate a Unicode file; for example, a file extension or a magic number (Suggestions ?). The programs should also be capable of mapping Unicode to other character sets; for example, to ASCII, ISO 8859-1, PC1, etc. When the target character set is poorer, the mapping should be reasonable; for example, mapping "é" to "e", or unmappable character to "_" or the code number . The different cases should be considered:

1. non-multilingual client ---- non-multilingual server
2. non-multilingual client ---- multilingual server
3. multilingual client ---- non-multilingual server
4. multilingual client ---- multilingual server

2.1. HTML tagging with Unicode

2.2. HTTP/1.0

2.2.1. Accept-Charset

2.2.2. Accept-Encoding

3. Multilingual Aligned Hypertext (MAH)

3.1. Some functionalities

• Making a list of available languages for displaying in a drop menu or similar
• Indicating the level of alignness (paragraph, sentence, etc)
• Display at least two aligned documents side by side and move them in sync
• Give polite error messages such as the document is not available in Swedish
• Most of the Services proposed in Dragoman should be implemented

3.2. Data structure

• Directory based scheme for a single set of files
• Directory based scheme for several sets of files
• SGML (Suggested by Jean Paoli, GRIF, JeanPaoli@grif.fr)
• (Suggestions ? : other schemes; for example tar, cpio)

The default for a single set of files is:

mahDocName.html

DocName.mah                          (directory)
           /en.html                   English
           /es.html                   Spanish
           /de.html                   German


The default for several sets of files is:

mahDatabaseName.html

DatabaseName.mah                      (directory)
                /en/DocName1.html      English
                /en/DocName2.html      English

                /es/DocName1.html      Spanish
                /es/DocName2.html      Spanish

                /de/DocName1.html      German
                /de/DocName2.html      German

The mahName.html should be usable directly by the present clients (browsers) and/or indirectly to generate html files of the fly. Multilingual clients should use the information to access the documents in a transparent way.

3.3. Anchoring Strategy

The anchors would have to be at least at sentence level. It would be hard to place implicit anchors in part of a sentence without tagging: the second text should have null anchors; named null anchors if there are several in one sentence.

examples:

• No need for null anchoring in the second text. A whole sentence is anchored in the first text and finding the place for the implicit anchor in the second text is easy. The white table. <a href="MyURL"> The black table </a>. The green table. La mesa blanca. La mesa negra. La mesa verde. (implicit anchor)
• It needs a null anchor in the second text. Only part of a sentence is anchored in the first text and finding the place for the implicit anchor in the second text is hard. The white table. The black <a href="MyURL"> table </a>. The green table. La mesa blanca. La <a href="Null"> mesa </a> negra. La mesa verde. (null anchor)

3.4. HTTP/1.0

3.4.1. Language tag

3.4.2. Accept-Language

The server should send one language request, rather than a set of languages as the request should be for the Spanish linguistic version and not for Spanish or German. Also, there should be a mechanism to request the list of available linguistic versions of a document; the client should have language drop menu (or similar) listing the available linguistic versions, rather than anchoring pointing to other linguistic versions.

3.4.3. Link header

3.4.4. vary (in URI)

3.5. Applications

• The requester of a translation could send only the URL of the document to be translated, perhaps in a mah file. The document would have to be in a document repository; i.e., it must be guaranteed that no further changes are allowed. Note that the repository does not have to be necessarily controlled by the translation department.

• The requester can pass the background documentation as anchors.

• The Translation Folder is just a mah file that could be at the origin the one sent by the requester and augmented latter.

• The translator could look at the work of his collegues in other languages; i.e., if an English document has to be translated into Spanish and German, the Spanish translator could look at the ongoing work of the German translator.

• The Translator could verify the alignness of his translation.

• The requester of a publication could use a similar paradigm to the requester of translation.

• The publishing department could generate the publications camera ready in several linguistic versions.

3.6. Defaults

• Structure as above
• English documents have the explicit anchors
• The terminology database is DefaultTerminologyDatabase

3.7. Precedence

• Defaults (the lowest precedence)
• Common preference.html (see present practice)
• HTML document
• Private preference.html

3.8. mahName.html

3.9. Preference