UTS #35: Unicode Locale Data Markup Language

The calendar data provides locale-independent data about calendars and usage. Example:




...
The common values provide a list of the calendars that are in common use, and thus should be shown in UIs that provide choice of calendars. (An 'Other...'
button could give access to the other available calendars.)





...
These values provide information on how a calendar is used in a particular territory. It may also be used in computing week boundaries for other
purposes. The default is provided by the element with territories="001".
The minDays indicates the minimum number of days to count as the first week (of a month or year). The first day of the week is typically used
for calendar presentation.
What is meant by the weekend varies from country to country. It is typically when most non-retail businesses are closed. The time should not be specified
unless it is a well-recognized part of the day.
The weekendStart day defaults to "sat", and weekendEnd day defaults to "sun".
For more information, see
Section 5.2.1
Dates and Date Ranges
C.6
Measurement System Data







The measurement system is the normal measurement system in common everyday use (except for
date/time). For example:






The values are "metric", "US", or "UK";
others may be added over time. The "metric" value indicates the use of SI [
ISO1000
] base or derived units,
or non-SI units accepted for use with SI: For example, meters, kilograms, liters, and degrees Celsius.
The "US" value indicates the customary system of measurement as
used in the United States: feet, inches, pints, quarts, degrees Fahrenheit,
and so on. The "UK" value indicates the customary system of measurement
as used in the United Kingdom: feet, inches, pints, quarts, and so on.
It is also called the Imperial system: the pint, quart, and so on are different sizes than in "US".
The paperSize attribute gives the height and width of paper used for normal business letters. The values are "A4" and "US-Letter".
For both measurementSystem entries and paperSize entries, later entries for specific territories such as "US" will override the
value assigned to that territory by earlier entries for more inclusive territories such as "001".
The measurement information was formerly in the main LDML file, and had a somewhat different format.
C.7
Supplemental Time Zone Data














The following is data that can be used to get a single timezone id from a set of modern equivalents.
It provides a stable identifier for use with the standard timezone database.


...
The following subelement of supplies information used by Appendix J:
Time Zone Display Names






The multizone attribute lists the territories that that have multiple TZIDs, which is used in step #5 of Appendix J:
Time Zone
Display Names
. The zoneItem type is the canonical ID for CLDR. The aliases map to that canonical ID; this is used in step #1 in Appendix J:
Time Zone Display Names
. The territory is also used in step #5.
The following data can be used to provide mappings between
TZ
IDs and other platforms. The purpose is to assist with migration and vetting.




...
C.8
Supplemental Character Fallback Data





The characters element provides a way for non-Unicode systems, or systems that only support a subset of Unicode characters, to transform CLDR data. It gives
a list of characters with alternative values that can be used if the main value is not available. For example:



ss


Ö
O


Pts


Fr.



The ordering of the substitute elements indicates the preference among them.
That is, this data provides recommended fallbacks for use when a charset or supported repertoire
does not contain a desired character. There is more than one possible
fallback: the recommended usage is that when a character
value
is not in the desired repertoire
the following process is used, whereby the first value that is wholly in the desired repertoire
is used.
toNFC
value
other canonically equivalent sequences, if there are any
the explicit
substitutes
value (in order)
toNFKC
value
C.9
Supplemental Code Mapping









The code mapping information provides mappings between the subtags
used in the CLDR locale IDs (from BCP 47) and other coding systems or related information. The
language codes are only provided for those codes that have two letters in BCP 47 to their ISO
three-letter equivalents. The territory codes provide mappings to numeric (UN
M.49 [
UNM49
] codes,
equivalent to ISO numeric codes), ISO three-letter codes, FIPS 10 codes, and the internet
top-level domain codes. The alphabetic codes are only provided where different from the type.
For example:



...

...

C.10
Likely Subtags



There are a number of situations where it is useful to be able to
find the most likely language, script, or region. For example, given the language "zh" and the
region "TW", what is the most likely script? Given the script "Thai" what is the most likely
language or region? Given the region TW, what is the most likely language and script?
Conversely, given a locale, it is useful to find out which fields
(language, script, or region) may be superfluous, in the sense that they contain the likely
tags. For example, "en_Latn" can be simplified down
to "en" since "Latn" is the likely script for "en"; "ja_Japn_JP"
can be simplified down to "ja".
The
likelySubtag
supplemental data provides default information for
computing these values. This data is based on the default content data, the population data, and
the the suppress-script data in [
BCP47
]. It is heuristically derived, and
may change over time. To look up data in the table, see if a locale matches one of the
from
attribute values. If so, fetch the corresponding
to
attribute value. For example, the
Chinese data looks like the following:






So looking up "zh_TW" returns "zh_Hant_TW", while looking up "zh"
returns "zh_Hans_CN". In the following text, the components of such a result will be
be designated with language
², region², and script².
The data is designed to be used in the following operations. It can
also be used with language tags using [
BCP47
] syntax, with a few changes.
Add Likely Subtags:
Given a locale, to fill in the most
likely other fields.
This operation is performed in the following way.
Canonicalize.
Make sure the
input locale is in canonical form: uses the right separator, and has the right casing.
Replace any
deprecated subtags with their canonical values using the data in supplemental
metadata. Use the first value in the replacement list, if it exists.
If the tag is grandfathered (see type="choice"> in the supplemental data), then return it.
Remove the script code 'Zzzz' and the region code 'ZZ' if they occur; change an empty language subtag to
'und'.
Get the components of the cleaned-up tag (language
script
¹, and
region
), plus any variants
if they exist
(including keywords).
Try each of
the following in order (where the fields exist). The notation field
means field¹ if it exists, otherwise field².
Lookup
language
_ script
_ region
. If in the table, return the language
_ script
_ region
+ variants.
Lookup
language
_ script
If in the table, return language
_ script
_ region
+ variants.
Lookup
language
_ region
. If in the table, return language
_ script
_ region
+ variants.
Lookup
language
. If in the table, return language
_ script
_ region
+ variants.
If none of these succeed, signal an error.
Example:
Input is ZH-ZZZZ-SG.
Normalize to zh_SG.
Lookup in table. No match.
Remove SG, but remember it. Lookup zh, and get
the match (zh_Hans_CN). Substitute SG, and return zh_Hans_SG.
To find the most likely language for a country, or
language for a script, use "und" as the language subtag. For example, looking up "und_TW" returns zh_Hant_TW.
Remove
Likely Subtags:
Given a locale,
remove any fields that Add Likely Subtags would add.
The reverse operation removes fields that would be added by the
first operation.
First get max
= AddLikelySubtags(inputLocale). If an error is signaled, return it.
Remove the
variants from max.
Then for
trial
in {language, language _ region, language _ script}
If
AddLikelySubtags(
trial
) = max, then return
trial
+ variants.
If you do not
get a match, return max + variants.
Example:
Input is zh_Hant. Maximize to get zh_Hant_TW.
zh => zh_Hans_CN. No match, so continue.
zh_TW => zh_Hant_TW. Matches, so return zh_TW.
A variant of this favors the script over the region, thus using
{language, language_script, language_region} in the above. If that variant is used, then the
result in this example would be zh_Hant instead of zh_TW.
C.11
Language Plural Rules





This section defines certain plural forms that exist in a language—specifically, just the plural forms for nouns that express units such
as time, currency or distance, used in conjunction with a number expressed in decimal digits (i.e. "2", not "two", and not an indefinite number
such as "some" or "many"). For example, English has two forms:
form "one": 1 day
form "other": 0 days, 2 days, 10 days, 0.3 days
Other languages may have additional forms or only one form. CLDR provides the following tags for designating the various plural forms
of a language; for a given language, only the tags necessary for that language are defined, along with the specific numeric ranges covered by each tag (for
example,
the plural form "few" may be used for the numeric range 2-4 in one language and 3-9 in another):
zero
one
two
few
many
In addition, an "other" tag is always implicitly defined to cover the forms not explicitly designated by the tags defined
for a language. This "other" tag is also used for languages that only have a single form (in which case no plural-form tags are explicitly defined for
the language). For a more complex example, consider the rules for Russian and certain other languages:


n mod 10 is 1 and n mod 100 is not 11


n mod 10 in 2..4 and n mod 100 not in 12..14


These rules specify that Russian has a "one" form (for 1, 21, 31, 41, 51, …), a "few" form (for 2-4, 22-24, 32-34, …),
and implicitly an "other" form (for everything else: 0, 5-20, 25-30, 35-40, …, decimals). Russian does not need additional separate forms for zero, two, or
many, so these are not defined.
Plural rules syntax
The xml value for each pluralRule is a
condition
with a boolean result that specifies whether that rule
(i.e. that plural form) applies to a given numeric value
, where n can be expressed as a decimal fraction. Conditions have the following syntax:
condition = and_condition ('or' and_condition)*
and_condition = relation ('and' relation)*
relation = is_relation | in_relation | within_relation | 'n'
is_relation = expr 'is' ('not')? value
in_relation = expr ('not')? 'in' range
within_relation = expr ('not')? 'within' range
expr = 'n' ('mod' value)?
value = digit+
digit = 0|1|2|3|4|5|6|7|8|9
range = value'..'value
Whitespace (defined as Unicode Pattern_White_Space) can occur between or around any of the above tokens.
Rules should be mutually exclusive; for a given numeric value, only one rule should apply (i.e. the condition should only be
true for one of the pluralRule elements.
The difference between 'in' and 'within' is that 'in' only includes integers in the specified range,
while 'within' includes all values.
'mod' (modulus) is a remainder operation as defined in Java; for example, the result of "4.3 mod 3" is 1.3.
Examples:
one: n is 1
few: n in 2..4
This defines two rules, for 'one' and 'few'. The condition for 'one' is "n is 1"
which means that the number must be equal to 1 for this condition to pass. The
condition for 'few' is "n in 2..4" which means that the number must be between 2
and 4 inclusive for this condition to pass. All other numbers are assigned the
keyword 'other' by the default rule.
zero: n is 0 or n is not 1 and n mod 100 in 1..19
one: n is 1
Each rule must not overlap with other rules. Also note that a modulus is
applied to n in the last rule, thus its condition holds for 119, 219, 319...
one: n is 1
few: n mod 10 in 2..4 and n mod 100 not in 12..14
This illustrates conjunction and negation. The condition for 'few' has two
parts, both of which must be met: "n mod 10 in 2..4" and "n mod 100 not in
12..14". The first part applies a modulus to n before the test as in the
previous example. The second part applies a different modulus and also uses
negation, thus it matches all numbers
not
in 12, 13, 14, 112, 113, 114, 212,
213, 214...
Using the plural rules
Elements such as , and provide selection among subelements designating various
localized plural forms by tagging each of the relevant subelements with a different count value, or with no count value in some cases. Note that the plural forms
for a specific currencyFormat, unit type, or currency type may not use all of the different plural-form tags defined for the language. To format a currency or
unit type for a particular numeric value, determine the count value according to the plural rules for the language, then select the appropriate display form for
the currency format, currency type or unit type using the rules in those sections:
5.10.1
Number Symbols
(for currencyFormats elements)
5.10.2
Currencies
(for currency elements)
5.11
Unit Elements
C.12
Telephone Code Data







This data specifies the mapping between ITU telephone country codes [
ITUE164
] and CLDR-style territory codes (ISO 3166 2-letter codes or
non-corresponding UN
M.49 [
UNM49
] 3-digit codes). There are several things to note:
A given telephone country code may map to multiple CLDR territory codes; +1 (North America Numbering Plan) covers the US
and Canada, as well as many islands in the Caribbean and some in the Pacific
Some telephone country codes are for global services (for example, some satellite services), and thus correspond to territory code 001.
The mappings change over time (territories move from one telephone code to another). These changes are usually planned
several years in advance, and there may be a period during which either telephone code can be used to reach the territory. While the CLDR telephone code data
is not intended to include past changes, it is intended to incorporate known information on planned future changes, using "from" and "to"
date attributes to indicate when mappings are valid.
A subset of the telephone code data might look like the following (showing a past mapping change to illustrate the from and to
attributes):











>
Appendix D:
Unicode Language and Locale IDs
People have very slippery notions of what distinguishes a language code
versus a locale code. The problem is that both are somewhat nebulous concepts.
In practice, many people use [
BCP47
codes to mean locale codes instead of strictly language codes. It is easy to see why this came
about; because [
BCP47
includes an explicit region (territory) code, for most people it was sufficient for use as a
locale code as well. For example, when typical web software receives an [
BCP47
] code, it will use it as a locale code. Other typical software will
do the same: in practice, language codes and locale codes are treated interchangeably. Some people recommend distinguishing on the basis of "-"
versus "_" (for example,
zh-TW
for language code,
zh_TW
for locale code), but in practice that does not work because of the free variation out in the world in the use
of these separators. Notice that Windows, for example, uses "-" as a separator in its locale codes. So pragmatically one is forced to treat "-" and "_" as equivalent
when interpreting either one on input.
Another reason for the conflation of these codes is that
very
little data in most systems is distinguished by region alone; currency codes and measurement
systems being some of the few. Sometimes date or number formats are mentioned as regional, but that really
does not make much sense. If people see the sentence
"You will have to adjust the value to १,२३४.५६७ from ૭૧,૨૩૪.૫૬" (using Indic digits), they would say that sentence is simply not English. Number format is far
more closely associated with language than it is with region. The same is true for date formats: people would never expect to see intermixed a date in the format
"2003年4月1日" (using Kanji) in text purporting to be purely English. There are regional differences in date and number format — differences which can be important
— but those are different in kind than other language differences between regions.
As far as we are concerned —
as a completely practical matter
— two languages are different if they require substantially different localized resources.
Distinctions according to spoken form are important in some contexts, but the written form is by far and away the most important issue for data interchange.
Unfortunately, this is not the principle used in [
ISO639
], which has the fairly unproductive notion (for data interchange) that only spoken
language matters (it is also not completely consistent about this, however).
BCP47
can
express a difference if the use of written languages happens to correspond to region boundaries expressed
as [
ISO3166
] region codes, and has recently added codes that allow it to express some important cases that are not distinguished by [
ISO3166
codes. These written languages include simplified and traditional Chinese (both used in Hong Kong S.A.R.); Serbian in Latin script; Azerbaijani in Arab script,
and so on.
Notice also that
currency codes
are different than
currency localizations
. The currency localizations should largely be in the language-based
resource bundles, not in the territory-based resource bundles. Thus, the resource bundle
en
contains the localized mappings in English for a range of
different currency codes: USD → US$, RUR → Rub, AUD → $A and so on. Of course, some currency symbols are used for more than one currency, and in such cases specializations
appear in the territory-based bundles. Continuing the example,
en_US
would have USD → $, while
en_AU
would have AUD → $. (In protocols, the currency
codes should always accompany any currency amounts; otherwise the data is ambiguous, and software is forced to use the user's territory to guess at the currency.
For some informal discussion of this, see
JIT Localization
.)
D.1
Written Language
Criteria for what makes a written language should be purely pragmatic;
what would copy-editors say?
If one gave them text like the following, they
would respond that is far from acceptable English for publication, and ask for it to be redone:
"Theatre Center News: The date of the last version of this document was 2003年3月20日. A copy can be obtained for $50,0 or 1.234,57 грн. We would
like to acknowledge contributions by the following authors (in alphabetical order): Alaa Ghoneim, Behdad Esfahbod, Ahmed Talaat, Eric Mader, Asmus Freytag,
Avery Bishop, and Doug Felt."
So one would change it to either B or C below, depending on which orthographic variant of English was the target for the publication:
"Theater Center News: The date of the last version of this document was 3/20/2003. A copy can be obtained for $50.00 or 1,234.57 Ukrainian Hryvni. We
would like to acknowledge contributions by the following authors (in alphabetical order): Alaa Ghoneim, Ahmed Talaat, Asmus Freytag, Avery Bishop, Behdad
Esfahbod, Doug Felt, Eric Mader."
"Theatre Centre News: The date of the last version of this document was 20/3/2003. A copy can be obtained for $50.00 or 1,234.57 Ukrainian Hryvni. We
would like to acknowledge contributions by the following authors (in alphabetical order): Alaa Ghoneim, Ahmed Talaat, Asmus Freytag, Avery Bishop, Behdad
Esfahbod, Doug Felt, Eric Mader."
Clearly there are many acceptable variations on this text. For example, copy editors might still quibble with the use of first
versus last name sorting in the
list, but clearly the first list was
not
acceptable English alphabetical order. And in quoting a name, like "Theatre Centre News", one may leave it in
the source orthography even if it differs from the publication target orthography. And so on. However, just as clearly, there limits on what is acceptable English,
and "2003年3月20日", for example, is
not
Note that the language of locale data may differ from the language of localized software or web sites, when those latter are not localized into the user's
preferred language. In such cases, the kind of incongruous juxtapositions described above may well appear, but this situation is usually preferable to forcing
unfamiliar date or number formats on the user as well.
Appendix E:
Unicode Sets
A UnicodeSet is a set of Unicode characters (and possibly strings) determined by a pattern, following
UTS #18: Unicode Regular Expressions
URegex
],
Level 1 and RL2.5, including the syntax where given. For an example of a concrete implementation of this, see [
ICUUnicodeSet
].
Patterns are a series of characters bounded by square brackets that contain lists of characters and Unicode property sets. Lists are a sequence of characters
that may have ranges indicated by a '-' between two characters, as in "a-z". The sequence specifies the range of all characters from the left to the right,
in Unicode order. For example,
[a c d-f m]
is equivalent to
[a c d e f m]
. Whitespace can be freely used for clarity, as
[a c d-f m]
means
the same as
[acd-fm]
Unicode property sets are specified by any Unicode property and a value of that property, such as
[:General_Category=Letter:]
. The property names
are defined by the PropertyAliases.txt file and the property values by the PropertyValueAliases.txt file. For more information, see [
UCD
].
The syntax for specifying the property sets is an extension of either POSIX or Perl syntax, by the addition of "=". For example, you can match letters
by using the POSIX-style syntax:
[:General_Category=Letter:]
or by using the Perl-style syntax
\p{General_Category=Letter}
Property names and values are case-insensitive, and whitespace, "-", and "_" are ignored. The property name can be omitted for the Category and Script properties,
but is required for other properties. If the property value is omitted, it is assumed to represent a boolean property with the value "true". Thus
[:Letter:]
is equivalent to
[:General_Category=Letter:]
, and
[:Wh-ite-s pa_ce:]
is equivalent to
[:Whitespace=true:]
The table below shows the two kinds of syntax: POSIX and Perl style. Also, the table shows the "Negative", which is a property that excludes all characters
of a given kind. For example,
[:^Letter:]
matches all characters that are not
[:Letter:]
Positive
Negative
POSIX-style Syntax
[:type=value:]
[:^type=value:]
Perl-style Syntax
\p{type=value}
\P{type=value}
These following low-level lists or properties then can be freely combined with the normal set operations (union, inverse, difference, and intersection):
To union two sets, simply concatenate them. For example,
[[:letter:] [:number:]]
To intersect two sets, use the '&' operator. For example,
[[:letter:] & [a-z]]
To take the set-difference of two sets, use the '-' operator. For example,
[[:letter:] - [a-z]]
To invert a set, place a '^' immediately after the opening '['. For example,
[^a-z]
. In any other location, the '^' does not have a special meaning.
The binary operators '&', '-', and the implicit union have equal precedence and bind left-to-right. Thus
[[:letter:]-[a-z]-[\u0100-\u01FF]]
is equal
to
[[[:letter:]-[a-z]]-[\u0100-\u01FF]]
. Another example is the set
[[ace][bdf] - [abc][def]]
, which is not the empty set, but instead equal to
[[[[ace] [bdf]] - [abc]] [def]]
, which equals
[[[abcdef] - [abc]] [def]]
, which equals
[[def] [def]]
, which equals
[def]
One caution: the '&' and '-' operators operate between sets. That is, they must be immediately preceded and immediately followed by a set. For example, the
pattern
[[:Lu:]-A]
is illegal, since it is interpreted as the set
[:Lu:]
followed by the incomplete range
-A
. To specify the set of
upper case
letters except for 'A', enclose the 'A' in a set:
[[:Lu:]-[A]]
A multi-character string can be in a Unicode set, to represent a tailored grapheme cluster for a particular language. The syntax uses curly braces for that
case.
In Unicode Sets, there are two ways to quote syntax characters and whitespace:
E.1 Single Quote
Two single quotes represents a single quote, either inside or outside single quotes. Text within single quotes is not interpreted in any way (except for
two adjacent single quotes). It is taken as literal text (special characters become non-special).
E.2 Backslash Escapes
Outside of single quotes, certain backslashed characters have special meaning:
\uhhhh
Exactly 4 hex digits; h in [0-9A-Fa-f]
\Uhhhhhhhh
Exactly 8 hex digits
\xhh
1-2 hex digits
\ooo
1-3 octal digits; o in [0-7]
\a
U+0007 (BELL)
\b
U+0008 (BACKSPACE)
\t
U+0009 (HORIZONTAL TAB)
\n
U+000A (LINE FEED)
\v
U+000B (VERTICAL TAB)
\f
U+000C (FORM FEED)
\r
U+000D (CARRIAGE RETURN)
\\
U+005C (BACKSLASH)
\N{name}
The Unicode character named "name".
Anything else following a backslash is mapped to itself, except in an environment where it is defined to have some special meaning. For example,
\p{uppercase}
is the set of upper case letters in Unicode.
Any character formed as the result of a backslash escape loses any special meaning and is treated as a literal. In particular, note that \u and \U escapes
create literal characters. (In contrast, Java treats Unicode escapes as just a way to represent arbitrary characters in an ASCII source file, and any resulting
characters are
not
tagged as literals.)
The following table summarizes the syntax that can be used.
Example
Description
[a]
The set containing 'a' alone
[a-z]
The set containing 'a' through 'z' and all letters in between, in Unicode order.
Thus it is the same as [\u0061-\u007A].
[^a-z]
The set containing all characters but 'a' through 'z'.
Thus it is the same as [\u0000-\u0061 \u007B..\U0010FFFF].
[[pat1][pat2]]
The union of sets specified by pat1 and pat2
[[pat1]&[pat2]]
The intersection of sets specified by pat1 and pat2
[[pat1]-[pat2]]
The asymmetric difference of sets specified by pat1 and pat2
[a {ab} {ac}]
The character 'a' and the multi-character strings "ab" and "ac"
[:Lu:]
The set of characters with a given property value, as defined by PropertyValueAliases.txt. In this case, these are the Unicode
upper case letters.
The long form for this is
[:General_Category=Uppercase_Letter:]
[:L:]
The set of characters belonging to all Unicode categories starting with 'L', that is,
[[:Lu:][:Ll:][:Lt:][:Lm:][:Lo:]]
. The long form for
this is
[:General_Category=Letter:]
Appendix F:
Date Format Patterns
A date pattern is a string of characters, where specific strings of characters are replaced
with date and time data from a calendar when formatting or used to generate data for a calendar
when parsing. The following
are examples:
Pattern
Result (in a particular locale)
yyyy.MM.dd G 'at' HH:mm:ss zzz
1996.07.10 AD at 15:08:56 PDT
EEE, MMM d, ''yy
Wed, July 10, '96
h:mm a
12:08 PM
hh 'o''clock' a, zzzz
12 o'clock PM, Pacific Daylight Time
K:mm a, z
0:00 PM, PST
yyyyy.MMMM.dd GGG hh:mm aaa
01996.July.10 AD 12:08 PM
When parsing using a pattern, a lenient parse should
be used; see
Lenient Parsing
The Date Field Symbol Table below contains the characters used in
patterns to show the appropriate formats for a given locale, such as yyyy for the year. Characters may be used multiple times. For example, if y is used for the year, 'yy' might produce '99', whereas 'yyyy' produces '1999'. For most numerical
fields, the number of characters specifies the field width. For example, if h is the hour, 'h' might produce '5', but 'hh' produces '05'. For some characters,
the count specifies whether an abbreviated or full form should be used, but may have other choices, as given below.
Two single quotes represents a literal single quote, either inside or outside single quotes. Text within single quotes is not interpreted in any way (except
for two adjacent single quotes). Otherwise all ASCII letter from a to z and A to Z are reserved as syntax characters, and require quoting if they are to represent
literal characters. In addition, certain ASCII punctuation characters may become variable in the future (for
example, ":" being interpreted as the time separator and
'/' as a date separator, and replaced by respective locale-sensitive characters in display).
Note: the counter-intuitive use of 5 letters for the narrow form of weekdays and months is forced by backwards compatibility.
Date Field Symbol Table
Field
Sym.
No.
Example
Description
era
1..3
AD
Era - Replaced with the Era string for the current date. One to three letters for the
abbreviated form, four letters for the long form, five for the narrow form.
Anno Domini
year
1..n
1996
Year. Normally the length specifies the padding, but for two letters it also specifies the maximum length. Example:
Year
yy
yyy
yyyy
yyyyy
AD 1
01
001
0001
00001
AD 12
12
12
012
0012
00012
AD 123
123
23
123
0123
00123
AD 1234
1234
34
1234
1234
01234
AD 12345
12345
45
12345
12345
12345
1..n
1997
Year (in "Week of Year" based calendars). This year designation is used in ISO year-week calendar as defined by ISO 8601, but can be used in non-Gregorian based calendar systems where week date processing is desired. May not always be the same value as calendar year.
1..n
4601
Extended year. This is a single number designating the year of this calendar system, encompassing all supra-year fields. For example, for the Julian
calendar system, year numbers are positive, with an era of BCE or CE. An extended year value for the Julian calendar system assigns positive values
to CE years and negative values to BCE years, with 1 BCE being year 0.
quarter
1..2
02
Quarter - Use one or two for the numerical quarter, three for the abbreviation, or four
for the full name.
Q2
2nd quarter
1..2
02
Stand-Alone
Quarter - Use one or two for the numerical quarter, three for the abbreviation,
or four for the full name.
Q2
2nd quarter
month
1..2
09
Month - Use one or two for the numerical month, three for the abbreviation, or four for
the full name, or five for the narrow name.
Sept
September
1..2
09
Stand-Alone
Month - Use one or two for the numerical month, three for the abbreviation,
or four for the full name, or 5 for the narrow name.
Sept
September
Special symbol for Chinese leap month, used in combination
with M. Only used with the Chinese calendar.
week
1..2
27
Week of Year.
Week of Month
day
1..2
Date - Day of the month
1..3
345
Day of year
Day of Week in Month. The example is for the 2nd Wed in July
1..n
2451334
Modified Julian day. This is different from the conventional Julian day number in two regards. First, it demarcates days at local zone midnight,
rather than noon GMT. Second, it is a local number; that is, it depends on the local time zone. It can be thought of as a single number that encompasses
all the date-related fields.
week
day
1..3
Tues
Day of week - Use one through three letters for the short day, or four for the full name,
or five for the narrow name.
Tuesday
1..2
Local day of week. Same as E except adds a numeric value that will depend on the local
starting day of the week, using one or two letters. For this example, Monday is the first day of the week.
Tues
Tuesday
Stand-Alone
local day of week - Use one letter for the local numeric value (same
as 'e'), three for the short day, or four for the full name, or five for the narrow name.
Tues
Tuesday
period
AM
AM or PM
hour
1..2
11
Hour [1-12].
1..2
13
Hour [0-23].
1..2
Hour [0-11].
1..2
24
Hour [1-24].
1..2
n/a
This is a special-purpose symbol. It must not occur in patterns, but is reserved for use in APIs doing flexible date pattern
generation, and requests the preferred format (12 versus 24 hour) for the language in question.
minute
1..2
59
Minute. Use one or two for zero padding.
second
1..2
12
Second. Use one or two for zero padding.
1..n
3457
Fractional Second - rounds to the count of letters. (example is for 12.34567)
1..n
69540000
Milliseconds in day. This field behaves
exactly
like a composite of all time-related fields, not including the zone fields. As such, it also
reflects discontinuities of those fields on DST transition days. On a day of DST onset, it will jump forward. On a day of DST cessation, it will jump
backward. This reflects the fact that is must be combined with the offset field to obtain a unique local time value.
zone
1..3
PDT
fallbacks:
HPG-8:00
GMT-08:00
Time
Zone -
with
the s
pecific non-location format
. Where that is unavailable, falls back to
localized GMT format
. Use one to three letters for the short
format
or four for the full format. In the short format,
metazone names are not used unless the commonlyUsed flag is on in the locale.
For more
information about timezone formats, see
Appendix J: Time Zone Display Names
Pacific Daylight Time
fallbacks:
HPG-8:00
GMT-08:00
1..3
-0800
Time
Zone - Use one to three letters for RFC 822
format, four letters for
the localized GMT format.
For more information about
timezone formats, see
Appendix J: Time Zone Display Names
HPG+8:00
fallbacks:
GMT-08:00
PT
Time Zone - with the
generic
non-location format
. Where that is unavailable,
uses special fallback rules given in
Appendix J
Use one letter for short format, four for long
format.
For more information about timezone formats, see
Appendix J: Time Zone Display Names
Pacific Time
fallbacks:
Pacific Time (Canada)
Pacific Time (Yellowknife)
United States (Los Angeles) Time
HPG-8:35
GMT-08:35
PST
fallbacks:
HPG-8:00
GMT-08:00
Time Zone -
with the same format as z,
except that metazone timezone abbreviations are to be displayed
if available, regardless of the value of commonlyUsed.
For more information about timezone formats, see
Appendix J: Time Zone Display Names
United
States (Los Angeles) Time
fallbacks:
HPG-8:35
GMT-08:35
Time Zone -
with the
generic
location format
. Where that is unavailable,
falls back to the localized GMT format. (Fallback is only necessary with a GMT-style
Time Zone ID, like Etc/GMT-830.)
This is especially
useful when presenting possible timezone choices for user selection,
since the naming is more uniform than the v format.
For more information about timezone formats, see
Appendix J: Time Zone Display Names
All non-letter character represent themselves in a pattern, except for the single quote. It is used to 'escape' letters. Two single quotes in a row, whether
inside or outside a quoted sequence, represent a 'real' single quote.
F.1
Localized Pattern Characters
(deprecated)
These are characters that can be used when displaying a date pattern to an end user. This can occur, for example, when a spreadsheet allows users to specify
date patterns. Whatever is in the string is substituted one-for-one with the characters "
GyMdkHmsSEDFwWahKzYe", with the above meanings
Thus, for example, if "J" is to be used instead of "Y" to mean Year, then the string would be: "
GyMdkHmsSEDFwWahKz
e".
This element is deprecated. It is recommended instead that a more sophisticated UI be used for localization, such as using icons to represent the different
formats (and lengths) in the
Date Field Symbol Table
F.2 AM / PM
Even for countries where the customary date format only has a 24 hour format, both the am and pm localized strings must be present and must be distinct from
one another. Note that as long as the 24 hour format is used, these strings will normally never be used, but for testing and unusual circumstances they must
be present.
F.3 Eras
There are only two values for an era in a Gregorian calendar, "BC" and "AD". These values can be translated into other languages, like "a.C." and and "d.C."
for Spanish, but there are no other eras in the Gregorian calendar. Other calendars have a different numbers of eras. Care should be taken when translating
the era names for a specific calendar.
F.4 Week of Year
Values calculated for the Week of Year field range from 1 to 53 for the Gregorian calendar (they may have different ranges for other calendars). Week 1 for
a year is the first week that contains at least the specified minimum number of days from that year. Weeks between week 1 of one year and week 1 of the following
year are numbered sequentially from 2 to 52 or 53 (if needed). For example, January 1, 1998 was a Thursday. If the first day of the week is MONDAY and the minimum
days in a week is 4 (these are the values reflecting ISO 8601 and many national standards), then week 1 of 1998 starts on December 29, 1997, and ends on January
4, 1998. However, if the first day of the week is SUNDAY, then week 1 of 1998 starts on January 4, 1998, and ends on January 10, 1998. The first three days
of 1998 are then part of week 53 of 1997.
Values are similarly calculated for the Week of Month.
F.5 Week Elements
firstDay
A number indicating which day of the week is considered the 'first' day, for calendar purposes. Because the ordering of days may vary between calendar,
keywords are used for this value, such as sun, mon,... These values will be replaced by the localized name when they are actually used.
minDays (Minimal Days in First Week)
Minimal days required in the first week of a month or year. For example, if the first week is defined as one that contains at least one day, this value
will be 1. If it must contain a full seven days before it counts as the first week, then the value would be 7.
weekendStart, weekendEnd
Indicates the day and time that the weekend starts or ends. As with firstDay, keywords are used instead of numbers.
Appendix G:
Number Format Patterns
G.1 Number Patterns
The
NumberElements
resource affects how these patterns are interpreted in a localized context. Here are some examples, based
on the French locale. The "." shows where the decimal point should go. The "," shows where the thousands separator should go. A "0" indicates zero-padding:
if the number is too short, a zero (in the locale's numeric set) will go there. A "#" indicates no padding: if the number is too short, nothing goes there.
A "¤" shows where the currency sign will go. The following illustrates the effects of different patterns for the French locale, with the number "1234.567".
Notice how the pattern characters ',' and '.' are replaced by the characters appropriate for the locale.
Pattern
Currency
Text
#,##0.##
n/a
1 234,57
#,##0.###
n/a
1 234,567
###0.#####
n/a
1234,567
###0.0000#
n/a
1234,5670
00000.0000
n/a
01234,5670
# ##0.00 ¤
EUR
1 234,57 €
JPY
1 235 ¥
The number of # placeholder characters before the decimal do not matter, since no limit is placed on the maximum number of digits. There should, however,
be at least one zero someplace in the pattern. In currency formats, the number of digits after the decimal also do not matter, since the information in the
supplemental data (see
Appendix C: Supplemental Data
is used to override the number of decimal places — and the rounding
— according to the currency that is being formatted. That can be seen in the above chart, with the difference between Yen and Euro formatting.
When parsing using a pattern, a lenient parse should
be used; see
Lenient Parsing
G.2 Special Pattern Characters
Many characters in a pattern are taken literally; they are matched during parsing and output unchanged during formatting. Special characters, on the other
hand, stand for other characters, strings, or classes of characters. For example, the '#' character is replaced by a localized digit. Often the replacement
character is the same as the pattern character; in the U.S. locale, the ',' grouping character is replaced by ','. However, the replacement is still happening,
and if the symbols are modified, the grouping character changes. Some special characters affect the behavior of the formatter by their presence; for example,
if the percent character is seen, then the value is multiplied by 100 before being displayed.
To insert a special character in a pattern as a literal, that is, without any special meaning, the character must be quoted. There are some exceptions to
this which are noted below.
Symbol
Location
Localized?
Meaning
Number
Yes
Digit
1-9
Number
Yes
'1' through '9' indicate rounding.
Number
No
Significant digit
Number
Yes
Digit, zero shows as absent
Number
Yes
Decimal separator or monetary decimal separator
Number
Yes
Minus sign
Number
Yes
Grouping separator
Number
Yes
Separates mantissa and exponent in scientific notation.
Need not be quoted in prefix or suffix.
Exponent
Yes
Prefix positive exponents with localized plus sign.
Need not be quoted in prefix or suffix.
Subpattern boundary
Yes
Separates positive and negative subpatterns
Prefix or suffix
Yes
Multiply by 100 and show as percentage
(\u2030)
Prefix or suffix
Yes
Multiply by 1000 and show as per mille
¤ (\u00A4)
Prefix or suffix
No
Currency sign, replaced by currency symbol. If doubled, replaced by international currency symbol. If tripled, uses the long form of the decimal
symbol. If present in a pattern, the monetary decimal separator and grouping separators (if available) are used instead of the numeric ones.
Prefix or suffix
No
Used to quote special characters in a prefix or suffix, for example,
"'#'#"
formats 123 to
"#123"
. To create a single
quote itself, use two in a row:
"# o''clock"
Prefix or suffix boundary
Yes
Pad escape, precedes pad character
A pattern contains a postive and may contain a negative subpattern, for example, "#,##0.00;(#,##0.00)". Each subpattern has a prefix, a numeric part, and
a suffix. If there is no explicit negative subpattern, the negative subpattern is the localized minus sign prefixed to the positive subpattern. That is, "0.00"
alone is equivalent to "0.00;-0.00". If there is an explicit negative subpattern, it serves only to specify the negative prefix and suffix; the number of digits,
minimal digits, and other characteristics are ignored in the negative subpattern. That means that "#,##0.0#;(#)" has precisely the same result as "#,##0.0#;(#,##0.0#)".
Note:
The thousands separator and decimal separator in this pattern are always ',' and '.'. They are substituted by the code with the correct
local values according to other fields in CLDR.
The prefixes, suffixes, and various symbols used for infinity, digits, thousands separators, decimal separators,
and so on may be set to arbitrary values, and
they will appear properly during formatting.
However, care must be taken that the symbols and strings do not conflict, or parsing will be unreliable.
For example, either the positive and negative prefixes or the suffixes must be distinct for any parser using this data to be able to distinguish positive from
negative values. Another example is that the decimal separator and thousands separator should be distinct characters, or parsing will be impossible.
The
grouping separator
is a character that separates clusters of integer digits to make large numbers more legible. It commonly used for thousands,
but in some locales it separates ten-thousands. The
grouping size
is the number of digits between the grouping separators, such as 3 for "100,000,000"
or 4 for "1 0000 0000". There are actually two different grouping sizes: One used for the least significant integer digits, the
primary grouping size
and one used for all others, the
secondary grouping size
. In most locales these are the same, but sometimes they are different. For example, if the
primary grouping interval is 3, and the secondary is 2, then this corresponds to the pattern "#,##,##0", and the number 123456789 is formatted as "12,34,56,789".
If a pattern contains multiple grouping separators, the interval between the last one and the end of the integer defines the primary grouping size, and the
interval between the last two defines the secondary grouping size. All others are ignored, so "#,##,###,####" == "###,###,####" == "##,#,###,####".
For consistency in the CLDR data, the following conventions should be observed so as to have a canonical representation:
All number patterns should be minimal: there should be no leading # marks except to specify the position of the grouping separators (for
example, avoid
##,##0.###).
All formats should have one 0 before the decimal point (for example, avoid #,###.##)
Decimal formats should have three hash marks in the fractional position (for
example, #,##0.###).
Currency formats should have two zeros in the fractional position (for
example, ¤ #,##0.00).
The exact number of decimals is overridden with the decimal count in supplementary data.
The only time two thousands separators needs to be used is when the number of digits varies, such as for Hindi: #,##,##0.
G.3 Formatting
Formatting is guided by several parameters, all of which can be specified either using a pattern or using the API. The following description applies to formats
that do not use
scientific notation
or
significant digits
If the number of actual integer digits exceeds the
maximum integer digits
, then only the least significant digits are shown. For example, 1997
is formatted as "97" if the maximum integer digits is set to 2.
If the number of actual integer digits is less than the
minimum integer digits
, then leading zeros are added. For example, 1997 is formatted
as "01997" if the minimum integer digits is set to 5.
If the number of actual fraction digits exceeds the
maximum fraction digits
, then half-even rounding it performed to the maximum fraction digits.
For example, 0.125 is formatted as "0.12" if the maximum fraction digits is 2. This behavior can be changed by specifying a rounding increment and a rounding
mode.
If the number of actual fraction digits is less than the
minimum fraction digits
, then trailing zeros are added. For example, 0.125 is formatted
as "0.1250" if the mimimum fraction digits is set to 4.
Trailing fractional zeros are not displayed if they occur
positions after the decimal, where
is less than the maximum fraction
digits. For example, 0.10004 is formatted as "0.1" if the maximum fraction digits is four or less.
Special Values
NaN
is represented as a single character, typically
(\uFFFD)
. This character is determined by the localized number symbols. This
is the only value for which the prefixes and suffixes are not used.
Infinity is represented as a single character, typically
(\u221E)
, with the positive or negative prefixes and suffixes
applied. The infinity character is determined by the localized number symbols.
G.4
Scientific Notation
Numbers in scientific notation are expressed as the product of a mantissa and a power of ten, for example, 1234 can be expressed as 1.234 x 10
The mantissa is typically in the half-open interval [1.0, 10.0) or sometimes [0.0, 1.0), but it need not be. In a pattern, the exponent character immediately
followed by one or more digit characters indicates scientific notation. Example: "0.###E0" formats the number 1234 as "1.234E3".
The number of digit characters after the exponent character gives the minimum exponent digit count. There is no maximum. Negative exponents are formatted
using the localized minus sign,
not
the prefix and suffix from the pattern. This allows patterns such as "0.###E0 m/s". To prefix positive exponents
with a localized plus sign, specify '+' between the exponent and the digits: "0.###E+0" will produce formats "1E+1", "1E+0", "1E-1",
and so on. (In localized
patterns, use the localized plus sign rather than '+'.)
The minimum number of integer digits is achieved by adjusting the exponent. Example: 0.00123 formatted with "00.###E0" yields "12.3E-4". This only happens
if there is no maximum number of integer digits. If there is a maximum, then the minimum number of integer digits is fixed at one.
The maximum number of integer digits, if present, specifies the exponent grouping. The most common use of this is to generate
engineering notation
in which the exponent is a multiple of three, for example, "##0.###E0". The number 12345 is formatted using "##0.####E0" as "12.345E3".
When using scientific notation, the formatter controls the digit counts using significant digits logic. The maximum number of significant digits limits
the total number of integer and fraction digits that will be shown in the mantissa; it does not affect parsing. For example, 12345 formatted with "##0.##E0"
is "12.3E3". See the section on significant digits for more details.
Exponential patterns may not contain grouping separators.
G.5
Significant Digits
There are two ways of controlling how many digits are shows: (a) significant digits counts, or (b) integer and fraction digit counts. Integer and fraction
digit counts are described above. When a formatter is using significant digits counts, the number of integer and fraction digits is not specified directly,
and the formatter settings for these counts are ignored. Instead, the formatter uses however many integer and fraction digits are required to display the specified
number of significant digits. Examples:
Pattern
Minimum significant digits
Maximum significant digits
Number
Output
@@@
12345
12300
@@@
0.12345
0.123
@@##
3.14159
3.142
@@##
1.23004
1.23
In order to enable significant digits formatting, use a pattern containing the
'@'
pattern character. In order to disable significant digits
formatting, use a pattern that does not contain the
'@'
pattern character.
Significant digit counts may be expressed using patterns that specify a minimum and maximum number of significant digits. These are indicated by the
'@'
and
'#'
characters. The minimum number of significant digits is the number of
'@'
characters. The maximum number
of significant digits is the number of
'@'
characters plus the number of
'#'
characters following on the right. For example, the
pattern
"@@@"
indicates exactly 3 significant digits. The pattern
"@##"
indicates from 1 to 3 significant digits. Trailing zero
digits to the right of the decimal separator are suppressed after the minimum number of significant digits have been shown. For example, the pattern
"@##"
formats the number 0.1203 as
"0.12"
If a pattern uses significant digits, it may not contain a decimal separator, nor the
'0'
pattern character. Patterns such as
"@00"
or
"@.###"
are disallowed.
Any number of
'#'
characters may be prepended to the left of the leftmost
'@'
character. These have no effect on the minimum
and maximum significant digits counts, but may be used to position grouping separators. For example,
"#,#@#"
indicates a minimum of one significant
digits, a maximum of two significant digits, and a grouping size of three.
The number of significant digits has no effect on parsing.
Significant digits may be used together with exponential notation. Such patterns are equivalent to a normal exponential pattern with a minimum and maximum
integer digit count of one, a minimum fraction digit count of
Minimum Significant Digits - 1
, and a maximum fraction digit count of
Maximum
Significant Digits - 1
. For example, the pattern
"@@###E0"
is equivalent to
"0.0###E0"
G.6 Padding
Patterns support padding the result to a specific width. In a pattern the pad escape character, followed by a single pad character, causes padding to be
parsed and formatted. The pad escape character is '*'. For example,
"$*x#,##0.00"
formats 123 to
"$xx123.00"
, and 1234 to
"$1,234.00"
When padding is in effect, the width of the positive subpattern, including prefix and suffix, determines the format width. For example, in the pattern
"* #0 o''clock"
, the format width is 10.
Some parameters which usually do not matter have meaning when padding is used, because the pattern width is significant with padding. In the pattern
"*
##,##,#,##0.##", the format width is 14. The initial characters "##,##," do not affect the grouping size or maximum
integer digits, but they do affect the format width.
Padding may be inserted at one of four locations: before the prefix, after the prefix, before the suffix, or after the suffix. No padding can be specified
in any other location. If there is no prefix, before the prefix and after the prefix are equivalent, likewise for the suffix.
When specified in a pattern, the code point immediately following the pad escape is the pad character. This may be any character, including a special
pattern character. That is, the pad escape
escapes
the following character. If there is no character after the pad escape, then the pattern is
illegal.
Rounding
Patterns support rounding to a specific increment. For example, 1230 rounded to the nearest 50 is 1250. Mathematically, rounding to specific increments is
performed by multiplying by the increment, rounding to an integer, then dividing by the increment. To take a more bizarre example, 1.234 rounded to the nearest
0.65 is 1.3, as follows:
Original:
1.234
Divide by increment (0.65):
1.89846...
Round:
Multiply by increment (0.65):
1.3
To specify a rounding increment in a pattern, include the increment in the pattern itself. "#,#50" specifies a rounding increment of 50. "#,##0.05" specifies
a rounding increment of 0.05.
Rounding only affects the string produced by formatting. It does not affect parsing or change any numerical values.
An implementation may allow the specification of a
rounding mode
to determine how values are rounded. In the absence of such choices, the default
is to round "half-even", as described in IEEE arithmetic. That is, it rounds towards the "nearest neighbor" unless both neighbors are equidistant, in which
case, it rounds towards the even neighbor. Behaves as for round "half-up" if the digit to the left of the discarded fraction is odd; behaves as for round
"half-down" if it's even. Note that this is the rounding mode that minimizes cumulative error when applied repeatedly over a sequence of calculations.
Some locales use rounding in their currency formats to reflect the smallest currency denomination.
In a pattern, digits '1' through '9' specify rounding, but otherwise behave identically to digit '0'.
decimalFormats
The normal locale specific way to write a base 10 number.
currencyFormats
Use \u00A4 where the local currency symbol should be. Doubling the currency symbol (\u00A4\u00A4) will output the international currency symbol (a 3-letter
code).
percentFormats
Pattern for use with percentage formatting
scientificFormats
Pattern for use with scientific (exponent) formatting.
G.7
Quoting Rules
Single quotes, (
), enclose bits of the pattern that should be treated literally. Inside a quoted string, two single quotes ('') are replaced
with a single one ('). For example:
'X '
' Q '
->
X 1939 Q
(Literal strings
underlined
.)
G.8
Number Elements
Localized symbols used in number formatting and parsing.
decimal
- separates the integer and fractional part of the number.
group
- groups (for example) units of thousands: 10
= 1,000,000. The grouping separator is commonly used for thousands, but in some countries
for ten-thousands. The interval is a constant number of digits between the grouping characters, such as 100,000,000 or 1,0000,0000. If you supply a pattern
with multiple grouping characters, the interval between the last one and the end of the integer is the one that is used. So "#,##,###,####" == "######,####"
== "##,####,####".
list
- separates lists of numbers
percentSign
- symbol used to indicate a percentage (1/100th) amount. (If present, the value is also multiplied by 100 before formatting. That way 1.23 → 123%)
nativeZeroDigit
- Symbol used to indicate a digit in the pattern, or zero if that place would otherwise be empty. For example, with the digit of '0', the pattern "000"
would format "34" as "034", but the pattern "0" would format "34" as just "34". As well, the digits 1-9 are expected to follow the code point of this specified
0 value.
patternDigit
- Symbol used to indicate any digit value, typically #. When that digit is zero, then it is not shown.
minusSign
- Symbol used to denote negative value.
plusSign
- Symbol used to denote positive value.
exponential
- Symbol separating the mantissa and exponent values.
perMille
- symbol used to indicate a per-mille (1/1000th) amount. (If present, the value is also multiplied by 1000 before formatting. That way 1.23 → 1230 [1/000])
infinity
- The infinity sign. Corresponds to the IEEE infinity bit pattern.
nan - Not a number
- The NaN sign. Corresponds to the IEEE NaN bit pattern.
currencyDecimal
This is used as the decimal separator in currency formatting/parsing, instead of the DecimalSeparator from the
NumberElements
list. This item is optional in the CLDR.
currencyGroup
This is used as the grouping separator in currency formatting/parsing, instead of the DecimalSeparator from the
NumberElements
list. This item is optional in the CLDR.
Appendix H:
Choice Patterns
A choice pattern is a string that chooses among a number of strings, based on numeric value. It has the following form:
= ( '|' )*
=
= ('+' | '-')? (
'∞' | [0-9]+ ('.' [0-9]+)?)
= '<' | '
≤'
The interpretation of a choice pattern is that given a number N, the pattern is scanned from right to left, for each choice evaluating
N. The first choice that matches results in the corresponding string. If no match is found, then the first string is used. For example:
Pattern
Result
0≤Rf|1≤Ru|1∞,
-3, -1, -0.000001
Rf (defaulted to first string)
0, 0.01, 0.9999
Rf
Ru
1.00001, 5, 99,
Re
Quoting is done using ' characters, as in date or number formats.
Appendix I:
Inheritance and Validity
The following describes in more detail how to determine the exact inheritance of elements, and the validity of a given element in LDML.
I.1 Definitions
Blocking
elements are those whose subelements do not inherit from parent locales. For example, a element is a blocking element: everything
in a element is treated as a single lump of data, as far as inheritance is concerned.
For more information, see
Appendix K: Valid Attribute Values
Attributes that serve to distinguish multiple elements at the same level are called
distinguishing
attributes.
For example, the
type
attribute distinguishes different elements in lists of
translations, such as:
Afar
Abkhazian
Distinguishing attributes affect inheritance; two elements with different distinguishing
attributes are treated as different for purposes of inheritance. For more information, see
Appendix K: Valid Attribute Values
Other attributes are called nondistinguishing (or informational) attributes. These carry
separate information, and do not affect inheritance.
For any element in an XML file,
an element chain
is a resolved [
XPath
] leading from the root to an element, with attributes on each element in alphabetical
order. So in, say,
we may have:








Αραβικά
...
Which gives the following element chains (among others):
//ldml[@version="1.1"]/identity/version[@number="1.1"]
//ldml[@version="1.1"]/localeDisplayNames/languages/language[@type="ar"]
An element chain A is an
extension
of an element chain B if B is equivalent to an initial portion of A. For example, #2 below is an extension of #1.
(Equivalent, depending on the tree, may not be "identical to". See below for an example.)
//ldml[@version="1.1"]/localeDisplayNames
//ldml[@version="1.1"]/localeDisplayNames/languages/language[@type="ar"]
An LDML file can be thought of as an ordered list of
element pairs
: , where the element chains are all the chains for the end-nodes.
(This works because of restrictions on the structure of LDML, including that it
does not allow mixed content.) The ordering is the ordering that the element
chains are found in the file, and thus determined by the DTD.
For example, some of those pairs would be the following. Notice that the first has the null string as element contents.
//ldml[@version="1.1"]/identity/version[@number="1.1"]
""
//ldml[@version="1.1"]/localeDisplayNames/languages/language[@type="ar"]
"Αραβικά"
Note:
There are two exceptions to this:
Blocking nodes and their contents are treated as a single end node.
In terms of computing inheritance, the element pair consists of the element chain
plus all distinguishing attributes; the value consists of the value (if any) plus any
nondistinguishing attributes.
Thus instead of the element pair being (a) below, it is (b):
//ldml[@version="1.1"]/dates/calendars/calendar[@type='gregorian']/week/weekendStart[@day='sun'][@time='00:00']
"">
//ldml[@version="1.1"]/dates/calendars/calendar[@type='gregorian']/week/weekendStart
[@day='sun'][@time='00:00']
Two LDML element chains are
equivalent
when they would be identical if all attributes and their values were removed
except
for distinguishing attributes. Thus the following are equivalent:
//ldml[@version="1.1"]/localeDisplayNames/languages/language[@type="ar"]
//ldml[@version="1.1"]/localeDisplayNames/languages/language[@type="ar"][@draft="unconfirmed"]
For any locale ID, an
locale chain
is an ordered list starting with the root and leading down to the ID. For example:

I.2 Resolved Data File
To produce fully resolved locale data file from CLDR for a locale ID L, you start with L, and successively add unique items from the parent locales until
you get up to root. More formally, this can be expressed as the following procedure.
Let Result be initially empty.
For each Li in the locale chain for L, starting at L and going up to root:
Let Temp be a copy of the pairs in the LDML file for Li
Replace each alias in Temp by the list of pairs it points to.
That alias now blocks any inheritance from the parent. (See
Section 5.1 Common Elements
for an example.)
For each element pair P in Temp:
If P does not contain a blocking element, and Result does not have an element pair Q with an equivalent element chain, add P to Result.
Note:
when adding an element pair to a result, it has to go in the right order for it to be valid according to the DTD.
I.3
Valid Data
The attribute
draft="x"
in LDML means that the data has not been approved by the
subcommittee. (For more information, see
Process
). However, some data
that is not explicitly marked as
draft
may be implicitly
draft
, either because it inherits
it from a parent, or from an enclosing element.
Example 2.
Suppose that new locale data is added for af (Afrikans). To indicate that all of the data is
unconfirmed
, the attribute can be added
to the top level.






...
...

Any data can be added to that file, and the status will all be draft=
unconfirmed
Once an item is vetted—
whether it is inherited or explicitly in
the file
—then its status can be changed to
approved
. This can be done either by leaving draft="unconfirmed" on the enclosing element and marking
the child with draft="approved", such as:






...
...




However, normally the draft status should be canonicalized, which means it is pushed down to leaf nodes: see
Appendix L: Canonical
Form
Note:
A missing draft attribute is
not
the same as either a true or false value. A missing attribute means instead:
inherit
the
draft status from enclosing elements and parent locales.
The attribute
validSubLocales
allows sublocales in a given tree to be treated as though a file for them were present when there
is not one. It can
be applied to any element. It only has an effect for locales that inherit from the current file where a file is missing, and the elements would
not otherwise
be draft.
Example 1.
Suppose that in a particular LDML tree, there are no region locales for German,
for example, there is a de.xml file, but no files for de_AT.xml,
de_CH.xml, or de_DE.xml. Then no elements are valid for any of those region locales. If we want to mark one of those files as having valid elements, then we
introduce an empty file, such as the following.








With the
validSubLocales
attribute, instead of adding the empty files for de_AT.xml, de_CH.xml, and de_DE.xml, in the de file we can add to the parent
locale a list of the child locales that should behave as if files were present.






...

More formally, here is how to determine whether data for an element chain E is implicitly or explicitly draft, given a locale L. Sections 1, 2, and 4 are
simply formalizations of what is in LDML already. Item 3 adds the new element.
I.4 Checking for Draft Status:
Parent Locale Inheritance
Walk through the locale chain until you find a locale ID L' with a data file D. (L' may equal L).
Produce the fully resolved data file D' for D.
In D', find the first element pair whose element chain E' is either equivalent to or an extension of E.
If there is no such E', return
true
If E' is not equivalent to E, truncate E' to the length of E.
Enclosing Element Inheritance
Walk through the elements in E', from back to front.
If you ever encounter draft=
, return
If L' = L, return
false
Missing File Inheritance
Otherwise, walk again through the elements in E', from back to front.
If you encounter a validSubLocales attribute:
If L is in the attribute value, return
false
Otherwise return
true
Otherwise
Return
true
The validSubLocales in the most specific (farthest from root file) locale file "wins" through the full resolution step (data from more specific files replacing
data from less specific ones).
I.5 Keyword and Default Resolution
When accessing data based on keywords, the following process is used. Consider the following example:
The locale 'de' has collation types A, B, C, and no element
The locale 'de_CH' has
Here are the searches for various combinations.
1.
de_CH
not found
de
not found
root
not found: so get the default type in de_CH
de@collation=B
found
2.
de
not found
root
not found: so get the default type in de, which itself falls back to root
de@collation=standard
not found
root@collation=standard
found
3.
de@collation=A
found
4.
de@collation=standard
not found
root@collation=standard
found
Note:
It is an invariant that the default in root for a given element must
always be a value that exists in root. So you can not have the following in root:


...
...


For identifiers, such as language codes, script codes, region codes, variant codes, types, keywords, currency symbols or currency display names, the default
value is the identifier itself whenever if no value is found in the root. Thus if there is no display name for the region code 'QA' in root, then the display
name is simply 'QA'.
Appendix J:
Time Zone Display Names
There are three main types of formats for zone identifiers: GMT, generic (wall time), and standard/daylight. Standard and daylight are equivalent to a particular
offset from GMT, and can be represented by a GMT offset as a fallback. In general, this is not true for the generic format, which is used for picking
timezones or for conveying a timezone for specifying a recurring time (such as a meeting in a calendar). For either purpose, a GMT offset would lose information.
Time Zone Format Terminology
The following
terminology defines more precisely the formats that are used.
Generic
non-location format:
Reflects "wall
time" (what is on a clock on the wall): used for recurring events, meetings, or anywhere people do
not want to be overly
specific. For example, "10 am Pacific Time" will be GMT-8 in the winter, and GMT-7 in the
summer.
"Pacific Time"
"PT"
Generic
partial location format:
Reflects "wall
time": used as a fallback format when the generic non-location format is not specific
enough.
"Pacific Time (Canada)"
"Pacific Time (Yellowknife)"
Generic
location format:
Reflects "wall
time": mostly used in populating choice lists for timezones, since the naming is more uniform
than the generic non-location format. It is also a fallback format when there is no translation
for the generic non-location format.
"United States (Los Angeles) Time"
"Italy Time".
Specific
non-location format:
Reflects a specific standard or daylight time, which may or may not
be the wall time. For example, "10 am Pacific Standard Time" will be GMT-8 in the winter and in
the summer.
"Pacific Standard Time"
"PST"
"Pacific Daylight Time"
"PDT"
Localized
GMT format:
A constant, specific offset from GMT (or UTC), which may be in a translated
form.
"HMG+03.30"
"GMT+03:30"
Гриинуич
+03:30
RFC 822 GMT
format:
A constant, specific offset from GMT (or UTC), which always has the same format.
"-0800"
Raw Offset
- an offset from GMT that does not include any daylight savings behavior. For
example, the raw offset for Pacific Time is -8, even though the
observed offset
may be -8
or -7.
Metazone
- a collection of time zones that share the same behavior and same name during some
period. They may differ in
daylight behavior (whether they have it and when).
For example, the TZID America/Cambridge_Bay is in the following
metazones during various periods:







Zones may join or leave a metazone over
time. The data relating between zones and metazones is in the supplemental information; the
locale data is restricted to translations of metazones and zones, with one extra flag for usage
of abbreviations (commonlyUsed).
Invariants:
At any given
point in time, each zone belongs to exactly one metazone.
Except for daylight
savings
, at any given time, all zones in a metazone have the same offset at that time.
Golden Zone
- the TZDB zone that exemplifies a metazone.
For example, America/New_York is the golden zone for the metazone America_Eastern:
type="America/New_York"/>
Invariants:
The golden zones
are those in mapZone supplemental data under the territory "001".
Every metazone has exactly one golden zone.
Each zone has
at most one metazone for which it is golden.
The golden zone is in that metazone
during the entire life of the metazone. (The raw offset of the golden zone may change over
time.)
Each other zone must have the same raw offset as the golden
zone, for the entire period that it is in the metazone. (It might not have the same offset
when daylight savings is in effect.)
A golden zone in mapTimezones (supplementalData.xml) must have
reverse mapping in metazoneInfo (metazoneInfo.xml)
Preferred
Zone
- for a given TZID, the "best" zone out of a metazone for a given country or language.
Invariants:
The preferred zone for a given country XX
are those in mapZone supplemental data under the territory XX.
Every metazone has
at most one preferred zone for a given territory XX.
Each zone has
at most one metazone for which it is preferred for a territory XX.
The preferred zone
for a given metazone and territory XX is in a metazone M during any time when any other zone
in XX is also in M
A preferred
zone in mapTimezones (supplementalData.xml) must have reverse mapping in metazoneInfo (metazoneInfo.xml)
For example, for America_Pacific the preferred zone for Canada is
America/Vancouver, and the preferred zone for Mexico is America/Tijuana. The golden zone is
America/Los_Angeles, which is also also the preferred zone for any other country.
type="America/Los_Angeles"/>


fallbackFormat
a formatting string such as "{1} ({0})",
which is used to combine two pieces of zone information.
regionFormat:
a formatting string such as "{0} Time".
May use constructed pieces, such as where the {0} is produced from the fallbackFormat.
Goals
The timezones are designed so that:
For any given locale, every
time
round trips with all
patterns Z, ZZZZ, z, zzzz, v, vvvv, V, VVVV (but not necessarily every timezone). That is,
given a time and a format pattern with a zone string, you can format, then parse, and get
back the same time.
Note that the round-tripping is not just important for parsing; it
provides for formatting dates and times in an unambiguous way for users. It is also
important for testing.
There are exceptions to the above for transition times.
With
generic, during the transition when the local time maps to two possible GMT times.
For
example, Java works as follows, favoring standard time:
Source: Sun Nov 04 01:30:00 PDT 2007
=>
Formatted: "Sunday, November 4, 2007 1:30:00 AM"
=>
Parsed: Sun Nov 04 01:30:00 PST 2007
When the
timezone changes offset, say from GMT+4 to GMT+5, there can also be a gap.
The VVVV format will roundtrip not only the time, but the
canonical timezone.
When the data for a given format is not available, a fallback format
is used. The fallback order is given in the following by a list.
Specifics
z - [short form] specific non-location (but only if
commonlyUsed)
falling back to localized GMT
zzzz - [long form] specific non-location
falling back to localized GMT
Z - RFC 822 (no fallback necessary)
ZZZZ - Localized GMT (no fallback necessary)
V - specific non-location (ignores commonlyUsed)
falling
back to
localized GMT
Generics
v - [short form] generic non-location, if commonlyUsed
(however, the rules are more complicated, see #4 below)
falling back to generic location
falling back to localized GMT
vvvv - [long form] generic non-location
(however, the rules are more complicated, see #4 below)
falling back to generic location
falling back to localized GMT
VVVV - generic location
falling back to localized GMT
The following process is used for the particular formats, with the
fallback rules as above.
Some of the examples are drawn from real data, while others are for illustration. For illustration the region format is "Hora
de {0}".
The fallback format in the examples is "{1} ({0})", which is what is in root.
In
all
cases, first canonicalize the
TZ
ID according to the table in supplemental data. Use that canonical TZID in each of the following steps.
America/Atka → America/Adak
Australia/ACT → Australia/Sydney
For RFC 822 format ("Z") return the results according to the RFC.
America/Los_Angeles → "-0800"
Note:
The digits in this case are always from the western digits, 0..9.
For the localized GMT format,
use the gmtFormat (such as "GMT{0}" or "HMG{0}") with the hourFormat (such as "+HH:mm;-HH:mm"
or "+HH.mm;-HH.mm").
America/Los_Angeles → "GMT-08:00" //  standard time
America/Los_Angeles → "HMG-07:00" //  daylight time
Etc/GMT+3 → "GMT-03.00" // note that
TZ
tzids have inverse polarity!
Note:
The digits should be whatever are appropriate for the locale used to format the time zone, not necessarily from the western digits, 0..9.
For example, they might be from ०..९.
For the non-location formats (generic or specific),
if there is an explicit translation for the TZID in
timeZoneNames according to type (generic, standard, or daylight) in the resolved locale, return it.
America/Los_Angeles → "Heure du Pacifique (ÉUA)" // generic
America/Los_Angeles → 太平洋標準時 // standard
America/Los_Angeles → Yhdysvaltain Tyynenmeren kesäaika // daylight
Europe/Dublin  → Am Samhraidh na hÉireann // daylight
Note:
This translation may not at all be literal: it would be what is most recognizable for people using the target language.
Otherwise, if there is a metazone standard format, and the offset
and daylight offset do not change within 184 day +/- interval around the exact
formatted time, use the metazone standard format ("Mountain Standard Time" for
Phoenix). (184 is the smallest number that is at least 6 months AND the smallest
number that is more than 1/2 year (Gregorian)).
Otherwise, if there is a metazone generic format, then do the
following:
Compare offset at the requested time with the
preferred zone for the current locale; if same, we use the metazone generic
format. "Pacific Time" for Vancouver if the locale is en-CA, or for Los Angeles
if locale is en-US. Note that the fallback is the golden zone.
The metazone data actually supplies the preferred zone for a country. If the locale
does not have a country
the likelySubtags supplemental data is used
to get the most likely country.
If the zone is the preferred zone for its country
but not for the country of the locale, use the metazone generic format +
(country)
[Generic partial location]
"Pacific Time (Canada)" for the zone
Vancouver in the locale en_MX.
If all else fails, use metazone generic format +
(city).
[Generic partial location]:
"Mountain Time (Phoenix)", "Pacific
Time (Yellowknife)"
Otherwise, fall back.
Note:
In composing the metazone + city or country: use the
fallbackFormat
{1} will
be the city or country
{0} will be the generic metazone
Example:
United States (Phoenix) Time
For the generic location format:
Use as the country name, the explicitly localized country if available,
otherwise the raw country code. If the localized exemplar city is not available, use as the exemplar city the last field of the raw TZID, stripping off
the prefix and turning _ into space.
CU → "CU" // no localized country name for Cuba
America/Los_Angeles → "Los Angeles" // no localized exemplar city
From get the country code for the zone, and determine whether there is only one timezone in the country. If there is only one timezone
or the zone id is in the singleCountries list, format the country name with the regionFormat
(for example, "{0} Time"), and return it.
Europe/Rome → IT → Italy Time // for English
Africa/Monrovia → LR → "Hora de Liberja"
America/Havana → CU → "Hora de CU" // if CU is not localized
Note:
If a language does require grammatical changes when composing strings, then it should either use a neutral format such as what is in root,
or put all exceptional cases in explicitly translated strings.
Otherwise, get both the exemplar city and country name. Format  them with the fallbackFormat (for
example, "{1} ({0})"),
and substitute that in the regionFormat (for example, "{0} Time").
America/Buenos_Aires → "Argentina (Buenos Aires)" Time
// if the regionFormat is "{0} Time".
America/Buenos_Aires → "Аргентина (Буэнос-Айрес)"
// if both are translated, and the regionFormat is "{0}".
America/Buenos_Aires → "AR (Буэнос-Айрес)"
// if Argentina is not translated.
America/Buenos_Aires → "Аргентина (Buenos Aires)"
// if Buenos Aires is not translated.
America/Buenos_Aires → "AR (Buenos Aires)"
// if both are not translated.
Note:
As with the regionFormat, exceptional cases need to be explicitly translated.
Parsing
In parsing, an implementation will be able to either determine the zone id, or a simple offset from GMT for anything formatting according to the above process.
The following is a sample process for how this might be done. It is
only a sample; implementations may use different methods for parsing.
The sample describes the parsing of a zone as if it were an isolated string. In
implementations, the zone may be mixed in with other data (like the time), so the parsing
actually has to look for the longest match, and then allow the remaining text to be parsed for
other content. That requires certain adaptions to the following process.
Start with a string S.
If S matches the RFC 822 GMT format,
return it.
or example, "-0800" => Etc/GMT+8
If S matches the English or localized
GMT format, return the corresponding TZID
Matching should be lenient. Thus
allow for the number formats like: 03, 3, 330, 3:30, 33045 or 3:30:45. Allow +, -, or
nothing. Allow spaces after GMT, +/-, and before number. Allow non-Latin numbers. Allow
UTC or UT (per RFC 788) as synonyms for GMT in the English format.
For example, "GMT+3" or "UT+3" or "HPG+3" =>
Etc/GMT-3
If S matches the fallback format,
extract P = {0} [ie, the part in parens in the root format] and N = {1}.
If S does not match, set P = "" and N = S
If N matches the region format, then M = {0} from that format, otherwise M = N.
For example, "United States (Los
Angeles) Time" => N = "United States Time", M = "United States", P = "Los Angeles".
For example, "United States Time" => N =
"United States Time", M = "United States", P = "".
For example, "United States" => N = M =
"United States", P = "".
If P, N, or M is a localized country,
set C to that value. If C has only one zone, return it.
For example, "Italy Time (xxx)" or "xxx
(Italy)" => Europe/Rome
For example, "xxx (Canada)" or "Canada Time
(xxx)" => Sets C = CA and continues
If P is a localized TZID (and not
metazone), return it.
For example, "xxxx (Phoenix)" or "Phoenix
(xxx)" => America/Phoenix
If N, or M is a localized TZID (and not
metazone), return it.
For example, "Pacific Standard Time (xxx)" =>
"America/Los_Angeles" // this is only if "Pacific Standard Time" is not a metazone
localization.
If N or M is a localized metazone
If it corresponds to only one TZID,
return it.
If C is set, look up the Metazone +
Country => TZID mapping, and return that value if it exists
Get the locale's language, and get
the default country from that. Look up the Metazone + DefaultCountry => TZID mapping,
and return that value if it exists.
Otherwise, lookup Metazone + 001 =>
TZID and return it (that will always exist)
If you get this far, return an error.
Parsing can be more lenient than the above, allowing for different spacing, punctuation, or other variation.
stricter parse would check for consistency between the xxx portions above and the rest, so
"Pacific Standard Time (India)" would give an error.
Using this process, a correct parse will roundtrip the location
format (VVVV) back to the canonical zoneid.
Australia/ACT → Australia/Sydney → “Sydney (Australia)” → Australia/Sydney
The GMT formats (Z and ZZZZ) will return back an offset, and thus lose the original canonical zone id.
Australia/ACT → Australia/Sydney → "GMT+11:00" → GMT+11
The daylight and standard time formats, and the non-location
formats (z, zzzz, v, and vvvv) may either roundtrip back to the original canonical zone id,
to a zone in the same metazone that time, or to just an offset, depending on the available
translation data. Thus:
Australia/ACT → Australia/Sydney → "GMT+11:00" → GMT+11
PST8PDT → America/Los_Angeles → “PST” → America/Los_Angeles
America/Vancouver → “Pacific Time (Canada)” →
America/Vancouver
Note:
The hoursFormat, preferenceOrdering, and abbreviationFallback
items used in earlier versions of this appendix are deprecated.
Appendix K: Valid Attribute Values
The valid attribute values, as well as other validity information is contained in the metadata.xml file. (Some, but not all, of this information could have
been represented in XML Schema or a DTD.)
The following specify the ordering of elements / attributes in the file
ldml identity alias localeDisplayNames layout ...
type key registry alt source path day date...
The suppress elements are those that are suppressed in canonicalization.
The serialElements are those that do not inherit, and may have ordering
variable comment tRule reset p pc s sc t tc q qc i ic x extend first_variable last_variable first_tertiary_ignorable last_tertiary_ignorable
first_secondary_ignorable last_secondary_ignorable first_primary_ignorable last_primary_ignorable first_non_ignorable last_non_ignorable first_trailing
last_trailing
The validity elements give the possible attribute values. They are in the format of a series of variables, followed by attributeValues.

buddhist coptic ethiopic chinese gregorian hebrew islamic islamic-civil japanese arabic civil-arabic thai-buddhist persian

The types indicate the style of match:
choice: for a list of possible values
regex: for a regular expression match
notDoneYet: for items without matching criteria
locale: for locale IDs
list: for a space-delimited list of values
path: for a valid [
XPath
If the attribute order="given" is supplied, it indicates the order of elements when canonicalizing (see below).
The element lists elements, attributes, and attribute values that are deprecated. If any deprecatedItems element contains more than one attribute,
then only the listed combinations are deprecated. Thus the following means not that the draft attribute is deprecated, but that the true and false values for
that attribute are:

Similarly, the following means that the
type
attribute is deprecated, but only for the listed elements:



The blockingItems indicate which elements (and their child elements) do not inherit. For
example, because supplementalData is a blocking item, all paths containing the element
supplementalData
do not inherit.




The distinguishing items indicate which combinations of elements and attributes (in unblocked
environments) are
distinguishing
in performing inheritance. For example, the attribute
type is distinguishing
except
in combination with certain elements, such as in:
exclude="true"
elements="default measurementSystem mapping abbreviationFallback preferenceOrdering"
attributes="type"/>
Appendix L:
Canonical Form
The following are restrictions on the format of LDML files to allow for easier parsing and comparison of files.
Peer elements have consistent order. That is, if the DTD or this specification requires the following order in an element foo:




It can never require the reverse order in a different element bar.




Note that there was one case that had to be corrected in order to make this true. For that reason, pattern occurs twice under currency:
decimal?, group?, special*)) >
XML
files can have a wide variation in textual form, while representing precisely the same data. By putting the
LDML files in the repository into a canonical form, this allows us to use the simple diff tools used widely (and in CVS) to detect differences when vetting
changes, without those tools being confused. This is not a requirement on other uses of LDML; just simply a way to manage repository data more easily.
L.1 Content
All start elements are on their own line, indented by
depth
tabs.
All end elements (except for leaf nodes) are on their own line, indented by
depth
tabs.
Any leaf node with empty content is in the form .
There are no blank lines except within comments or content.
Spaces are used within a start element. There are no extra spaces within elements.

, not


, not

All attribute values use double quote ("), not single (').
There are no CDATA sections, and no escapes except those absolutely required.
no ' since it is not necessary
no 'a', it would be just 'a'
All attributes with defaulted values are suppressed. See the
Defaulted Attributes Table
XXX
The draft and alt="proposed.*" attributes are only on leaf elements.
The tzid are canonicalized in the following way:
All tzids as of as CLDR 1.1 (2004.06.08) in zone.tab are canonical.
After that point, the first time a tzid is introduced, that is the canonical form.
That is, new IDs are added, but existing ones keep the original form. The
TZ
timezone database keeps a set of equivalences in the "backward" file.
These are used to map other tzids to the canonical form. For example, when
America/Argentina/Catamarca
was introduced as the new name for the
previous
America/Catamarca
, a link was added in the backward file.
Link America/Argentina/Catamarca America/Catamarca
Example:











¤#,##0.00;(¤#,##0.00)





L.2 Ordering
Element names are ordered by the
Element Order Table
Attribute names are ordered by the
Attribute Order
Table
Attribute value comparison is a bit more complicated, and may depend on the attribute and type. Compare two values by using the following steps:
If two values are in the
Value Order Table
compare according to the order in the table. Otherwise if just one is, it goes first.
If two values are numeric [0-9], compare numerically (2 < 12). Otherwise if just one is numeric, it goes first.
Otherwise values are ordered alphabetically
An attribute-value pair is ordered first by attribute name, and then if the attribute names are identical, by the value.
An element is ordered first by the element name, and then if the
element names are identical, by the sorted set of attribute-value pairs
(sorted by #4). For the latter, compare the first pair in each (in
sorted order by attribute pair). If not identical, go to the second
pair, and so on.
Any future additions to the DTD must be structured so as to allow compatibility with this ordering.
See also Appendix K:
Valid Attribute Values
L.3 Comments
Comments are of the form .
They are logically attached to a node. There are 4 kinds:
Inline always appear after a leaf node, on the same line at the end. These are a single line.
Preblock comments always precede the attachment node, and are indented on the same level.
Postblock comments always follow the attachment node, and are indented on the same level.
Final comment, after
Multiline comments (except the final comment) have each line after the first indented to one deeper level.
Examples:

BC
...



...


L.4 Canonicalization
The process of canonicalization is fairly straightforward, except for comments. Inline comments will have any linebreaks replaced by a space. There may be
cases where the attachment node is not permitted, such as the following.




In those cases, the comment will be made into a block comment on the last previous leaf node, if it is at that level or deeper. (If there is one already,
it will be appended, with a line-break between.) If there is no place to attach the node (for example, as a result of processing that removes the attachment
node), the comment and its node's [
XPath
] will be appended to the final comment in the document.
Multiline comments will have leading tabs stripped, so any indentation should be done with spaces.
L.5
Element Order Table
The order of attributes is given by the elementOrder table in the supplemental metadata.
L.6
Attribute Order Table
The order of attributes is given by the attributeOrder table in the supplemental metadata.
L.7
Value Order Table
The order of attribute values is given by the order of the values in the attributeValues elements that have the attibute order="given". Numeric values are
sorted in numeric order, while tzids are ordered by country, then longitude, then latitude.
L.8
Defaulted Values Table
The defaulted attributes are given by the
suppress
table in the supplemental metadata. There is one special value _q; that is used on serial elements
internally to preserve ordering.
Appendix M: Coverage Levels
The following defines the coverage levels:
Level
Description
level 100
comprehensive
Has complete localizations (or valid inheritance) for every possible field
level 80
modern
Localizations (or valid inheritance) as given below
level 60
moderate
level 40
basic
level 30
minimal
level 20
posix
Only what is required for POSIX generation; example, only one
country name, only one currency symbol, and so on.
level 0
rudimentary
Does not meet any of the above levels.
Levels 40 through 80 are based on the following definitions and specifications.
The minimal coverage is POSIX plus the name of the language in itself,
territories using the language, plus a few key values such as the values in Section 3.1
Unknown or Invalid Identifiers
M.1 Definitions
Target-Language
is the language under consideration.
Target-Territories
is the list of territories found by looking up
Target-Language
in the elements in
supplementalData.xml
Language-List
is
Target-Language
, plus
basic:
Chinese, English, French, German, Italian, Japanese, Portuguese, Russian, Spanish (de, en, es, fr, it, ja, pt, ru, zh)
moderate:
basic + Arabic, Hindi, Korean, Indonesian, Dutch, Bengali, Turkish, Thai, Polish (ar, hi, ko, in, nl, bn, tr, th, pl). If an EU
language, add the remaining official EU languages, currently: Danish, Greek, Finnish, Swedish, Czech, Estonian, Latvian, Lithuanian, Hungarian, Maltese,
Slovak, Slovene (da, el, fi, sv, cs, et, lv, lt, hu, mt, sk, sl)
modern:
all languages that are official or major commercial languages of modern territories
Target-Scripts
is the list of scripts in which
Target-Language
can be customarily written (found by looking up
Target-Language
in the elements in
supplementalData.xml
Script-List
is the
Target-Scripts
plus the major scripts used for multiple languages
Latin, Simplified Chinese, Traditional Chinese, Cyrillic, Arabic (Latn, Hans, Hant, Cyrl, Arab)
Territory-List
is the list of territories formed by taking the
Target-Territories
and adding:
basic:
Brazil, China, France, Germany, India, Italy, Japan, Russia, United Kingdom, United States (BR, CN, DE, GB, FR, IN, IT, JP, RU, US)
moderate:
basic + Spain, Canada, Korea, Mexico, Australia, Netherlands, Switzerland, Belgium, Sweden, Turkey, Austria, Indonesia, Saudi Arabia,
Norway, Denmark, Poland, South Africa, Greece, Finland, Ireland, Portugal, Thailand, Hong Kong SAR China, Taiwan (ES, BE, SE, TR, AT, ID, SA, NO, DK,
PL, ZA, GR, FI, IE, PT, TH, HK, TW). If an EU language, add the remaining member EU countries: Luxembourg, Czech Republic, Hungary, Estonia, Lithuania,
Latvia, Slovenia, Slovakia, Malta (LU, CZ, HU, ES, LT, LV, SI, SK, MT).
modern:
all current ISO 3166 territories, plus the UN
M.49 [
UNM49
] regions in
supplementalData.xml
Currency-List
is the list of current official currencies used in any of the territories in
Territory-List
, found by looking at the region
elements in
supplementalData.xml
Calendar-List
is the set of calendars in customary use in any of
Target-Territories
, plus Gregorian.
Timezone-List
is the set of all timezones for multi-zone territories in
Target-Territories
, plus each of following timezones whose territories
is in
Territory-List
Argentina:
America/Buenos_Aires
Brazil:
America/Rio_Branco, America/Campo_Grande, America/Sao_Paulo
Australia:
Australia/Perth, Australia/Darwin, Australia/Brisbane, Australia/Adelaide, Australia/Sydney, Australia/Hobart
Canada:
America/Vancouver, America/Edmonton, America/Regina, America/Winnipeg, America/Toronto, America/Halifax, America/St_Johns
Mexico:
America/Tijuana, America/Hermosillo, America/Chihuahua, America/Mexico_City
US:
Pacific/Honolulu, America/Anchorage, America/Los_Angeles, America/Phoenix, America/Denver, America/Chicago, America/Indianapolis, America/New_York
Machine-readable information for this is contained in the element in the supplemental metadata file.
M.2 Data Requirements
The required data to qualify for the level is then the following.
identity
localeDisplayNames
languages:
localized names for all languages in
Language-List.
scripts:
localized names for all scripts in
Script-List
territories:
localized names for all territories in
Territory-List
variants, keys, types:
localized names for any in use in
Target-Territories
for example, a translation for PHONEBOOK in a German locale.
layout, orientation
exemplarCharacters
measurementSystem, paperSize
dates: all of the following for each calendar in
Calendar-List
calendars: localized names
monthNames & dayNames
context=format and width=narrow, wide, & abbreviated
plus context=standAlone and width=narrow, wide, & abbreviated,
if the grammatical forms of these are different than for context=format.
week: minDays, firstDay, weekendStart, weekendEnd
if some of these vary in territories in
Territory-List
, include territory locales for those that do.
am, pm, eraNames, eraAbbr
dateFormat, timeFormat: full, long, medium, short
timeZoneNames:
exemplar cities the timezones in
Timezone-List
hourFormat, hoursFormat, gmtFormat, regionFormat, fallbackFormat
numbers: symbols, decimalFormats, scientificFormats, percentFormats, currencyFormats
currencies: displayName and symbol for all currencies in
Currency-List
collation sequence
yesstr, nostr
transforms:
basic:
none
moderate
: transliteration between Latin and each other script in
Target-Scripts.
M.3 Default Values
Items should
only
be included if they are not the same as the default, which is:
what is in root, if there is something defined there.
for timezone IDs: the name computed according to
Appendix J: Time Zone Display Names
for collation sequence, the UCA DUCET (Default Unicode Collation Element Table)
however, in that case the locale must be added to the validSubLocale list in
collation/root.xml
for currency symbol, language, territory, script names, variants, keys, types, the internal code identifiers,
for example,
currencies: EUR, USD, JPY, ...
languages: en, ja, ru, ...
territories: GB, JP, FR, ...
scripts: Latn, Thai, ...
variants: PHONEBOOK,...
Appendix N:
Transform Rules









The transform rules are similar to regular-expression substitutions, but adapted to the specific domain of text transformations. The rules and comments in
this discussion will be intermixed, with # marking the comments. In the xml format these in separate elements:
comment
and
tRule
. The simplest rule is a conversion rule, which replaces one string of characters with another. The conversion rule takes
the following form:
xy → z ;
This converts any substring "xy" into "z". Rules are executed in order; consider the following rules:
sch → sh ;
ss → z ;
This conversion rule transforms "bass school" into "baz shool". The transform walks through the string from start to finish. Thus given the rules above "bassch"
will convert to "bazch", because the "ss" rule is found before the "sch" rule in the string (later, we'll see a way to override this behavior). If two rules
can both apply at a given point in the string, then the transform applies the first rule in the list.
All of the ASCII characters except numbers and letters are reserved for use in the rule syntax, as are the characters →, ←, ↔. Normally, these characters
do not need to be converted. However, to convert them use either a pair of single quotes or a slash. The pair of single quotes can be used to surround a whole
string of text. The slash affects only the character immediately after it. For example, to convert from an arrow signs to the word "arrow", use one of the following
rules:
\←   →  arrow\ sign ;
'←'   →   'arrow sign' ;
'←'   →   arrow' 'sign ;
Spaces may be inserted anywhere without any effect on the rules. Use extra space to separate items out for clarity without worrying about the effects. This
feature is particularly useful with combining marks; it is handy to put some spaces around it to separate it from the surrounding text. The following is an
example:
→ i ; # an iota-subscript diacritic turns into an i.
For a real space in the rules, place quotes around it. For a real backslash, either double it \\, or quote it '\'. For a real single quote, double it '',
or place a backslash before it \'.
Any text that starts with a hash mark and concludes a line is a comment. Comments help document how the rules work. The following shows a comment in a rule:
x → ks ; # change every x into ks
The "\u" notation can be used instead of any letter. For instance, instead of using the Greek π, one could write:
\u03C0 → p ;
One can also define and use variables, such as:
$pi = \u03C0 ;
$pi → p ;
N.1 Dual Rules
Rules can also specify what happens when an inverse transform is formed. To do this, we reverse the direction of the "←" sign. Thus the above example becomes:
$pi ← p ;
With the inverse transform, "p" will convert to the Greek p. These two directions can be combined together into a dual conversion rule by using the "↔" operator,
yielding:
$pi ↔ p ;
N.2 Context
Context can be used to have the results of a transformation be different depending on the characters before or after. The following means "Remove hyphens,
but only when they follow lower case letters":
[:lowercase letter:] } '-' → '' ;
The context itself ([:lowercase letter:]) is unaffected by the replacement; only the text between the curly braces is changed.
N.3 Revisiting
If the resulting text contains a vertical bar "|", then that means that processing will proceed from that point and that the transform will revisit part
of the resulting text. Thus the | marks a "cursor" position. For example, if we have the following, then the string "xa" will convert to "w".
x → y | z ;
z a → w;
First, "xa" is converted to "yza". Then the processing will continue from after the character "y", pick up the "za", and convert it. Had we not had the "|",
the result would have been simply "yza". The '@' character can be used as filler character to place the revisiting point off the start or end of the string.
Thus the following causes x to be replaced, and the cursor to be backed up by two characters.
x → |@@y;
N.4 Example
The following shows how these features are combined together in the Transliterator "Any-Publishing". This transform converts the ASCII typewriter conventions
into text more suitable for desktop publishing (in English). It turns straight quotation marks or UNIX style quotation marks into curly quotation marks, fixes
multiple spaces, and converts double-hyphens into a dash.
# Variables
$single = \' ;
$space = ' ' ;
$double = \" ;
$back = \` ;
$tab = '\u0008' ;
# the following is for spaces, line ends, (, [, {, ...
$makeRight = [[:separator:][:start punctuation:][:initial punctuation:]] ;
# fix UNIX quotes
$back $back → “ ; # generate right d.q.m. (double quotation mark)
$back → ‘ ;
# fix typewriter quotes, by context
$makeRight { $double ↔ “ ; # convert a double to right d.q.m. after certain chars
^ { $double → “ ; # convert a double at the start of the line.
$double ↔ ” ; # otherwise convert to a left q.m.
$makeRight {$single} ↔ ‘ ; # do the same for s.q.m.s
^ {$single} → ‘ ;
$single ↔ ’;
# fix multiple spaces and hyphens
$space {$space} → ; # collapse multiple spaces
'--' ↔ — ; # convert fake dash into real one
N.5 Rule Syntax
The following describes the full format of the list of rules used to create a transform. Each rule in the list is terminated by a semicolon. The list consists
of the following:
an optional filter rule
zero or more transform rules
zero or more variable-definition rules
zer or more conversion rules
an optional inverse filter rule
The filter rule, if present, must appear at the beginning of the list, before any of the other rules.  The inverse filter rule, if present, must appear
at the end of the list, after all of the other rules.  The other rules may occur in any order and be freely intermixed.
The rule list can also generate the inverse of the transform. In that case, the inverse of each of the rules is used, as described below.
N.6 Transform Rules
Each transform rule consists of two colons followed by a transform name, which is of the form source-target. For example:
:: NFD ;
:: und_Latn-und_Greek ;
:: Latin-Greek; # alternate form
If either the source or target is 'und', it can be omitted, thus 'und_NFC' is equivalent to 'NFC'. For compatibility, the English names for scripts can be
used instead of the und_Latn locale name, and "Any" can be used instead of "und". Case is not signficant.
The following transforms are defined not by rules, but by the operations in the Unicode Standard, and may be used in building any other transform:
Any-NFC, Any-NFD, Any-NFKD, Any-NFKC
- the normalization forms
defined by [
UAX15
].
Any-Lower, Any-Upper, Any-Title
- full case transformations, defined by [
Unicode
] Chapter 3.
In addition, the following special cases are defined:
Any-Null
- has no effect; that is, each character is left alone.
Any-Remove
- maps each character to the empty string; this, removes each character.
The inverse of a transform rule uses parentheses to indicate what should be done when the inverse transform is used. For example:
:: lower () ; # only executed for the normal
:: (lower) ; # only executed for the inverse
:: lower ; # executed for both the normal and the inverse
N.7 Variable Definition Rules
Each variable definition is of the following form:
$variableName = contents ;
The variable name can contain letters and digits, but must start with a letter. More precisely, the variable names use Unicode identifiers as defined by
[UAX31]. The identifier properties allow for the use of foreign letters and numbers.
The contents of a variable definition is any sequence of Unicode sets and characters or characters. For example:
$mac = M [aA] [cC] ;
Variables are only replaced within other variable definition rules and within conversion rules. They have no effect on transliteration rules.
N.8 Filter Rules
A filter rule consists of two colons followed by a UnicodeSet. This filter is global in that only the characters matching the filter will be affected by
any transform rules or conversion rules. The inverse filter rule consists of two colons followed by a UnicodeSet in parentheses. This filter is also global
for the inverse transform.
For example, the Hiragana-Latin transform can be implemented by "pivoting" through the Katakana converter, as follows:
:: [:^Katakana:] ; # do not touch any katakana that was in the text!
:: Hiragana-Katakana;
:: Katakana-Latin;
:: ([:^Katakana:]) ; # do not touch any katakana that was in the text
# for the inverse either!
The filters keep the transform from mistakenly converting any of the "pivot" characters. Note that this is a case where a rule list contains no conversion
rules at all, just transform rules and filters.
N.9 Conversion Rules
Conversion rules can be forward, backward, or double. The complete conversion rule syntax is described below:
N.9.1
Forward
A forward conversion rule is of the following form:
before_context { text_to_replace } after_context → completed_result | result_to_revisit ;
If there is no before_context, then the "{" can be omitted. If there is no after_context, then the "}" can be omitted. If there is no result_to_revisit,
then the "|" can be omitted. A forward conversion rule is only executed for the normal transform and is ignored when generating the inverse transform.
N.9.2
Backward
A backward conversion rule is of the following form:
completed_result | result_to_revisit ← before_context { text_to_replace } after_context ;
The same omission rules apply as in the case of forward conversion rules. A backward conversion rule is only executed for the inverse transform and is ignored
when generating the normal transform.
N.9.3
Dual
A dual conversion rule combines a forward conversion rule and a backward conversion rule into one, as discussed above. It is of the form:
a { b | c } d ↔ e { f | g } h ;
When generating the normal transform and the inverse, the revisit mark "|" and the before and after contexts are ignored on the sides where they do
not belong.
Thus, the above is exactly equivalent to the sequence of the following two rules:
a { b c } d  →  f | g  ;
b | c  ←  e { f g } h ;
N.10 Intermixing Transform Rules and Conversion Rules
Transform rules and conversion rules may be freely intermixed. Inserting a transform rule into the middle of a set of conversion rules has an important side
effect.
Normally, conversion rules are considered together as a group.  The only time their order in the rule set is important is when more than one rule matches
at the same point in the string.  In that case, the one that occurs earlier in the rule set wins.  In all other situations, when multiple rules match
overlapping parts of the string, the one that matches earlier wins.
Transform rules apply to the whole string.  If you have several transform rules in a row, the first one is applied to the whole string, then the second
one is applied to the whole string, and so on.  To reconcile this behavior with the behavior of conversion rules, transform rules have the side effect
of breaking a surrounding set of conversion rules into two groups: First all of the conversion rules before the transform rule are applied as a group to the
whole string in the usual way, then the transform rule is applied to the whole string, and then the conversion rules after the transform rule are applied as
a group to the whole string.  For example, consider the following rules:
abc → xyz;
xyz → def;
::Upper;
If you apply these rules to “abcxyz”, you get “XYZDEF”.  If you move the “::Upper;” to the middle of the rule set and change the cases accordingly,
then applying this to “abcxyz” produces “DEFDEF”.
abc → xyz;
::Upper;
XYZ → DEF;
This is because “::Upper;” causes the transliterator to reset to the beginning of the string. The first rule turns the string into “xyzxyz”, the second rule
upper cases the whole thing to “XYZXYZ”, and the third rule turns this into “DEFDEF”.
This can be useful when a transform naturally occurs in multiple “passes.”  Consider this rule set:
[:Separator:]* → ' ';
'high school' → 'H.S.';
'middle school' → 'M.S.';
'elementary school' → 'E.S.';
If you apply this rule to “high school”, you get “H.S.”, but if you apply it to “high  school” (with two spaces), you just get “high school” (with one
space).  To have “high  school” (with two spaces) turn into “H.S.”, you'd either have to have the first rule back up some arbitrary distance (far
enough to see “elementary”, if you want all the rules to work), or you have to include the whole left-hand side of the first rule in the other rules, which
can make them hard to read and maintain:
$space = [:Separator:]*;
high $space school → 'H.S.';
middle $space school → 'M.S.';
elementary $space school → 'E.S.';
Instead, you can simply insert “
::Null;
” in order to get things to work right:
[:Separator:]* → ' ';
::Null;
'high school' → 'H.S.';
'middle school' → 'M.S.';
'elementary school' → 'E.S.';
The “::Null;” has no effect of its own (the null transform, by definition,
does not do anything), but it splits the other rules into two “passes”: The first
rule is applied to the whole string, normalizing all runs of white space into single spaces, and then we start over at the beginning of the string to look for
the phrases.  “high    school” (with four spaces) gets correctly converted to “H.S.”.
This can also sometimes be useful with rules that have overlapping domains.  Consider this rule set from before:
sch → sh ;
ss → z ;
Apply this rule to “bassch” results in “bazch” because “ss” matches earlier in the string than “sch”.  If you really wanted “bassh”—that is, if you
wanted the first rule to win even when the second rule matches earlier in the string, you'd either have to add another rule for this special case...
sch → sh ;
ssch → ssh;
ss → z ;
...or you could use a transform rule to apply the conversions in two passes:
sch → sh ;
::Null;
ss → z ;
N.11 Inverse Summary
The following table shows how the same rule list generates two different transforms, where the inverse is restated in terms of forward rules (this is a contrived
example, simply to show the reordering):
Original Rules
Forward
Inverse
:: [:Uppercase Letter:] ;
:: latin-greek ;
:: greek-japanese ;
x ↔ y ;
z → w ;
r ← m ;
:: upper;
a → b ;
c ↔ d ;
:: any-publishing ;
:: ([:Number:]) ;
:: [:Uppercase Letter:] ;
:: latin-greek ;
:: greek-japanese ;
x → y ;
z → w ;
:: upper ;
a → b ;
c → d ;
:: any-publishing ;
:: [:Number:] ;
:: publishing-any ;
d → c ;
:: lower ;
y → x ;
m → r ;
:: japanese-greek ;
:: greek-latin ;
Note how the irrelevant rules (the inverse filter rule and the rules containing ←) are omitted (ignored, actually) in the forward direction, and notice how
things are reversed: the transform rules are inverted and happen in the opposite order, and the groups of conversion rules are also executed in the opposite
relative order (although the rules within each group are executed in the same order).
Appendix O: Lenient Parsing
O.1 Motivation
User input is frequently messy. Attempting to parse it by matching it exactly against a pattern is likely to be unsuccessful, even when the meaning of the
input is clear to a human being. For example, for a date pattern of "MM/dd/yy", the input "June 1, 2006" will fail.
The goal of lenient parsing is to accept user input whenever it is possible to decipher what the user intended. Doing so requires using patterns as data
to guide the parsing process, rather than an exact template that must be matched. This informative section suggests some heuristics that may be useful for lenient
parsing of dates, times, and numbers.
O.2 Loose Matching
Loose matching ignores attributes of the strings being compared that are not important to matching. It involves the following steps:
Remove "." from currency symbols and other fields used for matching, and also from the input string unless:
"." is in the decimal set, and
its position in the input string is immediately before a decimal digit
Ignore all format characters: in particular, ignore the RLM
and LRM used to control BIDI formatting.
Ignore all characters in [:Zs:] unless they occur between letters. (In the heuristics below, even those between letters are ignored except to delimit
fields)
Map all characters in [:Dash:] to U+002D HYPHEN-MINUS
Use the data in to map equivalent characters
(for example, curly to straight apostrophes).
Apply mappings particular to the domain (i.e., for dates or for numbers, discussed in more detail below)
Apply case folding (possibly including language-specific mappings such as Turkish i)
Normalize to NFKC; thus
no-break space
will map to
space
; half-width
katakana
will map to full-width.
Loose matching involves (logically) applying the above transform to both the input text and to each of the field elements used in matching, before applying
the specific heuristics below. For example, if the input number text is " - NA f. 1,000.00", then it is mapped to "-naf1,000.00" before processing. The currency
signs are also transformed, so "NA f." is converted to "naf" for purposes of matching. As with other Unicode algorithms, this is a logical statement of the
process; actual implementations can optimize, such as by applying the transform incrementally during matching.
O.3 Parsing Numbers
The following elements are relevant to determining the value of a parsed number:
A possible prefix or suffix, indicating sign
A possible currency symbol or code
Decimal digits
A possible decimal separator
A possible exponent
A possible percent or per mille character
Other characters should either be ignored, or indicate the end of input, depending on the application. The key point is to disambiguate the sets of characters
that might serve in more than one position, based on context. For example, a period might be either the decimal separator, or part of a currency symbol (for
example,
"NA f."). Similarly, an "E" could be an exponent indicator, or a currency symbol (the Swaziland Lilangeni uses "E" in the "en" locale). An apostrophe might
be the decimal separator, or might be the grouping separator.
Here is a set of heuristic rules that may be helpful:
Any character with the decimal digit property is unambiguous and should be accepted.
Note:
In some environments, applications may independently
wish to restrict the decimal digit set to prevent security problems. See
[UTR36]
The exponent character can only be interpreted as such if it occurs after at least one digit, and if it is followed by at least one digit, with only
an optional sign in between. A regular expression may be helpful here.
For the sign, decimal separator, percent, and per mille, use a set of all possible characters that can serve those functions. For example, the decimal
separator set could include all of [.,']. (The actual set of characters can be derived from the number symbols in the By-Type charts
[ByType]
which list all of the values in CLDR.) To disambiguate, the decimal separator for the locale must be removed from the "ignore" set, and the grouping separator
for the locale must be removed from the decimal separator set. The same principle applies to all sets and symbols: any symbol must appear in at most one
set.
Since there are a wide variety of currency symbols and codes, this should be tried before the less ambiguous elements. It may be helpful to develop
a set of characters that can appear in a symbol or code, based on the currency symbols in the locale.
Otherwise, a character should be ignored unless it is in the "stop" set. This includes even characters that are meaningful for formatting,
for example, the
grouping separator.
If more than one sign, currency symbol, exponent, or percent/per mille occurs in the input, the first found should be used.
A currency symbol in the input should be interpreted as the longest match found in the set of possible currency symbols.
Especially in cases of ambiguity, the user's input should be echoed back, properly formatted according to the locale, before it is actually used for
anything.
O.4 Parsing Dates and Times
Lenient parsing of date and time strings is more complicated, due to the large number of possible fields and formats. The fields fall into two categories:
numeric fields (hour, day of month, year, numeric month, and so on) and symbolic fields (era, quarter, month, weekday, AM/PM, time zone). In addition, the user may
type in a date or time in a form that is significantly different from the normal format for the locale, and the application must use the locale information
to figure out what the user meant. Input may well consist of nothing but a string of numbers with separators,
for example, "09/05/02 09:57:33".
The input can be separated into tokens: numbers, symbols, and literal strings. Some care must be taken due to ambiguity,
for example, in the Japanese locale the
symbol for March is "3 月", which looks like a number followed by a literal. To avoid these problems, symbols should be checked first, and spaces should be ignored
(except to delimit the tokens of the input string).
The meaning of symbol fields should be easy to determine; the problem is determining the meaning of the numeric fields. Disambiguation will likely be most
successful if it is based on heuristics. Here are some rules that can help:
Always try the format string expected for the input text first. This is the only way to disambiguate 03/07 (March 2007, a credit card expiration date)
from 03/07 (March 7, a birthday).
Attempt to match fields and literals against those in the format string, using loose matching of the tokens.
When matching symbols, try the narrow, abbreviated, and full-width
forms, including standalone forms if they are unique. You may want to
allow prefix matches too, or diacritic-insensitive, again, as long as
they are unique. For example, for a month, accept 9, 09, S, Se, Sep,
Sept, Sept., and so on.
When a field or literal is encountered that is not compatible with the pattern:
Synchronization is not necessary for symbolic fields, since they are self-identifying. Wait until a numeric field or literal is encountered before
attempting to resynchronize.
Ignore whether the input token is symbolic or numeric, if it is compatible with the current field in the pattern.
Look forward or backward in the current format string for a literal that matches the one most recently encountered. See if you can resynchronize
from that point. Use the value of the numeric field to resynchronize as well, if possible (for
example, a number larger than the largest month cannot be a
month)
If that fails, use other format strings from the locale (including those in ) to try to match the previous or next symbol or literal
(again, using a loose match).
Appendix P.
Supplemental Metadata
The supplemental metadata contains information about the CLDR file itself, used to test
validity and provide information for locale inheritance. A number of these elements are
described in
Appendix I:
Inheritance and Validity
Appendix K:
Valid Attribute Values
Appendix L:
Canonical Form
Appendix M:
Coverage Levels
P.1 Supplemental Alias Information
variantAlias*, zoneAlias* ) >















This element provides information as to parts of locale IDs that should be substituted when accessing CLDR data. This logical substitution should be done
to both the locale id, and to any lookup for display names of languages, territories,
and so on. As with the display names, the language type and replacement may
be any prefix of a valid locale id, such as "no_NO".




...

...

P.2 Supplemental Deprecated Information

The deprecated items can be used to indicate elements, attributes, and attribute values
that are deprecated. This means
that the items are valid, but that their usage is strongly discouraged. When the same deprecatedItems element contains combinations of elements, attributes,
and values, then the "least significant" items are only deprecated if they occur with the "more significant" items. For example:
Deprecated Items

A and B are deprecated

C and D are deprecated on all elements

C and D are deprecated, but only if they occur on elements A or B.

E is deprecated, but only if it is a value of C in an element A or B
In each case, multiple items are space-delimited.
P.3
Default Content


In CLDR, locales without territory information (or where needed,
script information) provide data appropriate for what is called the
default content locale
For example, the
en
locale contains data appropriate for
en-US
, while the
zh
locale contains content for
zh-Hans-CN
, and the
zh-Hant
locale contains content
for
zh-Hant-TW
. The default content locales themselves thus inherit all of their
contents, and are empty.
The choice of content is typically based on the largest literate
population of the possible choices. Thus if an implementation only provides the base language
(such as
en
), it will still get a complete and consistent set of data appropriate for a
locale which is reasonably likely to be the one meant.
Where other information is
available, such as independent country information, that information can always be used to pick
a different locale (such as
en-CA
for a website targeted at Canadian users).
If an implementation is to use a different default locale, then the
data needs to be
pivoted
; all of the data from the CLDR for the current default locale
pushed out to the locales that inherit from it, then the new default content locale's data moved
into the base. There are tools in CLDR to perform this operation.
References
Ancillary Information
To properly localize, parse, and format data requires ancillary information, which is not expressed in Locale Data
Markup Language. Some of the formats for values used in Locale Data Markup Language are constructed according to external specifications. The sources
for this data and/or formats include the following:
Charts
The online code charts can be found at
An index
to characters names with links to the corresponding chart is found at
DUCET
The Default Unicode Collation Element Table (DUCET)
For the base-level collation, of which all the collation tables in this document are tailorings.
FAQ
Unicode Frequently Asked Questions
For answers to common questions on technical issues.
FCD
As defined in UTN #5 Canonical Equivalences in Applications
Bugs
CLDR Bug Reporting form
Glossary
Unicode Glossary
For explanations of terminology used in this and other documents.
JavaChoice
Java ChoiceFormat
Olson
The
TZ
ID Database (aka Olson timezone database)
Time zone and daylight savings information.
ftp://elsie.nci.nih.gov/pub/
For archived data, see
ftp://munnari.oz.au/pub/oldtz/
For general information, see
Reports
Unicode Technical Reports
For information on the status and development process for technical reports, and for a list of technical reports.
UCA
UTS #10: Unicode Collation Algorithm
UCD
Unicode
Character Database
For an overview of the Unicode Character Database and a list of
its associated files, see:
Unicode
The Unicode Standard
For the latest version, see:
For the 5.1.0 version, see:
Versions
Versions
of the Unicode Standard
For information on version numbering, and citing and referencing
the Unicode Standard, the Unicode Character Database, and Unicode
Technical Reports.
XPath
Other Standards
Various standards define codes that are used as keys or values in Locale Data Markup Language. These include:
ISO639
ISO Language Codes
Actual List
ISO1000
ISO 1000: SI units and recommendations for the use of their multiples and of certain other units, International Organization for Standardization, 1992.
ISO3166
ISO Region Codes
Actual List
ISO4217
ISO Currency Codes
(Note that as of this point, there are significant problems with this list. The supplemental data file contains the best compendium of currency
information available.)
ISO15924
ISO Script Codes
Actual List
BCP47
The Registry
UNM49
UN M.49: UN Statistics Division
Country or area & region codes
Composition of macro geographical (continental) regions, geographical sub-regions, and selected economic and other groupings
XML Schema
W3C XML Schema
General
The following are general references from the text:
BIDI
UAX #9: The Bidirectional Algorithm
ByType
CLDR Comparison Charts
Calendars
Calendrical Calculations: The Millennium Edition by Edward M. Reingold, Nachum Dershowitz; Cambridge University Press;
Book and CD-ROM edition (July 1, 2001); ISBN: 0521777526. Note that the algorithms given in this book are copyrighted.
CharMapML
UTR #22: Character Mapping Tables
Comparisons
Comparisons between locale data from different sources
CurrencyInfo
UNECE Currency Data
DataFormats
CLDR Data Formats
Example
A sample in Locale Data Markup Language
ICUCollation
ICU rule syntax
ICUTransforms
Transforms
Transforms Demo
ICUUnicodeSet
ICU UnicodeSet
API
ITUE164
International Telecommunication Union: List Of ITU Recommendation E.164 Assigned Country Codes
available at
LocaleExplorer
ICU Locale Explorer
LocaleProject
Common Locale Data Repository Project
NamingGuideline
OpenI18N Locale Naming Guideline
RBNF
Rule-Based Number Format
RBBI
Rule-Based Break Iterator
Scripts
UAX #24: Script Names
UCAChart
Collation Chart
UAX14
UAX #14: Line Breaking Properties
UAX24
UAX #24: Script Names
UAX29
UAX #29: Text Boundaries
UAX31
UAX #31: Identifier and Pattern Syntax
URegex
UTR #18: Unicode Regular Expression Guidelines
UTR36
UTR #36: Unicode Security Considerations
UTCInfo
NIST Time and Frequency Division Home Page
U.S. Naval Observatory: What is Universal Time?
WindowsCulture
Windows Culture Info (with  mappings from [
BCP47
]-style codes to LCIDs)
Acknowledgments
Special thanks to the following people for their continuing overall contributions to the CLDR
project, and for their specific contributions in the following areas.
These descriptions only touch on the many contributions that they have made.
Mark Davis for creating the initial version of LDML, and adding to and maintaining this specification, and for his work on the LDML code and tests.
John Emmons for the POSIX conversion tool and metazones.
Deborah Goldsmith for her contributions to LDML architecture and this specification.
Chris Hansten for coordinating and managing data submissions and vetting.
Erkki Kolehmainen and his team for their work on Finnish.
Steven R. Loomis for development of the survey tool and database management.
Peter Nugent for his contributions to the POSIX tool and from Open Office, and for coordinating and managing data submissions and vetting.
George Rhoten for his work on currencies.
Roozbeh Pournader (روزبه پورنادر) for his work on South Asian countries.
Ram Viswanadha (రఘురామ్ విశ్వనాధ) for all of his work on LDML code and data integration, and for coordinating and managing data submissions and vetting.
Vladimir Weinstein (Владимир Вајнштајн) for his work on collation.
Yoshito Umaoka (馬岡 由人) for his work on the timezone architecture.
Rick McGowan for his work gathering language, script and region data.
Xiaomei Ji (吉晓梅) for her work on time intervals and plural formatting.
David Bertoni for his contributions to the conversion tools.
Mike Tardif for reviewing this specification and for coordinating and vetting data submissions.
Peter Edberg for work on this specification, telephone code data and plurals.
Other contributors to CLDR are listed on the
CLDR Project Page
Modifications
The following summarizes modifications from the previous revision of this document. Some of the modification
notes have an associated bug ticket number, which may be used to look up additional information about the modification; for further information,
see
Revision 11
Made a number of changes as the result of a copy-edit pass by Julie.
Added clarifications in
Unicode Language and Locale Identifiers
Revision 10
In
Section 1.1
Conformance
added UAX35-C2 and information on referencing particular components of Unicode locale or language
identifiers. [ticket #1801]
In
Section 3
Unicode Language and Locale Identifiers
Clarified the syntax and usage of the language and locale identifiers. [ticket #1801]
Replaced the use of the comma (U+0020 SPACE), as an options separator, with the semicolon (U+003B SEMICOLON).
This change was also reflected in the examples given. [ticket #1717]
Re-emphasized that key and type values are limited to ASCII lettters and digits,
and that they have to be unique within the first 8 letters and digits. [ticket #1772]
In
Section 4
Locale Inheritance
explained the different fallback processes for resource bundle lookup and resource item lookup. [ticket #1763]
In
Section 5
XML Format
clarified that no element value can start with a combining slash U+0338 (not a combining backslash). [ticket #1223]
In
Section 5.2
Common Attributes
updated the draft attribute value descriptions; added "contributed".
In
Section 5.3.1
Fallback Elements
clarified examples, and explained
how
implementations can provide a mechanism for overriding the fallbacks. [ticket #1763]
In
Section 5.4
Display Name Elements
re-emphasized the potential uses of these elements [ticket #1665], and added new localeDisplayPattern element [ticket #1448].
In
Section 5.9.1
Calendar Elements
Clarified handling of availableFormat patterns for calendars that require an era field if a year field is present [ticket #1346].
Described the format of a date-time format skeleton used with the availableFormats element [ticket #1611].
Added descriptions of intervalFormats [ticket #1813].
In
Section 5.9.2
Time Zone Names
indicated that timezone IDs are not limited to city names, and corrected/augmented the fallbackFormat examples. [ticket #1604]
In
Section 5.10.1
Number Symbols
updated to match current DTD.
In
5.10.2
Currencies
updated to match current DTD. Added section explaining use of "count" to format currency values for particular numeric values. [ticket #1550]
Added new
Section 5.11
Unit Elements
(renumbered the 5.x sections after it) describing element: Support for unit forms used (for example) in formatting durations [tickets #900, #973, #1009,
#1807, #1821]
In
Appendix C.6
Measurement System Data
clarified the meaning of "metric" and its relation to ISO 1000 [ticket #481], and corrected the values for paperSize [ticket #1712].
Added
Appendix C.11
Language Plural Rules
[tickets #1550, #1703]
Added
Appendix C.12
Telephone Code Data
[ticket #1542]
In
Appendix F:
Date Format Patterns
clarified the usage of the YYYY for week of year calendars. [ticket #1605]
In Section G.8 of
Appendix G:
Number Format Patterns
changed currencySeparator, which is not a valid field, to currencyDecimal. [ticket #997]
In
Appendix J:
Time Zone Display Names
amended the fallback example and corrected a typo; corrected root fallbackFormat and all discussions and examples that ensue from that change. [ticket #1604]
In the Parsing section of
Appendix J:
Time Zone Display Names
updated step 3 to allow UTC and UT as synonyms for GMT. [ticket #1582]
Fixed validation errors and broken links [tickets #1606, #1619].
Added contributors [ticket #1835]. In this Modifications section, added bug ticket information [ticket #1630].
Revision 9
Extensive rewrite of
Appendix J:
Time Zone Display Names
primarily due to refinements to the metazone process. This also caused some changes in Appendix F:
Date Format Patterns
. [ticket #1508]
Made the date
range handling uniform, with new
Section 5.2.1
Dates and Date Ranges
and related changes particularly to C.1 and C.5 in Appendix C:
Supplemental Data
Added
Appendix C10.
Likely Subtags
Added missing date pattern symbol "l" for Chinese calendar. [ticket #1557]
Revision 8
Reserved 'j' in date formats for distinguishing 12 and
24 formats.
Added section 5.1.2
Text Directionality
Added new conformance section: 1.1
Conformance
Revised text on loose matching to include BIDI control
characters: Appendix O:
Lenient Parsing
Revised text on distinguishing and blocking elements
Added currency exemplar sets
Added dateRangePatterns
Added language
fallbacks: 5.3.1
Fallback Elements
Clarified use of transliterator names
Added matching options for collation
Added currency change policy
Added description of character fallbacks, changed ordering of
NFC and NFKC.
Added DTD headers for supplemental data
Added supplemental metadata descriptions: Appendix P:
Supplemental Metadata
Added mappings to alternate
language and country codes
Added substantial data on
language and script usage in different countries
Added default content data
Added metazones
Clarified the before
and after elements in currency formatting.
Minor edits
Revision 7
Point at bug database instead of Unicode reporting form.
Add "root" as valid locale identifier, and clarify that "locale" in CLDR is really essentially language.
Added the list of private use language & script subtag codes that
will not be used by CLDR.
Corrected the dateTimeFormat assignments for {0} and {1}.
Revision 6
Incorporated Corrigendum 1 (see
) into Appendix F:
Date Format Patterns
and Section 5.4

Revamped Appendix J:
Time Zone Display Names
. Also changed "Fallback" to "Display Names" in the title of the Appendix,
and "Olson" to "TZ" in other places in the document.
Yesstr/nostr/yesexpr/noexpr changes in Section 5.12

Added Section 5.15

Moved week, measurement data to Appendix C:
Supplemental Data
Added coverage levels in Appendix M:
Coverage Levels
Added rule-based number formats and transforms, in Section 5.16
Transforms
, Appendix N:
Transform
Rules
Added metadata, replacing the contents of Appendix K:
Valid Attribute Values
Added availableFormats, dateFormatItem, and appendItem in

to support more flexible date/time formatting
Added measurementSystemNames and measurementSystemName in

for localized names of measurement
systems
Added quarters, quarterContext, quarterWidth, and quarter to

for names of calendar quarters
Extended possible values for
alt tag
Added ethiopic calendar to
allowed calendar values
Clarified usage of
quotation marks and alternate marks
Corrected example
ISBN
Added eraNarrow to

for one-character version of era names
Added
Appendix O
, on lenient parsing
Added
3.1
Unknown or Invalid Identifiers
Other editing
Updated descriptions to final DTD and metadata.
Revision 5
The canonical form for variants is upper case
Addition of UN M.49 codes
Addition of persian and coptic calendar IDs
Clarification of alias inheritance
New XML references section
Modified revision and generation field format
Use of language display names for whole initial segments of locale IDs names, such as nl-BE
Addition of the inList element
Clarification of 'narrow'
Additional dateTimeFormat description
Names of calendar fields, and relative times.
New element currencySpacing
Descriptions of POSIX yes/now
New supplemental data elements/attributes (end of Appendix C)
currency to/from
languageData
timezoneData
territoryContainment
mapTimezones
alias
deprecated
characters
in dateExtension of era to 1..3
Clarification of year padding
Deprecation of localizedPatternChars
Use of the singleCountries list
Appendix L: canonical form
Misc editing
Revision 1
(2005-06-30): added link to Corrigenda.
Copyright © 2001-2008 Unicode, Inc. All Rights Reserved. The Unicode Consortium makes no expressed or implied warranty of any kind, and
assumes no liability for errors or omissions. No liability is assumed for incidental and consequential damages in connection with or arising out of the use
of the information or programs contained or accompanying this technical report. The Unicode
apply.
Unicode and the Unicode logo are trademarks of Unicode, Inc., and are registered in some jurisdictions.