Dictionaries

Specifications

Version 2010

UNDL Foundation

December 2010

UNDL Foundation Dictionary Specifications

Dictionaries are lists of lexical items with their corresponding features.

Types

In the UNL framework, there are four different types of dictionaries:

UNL Dictionary (UD): the inventory of Universal Words. It is a flat list of UWs in alphabetical order with their corresponding semantic features.
NL Dictionary (ND): the inventory of lexical items of a given natural language (NL). It is a flat list of natural language entries with their corresponding grammatical features.
UNL-NL Dictionary (Generation Dictionary, GD): a bilingual dictionary linking entries of the UNL Dictionary to entries of the NL Dictionary.
NL-UNL Dictionary (Analysis Dictionary, AD): a bilingual dictionary linking entries of the NL Dictionary to entries of the UNL Dictionary.

General syntax

Dictionaries are plain text files with a single entry per line in the following format:

UNL Dictionary

[UCN] {ID} "UCL" (ATTR, ...) <unl, FRE, PRI>; COMMENTS

UNL-NL Dictionaries

[NLW] {ID} "UW" (ATTR, ...) <FLG, FRE, PRI>; COMMENTS

NL-UNL Dictionaries

[NLW] {ID} "UCN" (ATTR, ...) <FLG, FRE, PRI>; COMMENTS

Where:

UCN: Uniform Concept Name. It is a human-readable label for each Universal Word [cat(icl>animal)].
UCL: Uniform Concept Locator. It is a unique identifier for each Universal Word [31343243].
NLW: The lexical item of the natural language. It can be:

a multiword expression: [United States of America]
a compound: [hot-dog]
a simple word: [happiness]
a simple morpheme: [happ]
a non-motivated linguistic entity: [g]
a complex structure: [[bring] [back]]
a regular expression: [/colou{0,1}r/]

UW: The Universal Word of UNL, represented either by the corresponding UCN or UCL. This field can be empty if a word does not need a UW.
ID: The unique identifier (primary-key) of the entry (automatically generated).
ATTR: The list of features of the NLW, extracted from the UNDL Foundation tagset, separated by ",". Examples:
- simple features: NOU, MCL, SNG
- attribute-value pairs: POS=NOU, GEN=MCL, NUM=SNG
- inflection rules: FLX(PLR:="oo":"ee")
FLG: The three-character language code according to ISO 639-3.
FRE: The frequency of NLW in natural texts (0–255), used for natural language analysis.
PRI: The priority of the NLW (0–255), used for natural language generation.
COMMENTS: Any comment clarifying the mapping between NL and UNL entries.

Formal syntax

<UNL Dictionary entry>    ::= "["<UW>"]" "{"<ID>"}" "("<FEATURE LIST>")" "<unl,PRI,FRE>";
<NL Dictionary entry>     ::= "["<NLW>"]" "{"<ID>"}" "("<FEATURE LIST>")" "<FLG,PRI,FRE>";
<UNL-NL Dictionary entry> ::= "["<NLW>"]" "{"<ID>"}" """<UW>""" "("<FEATURE LIST>")" "<FLG,PRI,FRE>";

Where:

<NLW> ::= <SIMPLE NLW> | <COMPOUND NLW> | <REGULAR EXPRESSION>
<SIMPLE NLW> ::= <text>
<COMPOUND NLW> ::= ("["<text>"]")+
<ID> ::= <positive integer>
<UW> ::= <UCL> | <UCN>
<FEATURE LIST> ::= <FEATURE> ("," <FEATURE>)+

Complex structures as NLW*

In order to deal with multiple word expressions, the NLW can be represented as a complex structure comprising several sub-NLW entries:

[[sub-NLW][sub-NLW]...[sub-NLW]] {ID} "UW" (ATTR, ..., #01(ATTR, ...), #02(ATTR, ...), ...) <FLG,FRE,PRI>; COMMENTS

Example:

[[bring] [back]] {12343} "234234234" (pos=VER, #01(IFX(ET0:=4>"ought")), #02(pos=PRE)) <eng,0,0>;

In the entry above, the NLW has been split into two different sub-NLWs ([bring] and [back]). Each sub-NLW (#01 and #02) has different features.

Inflection rules inside dictionary entries

Dictionaries may contain rules for exceptions and irregular forms:

<RULE> ::= <ATTRIBUTE>("("<VALUE>:="<a-rule> (";"<VALUE>:="<a-rule>)* ")")
<ATTRIBUTE> ::= <HYPER-ATTRIBUTE> | <SIMPLE ATTRIBUTE>
<VALUE> ::= <VALUE LIST> | <SIMPLE VALUE>

Examples of inflection rules

NUM(PLR:="men")

If the node has the attribute "plural" (PLR) then replace the word by "men" in case of !NUM

POS(ORD:="1">"1st","2">"2nd","3">"3rd")

If the node has the attribute "ordinal" (ORD):

if last character is "1", replace with "1st";
if last character is "2", replace with "2nd";
if last character is "3", replace with "3rd".

FLX(3PS&PRS&IND:=0>"s")

If the word has the attribute 3PS,PRS,IND, add "s" to its end in case of !FLX

Regular expressions inside dictionary entries

Both NLW and UW may be replaced by regular expressions:

NLW: [/<RegEx>/] "<UW>" (<FEATURE LIST>) <FLG,FRE,PRI>;
UW:  [<NLW>] "/<RegEx>/" (<FEATURE LIST>) <FLG,FRE,PRI>;

Frequency and priority

Frequency is used for NL-to-UNL; priority is used for UNL-to-NL:

[nlw1] "uw" (A) <eng,0,1>
[nlw2] "uw" (A) <eng,0,2>
[nlw3] "uw" (A) <eng,0,3>

Examples of dictionary entries

[China]{24} "China(iof>Asian country)" (NOU, WRD, SNG, P0, F0) <eng,0,0>
[choose]{106} "to choose(icl>to decide)" (POS=VER, LEX=WRD, PAR=M1, FRA=Y76, FLX(3PS&PRS&IND:=0>"s"; PAS:="chose"; PTP:="chosen"; GER:="choosing";)) <eng,0,0>
[clear-eyed]{25} "clear-eyed(icl>discerning)" (POS=ADJ, LEX=WRD, PAR=M0, FRA=Y0) <en,0,0>
[Peter]{177} "Peter(iof>person)" (NOU) <eng,10,30>
[kill]{5987} "kill(icl>do)" (FLX(PAS:=0>"ed";)) <eng,70,80>
[[bring] [back]] {2345} "bring back" (POS=VER,VA(01>02),#01(POS=VER,FLX(PAS:=3>"ought";)),#02(POS=PRE)) <eng,50,34>
[/br(ing|ought)/] "bring(icl>do)" (POS=VER) <eng,0,0>
[[/br(ing|ought)/] [back]]{2345} "bring back(icl>do)" (POS=VER,#01(POS=VER),#02(POS=PRE)) <eng,50,34>
[/colo(u)?r/] "color" (POS=NOU) <eng,0,0> (NLW = {color, colour})
[/cit(y|ies)/] "city" (POS=NOU) <eng,0,0> (NLW = {city, cities})
[/(\d){4}/] "" (ENT=YEAR) <eng,0,0> (NLW = any sequence of four digits)
[city] "/city(.)*/" (POS=NOU) <eng,0,0> (UW = any UW that starts by the string "city")
[city] "/(.)+\(iof\>city\)/" (POS=NOU) <eng,0,0> (UW = any UW that ends by the string "(iof>city)")