English Disambiguation Grammar - Revision history

imported>Martins at 21:51, 29 October 2012

2012-10-29T21:51:47Z

← Older revision		Revision as of 21:51, 29 October 2012
Line 1:		Line 1:
	The English disambiguation grammars, or English d-grammars, are a part of the [[English grammar]] and are used to improve the results of the [[tokenization]] and to control the application of [[t-rule]]s. They follow the formalism described at [[UNL_Grammar_Specs#Disambiguation_Rules\|UNL Grammar Specs]] and are used both in natural language analysis ([[UNLization]]) and in natural language generation ([[NLization]]).		The English disambiguation grammars, or English d-grammars, are a part of the [[English grammar]] and are used to improve the results of the [[tokenization]] and to control the application of [[t-rule]]s. They follow the formalism described at [[UNL_Grammar_Specs#Disambiguation_Rules\|UNL Grammar Specs]] and are used both in natural language analysis ([[UNLization]]) and in natural language generation ([[NLization]]).

	== ~~UNLization~~ ==		== ENG->UNL Disambiguation Grammar ==
	In natural language analysis, the d-grammar is used to control the [[tokenization]] of the English sentences, i.e., to prevent wrong lexical choices and to induce the best matches. The d-grammar comprises two different types of disambiguation rules, or d-rules:		In natural language analysis, the d-grammar is used to control the [[tokenization]] of the English sentences, i.e., to prevent wrong lexical choices and to induce the best matches. The d-grammar comprises two different types of disambiguation rules, or d-rules:
	*'''Negative''' (blocking) rules, where the probability is equal to 0, prevent lexical choices		*'''Negative''' (blocking) rules, where the probability is equal to 0, prevent lexical choices

imported>Martins at 21:51, 29 October 2012

2012-10-29T21:51:05Z

← Older revision		Revision as of 21:51, 29 October 2012
Line 7:		Line 7:
	*'''Positive''' rules, where the probability is more than 0, force lexical choices		*'''Positive''' rules, where the probability is more than 0, force lexical choices
	*:For instance, the rule (['s],V)(BLK)(GER)=1; informs that the entry ['s] as a verb is to be preferred before a gerund (there are three entries ['s] in the dictionary: the contracted form of "is", the particle used to form the genitive and a plural suffix; if this rule is not stated, the system would simply select the first one appearing in the dictionary with the highest frequency)		*:For instance, the rule (['s],V)(BLK)(GER)=1; informs that the entry ['s] as a verb is to be preferred before a gerund (there are three entries ['s] in the dictionary: the contracted form of "is", the particle used to form the genitive and a plural suffix; if this rule is not stated, the system would simply select the first one appearing in the dictionary with the highest frequency)
	~~=== File ===~~
	The English d-grammar may be downloaded from [http://www.unlweb.net.br/resources/A1/eng_ana_dgrammar.txt eng_ana_dgrammar.txt]. The complete English d-grammar may be exported from the [[UNLarium]]: UNLWEB>UNLARIUM>GRAMMAR>ENGLISH>EXPORT.

	=== How to use d-grammars ===		=== How to use d-grammars ===
Line 14:		Line 12:

	=== Examples of disambiguation rules ===		=== Examples of disambiguation rules ===
			;TOKENIZATION OF TEMPORARY WORDS (used to control hyper-segmentation)
	;~~Preventing~~ hyper-segmentation ~~of temporary entries:~~		(TEMP,^DIGIT,^W)(^BLK,^PUT,^STAIL)=0;
	~~:"asdfg" must be tokenized as [asdfg] (one single temporary entry~~) ~~instead of [as][dfg], which would be the case, because [as] is in the dictionary~~		:there must be a blank, a punctuation sign or the end of the sentence after a temporary word, i.e., a temporary word cannot be followed by other word, except for digits, as in "1st"
	:(TEMP~~,^" "~~,^DIGIT,^W)(^~~" "~~,^STAIL)=0;		(^BLK,^PUT,^SHEAD)(TEMP,^W)=0;
	::(a temporary word, i.e., a word ~~not found in the dictionary, must~~ be followed by ~~a blank space or the end of the sentence)~~		:there must be a blank, a punctuation sign or the beginning of the sentence before a temporary word, i.e., a temporary word cannot be preceded by other word
	:(^~~" "~~,^PUT,^SHEAD)(TEMP~~,^" "~~,^W)=0;		(TEMP)(PUT)(TEMP)=0; there cannot be two temporary words separated by punctuation mark
	::(a temporary word, i.e., a word ~~not found in the dictionary, must~~ be preceded by ~~a blank space, a punctuation sign or the beginning of the sentence~~)		;DETERMINERS X PRONOUNS (used to disambiguate pronouns from determiners, which come first in the dictionary)
	:(~~^DIGIT~~)(~~{[st]\|[nd]\|[rd]\|[th]}~~)=0;		(D,^AFT)({PUT,^BLK\|STAIL})=0;
	~~::The subwords [st], [nd], [rd] and [th] may only appear after a number ("1st" or "tenth"). This prevents hyper-segmentation as in "asdfgst" = [asdfg][st], which would not~~ be ~~blocked~~ by ~~the rule above, because [st] is a suffix.~~		:determiners may not come at the end of the sentence or before a punctuation mark, except if their distribution is AFT, like "enough"
	;~~Determiners x~~ pronouns		(D,^AFT)(BLK)({V\|P\|AAV})=0;
	~~:There are many ambiguities in the dictionary between~~ determiners ~~and pronouns. The string "that"~~, ~~for instance, is represented~~ in the dictionary ~~as a demonstrative determiner ("that book"~~) ~~and as a demonstrative pronoun ("that is the book"). The following rules help differentiating them:~~		:determiners may not be precede verbs, prepositions or adjunct adverbs, except if their distribution is AFT
	:(D,^AFT)({PUT\|STAIL})=0;		;AUXILIARY VERBS X MAIN VERBS (have, be)
	:~~:Determiners cannot~~ come at the end of the sentence, except if their distribution is AFT ~~(after) ("He said that"~~, ~~but~~ "~~There are books~~ enough")		(AUX)(BLK)(^V,^[not])=0;
	:(D)(BLK)(V)=0;		:an auxiliary verb must be followed by a verb or the words "not" or "to"
	:~~:Determiners cannot come before a verb ("That~~ is ~~the book")~~		(AUX)(BLK)([not])(BLK)(^V)=0;
	;~~Auxiliary verbs x main verbs~~		:if followed by "not", the auxiliary must be followed by a verb
	~~:Many auxiliary verbs may also play the role of main verbs: "He has done that"~~ (~~"has" is an auxiliary~~) ~~x "He has a car" ("has" is the main verb). The following rule helps differentiating them:~~
	:(AUX~~,^COP~~)(BLK)(^V,^[not])=0;
	:~~:Auxiliary verbs which are not copula~~ must be followed by a verb or the ~~adverb~~ [not]

imported>Martins at 11:53, 5 September 2012

2012-09-05T11:53:44Z

← Older revision		Revision as of 11:53, 5 September 2012
Line 8:		Line 8:
	*:For instance, the rule (['s],V)(BLK)(GER)=1; informs that the entry ['s] as a verb is to be preferred before a gerund (there are three entries ['s] in the dictionary: the contracted form of "is", the particle used to form the genitive and a plural suffix; if this rule is not stated, the system would simply select the first one appearing in the dictionary with the highest frequency)		*:For instance, the rule (['s],V)(BLK)(GER)=1; informs that the entry ['s] as a verb is to be preferred before a gerund (there are three entries ['s] in the dictionary: the contracted form of "is", the particle used to form the genitive and a plural suffix; if this rule is not stated, the system would simply select the first one appearing in the dictionary with the highest frequency)
	=== File ===		=== File ===
	The English d-grammar ~~for the [[Corpus500]]~~ may be downloaded from [http://www.unlweb.net.br/resources/~~corpus500~~/eng_ana_dgrammar.txt eng_ana_dgrammar.txt]. The complete English d-grammar may be exported from the [[UNLarium]]: UNLWEB>UNLARIUM>GRAMMAR>ENGLISH>EXPORT.		The English d-grammar may be downloaded from [http://www.unlweb.net.br/resources/A1/eng_ana_dgrammar.txt eng_ana_dgrammar.txt]. The complete English d-grammar may be exported from the [[UNLarium]]: UNLWEB>UNLARIUM>GRAMMAR>ENGLISH>EXPORT.

	=== How to use d-grammars ===		=== How to use d-grammars ===
	D-grammars must be uploaded to or provided directly at the tab '''d-rules''' in [[IAN]].		D-grammars must be uploaded to or provided directly at the tab '''d-rules''' in [[IAN]].

imported>Martins at 18:04, 1 August 2012

2012-08-01T18:04:16Z

← Older revision		Revision as of 18:04, 1 August 2012
Line 16:		Line 16:
	;Preventing hyper-segmentation of temporary entries:		;Preventing hyper-segmentation of temporary entries:
	:"asdfg" must be tokenized as [asdfg] (one single temporary entry) instead of [as][dfg], which would be the case, because [as] is in the dictionary		:"asdfg" must be tokenized as [asdfg] (one single temporary entry) instead of [as][dfg], which would be the case, because [as] is in the dictionary
	~~:(^W,^" ",^PUT,^SBW,^DIGIT,^PFX,^SHEAD)(^W,^" ",^PUT,^SBW,^DIGIT,^SFX,^STAIL)=0;~~
	~~::This rule states that words must be isolated by blank spaces, punctuation sign or SHEAD and STAIL, except in the case of subwords, prefixes and suffixes.~~
	~~:(^DIGIT)({[st]\|[nd]\|[rd]\|[th]})=0;~~
	::The subwords [st], [nd], [rd] and [th] may only appear after a number ("1st" or "tenth"). This prevents hyper-segmentation as in "asdfgst" = [asdfg][st], which would not be blocked by the rule above, because [st] is a suffix.
	~~;Preventing the generation of two temporary words in sequence:~~
	~~:"asdfg hijkl" must be represented as a single temporary word [asdfg hijkl] instead of [asdfg][ ][hijkl], which would be the case, because [ ] is in the dictionary~~
	:(TEMP,^" ",^DIGIT,^W)(^" ",^STAIL)=0;		:(TEMP,^" ",^DIGIT,^W)(^" ",^STAIL)=0;
	::(a temporary word, i.e., a word not found in the dictionary, must be followed by a blank space or the end of the sentence)		::(a temporary word, i.e., a word not found in the dictionary, must be followed by a blank space or the end of the sentence)
	:(^" ",^PUT,^SHEAD)(TEMP,^" ",^W)=0;		:(^" ",^PUT,^SHEAD)(TEMP,^" ",^W)=0;
	::(a temporary word, i.e., a word not found in the dictionary, must be preceded by a blank space, a punctuation sign or the beginning of the sentence)		::(a temporary word, i.e., a word not found in the dictionary, must be preceded by a blank space, a punctuation sign or the beginning of the sentence)
			:(^DIGIT)({[st]\|[nd]\|[rd]\|[th]})=0;
			::The subwords [st], [nd], [rd] and [th] may only appear after a number ("1st" or "tenth"). This prevents hyper-segmentation as in "asdfgst" = [asdfg][st], which would not be blocked by the rule above, because [st] is a suffix.
	;Determiners x pronouns		;Determiners x pronouns
	:There are many ambiguities in the dictionary between determiners and pronouns. The string "that", for instance, is represented in the dictionary as a demonstrative determiner ("that book") and as a demonstrative pronoun ("that is the book"). The following rules help differentiating them:		:There are many ambiguities in the dictionary between determiners and pronouns. The string "that", for instance, is represented in the dictionary as a demonstrative determiner ("that book") and as a demonstrative pronoun ("that is the book"). The following rules help differentiating them:

imported>Martins at 14:09, 28 July 2012

2012-07-28T14:09:52Z

← Older revision		Revision as of 14:09, 28 July 2012
Line 2:		Line 2:

	== UNLization ==		== UNLization ==
	In natural language analysis, the d-grammar is used to control the [[tokenization]] of the English sentences, i.e., to prevent wrong lexical choices and to induce the best matches. The d-grammar comprises two different types of rules:		In natural language analysis, the d-grammar is used to control the [[tokenization]] of the English sentences, i.e., to prevent wrong lexical choices and to induce the best matches. The d-grammar comprises two different types of disambiguation rules, or d-rules:
	*'''Negative''' (blocking) rules, where the probability is equal to 0, prevent lexical choices		*'''Negative''' (blocking) rules, where the probability is equal to 0, prevent lexical choices
	*:For instance, the rule (D)(BLK)(V)=0; informs that the sequence determiner+blank space+verb is not allowed, i.e., there cannot be a determiner before a verb.		*:For instance, the rule (D)(BLK)(V)=0; informs that the sequence determiner+blank space+verb is not allowed, i.e., there cannot be a determiner before a verb.

imported>Martins at 14:09, 28 July 2012

2012-07-28T14:09:06Z

← Older revision		Revision as of 14:09, 28 July 2012
Line 10:		Line 10:
	The English d-grammar for the [[Corpus500]] may be downloaded from [http://www.unlweb.net.br/resources/corpus500/eng_ana_dgrammar.txt eng_ana_dgrammar.txt]. The complete English d-grammar may be exported from the [[UNLarium]]: UNLWEB>UNLARIUM>GRAMMAR>ENGLISH>EXPORT.		The English d-grammar for the [[Corpus500]] may be downloaded from [http://www.unlweb.net.br/resources/corpus500/eng_ana_dgrammar.txt eng_ana_dgrammar.txt]. The complete English d-grammar may be exported from the [[UNLarium]]: UNLWEB>UNLARIUM>GRAMMAR>ENGLISH>EXPORT.
	=== How to use d-grammars ===		=== How to use d-grammars ===
	~~The d~~-~~grammar~~ must be uploaded to or provided directly at the tab d-rules in [[IAN]].		D-grammars must be uploaded to or provided directly at the tab '''d-rules''' in [[IAN]].

	=== Examples of disambiguation rules ===		=== Examples of disambiguation rules ===

imported>Martins at 14:08, 28 July 2012

2012-07-28T14:08:05Z

← Older revision		Revision as of 14:08, 28 July 2012
Line 1:		Line 1:
	The English ~~Disambiguation Grammar~~, ~~which is~~ part of the [[English grammar]], is used to control the [[tokenization]] of the English sentences, i.e., to prevent wrong lexical choices and to induce the best matches. ~~It follows the formalism described at [[UNL_Grammar_Specs#Disambiguation_Rules\|UNL Grammar Specs]] and~~ comprises two different types of rules:		The English disambiguation grammars, or English d-grammars, are a part of the [[English grammar]] and are used to improve the results of the [[tokenization]] and to control the application of [[t-rule]]s. They follow the formalism described at [[UNL_Grammar_Specs#Disambiguation_Rules\|UNL Grammar Specs]] and are used both in natural language analysis ([[UNLization]]) and in natural language generation ([[NLization]]).

			== UNLization ==
			In natural language analysis, the d-grammar is used to control the [[tokenization]] of the English sentences, i.e., to prevent wrong lexical choices and to induce the best matches. The d-grammar comprises two different types of rules:
	*'''Negative''' (blocking) rules, where the probability is equal to 0, prevent lexical choices		*'''Negative''' (blocking) rules, where the probability is equal to 0, prevent lexical choices
	*:For instance, the rule (D)(BLK)(V)=0; informs that the sequence determiner+blank space+verb is not allowed, i.e., there cannot be a determiner before a verb.		*:For instance, the rule (D)(BLK)(V)=0; informs that the sequence determiner+blank space+verb is not allowed, i.e., there cannot be a determiner before a verb.
	*'''Positive''' rules, where the probability is more than 0, force lexical choices		*'''Positive''' rules, where the probability is more than 0, force lexical choices
	*:For instance, the rule (['s],V)(BLK)(GER)=1; informs that the entry ['s] as a verb is to be preferred before a gerund (there are three entries ['s] in the dictionary: the contracted form of "is", the particle used to form the genitive and a plural suffix; if this rule is not stated, the system would simply select the first one appearing in the dictionary with the highest frequency)		*:For instance, the rule (['s],V)(BLK)(GER)=1; informs that the entry ['s] as a verb is to be preferred before a gerund (there are three entries ['s] in the dictionary: the contracted form of "is", the particle used to form the genitive and a plural suffix; if this rule is not stated, the system would simply select the first one appearing in the dictionary with the highest frequency)
	== Examples of disambiguation rules ==		=== File ===
			The English d-grammar for the [[Corpus500]] may be downloaded from [http://www.unlweb.net.br/resources/corpus500/eng_ana_dgrammar.txt eng_ana_dgrammar.txt]. The complete English d-grammar may be exported from the [[UNLarium]]: UNLWEB>UNLARIUM>GRAMMAR>ENGLISH>EXPORT.
			=== How to use d-grammars ===
			The d-grammar must be uploaded to or provided directly at the tab d-rules in [[IAN]].

			=== Examples of disambiguation rules ===

	;Preventing hyper-segmentation of temporary entries:		;Preventing hyper-segmentation of temporary entries:

imported>Martins at 13:54, 28 July 2012

2012-07-28T13:54:33Z

← Older revision		Revision as of 13:54, 28 July 2012
Line 1:		Line 1:
	The English Disambiguation Grammar is used to control the [[tokenization]] of the English sentences, i.e., to prevent wrong lexical choices and to induce the best matches. It follows the formalism described at [[UNL_Grammar_Specs#Disambiguation_Rules\|UNL Grammar Specs]] and comprises two different types of rules:		The English Disambiguation Grammar, which is part of the [[English grammar]], is used to control the [[tokenization]] of the English sentences, i.e., to prevent wrong lexical choices and to induce the best matches. It follows the formalism described at [[UNL_Grammar_Specs#Disambiguation_Rules\|UNL Grammar Specs]] and comprises two different types of rules:
	*'''Negative''' (blocking) rules, where the probability is equal to 0, prevent lexical choices		*'''Negative''' (blocking) rules, where the probability is equal to 0, prevent lexical choices
	*:For instance, the rule (D)(BLK)(V)=0; informs that the sequence determiner+blank space+verb is not allowed, i.e., there cannot be a determiner before a verb.		*:For instance, the rule (D)(BLK)(V)=0; informs that the sequence determiner+blank space+verb is not allowed, i.e., there cannot be a determiner before a verb.

imported>Martins at 13:48, 28 July 2012

2012-07-28T13:48:42Z

← Older revision		Revision as of 13:48, 28 July 2012
Line 18:		Line 18:
	:(^" ",^PUT,^SHEAD)(TEMP,^" ",^W)=0;		:(^" ",^PUT,^SHEAD)(TEMP,^" ",^W)=0;
	::(a temporary word, i.e., a word not found in the dictionary, must be preceded by a blank space, a punctuation sign or the beginning of the sentence)		::(a temporary word, i.e., a word not found in the dictionary, must be preceded by a blank space, a punctuation sign or the beginning of the sentence)
	;Determiners x ~~Pronouns~~		;Determiners x pronouns
	:There are many ambiguities in the dictionary between determiners and pronouns. The string "that", for instance, is represented in the dictionary as a demonstrative determiner ("that book") and as a demonstrative pronoun ("that is the book"). The following rules help differentiating them:		:There are many ambiguities in the dictionary between determiners and pronouns. The string "that", for instance, is represented in the dictionary as a demonstrative determiner ("that book") and as a demonstrative pronoun ("that is the book"). The following rules help differentiating them:
	:(D,^AFT)({PUT\|STAIL})=0;		:(D,^AFT)({PUT\|STAIL})=0;

imported>Martins at 13:48, 28 July 2012

2012-07-28T13:48:07Z

← Older revision		Revision as of 13:48, 28 July 2012
Line 10:		Line 10:
	:(^W,^" ",^PUT,^SBW,^DIGIT,^PFX,^SHEAD)(^W,^" ",^PUT,^SBW,^DIGIT,^SFX,^STAIL)=0;		:(^W,^" ",^PUT,^SBW,^DIGIT,^PFX,^SHEAD)(^W,^" ",^PUT,^SBW,^DIGIT,^SFX,^STAIL)=0;
	::This rule states that words must be isolated by blank spaces, punctuation sign or SHEAD and STAIL, except in the case of subwords, prefixes and suffixes.		::This rule states that words must be isolated by blank spaces, punctuation sign or SHEAD and STAIL, except in the case of subwords, prefixes and suffixes.
			:(^DIGIT)({[st]\|[nd]\|[rd]\|[th]})=0;
			::The subwords [st], [nd], [rd] and [th] may only appear after a number ("1st" or "tenth"). This prevents hyper-segmentation as in "asdfgst" = [asdfg][st], which would not be blocked by the rule above, because [st] is a suffix.
	;Preventing the generation of two temporary words in sequence:		;Preventing the generation of two temporary words in sequence:
	:"asdfg hijkl" must be represented as a single temporary word [asdfg hijkl] instead of [asdfg][ ][hijkl], which would be the case, because [ ] is in the dictionary		:"asdfg hijkl" must be represented as a single temporary word [asdfg hijkl] instead of [asdfg][ ][hijkl], which would be the case, because [ ] is in the dictionary