next up previous contents
Next: Comparison with CLARE Up: Evaluation and Results Previous: Comparison with IBM

Comparison with Alvey

The evaluation in this section was done with the earlier 1995 release of the grammar. This section compares XTAG to the Alvey Natural Language Tools (ANLT) Grammar. We parsed the set of LDOCE Noun Phrases presented in Appendix B of the technical report ([#!Carroll93!#]) using XTAG. Table [*] summarizes the results of this experiment. A total of 143 noun phrases were parsed. The NPs which did not have a correct parse in the top three derivations were considered failures for either system. The maximum and average number of derivations columns show the highest and the average number of derivations produced for the NPs that have a correct derivation in the top three. We show the performance of XTAG both with and without the tagger since the performance of the POS tagger is significantly degraded on the NPs because the NPs are usually shorter than the sentences on which it was trained. It would be interesting to see if the two systems performed similarly on a wider range of data.
 
Table: Comparison of XTAG and ANLT Parser
System # of # parsed % parsed Maximum Average
  NPs     derivations derivations
ANLT Parser 143 127 88.81% 32 4.57
XTAG Parser with 143 93 65.03% 28 3.45
POS tagger          
XTAG Parser without 143 120 83.91% 28 4.14
POS tagger          

 


next up previous contents
Next: Comparison with CLARE Up: Evaluation and Results Previous: Comparison with IBM
XTAG Project
1998-09-14