next up previous contents
Next: Bibliography Up: Evaluation and Results Previous: Comparison with Alvey

Comparison with CLARE

The evaluation in this section was done with the earlier 1995 release of the grammar. This section compares the performance of XTAG against that of the CLARE-2 system ([#!clare-report92!#]) on the ATIS corpus. Table [*] shows the performance results. The percentage parsed column for both systems represents the percentage of sentences that produced any parse. It must be noted that the performance result shown for CLARE-2 is without any tuning of the grammar for the ATIS domain. The performance of CLARE-3, a later version of the CLARE system, is estimated to be 10% higher than that of the CLARE-2 system.29.6
 
Table: Performance of CLARE-2 and XTAG on the ATIS corpus
System Mean length % parsed
CLARE-2 6.53 68.50%
XTAG 7.62 88.35%
 

In an attempt to compare the performance of the two systems on a wider range of sentences (from similar genres), we provide in Table [*] the performance of CLARE-2 on LOB corpus and the performance of XTAG on the WSJ corpus. The performance was measured on sentences of up to 10 words for both systems.
 
Table: Performance of CLARE-2 and XTAG on LOB and WSJ corpus respectively
System Corpus Mean length % parsed
CLARE-2 LOB 5.95 53.40%
XTAG WSJ 6.00 55.58%
 


next up previous contents
Next: Bibliography Up: Evaluation and Results Previous: Comparison with Alvey
XTAG Project
1998-09-14