Persian-English Tagged Test Corpus

Bilingual Corpus

The parallel test corpus consists of 3,000 Persian sentences with corresponding English translations. The sentences are representative of contemporary journalistic prose; they were all collected from the on-line Iranian newspaper Hamshahri. All sentences were manually translated at CRL.

Tagging

The Persian sentences have been manually tagged for Part-of-Speech and bracketed for Noun Phrases and Preposition Phrases. The English sentences have been tagged automatically for POS and phrases using Upenn's SuperTagger.
The sentences below represent an example of a tagged Persian sentence and its corresponding English translation:

Study_NN^A_NXN of_IN^B_nxPnx these_DT^B_Dnx stores_NNS^A_NXN shows_VBZ^A_nx0Vnx1 that_IN^B_Dnx England_NNP^A_NXN is_VBZ^B_Vvx increasingly_RB^B_ARBvx being_VBG^B_Vvx transformed_VBN^A_nx0Vnx1 into_IN^B_vxPnx a_DT^B_Dnx society_NN^A_NXN which_WDT^B_COMPs never_RB^B_ARBvx sleeps_VBZ^A_nx0Vnx1 ._.^B_sPU

Evaluations

The bilingual tagged corpus was used in the Shiraz project for testing the results of the morphological analyzer, the dictionary entries and the syntactic parser.
For testing the output of morphological analysis and dictionary lookup, the Glass-Box Evaluation system compared the results with the hand-produced annotations. The results were grouped as:

Non-Matching Structures if the part-of-speech marked in the corpus does not match the POS obtained from morphological analysis and dictionary look-up.
Non-Matching Spans if the range of the annotation in the corpus is different from the range produced by the system. This is the case, for instance, if a compound is not recognized and only the individual parts are analyzed by the sytem.
Matches if both the span and the structure of the corpus annotation match the span and structure produced by the system.

The results of the Glass-Box Evaluation component were used to correct and edit any mistakes in the dictionary, in the stemmer or in the morphological component.

Click to see a sample of the Glass-Box evaluations.

For problems with this Web site, send mailto webmaster@crl.nmsu.edu

Shiraz Home

Top of Page