... Klein and Manning, 2003;
Charniak and Johnson, 2005; Petrov and Klein,
2007), but also include dependency parsers (Mc-
Donald and Pereira, 2006; Nivre and Nilsson, 2005;
Sagae and Tsujii, 2007) and ... Proceedings of ACL-08: HLT, pages 46–54,
Columbus, Ohio, USA, June 2008.
c
2008 Association for Computational Linguistics
Task-oriented Evaluation of Syntactic Parsers...
... scripts
(Schank and Abelson, 1977) (structured represen-
tations of events, their causal relationships, and
their participants) and frames to drive interpreta-
tion of syntax and word use. Knowledge ... Proceedings of the 47th Annual Meeting of the ACL and the 4th IJCNLP of the AFNLP, pages 602–610,
Suntec, Singapore, 2-7 August 2009.
c
2009 ACL and AFNLP
Unsupervis...
... input are the basic units of
syntactic analysis. Standard evaluation procedures
and metrics (Black et al., 1991; Buchholz and Marsi,
2006) accordingly assume that the yield of the parse
tree is known ... Ballesteros and Nivre (2012), and the Easy-
First parser of Goldberg and Elhadad (2010) with the
features therein. Since these parsers cannot choose
their own tags, au...
... spelling, good grammar, rhythm and flow,
appropriateness of tone, and several other specific
characteristics of good text.
In terms of automatic evaluation, we are not aware
of any technique that measures ... evaluators: choosing the parsers and the
metrics derived from them; generating some texts
for human and parser evaluation; and, the key part,
getting human judgements...
... section, we present the evaluations of
ROUGE-L, ROUGE-S, and compare their per-
formance with other automatic evaluation meas-
ures.
5 Evaluations
One of the goals of developing automatic ... sequences X and
Y, the longest common subsequence (LCS) of X
and Y is a common subsequence with maximum
length. We can find the LCS of two sequences of
length m and n using...
... understanding of the
differences in the kinds of non-fluencies that occur, we are
left with a kind of grab bag of grammatical deviation that
can never be analyzed except by some sort of general ... corpus of over twenty hours of transcribed speech, in the
process of using the parser to search for various syntactic
constructions. Tht~ transcripts are of sociolinguistic...
... Proceedings of the 12th Conference of the European Chapter of the ACL, pages 112–120,
Athens, Greece, 30 March – 3 April 2009.
c
2009 Association for Computational Linguistics
Human Evaluation of a ... better understand what makes good
evaluation data (and metrics), we designed and im-
plemented an experiment in which human judges
evaluated German string realisations. The main...
... description of the approach, tips for
how to use AdWords for scientific research,
and results of pilot experiments on the impact
of affective text variations which confirm the
effectiveness of the approach.
1 ... the persuasive impact of a message.
The problem is that evaluation experiments repre-
sent a bottleneck: they are expensive and time con-
suming, and recruiting a hig...
... techniques
for syntactic analysis. Although it is now common
for researchers to rely on automatic morphosyntactic
analyses of transcripts to obtain part -of- speech and
morphological analyses, their use of syntactic ... from POS
tags and words alone, these are still hand-crafted
rules that need to be debugged and perfected over
time. This was the first evaluation of our sys...
...
processes one genre of discourse, that of news-
paper reports. The program creates summaries of
reports by relying on an expanded concept of text
grounding: certain syntactic structures and tense/ ... correlations between syntactic form and
information type and the syntactic means for
~ndicating episode boundaries must be determined.
The degree of correlation betw...