A Machine Learning Approach to Extract Temporal Information from Texts in Swedish and Generate Animated 3D Scenes Anders Berglund Richard Johansson Pierre Nugues Department of Computer Science, LTH Lund University SE-221 00 Lund, Sweden d98ab@efd.lth.se, {richard, pierre}@cs.lth.se Abstract Carsim is a program that automatically converts narratives into 3D scenes. Carsim considers authentic texts describing road accidents, generally collected from web sites of Swedish newspapers or transcribed from hand-written accounts by victims of accidents. One of the program’s key fea- tures is that it animates the generated scene to visualize events. To create a consistent animation, Carsim extracts the participants mentioned in a text and identifies what they do. In this paper, we focus on the extraction of tem- poral relations between actions. We first describe how we detect time expressions and events. We then present a machine learning technique to order the sequence of events identified in the narratives. We finally report the results we obtained. 1 Extraction of Temporal Information and Scene Visualization Carsim is a program that generates 3D scenes from narratives describing road accidents (Johansson et al., 2005; Dupuy et al., 2001). It considers au- thentic texts, generally collected from web sites of Swedish newspapers or transcribed from hand- written accounts by victims of accidents. One of Carsim’s key features is that it animates the generated scene to visualize events described in the narrative. The text below, a newspaper arti- cle with its translation into English, illustrates the goals and challenges of it. We bracketed the enti- ties, time expressions, and events and we anno- tated them with identifiers, denoted respectively o i , t j , and e k : En {bussolycka} e1 i södra Afghanistan krävde e2 {på torsdagen} t1 {20 dödsoffer} o1 . Ytterligare {39 personer} o2 skadades e3 i olyckan e4 . Bussen o3 {var på väg} e5 från Kanda- har mot huvudstaden Kabul när den o4 under en omkörning e6 körde e7 av vägbanan o5 och voltade e8 , meddelade e9 general Salim Khan, biträdande polischef i Kandahar. TT-AFP & Dagens Nyheter, July 8, 2004 {20 persons} o1 died e2 in a {bus accident} e1 in southern Afghanistan {on Thursday} t1 . In addition, {39 persons} o2 {were injured} e3 in the accident e4 . The bus o3 {was on its way} e5 from Kandahar to the capital Kabul when it o4 {drove off} e7 the road o5 while overtaking e6 and {flipped over} e8 , said e9 General Salim Khan, assistant head of police in Kandahar. The text above, our translation. To create a consistent animation, the program needs to extract and understand who the partici- pants are and what they do. In the case of the ac- cident above, it has to: 1. Detect the involved physical entities o3, o4, and o5. 2. Understand that the pronoun o4 refers to o3. 3. Detect the events e6, e7, and e8. 385 4. Link the participants to the events using se- mantic roles or grammatical functions and in- fer the unmentioned vehicle that is overtaken. 5. Understand that the order of the events is e6- e7-e8. 6. Detect the time expression t1 to anchor tem- porally the animation. In this paper, we describe how we address tasks 3, 5, and 6 w ithin the Carsim program, i.e., how we detect, interpret, and order events and how we process time expressions. 2 Previous Work Research on the representation of time, events, and temporal relations dates back the beginning of logic. It resulted in an impressive number of formulations and models. In a review of contem- porary theories and an attempt to unify them, Ben- nett and Galton (2004) classified the most influen- tial formalisms along three lines. A first approach is to consider events as transitions between states as in STRIPS (Fikes and Nilsson, 1971). A sec- ond one is to map events on temporal intervals and to define relations between pairs of intervals. Allen’s (1984) 13 temporal relations are a widely accepted example of this. A third approach is to reify events, to quantify them existentially, and to connect them to other objects using predicates based on action verbs and their modifiers (David- son, 1967). The sentence John saw Mary in Lon- don on Tuesday is then translated into the logical form: ∃[Saw(, j, m)∧P lace(, l)∧T ime(, t)]. Description of relations between time, events, and verb tenses has also attracted a considerable interest, especially in English. Modern work on temporal event analysis probably started with Re- ichenbach (1947), who proposed the distinction between the point of speech, point of reference, and point of event in utterances. This separation allows for a systematic description of tenses and proved to be very powerful. Many authors proposed general principles to extract automatically temporal relations between events. A basic observation is that the tempo- ral order of events is related to their narrative or- der. Dowty (1986) investigated it and formulated a Temporal Discourse Interpretation Principle to in- terpret the advance of narrative time in a sequence of sentences. Lascarides and Asher (1993) de- scribed a complex logical framework to deal w ith events in simple past and pluperfect sentences. Hitzeman et al. (1995) proposed a constraint- based approach taking into account tense, aspect, temporal adverbials, and rhetorical structure to an- alyze a discourse. Recently, groups have used machine learn- ing techniques to determine temporal relations. They trained automatically classifiers on hand- annotated corpora. Mani et al. (2003) achieved the best results so far by using decision trees to order partially events of successive clauses in En- glish texts. Boguraev and Ando (2005) is another example of it for English and Li et al. (2004) for Chinese. 3 Annotating Texts with Temporal Information Several schemes have been proposed to anno- tate temporal information in texts, see Setzer and Gaizauskas (2002), inter alia. Many of them were incompatible or incomplete and in an effort to rec- oncile and unify the field, Ingria and Pustejovsky (2002) introduced the XML-based Time markup language (TimeML). TimeML is a specification language whose goal is to capture most aspects of temporal rela- tions between events in discourses. It is based on A llen’s (1984) relations and a variation of Vendler’s (1967) classification of verbs. It de- fines XML elements to annotate time expressions, events, and “signals”. The SIGNAL tag marks sec- tions of text indicating a temporal relation. It includes function words such as later and not. TimeML also features elements to connect entities using different types of links, most notably tem- poral links, TLINKs, that describe the temporal re- lation holding between events or between an event and a time. 4 A System to Convert Narratives of Road Accidents into 3D Scenes 4.1 Carsim Carsim is a text-to-scene converter. From a nar- rative, it creates a complete and unambiguous 3D geometric description, which it renders visually. Carsim considers authentic texts describing road accidents, generally collected from web sites of Swedish newspapers or transcribed from hand- written accounts by victims of accidents. One of the program’s key features is that it animates the generated scene to visualize events. 386 The Carsim architecture is divided into two parts that communicate using a frame representa- tion of the text. Carsim’s first part is a linguistic module that extracts information from the report and fi lls the frame slots. The second part is a vir- tual scene generator that takes the structured rep- resentation as input, creates the visual entities, and animates them. 4.2 Knowledge Representation in Carsim The Carsim language processing module reduces the text content to a frame representation – a tem- plate – that outlines what happened and enables a conversion to a symbolic scene. It contains: • Objects. They correspond to the physical en- tities mentioned in the text. They also include abstract symbols that show in the scene. Each object has a type, that is selected from a pre- defined, finite set. An object’s semantics is a separate geometric entity, where its shape (and possibly its movement) is determined by its type. • Events. They correspond intuitively to an ac- tivity that goes on during a period in time and here to the possible object behaviors. We represent events as entities with a type taken from a predefined set, where an event’s se- mantics will be a proposition paired with a point or interval in time during which the proposition is true. • Relations and Quantities. They describe spe- cific features of objects and events and how they are related to each other. The most obvi- ous examples of such information are spatial information about objects and temporal in- formation about events. Other meaningful re- lations and quantities include physical prop- erties such as velocity, color, and shape. 5 Time and Event Processing We designed and implemented a generic com- ponent to extract temporal information from the texts. It sits inside the natural language part of Carsim and proceeds in two steps. The first step uses a pipeline of finite-state machines and phrase- structure rules that identifies time expressions, sig- nals, and events. This step also generates a feature vector for each element it identifies. Using the vectors, the second step determines the temporal relations between the extracted events and orders them in time. T he result is a text annotated using the TimeML scheme. We use a set of decision trees and a machine learning approach to find the relations between events. As input to the second step, the decision trees take sequences of events extracted by the first step and decide the temporal relation, possi- bly none, between pairs of them. To run the learn- ing algorithm, we manually annotated a small set of texts on which we trained the trees. 5.1 Processing Structure We use phrase-structure rules and finite state ma- chines to mark up events and time expressions. In addition to the identification of expressions, w e of- ten need to interpret them, for instance to com- pute the absolute time an expression refers to. We therefore augmented the rules with procedural at- tachments. We wrote a parser to control the processing flow where the rules, possibly recursive, apply regular expressions, call procedures, and create TimeML entities. 5.2 Detection of Time Expressions We detect and interpret time expressions with a two-level structure. The first level processes in- dividual tokens using a dictionary and regular ex- pressions. The second level uses the results from the token level to compute the meaning of multi- word expressions. Token-Level Rules. In Swedish, time expres- sions such as en tisdagseftermiddag ‘a Tuesday afternoon’ use nominal compounds. To decode them, we automatically generate a comprehensive dictionary with mappings from strings onto com- pound time expressions. We decode other types of expressions such as 2005-01-14 using regular expressions Multiword-Level Rules. We developed a grammar to interpret the meaning of multiword time expressions. It includes instructions on how to combine the values of individual tokens for ex- pressions such as {vid lunchtid} t1 {en tisdagefter- middag} t2 ‘{at noon} t1 {a Tuesday afternoon} t2 ’. The most common case consists in merging the to- kens’ attributes to form a more specific expression. However, relative time expressions such as i tors- dags ‘last Tuesday’ are more complex. Our gram- mar handles the most frequent ones, mainly those 387 that need the publishing date for their interpreta- tion. 5.3 Detection of Signals We detect signals using a lexicon and naïve string matching. We annotate each signal with a sense where the possible values are: negation, before, af- ter, later, when, and continuing. TimeML only de- fines one attribute for the SIGNAL tag, an identifier, and encodes the sense as an attribute of the LINKs that refer to it. We found it more appropriate to store the sense directly in the SIGNAL element, and so we extended it with a second attribute. We use the sense information in decision trees as a feature to determine the order of events. Our strategy based on string matching results in a lim- ited overdetection. However, it does not break the rest of the process. 5.4 Detection of Events We detect the TimeML events using a part-of- speech tagger and phrase-structure rules. We con- sider that all verbs and verb groups are events. We also included some nouns or compounds, which are directly relevant to Carsim’s application do- main, such as bilolycka ‘car accident’ or krock ‘collision’. We detect these nouns through a set of six morphemes. TimeML annotates events with three features: aspect, tense, and “class”, where the class corre- sponds to the type of the event. The TimeML spec- ifications define seven classes. We kept only the two most frequent ones: states and occurrences. We determine the features using procedures at- tached to each grammatical construct we extract. The grammatical features aspect and tense are straightforward and a direct output of the phrase- structure rules. To infer the TimeML class, we use heuristics such as these ones: predicative clauses (copulas) are generally states and verbs in preterit are generally occurrences. The domain, reports of car accidents, makes this approach viable. The texts describe sequences of real events. They are generally simple, to the point, and void of speculations and hypothetical scenarios. This makes the task of feature identifi- cation simpler than it is in more general cases. In addition to the TimeML features, we extract the grammatical properties of events. Our hypoth- esis is that specific sequences of grammatical con- structs are related to the temporal order of the de- scribed events. The grammatical properties con- sist of the part of speech, noun (NOUN) or verb (VB). Verbs can be finite (FIN) or infinitive (INF). They can be reduced to a single word or part of a group (GR). They can be a copula (COP), a modal (MOD), or a lexical verb. We combine these prop- erties into eight categories that we use in the fea- ture vectors of the decision trees (see EventStruc- ture in Sect. 6.2). 6 Event Ordering TimeML defines three different types of links: subordinate (SLINK), temporal (TLINK), and aspec- tual (ALINK). Aspectual links connect two event in- stances, one being aspectual and the other the ar- gument. As its significance was minor in the visu- alization of car accidents, we set aside this type of link. Subordinate links generally connect signals to events, for instance to mark polarity by linking a not to its main verb. We identify these links simul- taneously with the event detection. We augmented the phrase-structure rules to handle subordination cases at the same time they annotate an event. We restricted the cases to modality and polarity and we set aside the other ones. 6.1 Generating Temporal Links To order the events in time and create the tempo- ral links, we use a set of decision trees. We apply each tree to sequences of events where it decides the order between two of the events in each se- quence. If e 1 , , e n are the events in the sequence they appear in the text, the trees correspond to the following functions: f dt1 (e i , e i+1 ) ⇒ t rel (e i , e i+1 ) f dt2 (e i , e i+1 , e i+2 ) ⇒ t rel (e i , e i+1 ) f dt3 (e i , e i+1 , e i+2 ) ⇒ t rel (e i+1 , e i+2 ) f dt4 (e i , e i+1 , e i+2 ) ⇒ t rel (e i , e i+2 ) f dt5 (e i , e i+1 , e i+2 , e i+3 ) ⇒ t rel (e i , e i+3 ) The possible output values of the trees are: si- multaneous, after, before, is_included, includes, and none. These values correspond to the relations described by Setzer and Gaizauskas (2001). The first decision tree should capture more gen- eral relations between two adjacent events with- out the need of a context. Decision trees dt 2 and dt 3 extend the context by one event to the left re- spectively one event to the right. They should cap- ture more specific phenomena. However, they are not always applicable as we never apply a decision 388 tree when there is a time expression between any of the events involved. In effect, time expressions “reanchor” the narrative temporally, and we no- ticed that the decision trees performed very poorly across time expressions. We complemented the decision trees with a small set of domain-independent heuristic rules that encode common-sense knowledge. We as- sume that events in the present tense occur after events in the past tense and that all mentions of events such as olycka ‘accident’ refer to the same event. In addition, the Carsim event interpreter recognizes some semantically motivated identity relations. 6.2 Feature Vectors The decision trees use a set of features correspond- ing to certain attributes of the considered events, temporal signals between them, and some other parameters such as the number of tokens separat- ing the pair of events to be linked. We list below the features of f dt1 together with their values. The first event in the pair is denoted by a mainEvent pre- fix and the second one by relatedEvent: • mainEventTense: none, past, present, future, NOT_DETERMINED. • mainEventAspect: progressive, perfective, per- fective_progressive, none, NOT_DETERMINED. • mainEventStructure: NOUN, VB_GR_COP_INF, VB_GR_COP_FIN, VB_GR_MOD_INF, VB_GR_MOD_FIN, VB_GR, VB_INF, VB_FIN, UNKNOWN. • relatedEventTense: (as mainEventTense) • relatedEventAspect: (as mainEventAspect) • relatedEventStructure: (as mainEventStructure) • temporalSignalInbetween: none, before, after, later, when, continuing, several. • tokenDistance: 1, 2 to 3, 4 to 6, 7 to 10, greater than 10. • sentenceDistance: 0, 1, 2, 3, 4, greater than 4. • punctuationSignDistance: 0, 1, 2, 3, 4, 5, greater than 5. The four other decision trees consider more events but use similar features. The values for the Distance features are of course greater. 6.3 Temporal Loops The process described above results in an overgen- eration of temporal links. As some of them may be conflicting, a post-processing module reorganizes them and discards the temporal loops. The initial step of the loop resolution assigns each link w ith a score. This score is created by the decision trees and is derived from the C4.5 metrics (Quinlan, 1993). It reflects the accuracy of the leaf as well as the overall accuracy of the decision tree in question. The score for links generated from heuristics is rule dependent. The loop resolution algorithm begins with an empty set of orderings. It adds the partial order- ings to the set if their inclusion doesn’t introduce a temporal conflict. It first adds the links with the highest scores, and thus, in each temporal loop, the ordering with the lowest score is discarded. 7 Experimental Setup and Evaluation As far as we know, there is no available time- annotated corpus in Swedish, which makes the evaluation m ore difficult. As development and test sets, we collected approximately 300 reports of road accidents from various Swedish newspa- pers. Each report is annotated with its publishing date. Analyzing the reports is complex because of their variability in style and length. Their size ranges from a couple of sentences to more than a page. The amount of details is overwhelming in some reports, while in others most of the informa- tion is implicit. The complexity of the accidents described ranges from simple accidents with only one vehicle to multiple collisions with several par- ticipating vehicles and complex movements. We manually annotated a subset of our corpus consisting of 25 texts, 476 events and 1,162 tem- poral links. We built the trees automatically from this set using the C4.5 program (Quinlan, 1993). Our training set is relatively small and the num- ber of features we use relatively large for the set size. This can produce a training overfit. However, C4.5, to some extent, makes provision for this and prunes the decision trees. We evaluated three aspects of the temporal in- formation extraction modules: the detection and interpretation of time expressions, the detection and interpretation of events, and the quality of the final ordering. We report here the detection of events and the final ordering. 389 Feature N correct N erroneous Correct Tense 179 1 99.4% Aspect 161 19 89.4% Class 150 30 83.3% Table 1: Feature detection for 180 events. 7.1 Event Detection We evaluated the performance of the event detec- tion on a test corpus of 40 previously unseen texts. It should be noted that we used a simplified defi- nition of what an event is, and that the manual an- notation and evaluation were both done using the same definition (i.e. all verbs, verb groups, and a small number of nouns are events). The system detected 584 events correctly, overdetected 3, and missed 26. This gives a recall of 95.7%, a preci- sion of 99.4%, and an F-measure of 97.5%. The feature detection is more interesting and Table 1 shows an evaluation of it. We carried out this evaluation on the first 20 texts of the test cor- pus. 7.2 Evaluation of Final Ordering We evaluated the final ordering with the method proposed by Setzer and Gaizauskas (2001). Their scheme is comprehensive and enables to compare the performance of different systems. Description of the Evaluation Method. Set- zer and Gaizauskas carried out an inter-annotator agreement test for temporal relation markup. When evaluating the final ordering of a text, they defined the set E of all the events in the text and the set T of all the time expressions. They com- puted the set (E ∪ T) × (E ∪ T) and they defined the sets S  , I  , and B  as the transitive closures for the relations simultaneous, includes, and be- fore, respectively. If S  k and S  r represent the set S  for the an- swer key (“Gold Standard”) and system response, respectively, the measures of precision and recall for the simultaneous relation are: R = |S  k ∩ S  r | |S  k | P = |S  k ∩ S  r | |S  r | For an overall measure of recall and precision, Setzer and Gaizauskas proposed the following for- mulas: R = |S  k ∩ S  r | + |B  k ∩ B  r | + |I  k ∩ I  r | |S  k | + |B  k | + |I  k | P = |S  k ∩ S  r | + |B  k ∩ B  r | + |I  k ∩ I  r | |S  r | + |B  r | + |I  r | They used the classical definition of the F- measure: the harmonic means of precision and re- call. Note that the precision and recall are com- puted per text, not for all relations in the test set simultaneously. Results. We evaluated the output of the Car- sim system on 10 previously unseen texts against our Gold Standard. As a baseline, we used a sim- ple algorithm that assumes that all events occur in the order they are introduced in the narrative. For comparison, we also did an inter-annotator evalu- ation on the same texts, where we compared the Gold Standard, annotated by one of us, with the annotation produced by another member in our group. As our system doesn’t support comparisons of time expressions, we evaluated the relations con- tained in the set E × E. We only counted the reflexive simultaneous relation once per tuples (e x , e y ) and (e y , e x ) and we didn’t count relations (e x , e x ). Table 2 shows our results averaged over the 10 texts. As a reference, we also included Set- zer and Gaizauskas’ averaged results for inter- annotator agreement on temporal relations in six texts. Their results are not directly comparable however as they did the evaluation over the set (E ∪ T) × (E ∪ T ) for English texts of another type. Comments. T he computation of ratios on the transitive closure makes Setzer and Gaizauskas’ evaluation method extremely sensitive. Missing a single link often results in a loss of scores of gener- ated transitive links and thus has a massive impact on the final evaluation figures. As an example, one of our texts contains six events whose order is e 4 < e 5 < e 6 < e 1 < e 2 < e 3 . The event module automatically detects the chains e 4 < e 5 < e 6 and e 1 < e 2 < e 3 correctly, but misses the link e 6 < e 1 . This gives a recall of 6/15 = 0.40. When considering evaluations per- formed using the method above, it is meaningful to have this in mind. 8 Carsim Integration The visualization module considers a subset of the detected events that it interprets graphically. We 390 Evaluation Average n words Average n events P mean R mean F mean Gold vs. Baseline 98.5 14.3 49.42 29.23 35.91 Gold vs. Automatic " " 54.85 37.72 43.97 Gold vs. Other Annotator " " 85.55 58.02 68.01 Setzer and Gaizauskas 312.2 26.7 67.72 40.07 49.13 Table 2: Evaluation results for final ordering averaged per text (with P , R, and F in %). call this subset the Carsim events. Once the event processing has been done, Carsim extracts these specific events from the full set using a small do- main ontology and inserts them into the template. We use the event relations resulting from temporal information extraction module to order them. For all pairs of events in the template, Carsim queries the temporal graph to determine their relation. Figure 1 shows a part of the template represent- ing the accident described in Section 1. It lists the participants, with the unmentioned vehicle in- ferred to be a car. It also shows the events and their temporal order. Then, the visualization mod- ule synthesizes a 3D scene and animates it. Fig- ure 2 shows four screenshots picturing the events. Figure 1: Representation of the accident in the ex- ample text. 9 Conclusion and Perspectives We have developed a method for detecting time expressions, events, and for ordering these events temporally. We have integrated it in a text-to- scene converter enabling the animation of generic actions. The module to detect time expression and inter- pret events performs significantly better than the baseline technique used in previous versions of Carsim. In addition, it should to be easy to sep- arate it from the Carsim framework and reuse it in other domains. The central task, the ordering of all events, leaves lots of room for improvement. The accu- racy of the decision trees should improve with a larger training set. It would result in a better over- all performance. Switching from decision trees to other training methods such as Support Vector Ma- chines or using semantically motivated features, as suggested by Mani (2003), could also be sources of improvements. More fundamentally, the decision tree method we have presented is not able to take into account long-distance links. Investigation into new strate- gies to extract such links directly without the com- putation of a transitive closure would improve re- call and, given the evaluation procedure, increase the performance. References James F. Allen. 1984. Towards a general theo ry of action and time. Artificial Intelligence, 23(2):123– 154. Brandon Bennett and Antony P. Galton. 2004. A uni- fying semantics for time and events. Artificial Inte l- ligence, 153(1-2):13–48. Branimir Boguraev and Rie Kubota Ando. 2005. TimeML-com pliant text analysis for temporal rea- soning. In IJCAI-05, Proceed ings of the Nine teenth International Joint Conference on Artificial Intelli- gence, pages 997–10 03, Edinburgh, Scotland. Donald Davidson. 1967. The logical form of action sentences. In N. Rescher, editor, The Logic of Deci- sion and A ction. University of Pittsburgh Press. David R. Dowty. 1986. The effects of aspectual class on the temporal structure of discourse: Semantics or pragmatics? Linguistics and Philosophy, 9:37–61. 391 Figure 2: Animation of the scene and event visualization. Sylvain Dupu y, Arjan Egges, Vinc e nt Legendre, and Pierre Nugues. 2001. Generating a 3D simulation of a car accident from a written description in nat- ural language: The Carsim system. In ACL 2001, Workshop on Temporal and Spatial Information Pro- cessing, pages 1–8, Toulouse, France. Richard Fikes and Nils J. N ilsson. 1971. Strips: A new approach to the application of theorem p roving to problem solving. Artificia l Intelligence, 2:189– 208. Janet Hitzeman, Marc Noels Moens, and Clare Grover. 1995. Algorithm s for analyzing the temporal struc- ture of discourse. In Proceedings of the Annual Meeting of the European Chapter of the Associa- tion of Computational Linguistics, pages 253–260, Dublin, Ireland. Bob Ingria and James Pustejovsky. 2002. Specification for TimeML 1.0. Richard Johansson, Anders Berglund, Magnus Danielsson, and Pierre Nugues. 2005. Automatic text-to-scene conversion in the traffic accident domain. In IJCAI-05, Procee dings of the Nineteenth International Joint Conference on Artificial Intelli- gence, pages 1073–1 078, Edinburgh, Scotland. Alex Lascarides and Nicholas Asher. 1993. Tem- poral interpretation, discourse relations, and com- mon sense entailment. Linguistics & Philosophy, 16(5):437–493. Wenjie Li, Kam-Fai Wong, Guihong Cao, and Chunfa Yuan. 2004. Applying machine learning to Chinese temporal r elation r esolution. In Proceedings of the 42nd Meeting of the Association for Computational Linguistics (ACL’04), pages 582–588, Barcelona. Inderjeet Mani, Barry Schiffman, a nd Jianping Zhang. 2003. Inferring temporal ordering of events in news. In Human Language Technology Co nference (HLT’03), Edmonton, Canada. Inderjeet Man i. 2003. Recent developments in tempo- ral information extraction. In Nicolas Nicolov a nd Ruslan Mitkov, editors, Proceedings of RANLP’03. John Benjamins. John Ross Quinlan. 1993. C4.5: Programs for Ma- chine Learning. Morgan Kauffman. Hans Reichenbach. 1947. Eleme nts of Symbolic Logic. Academic Press, New York. Andrea Setzer and Ro bert Gaizausk as. 200 1. A pi- lot study on annotating temporal relations in text. In ACL 2 001, Workshop on Temporal an d Spatial Infor- mation Processing, pag e s 73 –80, Toulouse, France. Andrea Setzer a nd Robert Gaizauskas. 2002. On the importance of annotating temporal event-event rela- tions in text. In LREC 2002, Workshop on Annota- tion Standards for Temporal Information in Natural Language. Zeno Vendler. 1967. Lin guistics in Philosophy. Cor- nell University Press, Ithaca, New York. 392 . A Machine Learning Approach to Extract Temporal Information from Texts in Swedish and Generate Animated 3D Scenes Anders Berglund Richard Johansson Pierre. Annotating Texts with Temporal Information Several schemes have been proposed to anno- tate temporal information in texts, see Setzer and Gaizauskas (2002), inter alia. Many of them were incompatible. small do- main ontology and inserts them into the template. We use the event relations resulting from temporal information extraction module to order them. For all pairs of events in the template,