While the mutually beneficial role of interdisciplinarity in DH projects is often highlighted, the impulse to rethink one’s own discipline-centered dogmas and theory as a result of such cooperations is seldom addressed. In heureCLÉA, 1 a DH project involving a team of literary scholars and a team of computer scientists and machine learning specialists, both parties have encountered this phenomenon. In the following, we would like to demonstrate how interdisciplinary incompatibilities are brought to attention in the DH context, and how this realisation can help to improve a participant theory—in our example, the theory of narrative. 2

The Setting: heureCLÉA

heureCLÉA’s goal is to develop a ‘digital heuristic’: a discovery tool that supports literary scholars in analysing narrative texts. We focus on the generation of automated annotations concerning temporal aspects of novels, short stories, etc. 3 By automatically identifying time-related phenomena, our digital heuristic-tool shall provide bottom-up support for hermeneutic analyses.

Three steps lead toward this goal:

1. A small corpus of short stories is manually and collaboratively annotated with the help of a narratological tagset for temporal phenomena.

2. These annotations are then analysed and modelled in a computational approach that combines rule-based NLP methods with machine learning procedures. 4

3. Once the automatically generated output meets defined performance criteria, the digital heuristic is implemented as a functional module in the text analysis platform CATMA. 5

This looks straightforward, but as soon as the first manual annotations were discussed in phase 2, we encountered problems that point to fundamentally different, discipline-specific approaches to text annotation.

The Problem: Interdisciplinary Incompatibilities

To begin, heureCLÉA’s markup ‘philosophy’ is consciously non-prescriptive: 6 we do not enforce inter-annotator agreement. Instead, we invite human annotators to consider their annotation practice as an interpretive activity rather than a taxonomy-ruled declaration of objective phenomena. This approach is supported by CATMA, our web-based platform for collaborative text annotation. 7 Accordingly, our annotators made different and even contradictory interpretive decisions when tagging certain text segments.

From a computer science point of view, this renders parts of the relevant CATMA markup data noisy, i.e., too ambiguous to allow for reliable statistical ML-analysis and prediction. The literary studies’ perspective onto such annotator disagreement, however, is quite the opposite. 8 Rather than constituting a methodological obstacle, substantially different interpretations of texts derived from incompatible analyses are regarded as a natural consequence of the polyvalence of literary texts (cf. Jannidis et al., 2003), or to phrase it provocatively: by definition, a non-ambiguous text triggering uniform high-level interpretations lacks aesthetic quality (cf. Jakobson, 1979/1960). Of course, such judgement hinges on the definition of ‘high-level’—the interpretation of a text’s philosophical message will certainly fall into this category. But what about statements concerning more formal, seemingly uncontroversial characteristics of narrative, such as their temporal construction both in terms of the what (story) and the how (discourse) of narration?

Two examples:

1. Standard narratological taxonomy uses the category of order to identify whether a series of events is narrated in its original temporal order, or whether the chronologies of events and narration differ. The following passage was annotated as a prolepsis, i.e., ‘flash forward’, by some annotators: ‘Maybe you expect me there at The Post Inn. Then we go to Ammerland together. It’s going to be a grand trip’ (Wedekind, The Vaccination, our translation). 9 But not everybody agreed: a second group of annotators did not detect a deviation from the chronological representation of events.

2. The second example concerns the analysis of duration or speed of the narration: how long does it take to report events in relation to how long it takes for these events to happen? The following passage was sometimes annotated as scene, i.e., very slow narration, and sometimes as summary, i.e., very fast narration: ‘I have gazed outside all night, and methought this was how death must be like, or the after-death: over there and outside an infinite, hollowly roaring darkness. Will a thought, a notion of mine linger and weave on there and eternally hark to the intangible roaring?’ (Mann, Death, our translation). 10

From an ML perspective these interpretive variances produced noisy annotation data—but how about the humanist’s view: Isn’t this the hallmark of aesthetic originality? Are conflicting annotations not indicative of a semantic richness owed to a narrators’ ability to evoke, e.g., polyvalent temporal orders? If so, the phenomenon would indeed have been adequately encoded in nuce by conflicting manual annotations on the inconspicuous level of narrative form. 11

To test this hypothesis, we attempted to reconcile the two different annotation-centered methodological perspectives—that of the computer scientist and that of the literary scholar—by taking a decision-tree-inspired look at the human-generated annotation: What exactly had triggered the diverse output? Where were the choice points?

The Solution: Improving Narratological Theory via NLP-Parametrisation

Revisiting examples of noisy narratological annotation from a humanities perspective, we eventually realised that some did not result from genuine high-level polyvalence of literary texts. Instead, they could be traced back to one of two shortcomings—methodological and/or theoretical:

Ill-Defined Concepts

Some base-level narratological concepts proved simply ill-defined. Consider example one: The usual definition of the concept prolepsis (flash forward) does not specify whether temporal lookaheads must concern ‘actualized’ events in the fictional world. The annotation of the passage from example 1 thus varies according to how one defines the modal parameter ‘actualized’ vs. ‘hypothetical’. Problems of this kind were resolved by simply ‘pre-parametrising’ the relevant narratological categories so that previously non-explicit theoretical premises were now formally included in the category definition as functional parameters. This usually results in narrower definitions and, respectively, more consistent—less noisy—annotations. As for the definition of prolepsis, we confined this to ‘actualized’ anticipations. Accordingly, example 1 does not contain a prolepsis, for the future events in question—the meeting at the Post Inn, the journey together—are purely hypothetical.

Undetected Dependencies

Other contradictory annotations were traced back to opaque dependencies between specific base-level narratological concepts and over-arching theoretical assumptions in narratological theory . In a computational view, such non-explicit premises function as hidden parameters. This finding explains our second example: the analysis of duration depends on the underlying notion of what constitutes an event. If only specific occurrences in the fictional ‘outside world’ are considered an event, then the narration speed in example 2 is low. Since only the protagonist’s thoughts are reported here, nothing ‘happens’ empirically in the fictional world; it is only the narration that progresses. If, on the other hand, thoughts are considered events, too, then the narration speed is, of course, very high: a whole night of ‘gazing and thinking’ is narrated in just a few sentences!

To address such cases where annotation variance stemmed from non-explicated theoretical premises, we adopted a less deterministic approach: such incompatible markup decisions are now accepted on condition that their underlying assumptions are explained in the markup. Annotators must, for example, document their notion of event when analysing duration. High-level parametrisation of this type will be implemented in the heuristic module by offering the users to choose their own parameters. Automated markup suggestions will then be adapted to these choices. From a technical perspective, this is realised using transparent machine learning models (such as Decision Trees)—in contrast to less transparent models (e.g., Support Vector Machines). Transparency is reflected in the possibility of being able to retrace and visualise the concrete decision process, i.e., factors that are involved in the prediction, which not only allows for an adaptation of the underlying model based on user-input (so-called interactive or feedback-driven machine learning) but also facilitates a reasoning and discussion over the changes to the model due to user input.

Conclusion and Outlook

These examples demonstrate how interdisciplinary DH collaboration, apart from the defined project goals, can also yield more general theoretical and methodological insights. Methodological incompatibilities observed in the pragmatic domain can motivate one of the partner disciplines to optimize its elementary taxonomic definitions, and even stimulate theoretical revisions, as the example of inconsistent human annotation and the introduction of pre-parametrisation and flexible parametrisation of categories demonstrates.

DH-collaborations are particularly well suited to yield such benefits beyond the pragmatic: on the one hand, we practice the humanities’ hermeneutic approach, a methodology and epistemology that are phenomenological, synthesis-oriented, and historically contingent; on the other hand, we employ the formal approach of computational analysis and mathematical modelling, which strives for an abstract, fully parametrised representation and analysis of data structures and processes.

This methodological bifurcation is not a question of enduring a life in two cultures ( sensu C. P. Snow, cf. Snow [1993]); it is the fundamental and highly productive dialectic of DH’s epistemology and practice.

Notes

1. heureCLÉA is funded by the German Ministry for Education and Science (BMBF) as part of the eHumanities initiative. For further project details, see www .heureclea .de.

2. While this paper only addresses narrative theory as a beneficiary discipline, we have also experienced cases demonstrating the opposite constellation. For an illustration of the benefits that the interdisciplinary approach has brought for NLP, see Gius and Jacke (2015) as well as Bögel et al. (2015).

3. In addition to rather basic aspects like tense and dates, we are concerned with more complex temporal phenomena that are constitutive of all types of narrative representation: every narrative features a story, and this story is presented in a specific way. The more complex temporal phenomena with which we are concerned all refer to the temporal relation between story and representation. This relation can be analysed with regards to three aspects: order (when did an event happen? when is it told?), frequency (how often does it happen? how often is it told?), and duration (how long did it take to happen? how long did it take to tell about it?); cf. Genette (1972).

4. For first results concerning automated annotation, cf. Bögel et al. (2014).

5. Cf. www.digitalhumanities.it/catma.

6. heureCLÉA—like its markup tool CATMA—is based on the methodological premise of ‘hermeneutic markup’, as described in Piez (2010).

7. For CATMA, see www . catma . de.

8. For a more theoretical-methodological discussion of this divergence, cf. Gius and Jacke (2015).

9. ‘Vielleicht erwartet ihr mich dort im Gasthof zur Post. Dann fahren wir zusammen nach Ammerland. Das wird eine prächtige Tour’ (Wedekind, Die Schutzimpfung).

10. ‘Ich habe die ganze Nacht hinausgeblickt, und mich dünkte, so müsse der Tod sein oder das Nach dem Tode: dort drüben und draußen ein unendliches, dumpf brausendes Dunkel. Wird dort ein Gedanke, eine Ahnung von mir fortleben und -weben und ewig auf das unbegreifliche Brausen horchen?’ (Mann, Der Tod).

11. For a more detailed exploration of the phenomenon, see Meister (2003), who explores the resulting combinatorial potential of incrementally more complex and polyvalent action-constructs in Goethe’s novella cycle Unterhaltungen deutscher Ausgewanderten.