Converted from an OASIS Open Document
Music Information Retrieval offers new options for musicological research, particularly for methodologies which are hard (or impossible) to carry out manually, e.g., large corpus studies and measurings of acoustical properties. One such field of application is the mining of patterns. Patterns – and repetitions in general – play an important role in nearly all music styles and are thus of interest for many sub-fields of musicology. In particular, melodic patterns – or ‘licks’ as patterns are often called in jazz parlance - form a crucial component of jazz improvisation (Berliner, 1994; Norgaard, 2014; Owens, 1974). Given the significance of patterns and licks in jazz, several research questions arise, concerning historical issues, e.g., the oral tradition of melodic patterns and the development of a typical jazz language, or more systematic issues, e.g., the psychology of the creative process, where patterns can be regarded as necessary to accomplish the highly virtuoso feat of modern jazz improvisation:
This paper presents three novel user interfaces for investigating the pattern usage in monophonic jazz solos and exemplifies how these interfaces could be used for research on jazz improvisation.
Several web-based melody search engines have been developed in the past, e.g. Themefinder (
http://www.themefinder.org) which allows searching for patterns in both classical and folk music and
Musipedia (
https://www.musipedia.org
), a user-generated database of melodies, providing more sophisticated and user-friendly search interfaces like a virtual piano keyboard for entering melodic patterns and a query by tapping interface for rhythm search. A more thorough overview of search engines can be found in (Frieler et al., 2018). Unfortunately, many of these projects are discontinued or use out-dated web technology. An example for an up-to-date search interface including metadata filters is the RISM catalogue search (
https://opac.rism.info
).
The
Pattern History Explorer is a Shiny application that allows exploring a set of 653 patterns that are most commonly used by eminent players (
Pattern Search is a web interface for a general two-staged pattern search Martin2019-04-30T10:24:00
Similarity Search faces the challenge of finding and grouping similar patterns, i.e., patterns that differ in one or several tones (
The main goal of the
Pattern History Explorer is to enable the exploration of interval patterns in jazz solos by providing information from many different angles. It maintains an overview of interval patterns frequently used by a selected subset of performers and traces their usage in the Weimar Jazz Database (Pfleiderer, 2017), allowing for the discovery of cross-artist and cross-temporal relationships. Currently, 653 interval patterns with 11,630 instances are included. The pattern corpus was created by mining for interval patterns in solos of eminent performers using the partition mode of the
melpat module from the MeloSpySuite (Frieler, 2017). Subsequently, all instances of these patterns were searched for in the Weimar Jazz Database, and the results were included in the application. In general, the user of the Pattern History Explorer selects a certain interval pattern from the overall set of 653 patterns. Several options are available in order to filter the pattern set or to change the ordering of the patterns according to several criteria (e.g., filtering by performer, length, intrinsic characteristics such as Huron contour (Huron, 1996), tonality type, or tonal content). For the selected pattern, various kinds of information can be accessed in several tabs.
While pre-computing a set of patterns is helpful for exploratory investigations, searching for instances of arbitrary patterns of any length and frequency of occurrence within a database requires a different approach. The web-based
Pattern Search interface provides most of the functionality of the
melpat search module of the MeloSpySuite while also extending it with audio and score snippets (both as isolated patterns and within their melodic context) for visual and aural inspection. To execute a basic search, the user can enter a pattern on a virtual keyboard or as a space or comma separated list of elements and choose a corresponding transformation, that is, a mathematical mapping of the basic melodic representation (for details, see Frieler, 2017). Currently, ten pitch-related transformations for primary search are offered (e.g., MIDI pitch, semitone intervals, extended chordal diatonic pitch class). An additional 18 transformations, such as duration, inter-onset interval classes and various structural markers, are supplied for the optional secondary search. Additionally, the search space can be constrained by several metadata categories, like performer, style, or recording year. Search patterns can be regular expressions (in a hybrid syntax depending on the selected transformation) which allows searches for variants in a single run. The secondary search can be used to refine the result space, e.g., by filtering out certain rhythmic or metrical configurations or by confining instances to a single phrase. The underlying search algorithm is built upon the basic Python regular expression module, which is fed with virtual Unicode strings constructed from the different melodic representations (transformations) with different alphabets. Scores are generated with the help of Lilypond, while audio snippets are directly extracted from the solo audio files using
ffmpeg.
The latest addition to the web-based toolset is the
Similarity Search
web application. Basically, the interface follows the design and concept of the
Pattern Search
, however, with some significant differences. Not only identical pattern instances, but also similar patterns, that differ in one or more tones from the query, can be searched for. The calculation of search results differs from that of the
Pattern Search
in that the actual similarity calculation is done using the underlying PostgreSQL database system implementation of the Levenshtein
distance (edit distance). This distance measure has been shown in various studies (e.g., Frieler & Müllensiefen, 2006; Grachten & Arcos, 2004; Gulati, 2016) to be a good approximation to similarity judgements of melodies by human experts. The similarity search operates on a database of the complete set of pitch and interval n-grams of up to 20 elements that were previously extracted from the Weimar Jazz Database using the
melpat
module. This amounts to about 4 million n-grams (instances) for MIDI pitch and 3.5 million for interval n-grams. By entering a pattern (n-gram) as a space or comma separated list of elements and choosing a transformation, similar n-grams can be retrieved from the database. To further control the result set, the search interface provides options for parameters such as ‘minimum similarity’ (calculated using the inverse of Levenshtein distance), ‘maximum length difference’ (allowing for n-grams of differing length), or the preservation of melodic contour and pitch range. All searches can also be refined using metadata filters for performers, instruments, etc.
Search results are presented in tabular form together with two graphical representations allowing for visual inspection – an n-gram network graph (Figure 1) and a timeline chart (Figure 2), both generated using the
D3.js data visualization library (Data-Driven Documents, https://d3js.org/). In the case of the network graph utilizing a radial layout, n-grams can be grouped, e.g., by metadata attributes or same similarity value and n-gram class, resp.
The timeline chart depicts when and by which performer pattern instances were played. Audio snippets for all n-grams found are generated and can be listened to by clicking on the nodes in both the network graph and the timeline chart as well as via the play buttons in the result table.
The three applications are already usable interfaces for the Weimar Jazz Database and serve as prototypes for applications to explore large databases, which are going to be automatically extracted from large collections of jazz recordings, as aimed at within the transatlantic research project “Dig That Lick: Analysing Large-Scale Data for Melodic Patterns in Jazz Performances” (DTL). All three tools can be primarily viewed as bespoke interfaces for the specific needs of jazz researchers, but could easily be transferred to other melodic corpora, too. They could also be of interest to teachers, students and music fans as well as to training courses in computational music analysis.