Variability in phonological systems occurs on all levels — segmental, metrical as well as tonal. Critical sources of variation range from differences in vocal tract sizes, regular phonological alternation followed by the attrition of phonological contexts of regular rules and of course loans, leading to the maximal modification of reanalysis.

The level of variation as well as change, we will claim, is however, severely constrained. The hypothesis entertained is the following: phonological opacity may lead to varying choices for native speakers, and the resulting choice is governed by existing phonological preferences. Evidence that the native system plays a constraining influence comes from detailed examination of texts and poetry from Germanic languages (Dutch, English, German, Norwegian, Swedish) and Bengali. Phonological nonesuches (segmental, quantity and tonal) could change the statistical preferences but at each stage the phonological grammar has a restrictive effect. Neurolinguistic evidence also supports the notion that the native phonology governs processing.

In a book chapter published in 2014, we presented an idea about how Cardinal Categories (CCs: actor subject, topic) are processed from a neurophysiological and a neuroanatomical perspective. Based on their different functions (actor = centred in the event currently processed, subject = cataphoric “pointer”, topic = anaphoric “pointer”), we argued that the different CCs should correlate with different neuroanatomical activation patterns along the auditory dorsal stream. In my talk, I will present preliminary results from a new fMRI project using different types of movies, where we looked for empirical evidence for the above-mentioned idea. I will also talk about the problems and limitations inherent to using movies for this type of research. In addition, I will discuss the question of whether different genres in movies or texts modulate or even determine the activation pattern observed.

Speakers of Italian have at their disposal a variety of phrasal and clausal resources for questioning meaning, among which the format che cosa vuol dire X? ‘what does X mean?’. Likewise, recent studies on English and German about similar resources have shown that speakers use them to identify a meaning problem. This contribution takes a step further, by showing that the ‘what does X mean?’-format allows speakers to accomplish a variety of actions. These may be related to a) the negotiation of understanding, and b) the display of a negative stance. In many occurrences, the interactants display clear orientation to either a problem of understanding or a commonly shared negative stance. However, in sensitive environments (such as conflictual discussions), the resource allows speakers to frame their negative stance as a problem of understanding, thereby resisting escalation of the conflict. The ‘what does X mean’-format may or may not be produced with concomitant embodied behaviour. When the format is used to problematise understanding, no specific embodied conduct is observed. Yet, when it is used to display a negative stance, speakers may be seen to perform the grappolo-gesture.

A careful study of multimodal communication demands the consideration of various modes of signalling including but not limited to the gestures of face and hand. The enormity and the complexity of the annotation/analysis of these can be intimidating. We as researchers still largely rely on our qualitative insight and eyes to deal with such essential tasks despite the availability of reliable quantitative tools. MediaPipe is one such tool that can assist us in capturing gesture kinematics. In this talk, I will discuss the prerequisites and the basics of video-based motion capture and look at a few examples with MediaPipe, making use of existing public toolkits (e.g., The Envision Toolbox)^[1].

^[1] Cwiek, A., De Melo, G., Edelman, J., Owoyele, B., Pouw, W., Santuber, J. Trujillo, J.. Envision Toolbox: Multimodal (Signal) Processing and Analysis in Communication. https://github.com/WimPouw/EnvisionBootcamp2021 (**Alphabetical order)

I will start my talk with an overview of the most important fields of application in which eyetracking can contribute to multimodal interaction analysis. In the second part of my talk, I will focus on the advantages and possible challenges of collecting and analyzing eyetracking data as opposed to standard techniques of video-recording of ‘naturally’ occurring interaction. The ‘participant role’ of the camera (or camera-operating researcher) in the on-going interaction will be compared with the perspective of the eye-tracking glasses (scene camera plus overlaid tracker) and it will be argued that in many instances, the latter represent the ‘participants’ perspective’ better than the former. In the third part of my presentation I will report results of a reliability study on the transcription of standard video-recorded data vs. eye-tracking data, based on stationary, triadic interactions. They show that in a number of constellations, video-recordings do not allow to reconstruct participants’ interactionally relevant gaze patterns.

The assumption that natural language expressions have a single, discrete and clearly identifiable meaning in a given context, successfully challenged in lexical semantics by the rise of distributional models, still underlies much work in computational linguistics, including work based on distributed representations. In this talk I will first of all present the evidence that convinced us that the assumption that a single interpretation can always be assigned to anaphoric expression is no more than a convenient idealization. I will then present more recent work in the DALI project collecting more extensive evidence about disagreements in anaphoric interpretation both through online games such as Phrase Detectives (the recently released Phrase Detectives 3 corpus contains over 5 million judgments on almost 400,000 noun phrases, 13 judgments per item on average) and behavioral studies. I will then discuss other types of disagreement unearthed in NLP annotation project, and the issues this type of data raises for (computational) linguistics.

Next-speaker selection is a fundamental concept in studies of turn-taking in mundane conversation. The concept refers to the practices and mechanisms that participants in a conversation use and rely on to determine who should speak next. While the concept is central in Conversation Analysis and related disciplines, there is little empirical evidence to suggest that next-speaker selection has a psychological base in the sense that selected speakers register their being selected, orient to it, and, as a result, take the next turn. What relevant studies show is a co-occurrence of current speakers using next-speaker selection techniques such as gaze and addressed recipients taking the turns (as opposed to non-addressed recipients not taking the turn). This co-occurrence, however, does not allow us to make assumptions about participants’ intentions and states of mind. That is, speaking of next-speaker selection is a stretch, as, in actual fact, we do not know whether the questioner intends to select the recipient that does answer and we do not know whether the answerer answers because he or she feels selected because he/she was last gazed at. 
In this paper, we aim to fill this gap by drawing on a novel corpus, the Freiburg Multimodal Interaction Corpus (FreMIC). This corpus contains many hours of fine-grained transcriptions of unconstrained conversation and also, based on eyetracking, detailed data on gaze directions as well as large streams of pupillometric data. These are of central interest as changes in pupil size open up a window on processes in the mind, specifically planning activity.
The study focuses on questions in triadic interaction. Questions are an ideal testing ground for next-speaker selection, as asking a question puts pressure on the addressee of the question to respond. The question subset singled out for analysis consists of questions whose addressee was multiply selected in the sense that the answerer was ‘pre-selected’ sequentially and was additionally gaze-selected during the question. The study examines the Steeper Intensification Hypothesis: it predicts a steeper increase in pupil dilation in the selected recipient than in the not-selected recipient. 
The results confirm the hypothesis: pupils in selected recipients dilate whereas pupils in not-selected recipients constrict during the question turn. We take this finding as evidence that speech planning intensifies in the recipient that is selected to respond and that speech planning de-intensifies in the recipient that is not selected.
Overall, the study suggests that next-speaker selection in conversational interaction has a cognitive base: participants orient to selection, synchronizing their speech planning in line with it, either stepping it up when selected or reducing it when seeing another recipient being selected for the next turn.

In this talk I will report on my most recent research project on multilingual encounters in tourism contexts in Croatia. I will present and discuss work in progress and first findings and insights about the importance – or insignificance – of English in international tourism contexts and the many shades in-between.

Croatia is one of the tourism hot spots in Europe and has become increasingly popular with international visitors, especially after parts of the famous HBO series Game of Thrones has been filmed there. I will introduce the country’s linguistic ecology and then zoom in to assess the language use in local tourism interactions. Based on first analyses of two sets of spoken data, i.e. 60 personal, semi-structured interviews and 74 authentic conversations, I discuss language choice and value and the role and use of English as part of multilingual encounters in Croatia’s tourism industry. I find that, although, due to the immense economic relevance of tourism in Croatia, many Croats work in this sector, English is not the default preferred language to be used in interactions with international tourists. Croats seem to be proud to be multilingual and maintain this ideology despite the growing local importance of English, valuing the ideological value of language over its communicative reach.

Since 2015, the increased migration of school-aged children to Germany has sparked heightened national interest in their language skills. Despite the size of this population (presently around 1.5 million students), they have received little attention in studies on language development and education (Marx, 2022).

In our talk, we will present a project that aimed to investigate language skills of immigrated students in German, the language of schooling, and English, a language taught in all schools, and the correlates thereof. Specifically, we gathered data from 344 students in Grades 5-8 on reading skills in German, vocabulary measures in English and German and metalinguistic awareness. We framed this study with two scoping reviews on studying language in newly immigrated students in Germany and with a secondary analysis of data from NEPS (the National Educational Panel Study), demonstrating the currently very limited evidence and database regarding the target population’s language skills.

In the first half of the lecture, we introduce the project and focus on the data from the reading and vocabulary measures as well as results from a comprehensive questionnaire. In the second half of the lecture, we turn to problems encountered when attempting to investigate this target population, looking especially at (1) defining, gaining access to, and determining the population of interest in the field, and (2) finding valid measures to investigate their language skills.

In the last decades, a growing body of research has pointed out the dynamic nature of the mind. To overcome limitations imposed by symbolic approaches, researchers from many disciplines have turned to the framework of dynamical systems describing a multitude of different cognitive processes including the production and perception of speech sounds and their cognitive representations as well as movement coordination.

One potential strength of dynamical systems is that they can handle a high amount of variability, because they do not separate between discrete symbolic representations and the continuous representations of the physical world. I will discuss the application of dynamical systems to capture prominence modulations of the speech systems in German and Persian. This includes the description of intonational and textual variation exhibiting systematic categorical and gradient changes. I will also discuss the application of dynamical systems to impaired speech in German. Speakers aim to compensate for problems of the speech motor system in a multidimensional phonetic space.

This talk will discuss the linguistic background and literary strategies of Deep Wheel Orcadia, a verse novel written in the Orkney language, which won the 2022 Arthur C Clarke Award for science fiction. The Orkney language is a dialect of Scots, itself a minority language: this paper will discuss the reasons for and difficulties in writing in a peripheral tongue, and what this can offer to literature in general. The talk will outline the history of the Orkney language and literature, and the context of Scots language literature more broadly. It will then discuss the problems encountered and decisions made in rendering a largely oral language without an extensive written literature in narrative verse, across orthography, vocabulary and grammar. It will also engage with the strategies used to create an English translation of the text while foregrounding the Orkney original. Finally, the talk will engage questions of science fictional linguistics, and how speculative fiction offers a powerful model for minor literatures.

This article provides a schematic overview of the relevance of deixis in dementia from the perspective of a phenomenological psychopathology with an inter- and transdisciplinary scope. It distinguishes between verbal, corporal and medial deixis in the therapeutic field and discusses their respective importance in the most common dementia, Alzheimer's dementia: Verbal indexicalia have on the one hand a deficit-indicating function for the loss of higher cognitive skills, such as language, orientation and memory, and are relevant for dementia diagnosis and its structural interpretation. The paper argues that on the other hand a) the corporal deixis is an important communicative resource, which helps to create, navigate and maintain attentionality and intentionality by its multimodal character (mimics, gestures and voice) and b) the medial deixis (arrows, signs, markings) is an important creative resource, which supports orientation despite cognitive impairments. The paper argues for more attention to the corporal and medial dimensions of deixis in the context of dementia in particular, and psychiatry and psychopathology in general.