ORCID
0000-0003-1910-8030
Date of Award
2025
Document Type
Dissertation
Degree Name
Doctor of Philosophy (PhD)
Department
Complex Systems and Data Science
First Advisor
Peter S. Dodds
Second Advisor
Christopher M. Danforth
Abstract
In this thesis, we explore language and cognition in both people and in computational models, through the lens of meaning construction at both the individual and collective level. We step through levels of linguistic structure and large language model (LLM) architecture --- from morphemes to stories --- in order to (begin to) build an understanding of the role these concepts play in human and machine cognition, and, in particular, in meaning construction and symbol grounding.
In Chapter~\ref{section:chapter1} and Chapter~\ref{section:chapter2}, we interrogate the interface between LLMs and the world, since it is this interface that facilitates the creation of \textit{idios kosmos} from \textit{koinos kosmos}. Specifically, we consider the role tokens --- the smallest units of textual structure accessible to LLM architecture --- play in LLM cognition. We argue that aspects of tokenization, including its objective function which is arguably meaningfully insulated from the main intelligence, can impact the system's cognition. We show that different kinds of words behave in different ways as tokens. We relate evidence that syntax, semantics, frequency or context, and formatting can be reflected in token content for LLMs. Based on a synthesis of our findings and previous results, we argue that relation is the fundamental mechanism for linguistic meaning construction.
In Chapter~\ref{section:chapter3}, we extrapolate from the results of the first two chapters to connect the mechanism of relation (metaphor) to two omnipresent aspects of cognition, locality and salience. We consider a variety of contexts in order to distill these concepts as omnipresent across different scales of biological and non-biological information technologies. To that end, we introduce and exploit the concepts of gnogeography and salioscape. Gnogeography is the sphere of ideas --- roughly, the representations, abstractions, models, facts, propositions, and descriptions of perceptual input --- that is accessible from the perspective of a being, given its experience and its architecture. Salioscape extends this idea with explicit embodiment. A context we consider in detail is the story: in Section~\ref{chapter3:characterspace}, we present our framework of 6 archetypes to describe fictional characters. This framework aligns with our previous results that elucidate power as a fundamental component of word meaning, which we use in our nascent theory of technology.
In the conclusion, we present high-level takeaways on cognition, language, and the future of learning machines as well as the science around them, based on the work put forth in the previous chapters.
Relatively tangible results include: (1) a framework for understanding the role tokens play in LLM cognition, anchored in the Distributional Hypothesis (DH); (2) evidence that the tokenization objective function can impact the system's cognition; (3) evidence that different kinds of words behave in different ways as tokens; (4) evidence of what information can be reflected in tokens, e.g. that information falling outside what might conventionally be considered word meaning can directly impact token content; (5) implications of (1-4) plus other key architectural choices for LLM cognition and its use of language (`flattening'); (6) an argument that LLMs demonstrate for the first time the non-trivial separability of something like a minimally-viable form of language from other cognitive processes (that is, that thinking and speaking can be significantly de-coupled), and that they validate and isolate the DH; (7) a framework for describing fictional characters in terms of 6 basic archetypes; (8) evidence for a connection between components of meaning across different linguistic scales; (9) the beginnings of a theory of technology in terms of its impact on problem-solving through the increase of power; (10) a theory of meaning construction in language; the theory that relation and locality are the fundamental components of problem-solving (relation being the core mechanism for subverting locality, the core constraint); (11) proposals for future work.
Language
en
Number of Pages
638 p.
Recommended Citation
Zimmerman, Julia Witte, "Locality, Relation, and Meaning Construction in Language, as Implemented in Humans and Large Language Models (LLMs)" (2025). Graduate College Dissertations and Theses. 2082.
https://scholarworks.uvm.edu/graddis/2082
