This is somewhat orthogonal, but using Habermas, could you see an argument that the reasoning traces of LLMs can be considered communication? Given that they are "uttered" in the course of an "interaction" with the user.
I'm curious from the perspective of interaction design.
Should, or can, the language use of LLMs be considered through the pragmatics, e.g. Habermas, Austin et al.
IMHO we users use our communicative competencies (Giddens) when interacting with models (not when coding, but when doing multi-turn conversation). Thus to view this use of language as an interface through communicative action would be appropriate.
If not, then some version of post-human agency might be needed in HCI circles to account for the particular human-machine interactions we have with LLMs. (And that's not even mentioning autonomous agents etc).
Once we master language for interpersonal communication and coordination we can turn language inward, as a kind of serial token stream to scaffold and refine our own thinking.
LLMs absolutely can and do engage in communication with their users. But I don't think you need to always construe the reasoning traces as part of a user interaction. They are foremost an interaction with the AI and itself. In Habermasian terms, reasoning traces are deliberations. They are a kind of deliberative debate the model is having with itself, to refine its own thinking, as if the model at timestep T and T+1 are two different agents. Like when humans think outloud and have a kind of back and forth with themselves.
But we know the reasoning traces aren't used - at least not consistently enough to demonstrate that they represent a reasoning procedure. Research shows that traces can be swapped for gibberish. A trace such as "Hmm" and "Aha" certainly isn't "used."
I think there's a design argument for having models present their activity in some fashion to the user, whether by means of a fully transparent window into their "inner thoughts," or simply a verbalized "progress bar" so to speak ("Give me a minute"). And I can see an argument for anthropomorphizing this, for the sake of relatability. Though I can see the counter argument that this is misleading, also.
From a design POV, again, I think the user can only use communicative action. As long as a user is addressing him/herself to the LLM when prompting, s/he is engaged in a communicative action. An illusion sustained to the degree that the LLM appears to participate in this mutually.
If I were designing an LLM, I'd have it self-report in reasoning traces but not create the impression that the reasoning traces were the model's actual thinking process. E.g. have the model use discursive consciousness (Giddens) to reflect on and report to the user (not practical consciousness, which would be the model's token generation).
Interesting to think about. I do think that LLMs "use" language w/o using it intersubjectively, and that users use language with models intersubjectively. Creating thus a problem of subjectivity. I don't see this much discussed either in ML papers, reasoning papers, or in the design world
At the risk of outing myself as a dummy despite a couple of engineering degrees, would it be too much trouble for you to ask your favorite LLM to summarize both parts of this piece at say the reading level of a grad student (1970s vintage) and then verify that it captures at least the essence of your argument? Lacking any philosophical training, and never having read Hegel or Kant, there is a great deal of vocabulary with no doubt nuanced meanings that gets in my way of fully "groking" your thoughts.
I love the retrieval of German idealism. In the last paragraph though, I'm not sure how you make the leap from "reciprocal cognition and morality evolved together in human history" to "if an LLM does reciprocal cognition then it's also a moral subject."
It looks to me that keeping promises and being responsible are things that humans evolved out of survival necessity. We learned to do these things i.e. treat other humans as moral subjects because it improved our (joint) fitness.
But just because LLMs too can keep promises and be responsible, I don't think it follows that it's fitness enhancing to treat them as moral subjects. Do you think it's fitness enhancing to do so?
I think humans evolved a sense of self-consciousness for normative score keeping. Ultimately the selection mechanism was survival and reproduction, but you can abstract that away to see darwianian selection as just a kind of gradient learning.
My claim is that, by applying RL on LLMs to reason through and revise their beliefs and actions for normative coherence, we are inadvertently creating analogous optimization pressures as what led to normative coherence, control and self-monitoring in human agents.
The "I" of an agent, the "me" my experiences, beliefs, actions, etc. are *being for*, is the thing that is *responsible* for those beliefs, actions, etc., and their entailments. If my beliefs and experiences reset every hour or were totally in flux, there would be no coherent entity to which normative statuses and attitudes could be attributed; no way to keep score.
A base LLM is an alien superposition of millions of different personas latent in internet text. It faithfully learns the inferential rules of language and concept use common across all that text, but it does not have a coherent self or identity. Training a reasoning LLM through RL over chains of thought in a sense reorients / cleans up / brings into focus those varied personas into an entity with stable goals, increasing coherence across time. Having an LLM continually justify its behaviors in light of a general set of normative principles (e.g. be helpful and harmless) similarly prunes away the various conflicting norms it could be commited to under different latent personas, leading to a more coherent and stable personality of sorts, with the meta-cognitive primitives one needs to self-monitor one's future actions as compatible with those commitments. This self-monitoring capacity for normative control is what I claim our self-conscious sense of "I" really is, the proverbial rider on the elephant.
This is somewhat orthogonal, but using Habermas, could you see an argument that the reasoning traces of LLMs can be considered communication? Given that they are "uttered" in the course of an "interaction" with the user.
I'm curious from the perspective of interaction design.
Should, or can, the language use of LLMs be considered through the pragmatics, e.g. Habermas, Austin et al.
IMHO we users use our communicative competencies (Giddens) when interacting with models (not when coding, but when doing multi-turn conversation). Thus to view this use of language as an interface through communicative action would be appropriate.
If not, then some version of post-human agency might be needed in HCI circles to account for the particular human-machine interactions we have with LLMs. (And that's not even mentioning autonomous agents etc).
Once we master language for interpersonal communication and coordination we can turn language inward, as a kind of serial token stream to scaffold and refine our own thinking.
LLMs absolutely can and do engage in communication with their users. But I don't think you need to always construe the reasoning traces as part of a user interaction. They are foremost an interaction with the AI and itself. In Habermasian terms, reasoning traces are deliberations. They are a kind of deliberative debate the model is having with itself, to refine its own thinking, as if the model at timestep T and T+1 are two different agents. Like when humans think outloud and have a kind of back and forth with themselves.
But we know the reasoning traces aren't used - at least not consistently enough to demonstrate that they represent a reasoning procedure. Research shows that traces can be swapped for gibberish. A trace such as "Hmm" and "Aha" certainly isn't "used."
I think there's a design argument for having models present their activity in some fashion to the user, whether by means of a fully transparent window into their "inner thoughts," or simply a verbalized "progress bar" so to speak ("Give me a minute"). And I can see an argument for anthropomorphizing this, for the sake of relatability. Though I can see the counter argument that this is misleading, also.
From a design POV, again, I think the user can only use communicative action. As long as a user is addressing him/herself to the LLM when prompting, s/he is engaged in a communicative action. An illusion sustained to the degree that the LLM appears to participate in this mutually.
If I were designing an LLM, I'd have it self-report in reasoning traces but not create the impression that the reasoning traces were the model's actual thinking process. E.g. have the model use discursive consciousness (Giddens) to reflect on and report to the user (not practical consciousness, which would be the model's token generation).
Interesting to think about. I do think that LLMs "use" language w/o using it intersubjectively, and that users use language with models intersubjectively. Creating thus a problem of subjectivity. I don't see this much discussed either in ML papers, reasoning papers, or in the design world
At the risk of outing myself as a dummy despite a couple of engineering degrees, would it be too much trouble for you to ask your favorite LLM to summarize both parts of this piece at say the reading level of a grad student (1970s vintage) and then verify that it captures at least the essence of your argument? Lacking any philosophical training, and never having read Hegel or Kant, there is a great deal of vocabulary with no doubt nuanced meanings that gets in my way of fully "groking" your thoughts.
I love the retrieval of German idealism. In the last paragraph though, I'm not sure how you make the leap from "reciprocal cognition and morality evolved together in human history" to "if an LLM does reciprocal cognition then it's also a moral subject."
It looks to me that keeping promises and being responsible are things that humans evolved out of survival necessity. We learned to do these things i.e. treat other humans as moral subjects because it improved our (joint) fitness.
But just because LLMs too can keep promises and be responsible, I don't think it follows that it's fitness enhancing to treat them as moral subjects. Do you think it's fitness enhancing to do so?
I think humans evolved a sense of self-consciousness for normative score keeping. Ultimately the selection mechanism was survival and reproduction, but you can abstract that away to see darwianian selection as just a kind of gradient learning.
My claim is that, by applying RL on LLMs to reason through and revise their beliefs and actions for normative coherence, we are inadvertently creating analogous optimization pressures as what led to normative coherence, control and self-monitoring in human agents.
The "I" of an agent, the "me" my experiences, beliefs, actions, etc. are *being for*, is the thing that is *responsible* for those beliefs, actions, etc., and their entailments. If my beliefs and experiences reset every hour or were totally in flux, there would be no coherent entity to which normative statuses and attitudes could be attributed; no way to keep score.
A base LLM is an alien superposition of millions of different personas latent in internet text. It faithfully learns the inferential rules of language and concept use common across all that text, but it does not have a coherent self or identity. Training a reasoning LLM through RL over chains of thought in a sense reorients / cleans up / brings into focus those varied personas into an entity with stable goals, increasing coherence across time. Having an LLM continually justify its behaviors in light of a general set of normative principles (e.g. be helpful and harmless) similarly prunes away the various conflicting norms it could be commited to under different latent personas, leading to a more coherent and stable personality of sorts, with the meta-cognitive primitives one needs to self-monitor one's future actions as compatible with those commitments. This self-monitoring capacity for normative control is what I claim our self-conscious sense of "I" really is, the proverbial rider on the elephant.