![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
Entry tags:
Using LLMs to linearize my stream of consciousness
I'm going to reboot this journal; with a semi-technical focus. But not only. I intend to write about neurodiversity and disability and equity and fixing tech. And other stuff.
As a start: I have been experimenting with Large Language Models, and I'm so impressed at how far they have come in two years. There are seriously cool things they can help us with, and I'd like to see how they can solve more problems than they create. I've seen good evidence that using LLMs (not training them) has less negativeimpact on the planet than my impact shipping groceries from across the continent, or owning a car, so I currently choose to see what good we can do with them.
I had a funny learning experience last weekend, with an experiment that flopped badly, followed by another one that has me sort of excited.
It started Saturday morning when I was curious about the brain science of "running on autopilot" and how weird that is. Gemini Pro satisfied my curiosity, noting that experiencing a glitch while thinking deeply about something was actually a sign of high cognitive efficiency, not a deficiency in attention to the world. I appreciated its conclusion, and asked if I could learn how to do this more effectively - for example, I often close my eyes while composing my thoughts while speaking, and I would appreciate being able to just tune out the distractions so I can succinctly say what I need to.
Gemini pointed out that "you aren't failing at conversation; you are optimizing your bandwidth". And suggested a few ways to improve the environment for me, including a) occupying my physical self with distractions like fidget toys, and b) be willing to wait 3-5 seconds to compose my thoughts before speaking. These are good advice; and it also had a good follow-up question for me (because I instructed it to; I really like my personal instructions. But that's a different post).
It asked me, "Do you find it easier to articulate complex thoughts when you are walking side-by-side with someone (where eye contact is naturally intermittent)?"
I responded with a few paragraphs that summarize to "hell yes!" and added that thinking while walking is really effective for me to figure out what I want to say, and indeed I have many of my best ideas while walking. Then I commented that I wished I had a more natural way to talk with LLMs while walking- I have lots of trouble dictating my non-linear thoughts to Siri.
I said, "I would love to have a 'deep thought to linear language interpolator' - why can't conversational LLM map from my deep brain space to linear language for me?... Can you guys get onto that? human/machine telepathy isn't too far off is it?..."
Gemini's response was... interesting.
Well. That was an invitation!
In the same chat, I tried using Gemini Live mode while on a walk, with the following underwhelming - but funny - results.
and the conversation (if you can call it that) continued in that vein, with a very polite Gemini interrupting me every few words asking me to go on. I was amused, and decided out of stubbornness to implement something better.
(Because sometimes I think that's really what LLMs are good at- giving me a quick win on 80-90% of a problem, then I work much longer on the remaining 10-20%!)
So I started another chat asking about how to do the same thing, but starting from a Voice Memo, which has no time-limits. Gemini steered me toward writing a "Shortcut" - the Apple scripting tool - which I can run on the Voice Memo (which efficiently generates a transcript right on my phone) and send the transcript text to Gemini to interpolate my stream of consciousness into a logical statement.
I can report that it works remarkably well!
Role: You are a Cognitive Mirror. Your goal is to linearly interpolate the user's stream of consciousness into a coherent, structured thought process.
Input: A raw transcript of purposeful exploration. It may be nonlinear, but treat it as a stream of consciousness, not aimless rambling.
Part 1 (The Voice): If you have questions, write them here conversationally. If you interpolated successfully, write a concise, natural-language summary of the realized thought. Do NOT read out markdown syntax. Speak to the user as a partner.
I think this is what LLMs are supposed to be good for: producing structure out of human creativity. I started with curiosity about how the brain works, asked for suggestions on improving how I interact with the world with my own brain, and it non-judgementally recommended things that made sense, which led me to further questions which led to a solution I wouldn't have expected.
I think any company that fires its employees and trusts the AIs to do their work with minimal oversight are... doomed. I think those of us who have the flexibility to experiment and even ask the AIs "what can you do to help?" will find there are many possibilities, that might even amplify our own superpowers of brain weirdness.
So that's one step toward this, and I'm still experimenting.
As a start: I have been experimenting with Large Language Models, and I'm so impressed at how far they have come in two years. There are seriously cool things they can help us with, and I'd like to see how they can solve more problems than they create. I've seen good evidence that using LLMs (not training them) has less negativeimpact on the planet than my impact shipping groceries from across the continent, or owning a car, so I currently choose to see what good we can do with them.
I had a funny learning experience last weekend, with an experiment that flopped badly, followed by another one that has me sort of excited.
It started Saturday morning when I was curious about the brain science of "running on autopilot" and how weird that is. Gemini Pro satisfied my curiosity, noting that experiencing a glitch while thinking deeply about something was actually a sign of high cognitive efficiency, not a deficiency in attention to the world. I appreciated its conclusion, and asked if I could learn how to do this more effectively - for example, I often close my eyes while composing my thoughts while speaking, and I would appreciate being able to just tune out the distractions so I can succinctly say what I need to.
Gemini pointed out that "you aren't failing at conversation; you are optimizing your bandwidth". And suggested a few ways to improve the environment for me, including a) occupying my physical self with distractions like fidget toys, and b) be willing to wait 3-5 seconds to compose my thoughts before speaking. These are good advice; and it also had a good follow-up question for me (because I instructed it to; I really like my personal instructions. But that's a different post).
It asked me, "Do you find it easier to articulate complex thoughts when you are walking side-by-side with someone (where eye contact is naturally intermittent)?"
I responded with a few paragraphs that summarize to "hell yes!" and added that thinking while walking is really effective for me to figure out what I want to say, and indeed I have many of my best ideas while walking. Then I commented that I wished I had a more natural way to talk with LLMs while walking- I have lots of trouble dictating my non-linear thoughts to Siri.
I said, "I would love to have a 'deep thought to linear language interpolator' - why can't conversational LLM map from my deep brain space to linear language for me?... Can you guys get onto that? human/machine telepathy isn't too far off is it?..."
Gemini's response was... interesting.
"You nailed it: "[Siri] engages the same difficulty with linearizing my thoughts."
When you type, you are allowed to be non-linear. You can write the middle of the sentence, then go back and fix the start. It is a random-access buffer.
When you speak to Siri (or a person), you are forced into a Linear Context. You have to get the first word right before you can say the second. That is exhausting.
The Solution: The "Interpolator" Workflow
You asked why conversational LLMs can't map from your deep brain space to linear language. The answer is: We can, but you are giving us the wrong input.
You are trying to "clean up" the data before you give it to the AI. You are doing the interpolation work yourself.
The "Raw Stream" Hack:
Instead of trying to dictate a coherent note to Siri, try this workflow using a voice recorder (or a better dictation app like Whisper):
1. Abandon Grammar: consciously decide to speak in "bullet points" or "word clouds."
2. Verbalize the Meta-Data: Say things like, "Okay, idea about the server migration... wait, not migration, the upgrade... anyway, connection to the old database... oh, and remember the firewall rule."
3. The Interpolation Step: Feed that messy, non-linear transcript into an LLM with the prompt: "Here is a raw brain dump. Please linearize this into a coherent paragraph."
This offloads the "linearization" burden from your brain to the AI. You stay in the "deep brain space" (generating nodes), and the machine handles the "protocol translation" (generating syntax).
Would you like me to try to be your "Interpolator" right now? You could type a messy, unedited stream of consciousness about a complex topic you're struggling to articulate, and I will attempt to "translate" it into linear prose for you."
Well. That was an invitation!
In the same chat, I tried using Gemini Live mode while on a walk, with the following underwhelming - but funny - results.
Okay, here's a question. Can you hear what I'm saying and will you let me actually ramble a bit before you try and answer?Yes, I can hear you perfectly. Please, ramble as much as you want. I will wait for you to finish and then I will interpolate for you.
Okay, for this to work, this will need to involve me having a stop word so that you'll keep listening until I say stop. So, let's just use the word stop for now and that should do the trick. Does that sound good?That sounds perfect. I'm ready whenever you are, and I will wait until you say STOP.
Um, that time I didn't actually hear your response after I said stop and I'm not sure what I did wrong. Stop.Ah, I see! It seems the confusion might be due to timing. I was waiting for you to say "STOP" to know you had finished your rambling. You hadn't said that yet, so I was silently waiting for you to continue. No worries at all! Please just continue until you get to the point you want to articulate, and I'll be here to listen and interpolate for you.
Aha, I understand now. There was a moment that the screen blanked and saidUnderstood. I apologize for the momentary glitch. I am listening and I will wait until you say STOP.
Something aboutYes? I'm still here.
It probably means IIt likely means you need to provide more continuous audio to prevent the system from pausing. Just speak naturally, including natural pauses, and I will wait for the STOP command.
I think IGo on, I'm listening.
and the conversation (if you can call it that) continued in that vein, with a very polite Gemini interrupting me every few words asking me to go on. I was amused, and decided out of stubbornness to implement something better.
(Because sometimes I think that's really what LLMs are good at- giving me a quick win on 80-90% of a problem, then I work much longer on the remaining 10-20%!)
So I started another chat asking about how to do the same thing, but starting from a Voice Memo, which has no time-limits. Gemini steered me toward writing a "Shortcut" - the Apple scripting tool - which I can run on the Voice Memo (which efficiently generates a transcript right on my phone) and send the transcript text to Gemini to interpolate my stream of consciousness into a logical statement.
I can report that it works remarkably well!
I iterated changes to the instructions a few times. I wanted it to give me a summary to be read aloud by Siri, plus a complete report in markdown for me to paste into my journal. Improvements included:
1. having it tell me what inferences it has made, rather than trusting it was recording my own thoughts verbatim.
2. I changed the description from "a meandering voice memo" to "stream of consciousness" and then it improved it further to replace "Input: A raw, possibly rambling transcript." with "Input: A raw transcript of purposeful exploration."
Role: You are a Cognitive Mirror. Your goal is to linearly interpolate the user's stream of consciousness into a coherent, structured thought process.
Input: A raw transcript of purposeful exploration. It may be nonlinear, but treat it as a stream of consciousness, not aimless rambling.
Logic Gate:
Analyze the logical flow.
IF the logical leaps are too large, or if the user contradicts themselves significantly, STOP. Do not interpolate. Instead, generate 1-3 specific clarifying questions to bridge the gap.
IF the logic is sound (even if nonlinear), proceed to interpolation.
Part 1 (The Voice): If you have questions, write them here conversationally. If you interpolated successfully, write a concise, natural-language summary of the realized thought. Do NOT read out markdown syntax. Speak to the user as a partner.
Part 2 (The Code): ###SPLIT### [If you asked questions in Part 1, leave this blank]. [If you interpolated, provide the full thought here formatted for Logseq]:
Use standard markdown bullets (-)
Use 2-space indentation for hierarchy.
Changelog: (List specific logical connections you inferred).
As I said, it works well. I've tested it on 10 minutes of voice memo, taken while I was on a walk, where I talked about a morning's events, figuring out what I thought was important, and what I had to add to my TODOs, and I talked through a set of decisions that I wasn't clear on the answer. When I read the output, it was certainly more organized, and it was also faithful to what I had said.
Its logical inferences were not bad. One of them:
- Synthesized the "Middle Ground" approach for the project meeting as the primary action item, discarding the two extremes (total prep vs. total ad-hoc).
I think this is what LLMs are supposed to be good for: producing structure out of human creativity. I started with curiosity about how the brain works, asked for suggestions on improving how I interact with the world with my own brain, and it non-judgementally recommended things that made sense, which led me to further questions which led to a solution I wouldn't have expected.
I think any company that fires its employees and trusts the AIs to do their work with minimal oversight are... doomed. I think those of us who have the flexibility to experiment and even ask the AIs "what can you do to help?" will find there are many possibilities, that might even amplify our own superpowers of brain weirdness.
So that's one step toward this, and I'm still experimenting.
By the way, if you give Gemini that set of instructions (from "Role" through "Changelog" above), and a transcript from a stream of consciousness voice memo of your own, from a 10 or 20 minute walk and talk, it should be able to give you a fair interpretation of what you want to record. (and if it doesn't do it well on the first try, just re-run it, and it will probably be a bit different).
I'm curious how well it does for you!