Improvising with Generative AI
What happens when the machine Yes-Ands you back
Many of my art friends seem pessimistic about what generative AI will do to artists and the world. But I’m optimistic that it will be a net positive for creativity in the world. So I’ve been keeping an eye out for examples to highlight that possibility.
One popped up this morning while I listened to Ezra Klein interview Adam Moss (see here for transcript). Adam has a new book out called “The Work of Art: How Something Comes From Nothing.” But really, as they clarify in the conversation, the book could have been called “Editing” or “The Work of Editing”.
At one point, near the end, Ezra asks Adam what he thinks it means that most of the artists he profiled in the book use primarily analog means of editing - often pen and paper. Adam pushes back on this and points to a chapter on a generative artist named Tyler Hobbs.
ADAM MOSS: Well it’s not entirely right. There’s a whole chapter that’s the exception to that, which is about a guy named Tyler Hobbs, who’s a generative artist. And why I liked that chapter so much is that the machine is super important. It becomes the hand. He creates algorithms, gives it to the computer. The computer spits stuff out. He reacts to it, the same way that a painter might react to whatever they’ve painted with their hands. And then he changes the algorithm and just keeps going that way.
EZRA KLEIN: Well, he’s, in a way, become the editor of the machine —
ADAM MOSS: Absolutely.
EZRA KLEIN: — which I think I have a thing that A.I. is going to turn us much more into editors, because we’re going to have to know if the thing it is spitting at us is correct.
ADAM MOSS: Yeah, and evaluate it, yes.
I felt myself wanting to butt into the conversation and state the obvious. We’ll do more than evaluate the AI’s response, right? We’ll respond to the response! And when we respond to it we edit it — either directly (through correcting it) or indirectly (by saying something that will shape its subsequent responses).
Their conversation about editing (broadly construed) got me thinking about my old life as an improviser1 and my current hobby of talking to LLMs. Both involve a form of dialectical editing.
The more I think about it, the more I see a similarity between improv and conversing with generative AI. Sometimes, when I use generative technologies like LLMs, it starts to feel like I’m improvising. But instead of improvising with an individual human being it’s like improvising with a part of the written record of humanity.
I don’t think conversing or improvising with a GPT is a substitute for the magic of talking to a person with all their idiosyncrasies and in-the-moment aliveness. But that doesn’t mean there isn’t also something magical about improvising or conversing live with a monstrous and minuscule avatar of much of what has come before us. There’s something magical about both of these forms of improvising.
When I send a message to an LLM I have the same sensation from improv (and conversation) of making an offer and receiving a yes-and from the LLM.
And then I yes-and it back, and the cycle continues.
Yes-and with GPTs
When yes-and is referred to colloquially, it’s often oversimplified to focus more on the “yes” than the “and.”
But there's a deeper version of what happens when we yes-and (in improv and life). We iteratively point to the subset of the information space (dare I say, the Ruliad) that we want to explore with our interlocutor.
We do the same thing when talking to GPTs. In turn, the GPT responds with a subset of the Ruliad it thinks is most relevant based on the series of micro-associations our response raised for it.
For GPTs, these associations constitute their very being, but you might say the same is true for us.
It’s next token prediction all the way down.
I’ve found working with generative technologies has a similar beauty to improv that I rarely experienced working with previous non-generative technology.2
It's a new form of distributed cognition, an affordance that gives us instant access to a more dialectical way of thinking.
It's a whole new way to play with technology.
For seven years, through most of my 20s, improv was my main hobby. I went semi-pro and even performed improvised long-form (hour-long) musicals for a few years.
Although, certainly many previous forms of technology do indeed respond to our actions in more subtle ways. A guitar string vibrates one way or another depending on your choices, and you learn what sounds to create accordingly. Improvising guitar solos as a teenager gave me a similar thrill to improv (and conversation more broadly) at its best. But as far as solo-activities go generative AI amplifies and expands this dialectical feedback loop.