Sam Bleckley | Writing

The Consolation of Discernment

2025-02-21T00:00:00-05:00

Today I’m thinking about discernment.

In 1521 Íñigo López de Oñaz y Loyola, successful soldier and man-about-town, is finally injured badly in battle: his leg is fractured terribly by a cannonball. Sitting in bed, he daydreams about his future life. All the daydreams are fun and engaging — why daydream about boredom and unhappiness? — but sometimes his fantasy ends and he’s left engaged in the rest of the day enlivened and full of energy, and other times he’s left restless and hollow. After some experimentation and interior observation, he calls the former feeling consolation and the latter desolation; and finds they apply to all sorts of thoughts and actions, memories and plans. He gives the name discernment to the process of paying attention to which things result in consolation and which result in desolation.

Íñigo goes on to have holy visions and attribute consolation to the spirit of god and desolation to spiritual evil. He redirects his life and is eventually remembered as St Ignatius of Loyola, founder of the Jesuit order.

Those additional steps are culturally contextual; feel free to take them if they appeal to you, but they’re not necessary in order to make that initial observation: some things that are nice while they occur leave me feeling empty and hollow afterward, and some things leave me energized and full of delight even after they’re done. Discernment is a functional mental tool outside any religious context.

When my life is going well, and my daily actions are mostly part of larger plans, I don’t feel much need for discernment. I’m taking step 17 of 25 on my way to a goal, and perfectly satisfied. It is when life is somewhat rougher, when there is no plan, that discernment comes into its own. At moments when most actions feel like they will be mostly futile — when it’s unclear if there is any way to achieve worthwhile goals — discernment still functions very effectively at picking one course of action over another. Discernment ignores the ends and judges the means based on what they do internally rather than externally. Even if you’re Bill Murray in Groundhog Day and all your efforts will be washed away like sandcastles, consolation and desolation can provide some kind of compass.

(There is some relation to global motivations here, though discernment is nearly useless for storytelling purposes or team morale.)

The Inefficient Courtroom

2025-02-19T00:00:00-05:00

Today I’m thinking about trial by jury and efficiency.

In a courtroom, everyone has a specific role. Each lawyer is supposed to make the most effective and compelling argument they can, regardless of who they imagine to be guilty. The judge is supposed to constrain those arguments to the law, regardless of who they imagine to be guilty. Outside the courtroom, there’s an entire representative democracy making that law, which must apply to everyone, regardless of who the lawmakers imagine to be guilty. The jury is supposed to assess those arguments and come to a conclusion about the arguments, not the people involved. The stenographer is recording the proceeding so it can be vetted, and the decision appealed if anyone hasn’t done their job.

In reality, there are plenty of situations where the judge alone makes a (ahem) judgment call; but in the perfect vision of a jury trial, no one person makes the entire moral decision of right or wrong, guilty or not guilty, legal or illegal. Everyone has some smaller, less subjective job to do, and the final verdict is the sum of all those jobs without being exactly equal to any of them.

In this way, the court is a Chinese room. It is a system for making those decisions that is, in principle, more resilient than any single person, and indeed in spite of any single person. A person can be corrupt, or lazy, or biased; and even if it were humanly possible for a perfect judge to exist, undying, and able to preside over every dispute, our society is varied enough that there is no single ideal of moral judgment for that person to live up to.

A courtroom is, for that reason, much less efficient than a single person making decisions! A court is slow and plodding and expensive and resource-intensive. Robustness is redundancy, and redundancy is expensive.

I think this is a feature common among almost all long-lasting positive institutions; they are not efficient, they are robust. They work slowly and change slowly, and that slowness is, if not a feature in itself, a direct result of the most important feature.

Chesterton’s Fence urges us to understand the purpose of what we’re replacing but I think it is easy to look at slow robust systems and say “I understand the input and output of this process, and I can do it much faster and cheaper!” The robustness of social systems is a secret, hidden feature because the corruption and conflict it counteracts is rarely visible — because it’s ineffective, it goes away on its own! But if you remove that robustness, the corruption will return the moment your back is turned, and recreating the slow, plodding, but robust system is far harder than tearing it down — especially when it was built over centuries of real-world learning.

One Song in Your Musical Is Wrong

2025-01-31T00:00:00-05:00

I. Beauty and the Beast (1991)

Many musicals have an “I Wish” song — a song that establishes early on what the protagonist feels is missing from their life, a desire the story will both fulfill and complicate.

In Disney’s animated Beauty and the Beast, Belle’s “I Wish” song is very short and says, roughly: “I don’t want to be a little wife; I want adventure in the wider world; I want much more than is planned for me.” She releases dandelion seeds into the wind over a sun-drenched vista.

Belle’s story: she gets locked in the darl castle down the street, and eventually marries the guy she finds there.

What is most fascinating, to me, about that contrast is that there’s no need to change the story. The whole film would work fine if you swap out those 30 seconds of song, instead. She could wish for a quiet place to read, someone to talk to, and to not be mocked for her imagination — that would work with all the other evidence! And it would work in the “oh, getting my wish is not making me content” way, at first, too!

Instead, she sings an “I Wish” song that might reasonably suit the story of The Little Mermaid or Mulan — a story with travel and adventure in foreign lands.

II. Wicked

Many musicals have a show-stopping number involving a character girding their loins to make an important choice. In Wicked, that number is Defying Gravity. But as much as Defying Gravity is a banger, it always felt thematically unearned to me. What does gravity, or flying, have to do with the emotional journey of any of the characters? What does the metaphor built by the lyrics have to do with the internal decision Elphaba is making?

It’s the right moment in the musical for a song like that — the emotion is earned — but the actual content of the song is more related to this tiny bit of plot machinery (a spell that makes things fly) instead of any central theme or aspect of character.

The concept of defying gravity absolutely could have thematic resonance: she’s defying the laws of physics AND the laws of man. So what could you change leading up to it to make that connection work?

You could make two changes:

make Elphaba extremely rules-bound and Glinda “rules are for other people”; let that drive their conflict
and make flying a forbidden branch of magic

In that adjusted story, Elphaba seeing the cruelty of the laws towards non-humans, deciding to be an outlaw, AND experimenting with flight all thematically act together, and then whether or not to “Defy Gravity” can effectively represent the choice both characters are making.

Admittedly, equating “defying the laws of physics” with “defying the rules” could cause other problems, given that the head of state is A Wizard.

So, alternately, rather than change the story to suit the song, how could the song be adapted to better connect to the story given the themes that are already there?

The first act is almost entirely about perception: being seen as good vs being good; being seen as strong, or beautiful, or liked, or beloved vs actually being any of those things.

A physics metaphor about giving up on being perceived as something? That’s easy! Try one of these:

“I think I’ll try superposition”
“I think I’ll try quantum coherence”
“I think I’ll exist in several eigenstates”

The scansion and prosody of these options are, frankly, unassailable, and the language widely accessible. So that’s how I’d fix Wicked’s hit song and my version would be way more relatable and popular than the dud they’ve got, which no one likes.

III. Sheesh, Sam! What are you talking about?

I think a lot of my acquaintances think of me as a critical and negative person because I tend to think and talk about things I think don’t work, even when they’re a small detail in an otherwise successful whole (one short song, or even one lyric, in a popular and critically acclaimed musical, to pick an example at random.)

That’s not because I want to tear others down, though: I do exactly the same to my own work, both the technical and the artistic! Nor is it because I am a perfectionist — I am happy to release “good enough” into the world and work from there. My nitpickiness comes from just this: to me, how things fail is the most interesting part. That is the meat worth chewing on.

There are sometimes things to be learned in analyzing something that works well, to see how it works; but most often what you discover in the analysis is “it works because it follows guiding principles I already know.”

There is always a practical lesson available in examining something that doesn’t work, and investigating how it could be improved; because even when you discover that it is most easily improved by making it adhere to the principles you already know, you get the practice of mentally doing that work.

This urge to pick at the scabs of any undertaking has, in many ways, led to my current approach to consulting. I rarely accept a contract longer than 6 months. Why? Because while your teams and processes will grow and improve under my constant prodding at even small dysfunctions, they will not stabilize. To make a beautiful bonsai, you train and prune it aggressively and then leave it alone. I am constitutionally very good at the former and poor at the latter; so I try to provide the nitpicking as a service, rather than fight my instincts in order to effectively serve a long-term placement.

Still, it is good for me, sometimes, to play against type and spend some time admiring what works. If you’re my opposite and tend to shy away from analyzing smaller losses inside larger successes, let’s trade for a bit: I’ll set my nitpicks aside for a bit, and you try to see the failures as the most interesting part.

Why I Rewrote a Century-Old Artist’s Anatomy Book

2025-01-01T00:00:00-05:00

(If you’re not interested in the how or why of this project, and just want the result, it’s available here.)

Bridgman’s Constructive Anatomy is a classic study text for artists that has been in print for over a century. It’s time-tested and artist-proven: comic book artists, illustrators, and fine artists all study Bridgman. Other anatomy books may be more complete, more technical, more clear, or more accurate — but Bridgman’s drawings look like art. Any attempt to improve or correct Bridgman would be foolish! An act of hubris!

Last year, I set out to improve and correct Bridgman.

In my defense, this project didn’t start with that goal. I only wanted to do a more in-depth study of Bridgman for myself. Bridgman writes in a way that’s both dated and pretty convoluted. So, to prove to myself that I was truly digging in and understanding it, I decided to rewrite the text, paragraph by paragraph, into modern words. English-to-English translation: how hard could it be?

(Unavoidably, complications are sure to follow. This is a story of scope creep, as much as it is about writing or drawing.)

Almost as soon as I began, I discovered that some of the texts challenges were more subtle than the individual sentences being difficult to parse.

The Cryptic

Take, for example, the case of wedging. Bridgman has this to say, on the very first page of his text:

Masses of about the same size or proportion are conceived not as masses, but as one mass; those of different proportions, in respect to their movement, are conceived as wedging into each other, or as morticed or interlocking.

The effective conception is that of wedging.

Throughout the rest of the work, he will point out when forms should be thought of as wedging. Bridgman does not, however, ever define the word; he never illustrates the difference between two masses wedging into each other and any other form of contact.

I studied every time Bridgman used the word, both in this text and in others. I turned to other artists’ attempts at decoding the word. In the end, I came up with an analysis that I believe is effective, and that’s plausibly what Bridgman meant by the word, but there’s simply not enough evidence for my multi-page description, with new illustrations, to feel like a translation.

I hadn’t even gotten through Bridgman’s half-page introduction and already I was forced to think about how to distinguish between his ideas (if rephrased in modern words) and my own additions and analysis. This was the first hint that I would not end this project with a plain text file of notes.

Layout, Laid Out

Some problems with Constructive Anatomy are due to the constraints of affordable bookmaking in the 1920’s: none of the illustrations are in line with the text, and of course, they’re all in black and white. But other issues are deeper and more structural. Bridgman is miserly with his explanations: he rarely explicitly illustrates an idea if he’s described it, and doesn’t always describe an idea if he’s illustrated it. When the text does directly refer to an illustration, they are almost never found on the same page.

I couldn’t just rewrite the text and get anywhere. I had to connect the illustrations to the text, and (where necessary) make their labels easier to follow. I spent a week slicing out each of the 500+ illustrations and re-sorting them based on what they were attempting to show. This was another judgment call; the original text alternates pages of text with pages of dozens of uncaptioned drawings. It’s very likely many of those drawings were intended to demonstrate more than one idea. Moving them in line with the text makes it easier to connect them to one idea, but also easier to ignore any other lessons they might be trying to teach.

Due to the order he visits the body parts, Bridgman is forced to repeat himself to avoid using anatomical names he hasn’t yet defined. Once I was bought in enough to move all the illustrations, reordering the sections seemed obvious and necessary.

It was probably around this point that I started to consider the fact that George Bridgman died over 70 years ago, and his work is in the public domain, now: I could, legally, publish the result of all this work.

A Product of His Time

Other challenges are due to the social era: Bridgman’s anatomical vocabulary is sometimes outdated, and a few times completely incorrect (to be honest, I remain uncertain where he thought the brachioradialis or levator scapulae belonged; eventually I gave up trying to understand and put them in the correct places). His terminology is old-fashioned, and his anatomical analysis is, at least on one notable occasion, overtly racist.

Some important bits are completely missing. He ignores the female figure entirely – there are feminine faces and hands, but no feminine bodies appear anywhere in the entire book. In a course on anatomy for fine artists, he never mentions the breast. Some of his other books at least include a few female nudes, and I was able to pull in some of them to fill in gaps; but any discussion of how to make a body more stereotypically feminine or masculine in appearance, I had to write from scratch.

Wait, Is this product design? Again?

As I worked, I began asking other artists about their experiences with Bridgman. I started approaching the project less like a student and more like a product designer. When someone mentioned that their eyes glazed over at the Latin names, I started adding silly mnemonics to help make them easier to hang on to. I adjusted my color palette to be colorblind-friendly and began thinking about how the colors functioned across multiple illustrations, rather than just within one. I thought long and hard about how far I could go while remaining true to the meaning and spirit of Bridgman’s original curriculum.

Bridgman fills an important role, not because he’s the most technically accurate or comprehensive resource — quite the opposite! — but because he makes explicit aesthetic choices about how he wants to draw the human body. Studying his work is only partially about gaining the knowledge of a med student, and much more about discovering and showing your own sense of beauty in the same set of parts. His text has many insights about how to think about (and draw) the human body, and I’ve tried to preserve and expose those insights. But there are lots of texts out there that will name the muscles and bones and visible landmarks. Bridgman is special because his drawings are beautiful, rather than just illustrative.

The more I honed in on the experience I wanted people to have when studying Bridgman, the easier it was to make difficult decisions about what to keep and what could be left in the past.

It’s a book!

I’m proud to say that the result of all this work is now available for purchase, in a luxurious studio reference edition, with full color where necessary and big, note-taking-friendly margins. You can pick up a copy here:

Bridgman’s Constructive Anatomy:
A Modern Revision on Amazon

It’s (not) a Living

“Wow, Sam.” I hear you say. “Is rewriting and improving public-domain texts a way to earn a living? A passive income?”

No. This project is turning out to be more successful than I expected; I’ve sold one or two copies a day since I launched it. There was no need for the complicated pre-launch strategies a normal book needs because people are already searching for (and buying) “Bridgman’s Constructive Anatomy.” Because the text is in the public domain, it’s not hard to siphon off some of that traffic. There are a handful of people doing so (less successfully) without having even bothered with improving the text.

But it’s a beer money project. If no one bought the original Constructive Anatomy and they all bought mine instead — complete market saturation — I would still net less than $5000 a year. And even if I wanted to repeat the trick with a different book, to double up that payday, there just aren’t that many books in a similar position: public domain, relatively well-known, and deeply flawed in ways I have the skills to recognize and correct.

That’s not a disappointment — I’ve known that this wasn’t a money-maker from the start — but I’ve seen a few people get an entreprenurial glint in their eye as I’ve talked about this project. Keep your feet on the ground.

Appendix: Runes

Almost every illustration from the original book made it into my revision. A number of illustrations from Bridgman’s other texts also made it in and I created a few dozen new illustrations to fill in gaps. A handful of originals, however, didn’t make the cut. Here are a few Bridgman doodles that, even after many months of extensive study, I remain unable to interpret, at least in terms of what lesson they might convey to a student of art or anatomy.

On Tools

2024-11-19T00:00:00-05:00

There are names “tools” many different contexts; there are tools for physical work, tools for mental work, for office work, for technical work, and for teamwork. There are so many things in our lives called “tools” that damaging ideas sometimes sneak in wearing a tool-shaped trenchcoat.

To be a real tool, it must accomplish one of these three transformations:

Transform work that required many people into work that requires fewer. These are tools like a skid steer (replacing a team of hand-shovelers), or a design system (reducing the custom per-feature design load). In military contexts this kind of tool is called a “force multiplier” — if what required a force of 10 people now only requires 5, the tool is a 2x force multiplier.
Transform hard work for one person into easy work for one person. These are tools like a chainsaw, or a targeted single-use library. They’re not quite force multipliers, because a task can never use half a person; but a hard task leaves you exhausted, and an easy task leaves you ready for more.
Transform impossible work into possible work. These are tools like calculus, a welder, or Unreal Engine. Before Leibniz and Newton invented calculus, there were problems which were impossible to solve; now they can be solved by a motivated high schooler.

Every one of these tools requires initial expense, education, and ongoing maintenance; they are worth the cost because they are tools by one of the above three definitions. Things that look and sound like tools but don’t actually make hard work easier are not tools, they’re mistakes. Don’t introduce complications to your team unless they’re actually tools.

The Anatomist’s Fallacy

2024-10-29T00:00:00-04:00

I tend to be shy about using the words “purpose” and “intention”. There are lots of processes that make it tempting to ascribe intention, but which have none. I think the easiest example is evolution. “Wide-spread eyes evolved in prey animals in order to detect stalking predators,” is an easy thought to have, but evolution as a process doesn’t plan; nothing is “in order to”. Everything is “continues because”. A more awkward but precise statement is “wider-spread eyes persist in prey animals in part because they make animals more likely to detect stalking predators and survive to reproduce, but they may provide other advantages, or just provide no disadvantages.”

I call this “the anatomists fallacy” — ascribing purpose to something simply because it’s effective at something, even when it’s the result of a system that might not be capable of intention at all; even when it has dozens of effects, not just the one.

The corrected statement is a mouthful, though, and I can understand why we hear the former more often. When you’re not just observing, though, when you’re trying to manipulate a complex system, it’s important to think and speak very clearly about how it works.

And natural processes, like evolution, are not the only ones where this applies.

In my eyes, there is a spectrum of intention-having ability. Natural processes obviously have none. A sunset isn’t for anything. On the other hand, when I am alone in a room, writing this paragraph, I have a fairly well-defined set of objectives, things I want to communicate, results I want to obtain by writing. If we couldn’t describe the conscious and planned actions of a single person as having purpose and intention, then the words wouldn’t have any use at all.

But even human systems, once enough humans are involved, start to feel similar: the output of big complex systems like corporations, governments, and economies don’t feel like they can be ascribed a single purpose to me. They are the result of hundreds or thousands or millions of people, all with different purposes, acting on incentives and whims, negotiating and working together or against each other. The effects may be better described as just consequences, leaving off the leading adjectives of “intended” or “unintended”.

And between a single human and an entire society are smaller groups – work teams, local organizations, families, friend groups – these are entities one can draw a reasonably clear line around, and which take actions and have effects. They are in some blurry area; if everyone involved is of one mind, and has communicated clearly, they could easily be acting to one purposeful end. But it’s just as possible that everyone is acting independently, and the resulting group effects are, happy or unhappy, just effects.

So here is the practical exercise at the end of this philosophical maundering: as you go about your life this week, think about each part of it the way a careful biologist might think about a new anatomical structure — not in terms of what it’s supposed to do, but all the things it actually does.

Don’t ask “What is this meeting for?” — observe, and see what its effect actually is. Where does information flow? Who feels what, who learns what? What’s the difference after the meeting vs before?

Don’t ask “Is this initiative succeeding?” — Observe what it’s doing, and everything it’s doing.

Outside of work, consider family dinner, lunch with friends, a traffic jam, a crowded sports bar.

Then, when you see negative effects, you can think about interventions not in terms of changing people’s minds to want different things, but in terms of nudging the entire system to produce different effects without touching the intentions involved at all.

Build a Laboratory

2024-10-07T00:00:00-04:00

A product is something someone will pay for.
A viable product is one you can sell for more than it costs to produce.
A minimum viable product is the easiest thing you can make with those properties, where “ease” depends on your specific circumstances: the time, money, resources, and skills you have available to you

In those terms, it’s inescapable that an MVP is the best thing for your business to build. It’s nearly tautological; a business can’t exist without a viable product, and the least risky way to get there would be the one with the least cost and complexity.

Nevertheless, in the past year, in meetups, conferences talks, conversations with clients, and online, I’ve heard a fair amount of pushback against the MVP concept, with folks often trying to replace just one letter — recommending a “minimum viable test”, or a “minimum useful product”. And there are problems: before you build it, you don’t know what people will pay! You don’t know how much it will cost! You don’t know what features it needs and which can be eschewed! Building an entire business is an expensive way to learn that your idea isn’t viable.

But when addressed on its own terms, the idea of an MVP feels unassailable — because it’s tautological, it’s hard to argue against.

In order to address that tension and frustration, I think it might be easier to approach the idea side-on, by translating the concept of the MVP into a slightly different language, where there’s more room to examine it. I have found a lot of value in an experimental mindset: make a hypothesis, find a way to test it, adjust, repeat — so I instinctively turn to that framework when I need a way to think about actions and motivations.

Let me invent some vocabulary to fit that perspective:

A market hypothesis is a guess that a sufficiency of customers will pay some amount for some good or service. (This should not be confused with the broader “efficient market hypothesis”).

An operations hypothesis is a guess that you can produce a product at a given cost (in money, in labor, in time, etc).

A complete business hypothesis is a matching pair of a market hypothesis and an operations hypothesis.

In this language, an MVP is just the simplest experiment that demonstrates the validity of a complete business hypothesis.

Importantly, that experiment may still be too complex, too expensive, or too risky to design or implement straight away. And that’s OK — by framing it this way, we already know that our big guess is made up of smaller guesses; every complex hypothesis is made up of lesser assumptions.

In the original language of the MVP, an MVP is atomic — it cannot be broken into smaller bits without losing its fundamental properties, because it’s already, by name and definition, “minimal”.

But in the experimental mindset, we can envision sub-hypotheses — the assumptions from which our larger guesses are built — and we can design and run experiments to test those sub-hypotheses. A proof of your complete business hypothesis is not the least thing you can do; it’s simply the least you can do to be home free. Those smaller experiments along the way aren’t viable products, or even necessarily products at all — if what you’re testing is an underlying assumption in your operations hypothesis, you may not even need to put it in front of users.

The end goal is still always to develop and prove a complete business hypothesis; it’s still just as tautologically true that you’re not a successful business until you have a viable product. Because we’re testing a hypothesis, a disproof is as genuine an outcome as a proof.

User interviews, user tests, market tests, prototypes — all the tools we’re already familiar with can be cast in this experimental light, as ways of testing sub-hypotheses. If an interview or a mockup can cheaply disprove one small assumption on which you based your larger business hypotheses, then you’ve cheaply disproved the whole thing; you know you need to make adjustments without having spent the cost of an entire MVP. (This isn’t a new conclusion, only a new lens: translated back into startup-speak, small, low-risk disproofs are covered by the mantra “fail fast”).

And that is part of the value of this way of thinking: it’s clear that our hypothesis, like most guesses, is probably wrong; rather than trying to build the easiest thing that is a successful business — rather than “build an MVP” — we want to discover a successful business in the easiest way — we want to develop a correct business hypothesis as simply as possible.

Once we acknowledge that our first experiment won’t be the last, the sensible thing to do is to plan for many experiments. A smart inventor doesn’t build a new workshop for every prototype; a smart scientist doesn’t buy all new equipment for every experiment.

So don’t build an MVP; build a laboratory.

Don’t build a product; build yourself tools that will allow you to keep testing and improving hypotheses until you find ones that work.

A lot of my clients balk at the idea of building a componentized design system; but if a slick brand or elegant UI are part of your market hypothesis, then a design system is a UI laboratory. Custom-built one-off UIs are useful only for a single experiment; a design system turns your product into an optical table, where you can stage a thousand experiments using the same building blocks.

A lot of my clients, especially those with small teams don’t want to spend time and energy on robust feature flagging; it doesn’t feel like progress the same way that actually building features does. But being able to add, remove, and recombine entire features turns the product into an inventor’s workbench. It seems too expensive for an MVP, but it’s an obvious step for cheaply running a series of experiments.

(Note that I’m explicitly not talking about A/B testing when I am talking about “experiments” — that is a kind of experiment, but it is more useful for refining and improving an already successful business hypothesis. A/B testing can extract a weak signal from a noisy channel, which is great when you’ve got a real business already, but tight margins and high volume. The gap between a disproven complete business hypothesis and a proven one, on the other hand, is usually visible to the naked eye. So when I say “build a laboratory” I don’t mean “construct a set of rigorous statistical tools like giant enterprises use”; I mean “provide yourself with building blocks that are easy to assemble in lots of ways, and a platform for exposing the results to users where you can watch what happens”. This is part of the difference between a “scientific” mindset and an “experimental” one — I am advocating for the latter.)

If this way of thinking seems appealing, here are some questions to ask yourself:

What are some ways in which your current project is like a laboratory? What experiments have you anticipated and left room for?
What are some ways in which your current project is a one-off? If it turns out to be unsuccessful, what will you have to throw away?
Reframe your business as a complete business hypothesis; how would you say it? What are the most vulnerable assumptions underlying that hypothesis? Are you testing those assumptions in the easiest way? Or are you building the machinery of a business while leaving the assumptions untested?

AI in 2024: Eat the fruit, leave the rind

2024-08-31T00:00:00-04:00

In my opinion, the most exciting uses of LLMs don’t involve generating text at all.

As one step of guessing what the next word of a text might be, existing LLMs turn all the previous text into a set of numbers.

There are the same number of numbers, no matter how short or long the text (with a strict upper limit of the context window). If 5 words produce 50 numbers, 500 words also produce 50 numbers – a different 50, but still 50 of ‘em. These numbers are called a ‘vector embedding’ of that text.

You can measure how similar or how distant those vector embeddings are, and that corresponds well to how similar or different the texts are – not just in a basic “how many words do they share” kind of way, but in a “do they mean the same thing; are they about the same topic” kind of way.

Multi-modal models can embed text, images, and other kinds of data all in the same space, allowing that measurement between very disparate sources.

That’s really cool! That’s a big deal! The ability to pick out related documents, to search by meaning instead of by word, to cluster documents in high-dimensional space… that’s an incredible, exciting tool, and one that can be used disconnected from generation entirely.

Most tutorials suggest using vector embeddings as a search engine, concealed behind retrieval augmented generation – but I say, why bother with the generative part? A more effective search is the win; allowing a generative process to rephrase the results is mostly risk with little to no upside.

Vector embeddings can’t produce false text, because they only organize the text you give them.
Vector embeddings can’t secretly produce copyrighted content because they don’t produce content.
Vector embeddings can still help your users find the answers they’re looking for, but you have complete control over the wording of the results
They have much lower energy costs than generation

Every major LLM provider also offers an API for vector embeddings, and they’re cheap as dirt because they require far less compute than generation.

And that’s it for part III! You’ve now got a pretty complete picture of my opinions about large-model AI in 2024:

only use generation when bullshit is acceptable
know which arguments against it have weight, and
vector embedding is cheap, low-risk, and underutilized

If you’d like to discuss how any of these thoughts relate to your team, product, and business, reach out!

AI in 2024: Making a Case

2024-08-26T00:00:00-04:00

I hang out with writers and artists along with technical people, and the conversation frequently turns to generative AI, both the text and image kind. The sentiment is rarely positive, and I am sympathetic. I think generative AI is overhyped, the wrong choice for many situations, and is already costing people their livelihoods.

I am a pedant, though, so while I am sympathetic, I get frustrated when I hear that negative sentiment carried by specious and ineffective arguments.

My favorite reason to be cautious is the one I gave last week — What Good is Bullshit? — but today I thought I’d take on some other ways people have of saying “generative AI is a problem”, and fine-tune those arguments to align with the truth as best I understand it.

Tired of this relentless negativity? In Part III, I’ll change my tune and talk about some of the LLM use cases I’m most excited about (mostly ignoring the generative aspect entirely).

“LLMs Cost Too Much Energy”

This is, I think, mostly a hold-over argument from fighting against crypto. LLMs do cost energy: training, especially, is expensive, and individual queries are certainly more costly than an average web query; but scary articles about the immense increase in energy usage by major players don’t seem to hold up to scrutiny. While specific energy figures for the major LLMs are largely unavailable, here’s Google’s total energy consumption over the last decade; can you spot the moment Gemini started having its dramatic and damaging effect? (Remember that ChatGPT was released in 2022)

I can’t either.

Too, crypto was doing things that we could already do another way, much faster and much cheaper, with a central database; LLMs are doing something computers couldn’t do before. It’s perfectly fair to argue we don’t need that thing, but it’s a weaker argument, and hard to make the data speak loudly enough for this to be an effective deterrent, I think.

“Training generative models on scraped data is theft.”

Note from Feb 12, 2024: If Judge Bibas’ judgement holds, this section is inaccurate — a direct commercial impact outweighs the fact that the trained work doesn’t appear in the delivered result.

This has a good argument at its core, but it’s almost always phrased in a way that I think is morally complicated and potentially legally unsuccessful.

I’ll pursue the legal argument in the US because that’s the law I’m most familiar with, but I believe a similar framework would apply to any attempt at allowing or banning AI training on open web data in the EU, too. Copyright law is complex, the existing precedent can seem contradictory and confusing, and I’m not a lawyer let alone an international intellectual property lawyer, so take all of this with a grain of salt.

In both cases, the only way an action can legally be theft is if it is by distributing a reproduction. The act of scraping published work and storing it in a private database is not, as I understand it, an infringement. If an article is openly published, you’re allowed to save that article and read it later — in fact just by reading it a copy was made, stored on your computer, and displayed. You’re allowed to print it out, to cut it up, to lick it, whatever. What you’re not allowed to do is sell access to your copy, or make a billion copies to give away: distribution is an infringement.

The question of theft, then, is not in the training process at all; I am not a lawyer, but I’m pretty sure doing even very complicated statistics on open-web data is allowed. The question is whether the trained LLM “contains” the original copyrighted work, in whole or in part; and whether whatever gets distributed (the entire model or merely its output) will contain that data.

The answer is “yes, probably, maybe.” It’s reasonably easy to prove that at least some training data is ‘memorized’ by the model – not just analyzed and abstracted but memorized verbatim. The major players, like OpenAI, are doing their best to add layers on top of the LLM to prevent simple exposure of copyrighted training data – it’s easy to run into a “against terms of service” warning if you try obvious ways of prompting for such things – but the data is there, and can be exposed in more creative ways that don’t involve direct prompting at all: see this paper from late last year.

Because these methods produce random memorized data, rather than specific info, it might be hard to find someone with standing to sue; but in the US, a lawsuit about this would mean a fair use factor analysis, and the ‘market impact’ factor is as large as I think it’s possible to imagine a derivative work having.

I understand that this is a disappointing version of the argument — what creators want is to prevent the training at all, not to allow distribution of a version of the LLM with very careful shackles — but if you made me place a bet on a legal argument working in court, it would be this one. If you can provide court-approved proof that the work is present in the model verbatim, the simplest way for an LLM-builder to prove that it cannot appear in the output is to allow opting out as input.

So I’d rephrase the argument this way: Generative models memorize and reproduce copyrighted material, which is illegal; and they do so in unexpected ways, so the user might not even know the response contains plagiarized material.

“Generative AI can’t be creative, it can only regurgitate old ideas” or “It’s just autocomplete”

This is just straightforwardly a bad take. It’s trivial to get LLMs to generate new ideas. Creativity is easy to extract from randomness; it’s more technically impressive that generative AI can be uncreative, and produce trope-ridden schlock.

Try prompting an LLM with “invent and briefly describe a new art movement” or “describe an unusual artwork made by an imaginary artist”. The results are not necessarily thrilling artistic achievements, and I have no fear of LLMs taking over the conceptual art scene unaided, but they’re not descriptions of anything that exists.

Or, more prosaically, “write a sentence that has never been uttered before”.

You can make an argument about the quality, value, and validity of the ideas an LLM generates, but even shuffling a deck of index cards can generate new ideas from old ones.

“Using it is lazy” or “it takes no skill”

This is an old take. Many clever tools are most often used in lazy ways; but it is more morally sound to criticize the laziness, not the tool. A camera is a lazy way of making a picture — except when it’s not.

The LLM doesn’t determine how much or how little effort goes into the work around the act of generation; the user does.

I hate listening to people talk about using LLMs to write entire stories. But I have no objections to someone using an LLM to find 5 more ways of expressing a sentence when their first two tries didn’t work. Mechanically producing infinite variations on a single theme is something artists have been doing for ages, with whatever technology was available to do so. If the other objections can be dealt with, there’s no reason creative people can’t spend just as much sweat and blood with this in their toolbox as without it.

“Generative AI will cost us jobs”

This is a good one! I think the only tweaking it needs is to be more immediate and particular: “Generative AI is costing jobs.” The scaremongering of “art is dead, creative writers will have no jobs, and it’ll come for you next!” is maybe less honest than “copywriters are already losing work; so are stock photographers and some kinds of illustrators.” An argument about the future implications is hard, and it’s easy to be wrong. An argument about what is already occurring is undeniable, with no risk of being proven wrong in a year.

fin

Are there other arguments against LLM use that you think are more effective? Is your team currently building an LLM into your product, and you think it’s the right choice despite these arguments? Feel free to reach out and bend my ear.

AI in 2024: What Good is Bullshit?

2024-08-16T00:00:00-04:00

It’s been a year since I last wrote about generative AI and its practical application, and some of my thoughts have solidified in that time, so I thought I might be due to revisit the topic. I expect this to come in three relatively short parts, of which this is part one.

Generative AI is Bullshit

One of my favorite papers this year so far has been Generative AI is Bullshit, in Ethics and Information Technology by Hicks, Humphries, and Slater. Despite its inflammatory title, it presents a straightforward and largely semantic argument that I think is very useful when thinking about potential uses of generative AI.

I find myself making reference to it over and over again, and I don’t want to force people to read a lengthy academic paper, so here’s a 3-point summary (the paper itself is very readable, though, and if that sort of thing is your idea of fun, click through and read it instead. If you’ve done that, or are already familiar with the paper, feel free to skip ahead to “What is bullshit good for?”)

“Bullshit” as a technical term

For the sake of this argument, “bullshit” means anything that is intended to be plausible but is unconcerned with truth. Something purely and purposefully false is a lie; but when it doesn’t matter whether it’s true or false as long as it is believable, that’s bullshit.
LLMs are explicitly trained to bullshit

An LLM is given all the text its trainers can get their hands on, true, false, fiction, non-fiction, opinion, fantasy, shitpost, diatribe, peer-reviewed academic study…

Training optimizes the LLM’s ability to predict the next word of that text, given some chunk of it. If the text is a complete fabrication written by a crackpot, it still predicts the next word. If the text is truth handed down by a god, it predicts the next word. If the text is a dadaist poem, it predicts the next word. It does not have the ability to say “hold on, that’s nonsense!” it can only say “it is unlikely anyone would ever say that”.
‘Hallucinations’ and correct answers are the same side of the same coin

It’s so tempting to think about the times an LLM generates blatantly untrue text as somehow different from when it generates true text; that we can work to fix the former and keep the latter.

But the same process generates both, and that process is inherently indifferent to truth. Both are bullshit!

What is bullshit good for?

Just because generative LLMs produce bullshit doesn’t mean they’re useless. You just need to carefully pick tasks where good, high-quality bullshit is all you need.

That sounds very limiting, but one key quality of bullshit is that it’s internally consistent; as soon as it contradicts itself, it loses plausibility. LLMs are not perfect bullshit generators, but a lot of their odd behavior can be understood in terms of this pressure towards consistency; very little training text, even crackpot text, outright changes its tune mid-thought.

Tasks that require agreement with the real world will sometimes fail catastrophically. Tasks that only require plausible consistency with the prompt are safer bets. Some examples:

Walking someone through handling a medical crisis? Hell no!
Explaining scientific concepts to students? No, probably dangerous.
Summarizing a correct but lengthy description of a scientific topic down to a couple sentences? This is on the edge of its ability; it’s much harder to make a summary that’s plausible but incorrect when the document is right there.
Rephrasing a correct short summary in different words, or according to some constraint? Now we’re talking!

This is one reason why retrieval-augmented generation makes for more effective chatbots than training-based specialization: retrieving existing (and truthful) documents is a search task, and then rephrasing their contents to fit the conversation merely requires consistency, and so plausibility is often enough of a constraint to maintain that truth.

Is a generative LLM a good choice for my task? Questions to ask yourself:

Am I OK with producing falsehoods with some frequency? or
Can I provide enough context that even limited internal consistency of the response with the prompt will incidentally ensure correctness? and
Am I in total control of the prompt, or will users or other parties be able to make the prompt inconsistent, leading to poor results?

This is considerably simpler than most of my question-based discernment tools! I suspect you can guess from the last of these questions that I’m still bearish on AI-powered chatbots. I don’t think exposing a raw LLM to users is a sensible approach for reliability, security, expense, or user experience – even with RAG involved.

That’s not to say I am against the vector database portion of RAG. These questions apply only to generative uses of LLMs. It is the non-generative uses which I think are far more exciting in terms of application, and I intend to return to that subject in part III of this series.

The conclusion of this part, however, is that I still believe, like I did a year ago, that too many companies and individuals are relying on LLMs to produce concise, human-readable truth, when what they’re good at is producing plausible filler. LLMs don’t have to be a flash-in-the-pan fad forced into your product and then abandoned; play to their strengths, and you’ll see longer-lasting success.