Class 4 · Coding With AI

It will write in four seconds what would have taken you four hours, and it will do it with exactly the same confidence whether it is right or catastrophically wrong. Everything in this class is about the second half of that sentence.

Logic gym#

Fifteen minutes, on paper, in pairs, and nothing runs. That last part is the point: this whole class is about judging code you did not write, and judging it by running it until it looks right is the failure mode.

1. Review it without running it#

def universes_needed(pixels):
    return pixels * 3 / 512

Say three things wrong with it. One is arithmetic, one is a type, and one is a missing decision. Then say which of the three a test on 170 pixels would catch, and what that tells you about tests written by the same person who wrote the bug.

2. Which of these can it know#

For each, say whether a language model can reliably answer it, and why:

• "Write a function that converts timecode to frames."
• "What is the DMX address of the fourth mover on our rig?"
• "Why does my sACN packet get rejected by this node?"
• "What did we agree in the production meeting on Tuesday?"
• "Is this circuit safe?"

The pattern in your answers is the useful thing, not the individual answers. Say it in one sentence.

3. The confident wrong answer#

Somebody brings you generated code that calls a library function you have never heard of, with a plausible name and plausible arguments.

Say, in order, the three things you check. The first one takes eleven seconds and catches most of these.

4. What never leaves the building#

Sort these into "may go to a model" and "may not", and say the rule you used:

• A messy CSV of fixture addresses.
• The cast list with dates of birth.
• A photo of a schematic from a supplier, marked confidential.
• A script excerpt for an unreleased show.
• An error message from a console.

The rule is one sentence and it is not "anything sensitive". Get to the sentence.

5. Who signs#

A generated function controls a winch. It is reviewed by you, tested by you, and it works.

Name every person who is responsible if it moves at the wrong moment. Say what "responsible" means for each of them, and say which parts of that list would change if a person had written it instead.

The correct answer is that nothing on the list changes, and being able to say that clearly in a production meeting is worth more than anything else in this class.

What the gym is for#

The other gyms trained running a program forward in your head. This one trains reading a program and asking what it cannot know, which is a different move and the one that keeps generated code useful instead of dangerous.

What the model is actually doing#

You cannot use a tool well while believing something false about how it works. So, without the mathematics:

What the model is actually doing#

A large language model is a function that takes a sequence of text and produces a probability distribution over what comes next. That is the entire operation. It is repeated, one piece of text at a time, with each output fed back in as part of the next input.

"One piece of text" is a token, roughly three to four characters of English, a bit less for code, considerably less for Chinese. set_light might be three tokens. A rule of thumb that is good enough for planning: a token is about 0.75 of an English word.

What produced those probabilities is training on an enormous amount of text, including a very large amount of code, adjusting billions of internal numbers until the predictions got good. Nothing was stored as facts you can look up. What was learned is structure: that after def fade( there is very likely a parameter list, that a function called read_csv probably takes a filename, that code with if usually has a matching branch.

Four consequences follow directly, and all four are practical.

1. It is extremely good at the shape of things. Boilerplate, conventions, the standard way to open a file, the usual structure of an Arduino sketch. These are exactly what "what usually comes next" is best at, and this is genuinely most of the code anybody writes.

What the model is actually doing#

2. Plausibility and truth are not distinguished. A function name that looks exactly like a real one, with the arguments a real one would have, in a library that exists, is highly probable text. It does not need to exist. This is the failure everybody calls hallucination, and it is not a bug being fixed later; it is the same mechanism that makes the tool work at all, seen from the other side.

3. It is best where there was most training data, worst where there was least. Python and JavaScript: excellent. Common web frameworks: excellent. The scripting dialect of one lighting console, documented in a PDF behind a dealer login: it will produce something confident and fictional. This describes most of our industry's software.

What the model is actually doing#

4. It does not know what it does not know. There is no internal signal that says "low confidence here". The wrong answer arrives in the same tone as the right one. Your task 2 preparation is the evidence, and it is worth more than my saying it.

What it also cannot know#

Not because it is not smart enough. Because the information was never anywhere it could learn it:

• What is actually patched in your rig
• What that specific fixture does on power-up, because the manual is a scanned PDF from 2011
• Which of the three cue stacks is the one being used tonight
• That the sensor on the door has been intermittent since Tuesday
• Anything that happened after its training ended, unless a tool gives it a way to look

Everything in that list is what a technical director knows and the model does not. That is not a weakness in the tool. It is a fair description of the division of labour, and it is a much better job description than "person who types code".

Context, and why it forgot#

Context, and why it forgot#

The model reads a fixed-size window of text: the context window. Current frontier models take around a million tokens, which is very large, and older or cheaper ones take a fraction of that. In practice everything the model can use has to be in that window: your instructions, the code you pasted, the files the tool decided to include, the whole conversation so far.

This explains behaviour that otherwise looks like carelessness:

It forgot what I told it at the start. In a long conversation, early content is either genuinely out of the window or so far from the current position that it carries little weight. Restate what matters. Do not assume anything said forty exchanges ago is still operative.

It does not know about the other file. If the tool did not put that file in the window, it does not exist. Everything the model appears to "know" about your project is what it was shown.

It contradicts itself between sessions. A new session is a new window. There is no memory between them except what the tool deliberately carries over.

It gets slower and dearer as the conversation grows. Every request re-reads the whole window, and you pay per token read. This is why starting a fresh conversation for a new task is not just tidiness, it is cheaper and it is usually better output.

The practical habit: treat the context window as a workbench. Put on it exactly what is needed for the job in hand, take off what is finished. A cluttered bench produces bad work here for the same reason it does at a real bench.

The loop#

Working with these tools well is a loop, not a request. Six steps, and the tempting ones to skip are the ones that cost you.

1. Describe the job, including what "correct" means. Not "write a script to check DMX addresses" but "read this CSV, which has columns Fixture, Universe, Address and Footprint. Report any fixture whose address range overlaps another's in the same universe. Addresses are 1-based. Print the two fixture names and the overlapping range."

That difference is most of the quality difference. Notice what the good version contains: the shape of the input, the definition of the fault, and a fact the model cannot know (1-based). Writing that paragraph is the specifying skill from Class 1, and it is the part that does not get automated.

2. Get the draft. Fast. Do not agonise over the prompt. A second attempt with a correction is usually better and quicker than a perfect first prompt.

3. Read it. All of it. This is the step people skip and it is the entire point of the class. You are looking for the four questions from Class 1: what state, what per pass, what assumed, what on failure. If you cannot answer those four, you cannot use this code, and the correct response is to ask the model to explain it or to simplify it until you can.

The loop#

4. Run it on something you can afford to lose. A copy. Always a copy. The failure mode of generated code is not a syntax error, which announces itself. It is confident, plausible action on a wrong assumption, applied to two hundred files.

5. Prove it against a case you worked out by hand. Give it an input where you already know the answer. Include an ugly one: the empty field, the duplicate, the fixture at address 509 with a footprint of 8 that runs off the end of the universe. This is the step that catches the real faults, because generated code is usually right about the normal case and wrong at the edges, which is precisely where show software lives.

The loop#

6. Commit it, with a message saying it was generated and what you verified. Six months later, that message is the difference between trusting the tool and re-deriving it.

Reviewing generated code fast#

You will not read every line at the same depth. Read in this order:

1. The signature and the docstring. What does it claim to do?
2. Anything that deletes, overwrites or writes. Every destructive operation, first, always.
3. The boundaries. Loops: does it start at 0 or 1, and does it include the last item? Off-by-one in a DMX address is one wrong fixture. In a frame count it is a black frame in the show.
4. The assumptions. What does it require to be true? Does it check?
5. The failure path. Is there one? What does it do with the row it could not parse? Silently skipping bad rows is the single most common generated-code behaviour and the most dangerous, and it is dangerous precisely because everything looks like it worked.
6. The middle of the logic. Last, because it is the part most likely to be right.

The three questions to ask the model itself#

• "What does this code assume about its input?" It answers this well, and it is a fast way to get the assumption list you would otherwise have to derive.
• "What happens if the file is empty, the value is missing, or the number is out of range?"
• "Write me three test cases including the ugliest input you can think of." Then read the tests before you read the code, because the tests reveal what it believed the job was.

Do it now: review what the model wrote#

The starter is a real shape of generated answer: confident, readable, and wrong in two ways. One fault is a type that never got converted. The other is that nothing validates, because a model asked for a converter was not asked what to do with rubbish and so it does nothing with rubbish.

Read it before you run it. Getting a red row and changing things until it goes green is fixing a symptom, and this class is about the other thing. Name both faults out loud first, then press run and find out whether you were right.

What it can genuinely do now#

The pessimism above is a working method, not a verdict. What has actually changed is large, and this course exists because of it.

Prototyping in an afternoon. An effect that would have needed a programmer and a week can be proved on a Pi in a session. Not shipped. Proved, which is what unblocks a design decision.

Reading somebody else's code. Paste in the contractor's 400-line sketch and ask what it does, what it assumes, and what happens if the sensor disconnects. It is very good at this, and it is faster than the alternative, and it is the safest use of the tool because you are the one deciding whether the explanation holds up.

Translating between shapes. Convert this fixture export into that console's import format. Turn this cue list into a JSON structure. Endlessly tedious, entirely mechanical, exactly what it is for.

Explaining an error. Paste the traceback. It has seen a hundred thousand of them.

Writing the boring, dangerous parts correctly. Argument parsing, CSV quoting, timezone handling, retry logic. Areas where the standard approach is right and hand-rolling it is how you get the bug.

What it can genuinely do now#

Filling in a language you do not know. You can now write a Lua macro for a console you touched once, with the model, in an hour. Judgement about what the macro should do is still yours, which is why Classes 1 to 3 are the prerequisite for this one rather than the other way round.

What never goes to a model, and other lines#

Three separate concerns get muddled together. They have different answers.

Confidentiality. Show files, drawings, contracts, personal data, credentials, unreleased creative material. The default assumption for anything you paste into a service is that you have sent it to a third party. Whether it is used for training varies by service and by plan, and the answer is in a contract that changes. Read your organisation's policy, and if there is not one, that is a real finding worth raising, not a gap to route around.

Never, under any policy: passwords, API keys, .env files, private keys, the venue's network diagram with addresses in it, anything under NDA.

Safety. Nothing that moves, heats, carries mains, or restrains a person gets code from a model without a human competent to review it line by line, and without hardware protection that does not depend on the software being right. The interlock is a physical circuit. The limit switch is wired. The emergency stop is hardwired and removes power, and it does not ask any processor for permission. Electronics for Theatre Class 15 is the detail, and none of it changed because the first draft got faster.

What never goes to a model, and other lines#

A model is allowed to help you write the code that makes the effect happen. It is never the thing standing between a performer and a machine.

Licensing and provenance. Generated code can closely reproduce training material. For most production tooling nobody will care. For anything you ship, sell or contract, ask what your client's terms actually say. The honest position: this is unsettled, it varies by jurisdiction, and "I did not know" is a weak position to defend from. Ask before, not after.

---

Do it now: add the guards nobody asked for#

nextPreset works. It has worked in every rehearsal. Six cases make it survive the show, and three of them describe things that should never happen, which is exactly why they will.

No exception may escape this function. It is called from the cue engine: an exception there stops the show, and a sensible fallback merely looks slightly wrong from row H. Deciding which of those you are shipping is the whole job.

What it costs, and how to think about it#

You are charged per token, in and out, and output tokens cost several times what input tokens do. Rates move, so learn the arithmetic rather than a number.

Cost ≈ (input tokens × input rate) + (output tokens × output rate), per request, every request, including the ones where you asked it to try again.

Three consequences that actually change what you do:

Long conversations are expensive. The whole window is re-read every turn. A 100,000 token conversation costs about a hundred times what a 1,000 token one does, per message, for the same answer.

Caching is worth understanding. Most services can cache a stable prefix, so the unchanging part of the conversation is charged at a large discount on re-reads. It works on an exact prefix match: change one byte near the start and everything after it is charged again. The design consequence is simple. Put the stable material first, and the varying question last.

Bulk work is cheaper asynchronously. Batch interfaces for work that is not interactive typically cost about half. Renaming ten thousand files does not need an answer in two seconds.

For a student, all of this is small money. For a department deciding whether a tool is in the production budget, it is exactly the arithmetic that has to be in the paperwork, and the cost calculator does it with your own numbers.

What it costs, and how to think about it#

Buy, build, or glue, now that generating is cheap#

The honest change is not that you can write more code. It is that the cost of the "build" option dropped, and the cost of maintaining it did not.

A tool that took a week now takes an afternoon. The README, the failure mode, the handover, the rewrite when the operating system changes and the person who fixes it at 23:40 all cost exactly what they did before. So the arithmetic moved, and it moved less than it feels like it moved.

Buy, build, or glue, now that generating is cheap#

Where the responsibility sits#

This section is short and it is the one to remember.

The tools are now agentic: they will run commands, edit many files, install packages, and work for minutes at a time without asking. That is a genuine step change in what one person can do in an afternoon. It is also a genuine step change in how much can go wrong before anybody looks.

The two habits that make it safe are both from earlier classes and neither is about AI:

• Version control (Class 3), so an agent's afternoon of work is a set of commits you can inspect and revert, not a folder in an unknown state.
• A copy (this class), so nothing irreplaceable is in the blast radius of a first attempt.

And the sentence:

"The AI wrote it" is not a sentence that appears in any incident report anybody has ever accepted. Your name is on the risk assessment. The tool is fast and it has no standing.

That is not a reason to avoid it. It is the reason the four questions are worth learning properly, and the reason this course spends five sixths of its time on things that are true whether or not a model wrote the draft.

Numbers from this class#