Canada built a cancer machine and left out the safety relays.

•  A Canadian machine treated cancer under computer control.

•  Its two predecessors had hardware interlocks. This one did not.

•  Six patients received doses nobody had prescribed.

•  The fix was a circuit the software cannot override.

AECL, a Canadian crown corporation, built medical linear accelerators. First the Therac-6, then the Therac-20, rated at 6 and 20 MeV, the figure that measures the energy of the beam. Both were computerised versions of machines that already worked on their own. The computer added convenience, and the hardware interlocks that policed the beam stayed where they were.

The Therac-25 was a different proposition. It was designed around the computer from the outset at 25 MeV, and it was not built on a machine that could stand by itself. AECL chose not to duplicate the hardware interlocks, and handed the software the work those circuits had been doing.

The first prototype, in 1976, was hardwired throughout. The fully computerised version reached customers late in 1982. In March 1983 AECL ran a safety analysis on it, in the form of a fault tree, and that analysis left the software out. Residual programming errors, it stated, were not included.

Every row below comes from one document, the investigation of the accidents that Nancy Leveson and Clark Turner published in IEEE Computer in July 1993. They worked from the regulators' files and the manufacturer's own correspondence, and theirs is the version the field has worked from ever since.

Read the fourth row first. On 21 March 1986, at a clinic in Tyler, Texas, a man came in for the ninth session of a course of 6,000 rads spread over six and a half weeks. That session was prescribed at 180 rads. Later reconstruction put what he received at 16,500 to 25,000 rads, delivered in under a second into an area about a centimetre across.

Four decisions sat behind the six, and none was the programming error itself.

•  The interlocks were not duplicated. The Therac-6 and the Therac-20 kept the mechanical and electrical interlocks of the machines they were built on. The Therac-25 was designed without them and the software inherited the job.

•  Error sat elsewhere, harmless. The Therac-20 carried a related programming error. Its independent hardware circuits would not let the beam turn on, so nobody was exposed, and the error registered as a nuisance that blew fuses.

•  The screen said no dose. At Hamilton the machine stopped after five seconds with an error code, and the display read no dose and treatment pause. The operator followed the procedure for that state and carried on.

•  The analysis had a boundary. The 1983 fault tree assumed residual programming errors away. The one part of the machine now carrying the safety work was the one part the analysis did not examine.

Six patients, between June 1985 and January 1987, in Georgia, Ontario, Washington and Texas. The paper calls it the worst series of radiation accidents in the thirty-five-year history of medical accelerators, and the injuries ran to tissue death, an arm that stopped working, and deaths.

The first ones were not believed. At Marietta the manufacturer did not accept that the machine had injured the woman until long afterwards, despite what she said, the burns on her and the physicist's suspicion. At Yakima in December 1985 a striped reddening of the skin was not treated as abnormal until the following year.

The regulator then declared the Therac-25 defective and ordered the firm to notify every purchaser and submit a plan. The final plan ran to more than twenty changes.

•  Ask what was removed. A new system arrives described by what it can do. Ask which safeguard stopped being fitted.

•  Count what is visible. A relay sits on the rail or it does not, and anyone can walk over and look. A check inside software is a statement about a file.

•  The display is not the machine. At Hamilton the panel reported no dose while the beam had gone, and the operator had nothing else.

•  The analysis had a boundary. The 1983 fault tree was honest work and drew a line around the hardware. Everything that mattered had moved past it.

I sat in a dealing room in 1998 where the limits lived in a spreadsheet and the only hard stop was a man.

When something is called safer because it is cleverer, ask which physical part came out in exchange. Gold is weighed on a machine nobody can reprogram to weigh something else.