Colossus - the Orignal Agile Project?

I have been wanting to visit Bletchley Park and the National Museum of Computing for a while now and decided to take the trip down to Milton Keynes to take a look around.

The story of Bletchley Park is not only a fascinating tale of British ingenuity and brilliance against the Nazi threat of the Second World War, it's also the story of the birth of modern computing.

For those of who don't know, Bletchley Park was the home of the UK Government Code and Cypher School (GCCS), the forerunner of GCHQ, during World War II. GCCS was set-up to crack and decipher German signals captured by numerous Station-Y listening posts around the UK. It's been the subject of numerous books and films, including the novel by Robert Harris and a film (Enigma) staring Dougray Scott and Kate Winslet.

One of the main reasons I went to Bletchley Park was to see the rebuilt Colossus Mk2. The story behind the design and construction of this machine is an inspiration.

GCCS had not only successfully broke the code of the German Enigma machine, but also highly automated the key and message decoding through the construction of a machine call the Bombe, designed by Alan Turing. Hitler, though, wanted a higher encryption capability for signals to his high command and his scientific team came up with a teleprinter based system developed by the Lorenz company.

The Lorenz machine worked with a 32 symbol baudot code system, messages were then encrypted with a sequence of obscuring characters using modulo 2 addition (exclusive NOR in boolean terms). If the obscuring characters were truly random, the cipher would of been near on impossible to break at that time, but the Lorenz machine used a series of mechanical rotors to generate a pseudo-random key. The breakthrough was when a 4000 character message was being sent in the German High Command, the receiver did not fully get the message and asked the sender to repeat it. The radio operator committed the cardinal sin and sent the message again with the same Lorenz settings.

Brigadier John Tiltman and the Cambridge graduate Bill Tutte exploited mistakes made by German radio operators and began to reconstruct the pseudo-random sequence and discover how the Lorenz encoding machine worked. The Lorenz encoding was cracked, the problem was by long hand it look weeks to decode a message, far too long, so an automated and substantially quicker method was needed.

The Post Office Research Labs at Dollis Hill produced a machine based on relays that could read punch tape, but even this took six weeks to crack the average message - still too long. One of the mathematicians working at Bletchley, Max Newman, worked out that using electronic logic circuits working in parallel, the messages could be broken quicker. Max approached one of the Post Office engineers Tommy Flowers to design a build a electronic machine to process the Lorenz messages. The first attempt was called the Heath Robinson, it proved both Max's theory and the electronic circuit design were correct - the problem was it's reliability.

Tommy's new design was based on using around a 1000 valves. None of his management believed it was feasible and told him to abandon the project. Luckily, Tommy ignored the doubters, and he and his team worked shifts round the clock to design and build Colossus in less than 9 months! Colossus went operational in January 1944. The machine was a success and Colossus was decoding Lorenz messages at a rate of 5000 characters per second. The Mk2 quickly followed which used around 2500 valves and was substantially quicker than the Mk1. In total ten machines were built and delivered to Bletchley. Through 1944 and 45 they worked around the clock decoding German messages. The success of D-Day was, in part, down to the Colossus, as German High Command messages decoded assured the Allies that the D-Day diversion plans had been believed by Hitler, without these decoded intercepts, the Allies would not of had the confidence that the Germans had taken the D-Day diversion bait.

In computing terms, Colossus can be regarded as a programmable special purpose computer. AND, OR and XOR logic gates could be configured in numerous combinations with a plug board system. Colossus had a 5-bit shift register, the first computing machine to use such an electronic circuit. The diagram below is an original schematic showing the architecture of Colossus


At the end of the Second World War Churchill ordered that 8 of the 10 Colossus's were to be completely destroyed, along with all schematics and technical documentation. Two survived and were taken on to, what is now, GCHQ in Cheltenham. These two machines were destroyed in the early 1960s.

The existence of Colossus remained secret until the 1970s when small snippets of information about the machine began to emerge. Ironically, most of the information about Colossus was released by the US Government under the Freedom of Information Act. Obviously, as Allies during the war, the US had knowledge and access to Colossus and a number of US service personnel were seconded to Bletchley Park.

In the early 90s a computing enthusiast Tony Sale, who was part of the group that helped save Bletchley Park from sure destruction, had the dream of rebuilding Colossus. I say again, rebuilding, not a replica! A number of the original team were still around, including Tommy Flowers. Luckily, they kept scraps of information about the original machine. The rebuild has taken over 15 years, but the machine is now up to a standard that it can decode Lorenz transmissions to the same speed and standard as the original. The BBC News clip below records an event in 2007 when Bletchley Park Trust held a competition to see whether anyone could beat Colossus on decoding an message encrypted by an original Lorenz machine.

Although comparison with modern 'general purpose' computers cannot be made directly, a scaled CPU clock speed for Colossus has been calculated at around 5.8MHz Pentium, not bad for a 65 year old computer! Seeing the machine 'in the flesh' it looks impressive.

So, back to the post title, what has Colossus got to do with Agile systems development? The team that designed and built Colossus, in my view, exhibited all the traits of a well performing systems team. They worked rapidly and iteratively, there was no 'big up-front requirements'. More importantly, it was the technical innovation, skill and persistence of Tommy Flowers and his team that won the day. Knowing how long typical IT projects take to get off the ground then deployed to production, it's absolutely amazing to think that this machine was designed and built in 9 months.

I highly recommend a visit to Bletchley Park. The pioneering work carried at Bletchley during the Second World War by the likes of Alan Turing, Max Newman and Tommy Flowers gave rise to the industry I work in today.

If you want further information on Colossus and Bletchley Park I'd recommend:

• Bletchley Park - the official site of the trust and museum.
• National Museum of Computing - the UK museum of computing.
• Codes and Ciphers - site maintained by Tony Sale the architect behind the rebuild of Colossus.
• Alan Turing - a site about Alan Turing maintained by his biographer Andrew Hodges