THE CODEBREAKERS Read online

  The cryptanalysts of S.I.S. and OP-20-G, then, matched these assumed plaintexts to their ciphertexts and looked for regularities from which they could derive a pattern of encipherment. This kind of work, particularly in the early stages of a difficult cryptanalysis, is perhaps the most excruciating, exasperating, agonizing mental process known to man. Hour after hour, day after day, sometimes month after month, the cryptanalyst tortures his brain to find some relationship between the letters that hangs together, does not dead-end in self-contradiction, and leads to additional valid results. “Most of the time he is groping in the darkest night,” one solver has written. “Now and again a little flicker of light gleams across the darkness, tantalizing him with a glimpse of a path. Hopefully he dashes to it only to find himself in another labyrinth. His knowledge that night is inevitably followed by day keeps his waning courage up, and he steers his course towards where the morning sun is soon to appear. Except that sometimes he is engulfed in an interminable polar night.”

  It must have seemed like that interminable night to the cryptanalysts who began attacking the new Japanese machine. The codebreakers went just so far—and for months could not push on further. As William Friedman recalled, “When the PURPLE system was first introduced it presented an extremely difficult problem on which the Chief Signal Officer [Mauborgne] asked us to direct our best efforts. After work by my associates when we were making very slow progress, the Chief Signal Officer asked me personally to take a hand. I had been engaged largely in administrative duties up to that time, so at his request I dropped everything else that I could and began to work with the group.”

  Friedman was (and is) the world’s greatest cryptologist. Then in his late forties, he was a quiet, studious man, well liked by his associates, of average height and build, and a natty dresser given to bow ties. Trained as a geneticist, he had become interested in cryptology in 1915 at a research institution in Illinois called the Riverbank Laboratories. He served as a cryptanalyst with the American Expeditionary Forces in World War I, and returned to River-bank to write an 87-page tract that revolutionized cryptanalysis by introducing statistical methods for the first time. Hired by the Signal Corps in 1921, he applied these methods to a cipher-machine solution that placed America in the forefront of world cryptology. During these years, his wife, the former Elizebeth Smith, whom he had met and married at Riverbank, was solving rumrunners’ codes for the Coast Guard. He wrote textbooks in cryptanalysis that are models of clarity. He became head of S.I.S. when it was founded and continued to exercise his extraordinary cryptanalytic abilities. His genius soon manifested itself in the attack on PURPLE.

  Lighting his way with some of the methods that he himself had developed, he led the cryptanalysts through the murky PURPLE shadowland. He assigned teams to test various hypotheses. Some prospected fruitlessly, their only result a demonstration that success lay in another direction. Others found bits and pieces that seemed to make sense, (OP-20-G cooperated in this work, with Harry L. Clark making especially valuable contributions, but S.I.S. did most of it.) Friedman and the other codebreakers began to segregate the ciphertext letters into cycles representing the rotation of the coding wheels—gingerly at first, then faster and faster as the evidence accumulated. The polyalphabetic class of ciphers, to which PURPLE belonged, is based ultimately upon an alphabet table, usually 26 letters by 26. To reconstruct the PURPLE tables, the cryptanalysts employed both direct and indirect symmetry of position—names only slightly less forbidding than the methods they denote. Errors, caused perhaps by garbled interceptions or simple mistakes in the crypt-analysis, jarred these delicate analyses and delayed the work. But slowly it progressed. A cryptanalyst, brooding sphinxlike over the cross-ruled paper on his desk, would glimpse the skeleton of a pattern in a few scattered letters; he tried fitting a fragment from another recovery into it; he tested the new values that resulted and found that they produced acceptable plaintext; he incorporated his essay into the over-all solution and pressed on. Experts in Japanese filled in missing letters; mathematicians tied in one cycle with another and both to the tables. Every weapon of cryptanalytic science—which in the stratospheric realm of this solution drew heavily upon mathematics, using group theory, congruences, Poisson distributions—was thrown into the fray.

  Eventually the solution reached the point where the cryptanalysts had a pretty good pencil-and-paper analog of the PURPLE machine. S.I.S. then constructed a mechanism that would do automatically what the cryptanalysts could do manually with their tables and cycles. They assembled it out of ordinary hardware and easily available pieces of communication equipment, such as the selector switches used for telephones. It was hardly a beautiful piece of machinery, and when not running just right it spewed sparks and made loud whirring noises. Though the Americans never saw the 97-shiki O-bun In-ji-ki, their contraption bore a surprising physical resemblance to it, and of course exactly duplicated it cryptographically.

  S.I.S. handed in its first complete PURPLE solution in August of 1940, after 18 or 20 months of the most intensive analysis. In looking back on the effort that culminated in this, the outstanding cryptanalytic success in the whole history of secret writing up to its time, Friedman would say generously:

  Naturally this was a collaborative, cooperative effort on the part of all the people concerned. No one person is responsible for the solution, nor is there any single person to whom the major share of credit should go. As I say, it was a team, and it was only by very closely coordinated teamwork that we were able to solve it, which we did. It represents an achievement of the Army cryptanalytic bureau that, so far as I know, has not been duplicated elsewhere, because we definitely know that the British cryptanalytic service and the German cryptanalytic service were baffled in their attempts and they never did solve it.

  Friedman, was, despite his partial disclaimer, the captain of that team. The solution had taken a terrific toll. The restless turning of the mind tormented by a puzzle, the preoccupation at meals, the insomnia, the sudden wakening at midnight, the pressure to succeed because failure could have national consequences, the despair of the long weeks when the problem seemed insoluble, the repeated dashings of uplifted hopes, the mental shocks, the tension and the frustration and the urgency and the secrecy all converged and hammered furiously upon his skull. He collapsed in December. After three and a half months in Walter Reed General Hospital recovering from the nervous breakdown, he returned to S.I.S. on shortened hours, working at first in the more relaxed area of cryptosecurity. By the time of Pearl Harbor he was again able to do some cryptanalysis, this time of German systems.

  Meanwhile, S.I.S. constructed a second PURPLE machine and gave it to the Navy. A third was sent to England in January of 1941 on King George V, Britain’s newest and largest battleship, which had just brought over her new ambassador to the United States, Lord Halifax. Two Army and two Navy cryptanalysts accompanied the machine. In return the United States received British cryptanalytic information, presumably about German codes and ciphers. This machine eventually reached the British codebreaking group at Singapore, and was evacuated with it to Delhi after the Japanese swarmed down Malaya. A fourth machine was sent to the Philippines, while a fifth was built as an extra for S.I.S. A machine for Hawaii was under construction at the time of Pearl Harbor; this became instead a second machine sent to England for use there by Great Britain.

  OP-20-G contributed importantly to the ease and speed of daily PURPLE solutions when 27-year-old Lieutenant (j.g.) Francis A. Raven discovered the key to the keys. After a number of PURPLE messages had been solved, Raven observed that the daily keys within each of the three ten-day periods of a month appeared to be related. He soon found that the Japanese simply shuffled the first day’s key to form the keys for the next nine days, and that the nine shuffling patterns were the same in all the ten-day periods. Raven’s discovery enabled the cryptanalysts to predict the keys for nine out of ten days. The cryptanalysts still had to solve for the first day’s key by straight
forward analysis, but this task and its delays were eliminated for the rest of the period. Furthermore, knowledge of the shuffles enabled the codebreakers to read all the traffic of a period even though they could solve only one of the daily keys.

  This fine piece of work, on the shoulders of the tremendous initial Fried-man-S.I.S. effort, resulted in the paradoxical situation of Americans reading the most difficult Japanese diplomatic system more quickly and easily than some lower-grade systems. They also became very facile in reading two-step systems in which PURPLE superenciphered an already coded message. The Japanese did this from time to time to provide extra security, usually with the CA code, the personal code of an ambassador or head of mission. A year after S.I.S. handed in its first PURPLE solution, the cryptanalysts solved a message enciphered in “the highest type of secret classification used by the Japanese Foreign Office.” The message was first enciphered in CA; this was then juggled according to the K9 transposition (normally used with the J19 code), and the transposed codetext was then enciphered on the PURPLE machine. The solution, which on the basis of the number of combinations involved might have been expected to take geologic eons, was completed in just four days.

  The question of who should receive this hard-won, easily-lost information was the knottiest, most nagging, most intractable problem in the whole operation of MAGIC. It involved a delicate balancing of security against utility. On the one hand was the need to turn the results to as much good effect as possible, and the more persons who saw it the greater its value would be. “I see no use in breaking a cipher,” one admiral remarked dryly, “unless you use its contents.” On the other hand was the danger that too wide a distribution would jeopardize this invaluable intelligence by increasing the possibility of a leak. In general, policy leaned heavily toward security, toward minimizing the risk as much as practicable by narrowly restricting the number of recipients.

  In an agreement dated January 23, 1941, the intelligence chiefs of the Army and the Navy listed those eligible to see MAGIC. The ten named comprised perhaps the most elite group in the American power structure of the day: the President, the secretaries of State, War, and Navy, the Chief of Staff, the Chief of Naval Operations, the heads of the Army and Navy War Plans divisions, and the heads of the Army and Navy intelligence divisions. In practice, of course, many others saw the intercepts, such as McCollum, the heads of the Army and Navy communications divisions (which controlled the cryptanalytic bureaus), and the cryptanalysts and translators themselves. In time so did others not on the original list nor involved in the processing. By December the Navy’s Assistant Chief of Naval Operations was regularly reading MAGIC. On the White House staff, President Roosevelt’s right-hand man, Harry Hopkins, and the President’s military and naval aides saw MAGIC; in fact, when Hopkins was confined to the Navy Hospital in November of 1941, Kramer brought it over to him specially. While Marshall interpreted the rules strictly and did not even entrust one of his closest assistants, Colonel Walter Bedell Smith, secretary to the general staff, with a key to the MAGIC briefcase, other officials, like Hull, Knox, and Stark, let their aides handle the details and so see the intercepts. In addition, at least four subordinate State Department officers saw MAGIC with fair regularity: Sumner Welles, the Under Secretary; Dr. Stanley K. Hornbeck, advisor on political relations; Maxwell M. Hamilton, chief of the Far Eastern desk, and Joseph W. Ballan-tine, a Far Eastern expert.

  Excluded from this tiny group were the field commanders of major military and naval forces. Security mainly controlled, but the feeling that this high-level, mainly political information should be analyzed in Washington contributed to this decision. But while the actual intercepts—indeed, the very existence of MAGIC—were kept from them, such intelligence extracted from it as Washington thought would help them was sent to them, usually attributed to “highly reliable sources.” For example, on July 8, Lieutenant General Walter C. Short, commanding in Hawaii, was told that “Movement of Jap shipping from Japan has been suspended and additional merchant vessels are being requisitioned.” This information came from MAGIC.

  The Philippines constituted a special case. Cavite was the Navy’s most favorably situated intercept post for Tokyo radio traffic, particularly Tokyo-Berlin, of which Hawaii, the East and West coasts, and England combined could not get more than 50 per cent. To cut the number of retransmissions of intercepts from Cavite to Washington, and thus reduce the danger of Japanese discovery of the MAGIC operation, the Navy in March sent out a PURPLE machine to the Philippines, OP-20-GY radioed the daily PURPLE and J19 keys to Fabian’s unit; he applied these to the messages intercepted by his and the Army’s intercept stations. He was then to forward the important solutions by radio. This procedure was practically abandoned later in the year, when almost every PURPLE message was important and all intercepts bearing its indicator were retransmitted to Washington. The Philippines were also regarded as the most threatened American outpost, and since diplomatic MAGIC was available right there because of a geographical accident, it went to General Douglas MacArthur and to Admiral Thomas C. Hart.

  In sending the MAGIC keys to Fabian, OP-20-GY employed a restricted cipher. Had the messages been sent using the general Navy keys, any of the many ships and shore installations holding those keys could have read them. Worse, had the Japanese worked an Oriental MAGIC of their own on these general keys, they would have learned of America’s most precious secret. The most secure naval cryptosystem was the E.C.M., or Electric Coding Machine, a device similar to but much stronger than PURPLE, which used a kind of code-wheel called a rotor. The MAGIC cipher used the E.C.M. with a special set of rotors, resulting, in effect, in a new cipher. Traffic in this cryptochannel, called COPEK, was kept down, and extra precautions were taken to guard against occurrences that might aid cryptanalysis. Only officers of the radio intelligence organizations in Washington, Cavite, and Honolulu held the rotors. They also used COPEK to exchange information on Japanese naval codes that they were solving.

  Rochefort in Hawaii could read the COPEK messages sending diplomatic-code keys to Fabian, and it may have been from him that Lieutenant Commander Edwin T. Layton, intelligence officer for the Pacific Fleet, learned that the Asiatic Fleet had the diplomatic MAGIC. On March 11, 1941, he asked McCollum to send it out to him. The head of the Far Eastern branch of naval intelligence declined, expounding what might be called the official line. On April 22 he wrote:

  I thoroughly appreciate that you would probably be much helped in your daily estimates if you had at your disposal the DIP. This, however, brings up matters of security, et cetera, which would be very difficult to solve…. It seems reasonable to suppose that the Department should be the origin for evaluated political situations, as its availability of information is greater than that of any command afloat, however large, its staff is larger and it should be in a position to evaluate the political consequences…. I should think that the forces afloat should, in general, confine themselves to the estimate of the strategic and tactical situations with which they will be confronted when the time of action arrives. The material you mentioned can necessarily have but passing and transient interest as action in the political sphere is determined by the Government as a whole and not by the forces afloat…. In other words, while you and the Fleet may be highly interested in politics, there is nothing that you can do about it.

  The inconsistency of this position reflects Washington’s more basic inconsistency of, on the one hand, trying to keep MAGIC from the field commanders for security reasons and, on the other, constructing PURPLE machines for them.

  Nevertheless, despite Washington’s determination not to send MAGIC to the field, not to use the ordinary Navy cipher for it, and never to identify it as such in dispatches, the Navy in July wired Admiral Husband E. Kimmel, commanding the Pacific Fleet, a whole series of messages that gave the very serial numbers of the Japanese diplomatic messages in summarizing their contents! And on July 19, Washington began a message “PURPLE 14 July Canton to Tokyo” an
d continued with a quote from it. This practice ceased in August, suggesting tightened security, but again on December 3 the Navy clearly indicated Japanese intercepts as the source of its information.

  The tightening may have resulted from several scares that Washington had just had. In March, State lost MAGIC memorandum No. 9. A horrified Army intelligence officer once found another MAGIC memorandum casually discarded in the wastebasket of Brigadier General Edwin M. (Pa) Watson, the President’s military aide. In Boston the F.B.I, picked up a man connected with the cryptanalytic work who was attempting to sell information on it. The worst frights of all came in the spring of 1941.