back to article Declassified files reveal how pre-WW2 Brits smashed Russian crypto

Efforts by British boffins to thwart Russian cryptographic cyphers in the 1920s and 1930s have been declassified, providing fascinating insights into an obscure part of the history of code breaking. America's National Security Agency this week released papers from John Tiltman, one of Britain’s top cryptanalysts during the …

  1. John Sager

    That theory is testable

    With the volume of Venona decrypt available (only a few percent of the total AFAIR), it should be possible to verify if it's always or mostly a mix of KGB traffic with GRU traffic that decrypts.

  2. Alan Brown Silver badge

    > "They both selected a secure printing works that usually produced banknotes and gave strict instructions that only two copies of each pad should be printed," Lomas commented. "The printers decided to print four copies of each pad then send two each to the KGB and GRU.

    Who's to say they didn't do the same for banknotes?

    1. Anonymous Coward
      Anonymous Coward

      I've seen lots of Soviet era banknotes, as they still used them up until at least the mid 1990s. The one ruble note never seemed to have a serial number greater than seven digits, so they might not have been unique.

    2. TheVogon

      "Who's to say they didn't do the same for banknotes?"

      No one. But it would have devalued the currency exactly the same.

      1. Jim Mitchell Silver badge

        @TheVogon

        Wouldn't that only happen if people knew? I mean, print a run for the government, print some duplicates for you...

        1. eldakka Silver badge

          > Wouldn't that only happen if people knew? I mean, print a run for the government, print some duplicates for you...

          I can see the print shop operators now:

          "one for you, one for me, one for you, one for me"

          1. Ken 16
            Big Brother

            From each according to his ability

            to each according to his need?

          2. J.G.Harston Silver badge

            Nah, it goes: one you you one for me, two for you one two for me, three for you one two three for me....

        2. TheVogon

          "Wouldn't that only happen if people knew? I mean, print a run for the government, print some duplicates for you..."

          No, it it's basic economics that printing more currency will devalue it:

          https://www.economicshelp.org/blog/634/economics/the-problem-with-printing-money/

          1. Red Ted Silver badge
            FAIL

            Devaluation of the Ruble

            No, it it's basic economics that printing more currency will devalue it

            That's the ruble that Comrade Lenin specified the value of?

            Also in an economy the size of the USSR, you'd need a lot of extra bank notes to reduce it by much, so as long as you kept the numbers low relative the amount of money in circulation it wouldn't have much effect, but you could still be very very rich.

            1. TheVogon

              Re: Devaluation of the Ruble

              "That's the ruble that Comrade Lenin specified the value of?"

              He controlled the official foreign exchange rates and many prices. No one given a choice stuck to the official exchange rates. Or the Rouble for that matter.

              The point was that the devaluation effect does not require consumer knowledge that additional currency has been issued.

          2. strum

            >it's basic economics that printing more currency will devalue it

            Over-reliance on economic theory. If a ruble is worth what Stalin says it is, anyone who says otherwise won't need rubles no more. End of theory.

            1. TheVogon

              "Over-reliance on economic theory. If a ruble is worth what Stalin says it is"

              No, because then it doesnt function as currency in a normal market economy. It's effectively a scrip from a truck system of payment.

      2. Anonymous Coward
        Anonymous Coward

        "But it would have devalued the currency exactly the same."

        True but to the printer it would still be "free" if slightly devalued money.

  3. Anonymous Coward
    Anonymous Coward

    Find it difficult to believe

    Most USSR pads from that period were just bog standard books. The outgoing communications from Russia to spies abroad consisted of sequences of numbers which were page and symbol positions in it. This one repeats a gazillion times in literature (both historical and fiction) so there is quite likely to be a grain of truth in it. As a result there was absolutely nothing incriminating in a spy's house as the books in question were mostly fiction.

    I do not know about the communications back to USSR, but I would be surprised if they were different. If you have a system which works flawlessly in one direction, why bother with something in the other?

    1. Trygve Henriksen

      Re: Find it difficult to believe

      It's not so difficult to believe...

      The pads were used by embassies and such, as they often have a need to confer with home over a secure channel, knowing that anyone could be listening in on the conversation.

      Spies would mostly try to avoid any situation whereeven part of the conversation could be captured, as not only the message, but also the sender and receiver would be secret.

      OTPs and other codebooks is something you'd expect to find in an embassy or in the comms on a battleship, but not in someone's home. It's also very difficult to transfer a code books and OTPs to field agents without them being compromised.

      1. JeffyPoooh
        Pint

        Re:filling the OTP by restricted-purpose 2nd use

        TH noted, "It's also very difficult to transfer....OTPs to field agents without them being compromised."

        I suspect that a given section of One-Time Pad (OTP) could be safely reused a 2nd time, but *ONLY* to send some random gibberish to essentially refill the OTP itself at the far end. This would permit effectively-endless OTPs to be more easily distributed (i.e. continuously refilled) using the same remote communications channel, thus avoid the bother and peril of physical delivery.

        Here's why it seems safe: The fact that the 2nd use of the same OTP block is to carry random gibberish (i.e. more random bits to refill the OTP) means that the usual subtraction attack doesn't accomplish anything. Think about it...

        And, since each encrypted message remains fully-independent of the previous, they don't carry any residual information about other messages. So there's 'nobody home' in the statistical analysis department.

        The only obvious downside is that the resultant chain of communications would be like a house of cards. One decryption anywhere in the sequence, and the entire sequence of messages would fall open. Risk management would be required to evaluate the pros and cons.

        The same basic concept (assuming my suspicion is correct) would apply to both manual OTPs and the electronic high-speed equivalent; and that might be more widely applicable for modern applications, even endless streaming.

    2. Rich 11 Silver badge

      Re: Find it difficult to believe

      If you have a system which works flawlessly in one direction, why bother with something in the other?

      It only works flawlessly as long as the choice of book remains unknown: discover it and you can decrypt all the traffic you've intercepted in the past between those two parties. Users of a one-time pad are supposed to destroy each sheet after use so that even if the pad falls into enemy hands it can never be used to expose historic communications.

      1. David 164

        Re: Find it difficult to believe

        Now with the advent of digital library and very large computing capacity, I presume that using this method now insecure as places like GCHQ can brute force crack the code by running through every book in their digital library until they find the right one.

        1. phuzz Silver badge

          Re: Find it difficult to believe

          GCHQ can brute force crack the code by running through every book in their digital library

          Not every book is digitised, and different printings of books can vary enough to make them effectively unique as cipher pads. I do agree though that's it's more tricky than it used to be, but if Alice and Bob are sufficiently cryptic in how they define which page/paragraph/word/letter then they could defy purely automated analysis.

      2. Hans Neeson-Bumpsadese

        Re: Find it difficult to believe

        Users of a one-time pad are supposed to destroy each sheet after use so that even if the pad falls into enemy hands it can never be used to expose historic communications.

        Which works unless your method of disposal is to re-purpose them as bog roll because your supplies of that has run out. During the cold war, western spies went fishing used paper from the sewerage systems under at least one eastern bloc embassy to get hold of used one-time pads.

  4. JacobZ

    The clue is in the name

    "By reusing one-time pads..."

    There's a clue in the name, folks.

    1. Adam 1

      Re: The clue is in the name

      It is, but many people may not understand how it enables differential cryptanalysis. They may intuitively understand that it lowers their own security but totally misunderstand the threat model. In their minds, the risk is about whether their own message may be read, not whether they are enabling the reading of another message if the adversary holds both messages but not the key.

      1. DropBear

        Re: The clue is in the name

        Also, being aware that (properly used, properly random) one-time pads are the strongest encryption there is, laypeople might not grasp the magnitude of their gaffe when re-using it twice, possibly thinking "well maybe it's a bit weaker this way but surely it must still be plenty strong..."

        1. Antron Argaiv Silver badge

          Re: The clue is in the name

          The Russians are well known for their mathematical ability. It's surprising to me that the risks of re-use were not strongly impressed on the users.

          Oh, well, their loss. Maybe they've learned.

          1. Anonymous Coward
            Anonymous Coward

            Re: The clue is in the name

            "The Russians are well known for their mathematical ability. It's surprising to me that the risks of re-use were not strongly impressed on the users."

            The article implies that the pad printers sold duplicate pads to both agencies. The agencies and their users were probably not aware of this.

      2. keithpeter Silver badge
        Coat

        Re: The clue is in the name

        OK, so if I write a couple of short messages as plain ascii (7 bit) and then use

        xxd -b <message-files>

        to dump the binary (1s and 0s), reformat to mimic a paper tape or something, and then XOR the result to get rid of the two-time key, it should be a reasonable simulation of the problem facing the chaps in the 1950s?

    2. Jemma

      Re: The clue is in the name

      ... Doesn't matter - the NHS would still photostat them to save money.

      Reminds me of the mating call of the lesser-spotted incompetent teacher "I didn't photostat enough so it's one between two*" because I can't frigging count (and it's a maths lesson).

      *And not the fun type of one between two where one of the two is blonde and the other filipina.

  5. Milton

    Paranoia and hot pockets

    Paranoia about hyper-computers, quantum computers and rumoured breathroughs such as fast-factoring algorithms in the last five to 10 years seems to have fuelled a quiet resurgence in one time pads (OTPs).

    Thus Boris, politely invited to step out of the queue because he (a) travels alone, (b) has minimal luggage, (c) has a certain unmistakable bearing, emits a brief burning-plastic smell before he says "Bozhe moi, phone smokes!" and with practised humility explains in fractured English that his crappy East European phone must have a bad battery. Another quarter-gigabyte of OTP has just been roasted—with plausible deniability.

    And there are now many Borises, Jacks, Maurices, Joses and even a few Rachels and Tatianas, couriering the wondrous globe with excellent passports, over-rated language skills, lamentably giveaway body language (always the weak point) and tiny silicon chips the size of pinky-nails concealed hither, thither and even yon.

    We're close to inventing a (possibly quantum-tech) OTP which can be read only once, thereafter erasing itself without the need for Boris or Rachel to tickle the "Blown" button—useful, if only to relieve many small rooms in large airports of the smell of melted secrets.

    1. ArrZarr
      Coat

      Re: Paranoia and hot pockets

      When you started talking about "Boris", I though you meant Boris Johnson.

      The broken English part made sense and from there I had visions of BoJo being a Russian spy until I realised what you actually meant.

      1. Anonymous Coward
        Big Brother

        Re: Paranoia and hot pockets

        Oh, I'd been assuming that he was a Russian agent. It would really make a lot of sense: endlessly making damaging apparently-idiot-comments from the sidelines ('fuck business [... comrade]'), repeatedly destroying fragile consensus in the government, damaging our image abroad, travels abroad a lot where he no doubt has copious opportunities for assignations of various kinds, suspiciously supportive of Trump, and so on.

        I mean, obviously none of this is true and he's a good patriotic Englishman, of course. Of course.

      2. Anonymous Coward
        Anonymous Coward

        Stooge

        I can't imagine him being described as an intelligence asset nor even a useful idiot.

      3. Joe Gurman

        Re: Paranoia and hot pockets

        I thought the meant Boris Badenov, always foiled by moose and sqvirrel.

    2. DropBear

      Re: Paranoia and hot pockets

      I suggest that having your phone catch fire exactly when you're "invited to step out of the queue" would be the polar opposite of plausible deniability. Especially after it happened for the second time (with someone else).

      1. keithpeter Silver badge
        Coat

        Re: Paranoia and hot pockets

        My cheapo Android tablet gave me the choice of encrypting its storage when I set it up. Took a couple of minutes. I'm assuming the result is a 16Gb ssd filled with random numbers. Could a otp not just be made to look like an ssd with encrypted storage until the authorities started to compare a number of devices and realise the amazing coincidence of identical random numbers?

        Coat: Copy of MR-1418-RC in the (large) inside pocket

    3. frank ly

      Re: Paranoia and hot pockets

      "... lamentably giveaway body language (always the weak point)..."

      Can you tell me what you mean by this so I can work on improving my posture and behaviour?

    4. Lord Elpuss Silver badge

      Re: Paranoia and hot pockets

      @Milton

      Upvote purely for your writing style :D

    5. Sam Liddicott

      Re: Paranoia and hot pockets

      I hope you start writing for el Reg -- I mean not just in the comments section

    6. adam 40 Silver badge

      Re: Paranoia and hot pockets

      An OTP that can be read only once doesn't seem useful at all, it must be read precisely twice.

      Perhaps you mean two quantum-entangled OTP's? Now, THAT might be useful...

      1. Berwhale

        Re: Paranoia and hot pockets

        OTPs are produced in pairs, each one would only be read once (i.e. 1st copy of OTP held by agent used to encrypt, 2nd copy of OTP held by Control used to decrypt).

  6. YourNameHere

    persistence

    If you read some of the books like "Code Warriors" you will become to understand what word persistence and determination mean. They go through how some of these techniques were done. If I remember right they go through codes that were broken via this method. I just shook my head at how hard core, hard nosed and determined these types of people are.

  7. Hey Lobotoman! CALL -151!

    US FOIA request declassifying UK documents?

    I wonder how this FOIA disclosure was possible, as the US government has supplied source documents from its FIVE EYES partner. Presumably, the UK Govt had to declassify this first.

    1. Julz Silver badge

      Re: US FOIA request declassifying UK documents?

      # Hey Lobotoman! CALL -151!

      Our cousins over the pond routinely declassify stuff that is still classified here in Blighty. Causes considerable discomfort and shuffling around in seats among the select few.

    2. phuzz Silver badge

      Re: US FOIA request declassifying UK documents?

      If it's not harming US interests, then why not declassify? It's not like the UK can do much if they object.

      See also; the CIA declassifying details of U2 flights over the USSR in the 1960's, which confirmed the involvement of RAF pilots, whilst the UK files on the subject are still classified (if they even still exist).

  8. Rustbucket

    Alternative Venona

    The story I read is that as the Nazis advanced into Russia the part of the code section responsible for generating the random numbers for the one time pads was evacuated to the east but the printing staff were left behind, so they started reusing pages.

    They did not reuse whole books at a time but mixed up pages between new books. When the implications of the reuse were understood the staff were afraid to warn their superiors, because they would likely have been sent to Siberia or executed for the initial mistake.

    1. FrankAlphaXII

      Re: Alternative Venona

      That's basically how James Bamford described it in "Body of Secrets".

      There were 35,000 duplicate pages printed by the 8th Main Directorate of the KGB in early 1942, and they had 30,000 intercepts that were encrypted using said duplicate pages out of about a million intercepts. The duplicated one-time pads (I guess two-time pads) were used from 1942 to 1948.

      In the book, Bamford suggests that they were duplicated by Soviet cryptographers creating the one-time pads using carbon paper. It was careless and the Soviets paid for it.

  9. Anonymous Coward
    Anonymous Coward

    Russians! See? See?

    Like oh my God! I know, right?

    Respectfully,

    Rachel Maddow

    1. Anonymous Coward
      Anonymous Coward

      Re: Russians! See? See?

      Donald/Devin/Sarah,

      This is far beyond anything you can begin to hope to understand, so do yourself a favor and stop before you embarrass yourself further.

      Let the grown-ups and people with an IQ higher than their toothbrushes discuss it. Thanks.

      V/r,

      Signals Intelligence Collector

      1. Anonymous Coward
        Anonymous Coward

        Re: Russians! See? See?

        My apologies, will do!

        Just how much would you like your budget increased this year? Would another 50% work for you?

        No moral hazard there. No sir.

  10. John Smith 19 Gold badge
    Coat

    So kids, sometimes recycling is *bad*

    Uuhuu.

    1. Adam 1

      Re: So kids, sometimes recycling is *bad*

      Nonsense. All bits may be recycled. You just need to reuse them in random order.

      1. ChrisC Silver badge

        Re: So kids, sometimes recycling is *bad*

        "Nonsense. All bits may be recycled. You just need to reuse them in random order."

        OK, hands up who else finds it impossible to read this without it sounding exactly like Eric Morecambe?

  11. Anonymous Coward
    Anonymous Coward

    As any girl knows...

    ... you can't reuse your pads. And if you try, it gets messy.

  12. Anonymous Coward
    Anonymous Coward

    More recent communications

    Not to do with Russia directly, but fascinating is an article on the ANC's web site describing how their operatives communicated in Apartheid South Africa. The use of computers in the 1980s, basic encryption, then modems to transfer audio to cassette tape. Then find a phone and switch on the tape, to send screeches down a phone line. Receiving was the reverse of the above. Relevant to the article is the process this involved. A great read.

    "Talking To Vula"

    The Story of the Secret Underground Communications Network of Operation Vula

    by Tim Jenkin

    http://www.anc.org.za/content/talking-vula

    1. Anonymous Coward
      Anonymous Coward

      Re: More recent communications

      Reading through my post above, apologise I must for allowing my inner Yoda to over-ride my typing fingers. Not try to brief, must in future be done.

  13. wyatt
    Happy

    Happy to say that I've actually seen a one time letter pad in training about 15 years ago when I was in the army. Can't be many thatI'd have thought?

  14. Craig 8

    Interesting that Tiltman was involved, as this is very similar to the method he used to extract the Lorenz key stream, which lead to Tutte deducing the machine's mechanism and that lead to the design and construction of Colossus.

  15. Anonymous Coward
    Anonymous Coward

    Actually do-able

    There are some ways to do this without re-using you OTP, it is a fact that XOR ing two random numbers eg different sections/pages of a OTP together gives another (unbreakable) random number. Thus page 1 char 1 of your OTP can be used to modify each character of Page 2 etc extending your OTP length by the length squared, it is done for digital signatures on some embedded systems where the CRC is XOR'd by several different pages of the OTP (the more pages the better but XOR is fast on a micro)

    1. Anonymous Coward
      Anonymous Coward

      Re: Actually do-able

      AC wrote:

      "...Thus page 1 char 1 of your OTP can be used to modify each character of Page 2 etc extending your OTP length by the length squared ..."

      But surely if your OTP is just changed by a fixed value then combining two code-texts coded with the first and second version will remove the random element (which will be obvious because the text will reveal the statistical properties of combined plain-text - it will just be shifted by the fixed value) leaving you with a Caesar cipher which is trivial to solve.

    2. Anonymous Coward
      Anonymous Coward

      Re: Actually do-able

      This manifestly is not true, unless you choose the bits to xor together randomly in which case you are, in fact, adding more random information to the stream.

  16. msknight Silver badge

    It always strikes me as strange...

    ...that they didn't use another language other than their native, to communicate in.

    While everyone is trying to find patterns of Russian words, then if the correspondence was in English, for example, that would surely have made it more difficult for the people doing the cracking?

    Or is my thought pattern starting to show that it's nearing beer o'clock?

    1. keithpeter Silver badge
      Coat

      Re: It always strikes me as strange...

      @msknight

      https://en.wikipedia.org/wiki/Code_talker

      Use of minority languages was a thing when the need was for rapid communication of information that would be useless on a short time scale. Don't think people wanted to be fiddling with one time pads and doing mental arithmetic on battlefields.

    2. Robert Carnegie Silver badge

      A foreign language isn't code.

      It takes years to learn a foreign language properly, and people whose language it is can immediately understand you... unless you're dreadful.

      A dictionary of under 100 common words in any language liable to be used in this way should make it veey easy to detect.

    3. Anonymous Coward
      Anonymous Coward

      Re: It always strikes me as strange...

      Even if you don't know the language you can start looking for patterns which look like language based on the statistics of letters and n-grams of letters, which are very non-random in natural languages.

  17. Robert Carnegie Silver badge

    Outsourcing.

    Ass, you, bitten in.

  18. JeffyPoooh
    Pint

    Here is an explanation, with easy visuals...

    One of many explanations, but it's a good one.

    https://cryptosmith.com/2008/05/31/stream-reuse/

  19. maurizio.dececco

    Already known ..

    The Spy Catcher book by Peter Wright talk about this and a case of a decoded message that after something like 20 years of computing allowed to catch a spy.

    But probably that was classified material :-< ..

    Maurizio

  20. EveryTime

    One key element is that each communication started out with a cleartext header that identified the pad in use.

    This is the simplest way of keeping the sender and recipient in sync. Keeping things simple and reliable avoids the operators sending cleartext messages to recover when the inevitable screw-up occurs.

    This wouldn't have been a vulnerability with true one-time pads. Even reusing one-time pads wouldn't have been a problem if they had unique pad numbers.

  21. Antron Argaiv Silver badge
    Holmes

    The late Bob Morris, chief scientist at the NSA

    Now, *there's* an interesting family.

    Father of the (infamous) author of the first Internet worm:

    https://en.wikipedia.org/wiki/Robert_Tappan_Morris

    (apparently, he has redeemed himself, and now works for MIT)

    From Morris (the elder):

    Never underestimate the attention, risk, money and time that an opponent will put into reading traffic.

  22. Gel

    We are certain that this was not intentional by the Russians? Do not underestimate their knowledge and ability.

  23. herman Silver badge
    Facepalm

    Homer

    d'ой! - There FIFY

  24. Pat Harkin

    '"These could then be picked apart using a combination of statistics and predictable words" to decrypt the contents, he added.'

    But not by me, I added.

  25. cutterman

    Book based codes are very insecure (even if there are only two copies of the book in existence).

    The underlying language structure makes such codes intrinsically non-random and provides a wedge into the code.

    Only a _truly_ random sequence is _really_ secure, and surprisingly difficult to produce. Even pseudo-random numeric generators will eventually show a pattern which gives you a start, and that is all you need with a computer to do the heavy lifting.

    Mac

    1. Robert Sneddon

      PR(A)NG

      Pseudo-random number generators (PRNGs) aren't truly random, indeed they are peculiarly non-random. e.g. if the number 5 comes out it is followed by 24 every time. A PRNG will produce some or all of the numbers in its range exactly once before repeating the sequence but the repeat will be an exact copy of the first sequence and so on. PRNGs have their uses but cryptography isn't one of them.

      OTOH I know of a system that used radio noise to produce a RNG key for a digital device that had to be provably "random" under licencing restrictions -- slot machines.

  26. Mark Wallace

    Windows 10 would have been good for the one-time pads.

    Use it once; never even want to use it again.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2021