Hi everybody, bit of a warning here: The recovery key generated during the installation of Ubuntu 23.10 (if you select tpm-backed fde) cannot be used to unlock the disk outside of boot, as in any ‘cryptsetup’ command and so on will not accept the recovery key. unlocking when accessed from different system does not work etc.

You can use it to unlock the disk while booting if your tpm somehow fails, but ONLY in that specific situation.

I kind of purposefully broke my tpm keys to see if it could be restored with 23.10 and ended up having to reinstal, as I ended up having to enter the recovery key at boot every time and no way of adding additional unlock options to the volume, as cryptsetup would not accept the recovery key as passphrase.

This bug could be very bad for new users.

See this bug report: https://bugs.launchpad.net/ubuntu-desktop-installer/+bug/2039741

  • mvirts@lemmy.world
    link
    fedilink
    arrow-up
    16
    ·
    9 months ago

    Yet another reason why I wouldn’t put anything important on a tpm encrypted volume 😹 I just don’t trust it

    • Skull giver@popplesburger.hilciferous.nl
      link
      fedilink
      arrow-up
      11
      ·
      edit-2
      9 months ago

      I mean, the TPM did exactly what it was supposed to do. The firmware update warned about resetting the TPM, and it got reset. This way, if the NSA ever manages to compel TPM manufacturers to write key extraction firmware, they still can’t get to your secrets through software alone.

      This is just the installer not configuring a backup key for data recovery after such updates happen.

      • ichbinjasokreativ@lemmy.worldOP
        link
        fedilink
        arrow-up
        3
        ·
        9 months ago

        That’s true, but the issue is that cryptsetup does not accept the recovery key as a passphrase for the disk. Once the tpm gets reset, the user has to always enter the recovery key and cannot implement a new key to luks and the tpm.

        • Skull giver@popplesburger.hilciferous.nl
          link
          fedilink
          arrow-up
          5
          ·
          edit-2
          8 months ago

          cryptsetup is not at fault here, it’s the Snap people encoding their keys in a weird way. Obviously, this problem should be fixed, but it’s possible to recover from this failure at least. cryptsetup shouldn’t need to deal with the weird key formats the Ubuntu people have invented.

          You can add up to 32 key slots for a LUKS2 volume, and every one of those will allow you to add or remove keys. I have done so myself using a key file at one point. If you have any working key, you can add a new key to your system. Your recovery key would’ve worked if it was generated by systemd-enroll rather than whatever Ubuntu did.

          To fix the problem you have, “recovery keys” need to be turned into real keys. After a bit of conversion, Snap will just call cryptsetup like a normal Linux install would: *cmdAddRecoveryKey.Execute() -> AddRecoveryKeyToLUKSDevice -> AddRecoveryKeyToLUKSDeviceUsingKey -> AddKey.

          All the “recovery key” mechanism is doing is generate a key by itself (a very secure key, but just a normal key nonetheless) and pass it on through to cryptsetup. This key isn’t a normal string you can type in (as it’s purely random), which is why it’s encoded in a weird integer format that’s easy to type into a numpad.

          The snap commands aren’t printing the real key, they’re converting your input to keys during boot. However, if the drive unlocks at all, the user can add a new key, if they can just find how to get the real key out of their system. runFDERevealKeyCommand seems very suspect to me, I believe that command will dump the real key out, though I can’t find the source. I think you need to run fde-reveal-key and feed it something (JSON, I think?).

          This post contains a Go program that will decode the actual key from whatever Snap/Ubuntu turned it into and mount the partition; the parsing code was taken from Snap itself. I’ve taken the code and thrown together the following program to decode the key for your:

          package main
          
          import (
          	"encoding/binary"
          	"errors"
          	"fmt"
          	"os"
          	"strconv"
          )
          
          // ParseRecoveryKey Parse[16]byte interprets the supplied string and returns the corresponding [16]byte. The recovery key is a
          // 16-byte number, and the formatted version of this is represented as 8 5-digit zero-extended base-10 numbers (each
          // with a range of 00000-65535) which may be separated by an optional '-', eg:
          //
          // "61665-00531-54469-09783-47273-19035-40077-28287"
          //
          // The formatted version of the recovery key is designed to be able to be inputted on a numeric keypad.
          func ParseRecoveryKey(s string) (out [16]byte, err error) {
          	for i := 0; i < 8; i++ {
          		if len(s) < 5 {
          			return [16]byte{}, errors.New("incorrectly formatted: insufficient characters")
          		}
          		x, err := strconv.ParseUint(s[0:5], 10, 16) // Base 10 16 bit int
          		if err != nil {
          			return [16]byte{}, errors.New("incorrectly formatted")
          		}
          		binary.LittleEndian.PutUint16(out[i*2:], uint16(x))
          
          		// Move to the next 5 digits
          		s = s[5:]
          		// Permit each set of 5 digits to be separated by an optional '-', but don't allow the formatted key to end or begin with one.
          		if len(s) > 1 && s[0] == '-' {
          			s = s[1:]
          		}
          	}
          
          	if len(s) > 0 {
          		return [16]byte{}, errors.New("incorrectly formatted: too many characters")
          	}
          
          	return
          }
          
          func selfTest() bool {
          	_, e := ParseRecoveryKey("61665-00531-54469-09783-47273-19035-40077-28287")
          
          	if e != nil {
          		return false
          	} else {
          		return true
          	}
          }
          
          func main() {
          	if !selfTest() {
          		fmt.Println("Self-test failed, something went wrong during compilation")
          		return
          	}
          
          	fmt.Println("Please enter your Snap-encoded recovery key below:")
          	var recoveryKey string
          	_, err := fmt.Scanln(&recoveryKey)
          	if err != nil {
          		fmt.Println("Failed to read your key!")
          	}
          
          	if key, e := ParseRecoveryKey(recoveryKey); e != nil {
          		fmt.Printf("Failed to decode recovery key; %s\n", e)
          	} else {
          		fmt.Printf("Your recovery key is: ")
          		_, _ = os.Stderr.Write(key[:])
          		fmt.Println()
          	}
          }
          

          Steps to run that program:

          1. Install Go
          2. Save the file somewhere (recover.go)
          3. Run the command go run recover.go

          You will find the key dumped into the terminal, but there’s a good chance this is Unicode gibberish. Run the following command to save the key to a file: go run recover.go 2> key.txt; that will dump the key to key.txt rather than print it out. You can then use key.txt to add another key to a LUKS container, for example: cryptsetup luksAddKey --key-file=key.txt /dev/sda4 newkeyheremakesureitsverysecure.

          What I don’t get, is why the Ubuntu folks didn’t take the 16-byte key they generated, turned it into their own weird key format, and use that as a key string. That would keep their generation compatible with cryptsetup, would make the key equally easy to enter and equally secure, and wouldn’t expose these weird bugs. It would’ve cost what, 32 extra bytes in memory?

          • ichbinjasokreativ@lemmy.worldOP
            link
            fedilink
            arrow-up
            2
            ·
            8 months ago

            I wasn’t blaming cryptsetup, my mistake if it came across as though I did.

            Thank you for taking the time to look into a possible fix, I might just reinstall with tpm-backed fde again to see if this really works.

          • MAFoElffen@lemmy.world
            link
            fedilink
            arrow-up
            1
            ·
            edit-2
            8 months ago

            @Skull giver – I mentioned this above, but couldn’t link it to you.

            I like the code but the go run recover.go 2> key.text does not redirect the key to a file. It does not get input of the recovery key, so errors. Could you please add a few lines to write it directly to that file, instead of displaying it? (or output to both?)

            As someone thought, what is displayed onscreen is jibberish, because console cannot display raw hex characters… I’m thinking the LUKS key in the keyslot is raw or hex.

            As I now know, it is translated. And we can get the recovery key (hopefully) translated the other way around…

            If I can just find the valid key value, and write it to a file, then I can help @inchbinjasokreativ to write it back to his TPM. I’ve already written a BASH script to do that, but am just missing that key-file. I have the problem replicated to a VM, so can test it on that first.

            • Skull giver@popplesburger.hilciferous.nl
              link
              fedilink
              arrow-up
              2
              ·
              edit-2
              8 months ago

              I’m pretty sure they key is just random bytes so yeah I guess the console can trip up trying to display the key.

              Here’s the code to also dump the key to a file directly:

              		//fmt.Printf("Your recovery key is: ")
              		//_, _ = os.Stderr.Write(key[:])
              		//fmt.Println()
                              // ^ comment out the old code like that so it doesn't dump the key anymore
                              // new code starts here:
              		fmt.Println("Dumping to ./key.out for your convenience!")
              
              		if err = os.WriteFile("key.out", key[:], 0600); err != nil {
              			fmt.Printf("Error writing to key.out: %s", err)
              		}
              

              Comment out the old print line and add the new lines, and then the code should dump the output to a key.out directly. I think you can use that as a key file? If not, you can probably pipe it to stdin for cryptsetup.

              Edit: here’s a pastebin of the entire code dump: https://pastebin.com/WdFNRb7C

              I really hope Lemmy fixes the HTML character conversion bug, it’s getting annoying to have to look for workarounds every time I post code.

              • MAFoElffen@lemmy.world
                link
                fedilink
                arrow-up
                3
                ·
                8 months ago

                Success! I have a working key file…

                root@ubuntu:/home/ubuntu/Downloads# cryptsetup -v luksOpen /dev/sda3 luks-01 --key-file ./key-file.key
                No usable token is available.
                Warning: keyslot operation could fail as it requires more than available memory.
                Key slot 1 unlocked.
                Command successful.
                root@ubuntu:/home/ubuntu/Downloads# cryptsetup -v luksOpen /dev/sda4 luks-02 --key-file ./key-file.key
                No usable token is available.
                Warning: keyslot operation could fail as it requires more than available memory.
                Key slot 1 unlocked.
                Command successful.
                

                Success on the first volume, which I picked as first because it was only 53M in size. Mounted it to /mnt… And guess what I found inside it?

                root@ubuntu:/home/ubuntu/Downloads# ls -l /mnt/device/private-keys-v1/
                total 4
                -rw------- 1 root root 2459 Oct 18 18:29 O8CbAEpnfm7jGKkMqnokmdMBlE1oV6Xma_bUNudlshDYPxE4aJNhbhiGnF360Ze4
                

                That is a key, but not connected to either LUKS container there… I dumped the headers of both LUKS. There are 2 key-slots, and the key translated from the recovery key is in slot one of both containers, The second key-slot’s key must be the TPM’s key, which is unknown if that is stored anywhere except the TPM…

                But is shouldn’t matter now… Because that key-file did work to add a new passphrase to both LUKS containers.

                Thank you @Skull giver.

      • kautau@lemmy.world
        link
        fedilink
        arrow-up
        1
        ·
        edit-2
        9 months ago

        Yeah I’m hybrid Windows/Linux user, but many of my drives are Bitlocker encrypted. I need to install a bios update. To do so, it requires me to decrypt every bitlocker drive and not just the OS drive. Because the TPM keystore is based on the OS key store for each bitlocker drive. It’s frustrating, but it makes sense, so I’m all for additional security

  • MAFoElffen@lemmy.world
    link
    fedilink
    arrow-up
    12
    ·
    edit-2
    8 months ago

    As you can see by my UserName, I am the person who filed that Bug as a danger to users… Specifically brought up by @ichbinsokreativ, me trying to help him…

    Thank you Skull giver for the golang code, I had to debug your code, to get it to run, because of the forum transposing HTML char codes for some characters… Unfortunatley, displaying the result onto console displays ‘jibberish’, as you thought might happen. Console cannot display raw hex characters.

    Redirecting the output as you posted doesn’t work, as the script then doesn’t get the input of the recovery key, so errors.

    I have a request… I know a lot of languages, but GO isn’t one of them. Please… Could you please add a few lines to write the result directly to a file called recovery.key? Then if raw or hex, it would get to a key-file… Then I can test if that is going to work to add additional keys to the LUKS containers, and to be able to help people re-enroll the TPM key.

    If you could, they we would have a recovery work-around.

    Yes. They did something similar for past ZFS encryptions i their canned installs, but used native ZFS encryption, with a locked encrytped keyfile stored within a LUKS container, that had to be unlocked and mounted before unlocking the ZFS pools. Sometimes I don’t follow the logic behind some things.

    It seems like they often add complication to what should be simpler.

    • Skull giver@popplesburger.hilciferous.nl
      link
      fedilink
      arrow-up
      2
      ·
      8 months ago

      I picked Go because that’s the language the Snap people wrote the algorithm in. I’m far from an expert in the language, but I was too lazy to port it to Python or Rust.

  • stealthnerd@lemmy.world
    link
    fedilink
    arrow-up
    1
    ·
    9 months ago

    Okay so I just read up on this. It’s it true that TPM backed FDE only allows snaps?!?

    Debs are completely unsupported?

    • ichbinjasokreativ@lemmy.worldOP
      link
      fedilink
      arrow-up
      1
      ·
      9 months ago

      The ubuntu implementation is based on snap, but if you want to purge snapd (just install another distro at that point though), you can implement encryption as usual and add the tpm later on. Snap is only neccessary if you enable tpm-backed fde from setup.

  • Erik@lemmy.world
    link
    fedilink
    arrow-up
    1
    ·
    9 months ago

    Is it still possible to add an extra key (in another slot) to unlock it with cryptsetup? Adding this extra key might beat the purpose of using the TPM but if you choose a long random key and store it in a password manager that should still be pretty safe right?