Hi everybody, bit of a warning here: The recovery key generated during the installation of Ubuntu 23.10 (if you select tpm-backed fde) cannot be used to unlock the disk outside of boot, as in any ‘cryptsetup’ command and so on will not accept the recovery key. unlocking when accessed from different system does not work etc.

You can use it to unlock the disk while booting if your tpm somehow fails, but ONLY in that specific situation.

I kind of purposefully broke my tpm keys to see if it could be restored with 23.10 and ended up having to reinstal, as I ended up having to enter the recovery key at boot every time and no way of adding additional unlock options to the volume, as cryptsetup would not accept the recovery key as passphrase.

This bug could be very bad for new users.

See this bug report: https://bugs.launchpad.net/ubuntu-desktop-installer/+bug/2039741

  • Skull giver@popplesburger.hilciferous.nl
    link
    fedilink
    arrow-up
    5
    ·
    edit-2
    9 months ago

    cryptsetup is not at fault here, it’s the Snap people encoding their keys in a weird way. Obviously, this problem should be fixed, but it’s possible to recover from this failure at least. cryptsetup shouldn’t need to deal with the weird key formats the Ubuntu people have invented.

    You can add up to 32 key slots for a LUKS2 volume, and every one of those will allow you to add or remove keys. I have done so myself using a key file at one point. If you have any working key, you can add a new key to your system. Your recovery key would’ve worked if it was generated by systemd-enroll rather than whatever Ubuntu did.

    To fix the problem you have, “recovery keys” need to be turned into real keys. After a bit of conversion, Snap will just call cryptsetup like a normal Linux install would: *cmdAddRecoveryKey.Execute() -> AddRecoveryKeyToLUKSDevice -> AddRecoveryKeyToLUKSDeviceUsingKey -> AddKey.

    All the “recovery key” mechanism is doing is generate a key by itself (a very secure key, but just a normal key nonetheless) and pass it on through to cryptsetup. This key isn’t a normal string you can type in (as it’s purely random), which is why it’s encoded in a weird integer format that’s easy to type into a numpad.

    The snap commands aren’t printing the real key, they’re converting your input to keys during boot. However, if the drive unlocks at all, the user can add a new key, if they can just find how to get the real key out of their system. runFDERevealKeyCommand seems very suspect to me, I believe that command will dump the real key out, though I can’t find the source. I think you need to run fde-reveal-key and feed it something (JSON, I think?).

    This post contains a Go program that will decode the actual key from whatever Snap/Ubuntu turned it into and mount the partition; the parsing code was taken from Snap itself. I’ve taken the code and thrown together the following program to decode the key for your:

    package main
    
    import (
    	"encoding/binary"
    	"errors"
    	"fmt"
    	"os"
    	"strconv"
    )
    
    // ParseRecoveryKey Parse[16]byte interprets the supplied string and returns the corresponding [16]byte. The recovery key is a
    // 16-byte number, and the formatted version of this is represented as 8 5-digit zero-extended base-10 numbers (each
    // with a range of 00000-65535) which may be separated by an optional '-', eg:
    //
    // "61665-00531-54469-09783-47273-19035-40077-28287"
    //
    // The formatted version of the recovery key is designed to be able to be inputted on a numeric keypad.
    func ParseRecoveryKey(s string) (out [16]byte, err error) {
    	for i := 0; i < 8; i++ {
    		if len(s) < 5 {
    			return [16]byte{}, errors.New("incorrectly formatted: insufficient characters")
    		}
    		x, err := strconv.ParseUint(s[0:5], 10, 16) // Base 10 16 bit int
    		if err != nil {
    			return [16]byte{}, errors.New("incorrectly formatted")
    		}
    		binary.LittleEndian.PutUint16(out[i*2:], uint16(x))
    
    		// Move to the next 5 digits
    		s = s[5:]
    		// Permit each set of 5 digits to be separated by an optional '-', but don't allow the formatted key to end or begin with one.
    		if len(s) > 1 && s[0] == '-' {
    			s = s[1:]
    		}
    	}
    
    	if len(s) > 0 {
    		return [16]byte{}, errors.New("incorrectly formatted: too many characters")
    	}
    
    	return
    }
    
    func selfTest() bool {
    	_, e := ParseRecoveryKey("61665-00531-54469-09783-47273-19035-40077-28287")
    
    	if e != nil {
    		return false
    	} else {
    		return true
    	}
    }
    
    func main() {
    	if !selfTest() {
    		fmt.Println("Self-test failed, something went wrong during compilation")
    		return
    	}
    
    	fmt.Println("Please enter your Snap-encoded recovery key below:")
    	var recoveryKey string
    	_, err := fmt.Scanln(&recoveryKey)
    	if err != nil {
    		fmt.Println("Failed to read your key!")
    	}
    
    	if key, e := ParseRecoveryKey(recoveryKey); e != nil {
    		fmt.Printf("Failed to decode recovery key; %s\n", e)
    	} else {
    		fmt.Printf("Your recovery key is: ")
    		_, _ = os.Stderr.Write(key[:])
    		fmt.Println()
    	}
    }
    

    Steps to run that program:

    1. Install Go
    2. Save the file somewhere (recover.go)
    3. Run the command go run recover.go

    You will find the key dumped into the terminal, but there’s a good chance this is Unicode gibberish. Run the following command to save the key to a file: go run recover.go 2> key.txt; that will dump the key to key.txt rather than print it out. You can then use key.txt to add another key to a LUKS container, for example: cryptsetup luksAddKey --key-file=key.txt /dev/sda4 newkeyheremakesureitsverysecure.

    What I don’t get, is why the Ubuntu folks didn’t take the 16-byte key they generated, turned it into their own weird key format, and use that as a key string. That would keep their generation compatible with cryptsetup, would make the key equally easy to enter and equally secure, and wouldn’t expose these weird bugs. It would’ve cost what, 32 extra bytes in memory?

    • ichbinjasokreativ@lemmy.worldOP
      link
      fedilink
      arrow-up
      2
      ·
      9 months ago

      I wasn’t blaming cryptsetup, my mistake if it came across as though I did.

      Thank you for taking the time to look into a possible fix, I might just reinstall with tpm-backed fde again to see if this really works.

    • MAFoElffen@lemmy.world
      link
      fedilink
      arrow-up
      1
      ·
      edit-2
      9 months ago

      @Skull giver – I mentioned this above, but couldn’t link it to you.

      I like the code but the go run recover.go 2> key.text does not redirect the key to a file. It does not get input of the recovery key, so errors. Could you please add a few lines to write it directly to that file, instead of displaying it? (or output to both?)

      As someone thought, what is displayed onscreen is jibberish, because console cannot display raw hex characters… I’m thinking the LUKS key in the keyslot is raw or hex.

      As I now know, it is translated. And we can get the recovery key (hopefully) translated the other way around…

      If I can just find the valid key value, and write it to a file, then I can help @inchbinjasokreativ to write it back to his TPM. I’ve already written a BASH script to do that, but am just missing that key-file. I have the problem replicated to a VM, so can test it on that first.

      • Skull giver@popplesburger.hilciferous.nl
        link
        fedilink
        arrow-up
        2
        ·
        edit-2
        9 months ago

        I’m pretty sure they key is just random bytes so yeah I guess the console can trip up trying to display the key.

        Here’s the code to also dump the key to a file directly:

        		//fmt.Printf("Your recovery key is: ")
        		//_, _ = os.Stderr.Write(key[:])
        		//fmt.Println()
                        // ^ comment out the old code like that so it doesn't dump the key anymore
                        // new code starts here:
        		fmt.Println("Dumping to ./key.out for your convenience!")
        
        		if err = os.WriteFile("key.out", key[:], 0600); err != nil {
        			fmt.Printf("Error writing to key.out: %s", err)
        		}
        

        Comment out the old print line and add the new lines, and then the code should dump the output to a key.out directly. I think you can use that as a key file? If not, you can probably pipe it to stdin for cryptsetup.

        Edit: here’s a pastebin of the entire code dump: https://pastebin.com/WdFNRb7C

        I really hope Lemmy fixes the HTML character conversion bug, it’s getting annoying to have to look for workarounds every time I post code.

        • MAFoElffen@lemmy.world
          link
          fedilink
          arrow-up
          3
          ·
          9 months ago

          Success! I have a working key file…

          root@ubuntu:/home/ubuntu/Downloads# cryptsetup -v luksOpen /dev/sda3 luks-01 --key-file ./key-file.key
          No usable token is available.
          Warning: keyslot operation could fail as it requires more than available memory.
          Key slot 1 unlocked.
          Command successful.
          root@ubuntu:/home/ubuntu/Downloads# cryptsetup -v luksOpen /dev/sda4 luks-02 --key-file ./key-file.key
          No usable token is available.
          Warning: keyslot operation could fail as it requires more than available memory.
          Key slot 1 unlocked.
          Command successful.
          

          Success on the first volume, which I picked as first because it was only 53M in size. Mounted it to /mnt… And guess what I found inside it?

          root@ubuntu:/home/ubuntu/Downloads# ls -l /mnt/device/private-keys-v1/
          total 4
          -rw------- 1 root root 2459 Oct 18 18:29 O8CbAEpnfm7jGKkMqnokmdMBlE1oV6Xma_bUNudlshDYPxE4aJNhbhiGnF360Ze4
          

          That is a key, but not connected to either LUKS container there… I dumped the headers of both LUKS. There are 2 key-slots, and the key translated from the recovery key is in slot one of both containers, The second key-slot’s key must be the TPM’s key, which is unknown if that is stored anywhere except the TPM…

          But is shouldn’t matter now… Because that key-file did work to add a new passphrase to both LUKS containers.

          Thank you @Skull giver.