Exporting an EFS key requires being logged in as that user

One user wanted to move an EFS certificate/key to a different Windows installation. They were interested in taking the key out of the offline system, but this would be very difficult because of how the keys are stored. The key material is itself encrypted by another key derived from a variety of things. To export the key, one can use certmgr.msc or, if it doesn't want to export the private key, do a bit of PowerShell. Unfortunately, both methods require being logged in as the key's owner on the machine.