Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

Dear Hackers.

> Specifically in this case- I went back and tried to figure out what
> other database systems have an "encrypt EVERYTHING" option. I didn't
> have much luck finding one though. So I think we need to ask ourselves-
> the "check box" that we're trying to check off with TDE, do the other
> database system check that box? If so, then it looks like the "check
> box" isn't actually "encrypt EVERYTHING", it's more along the lines of
> "make sure all regular user data is encrypted automatically" or some
> such, and that's a very different requirement, which seems to be
> answered by the other systems by having a KMS + tablespace/database
> level encryption. We certainly shouldn't be putting a lot of effort
> into building something that is either overkill or won't be interesting
> to users due to limitations like "have to take the entire cluster
> offline to re-key it".
>
> Now, that KMS has to be encrypted using a master key, of course, and we
> have to make sure that it is able to survive across a crash, and it'd
> sure be nice if it was indexed.

Sorry, Does KMS here mean key Management System(or Service)?
I may be mistaken, but I know that KMS is managing cryptographic keys.
In other words, I kept the master key(or KEK) in KMS( not kept to
PostgreSQL server-side),
and PostgreSQL fetched the master key from KMS, and then encrypt or
decrypt it on the PostgreSQL server-side.
Of course, some KMS supports encryption function,
which is the function to encrypt plain text inside KMS. Is this
project aiming to use this function?

>
> A couple random ideas that probably won't work, but I'll put them out
> there for others to shoot down-
>
> Some kind of 2-phase WAL pass, where we do WAL replay for the
> non-encrypted bits first (which would include the KMS) and then go back
> and WAL replay the encrypted stuff. Seems terrible.

Sorry, Can you tell me an example what is the 2-phase WAL pass?
I know that WAL read process is decrypted WAL data when
reading an encrypted WAL page(per-page encrypt) or
WAL record(per-record encrypt) and then replay.
Is this a different case?

Best Regards.
Moon.

>
> An independent WAL for the KMS only. Ugh, do we need another walwriter
> then? and buffers, and lots of other stuff.
>
> Some kind of flat-file based approach with a temp file and renaming of
> files using durable_rename(), like what we used to do with
> pg_shadow/authid, and now do with replorigin_checkpoint and such?
>
> Something else?
>
> Thoughts?
>
> Thanks!
>
> Stephen