Pubkey Distribution Concept

Status: Shall be updated along with the current discussion. Feedback wanted (e.g. use gnupg-devel@).

See overview page.

Proposed solution

To find the pubkey of a given email-address an email client should:

1. Check its local cache
2. Ask the mail service provider via HTTPS, as specified in https://datatracker.ietf.org/doc/draft-koch-openpgp-webkey-service
3. Ask the mail service provider via DNS(SEC), as specified in https://datatracker.ietf.org/doc/draft-ietf-dane-openpgpkey
4. Consult a classic pubkey server (to see if there is a single match)
5. Resort to a fallback method using an initial email exchange.

Ask the mail service provider (MSP)

WKD draft spec 01 specifies the main features of how to ask the MSP. This part is called the web key directory (WKD).

2016-09-15: Smaller changes and additionals are still being discussed, but the main features are very likely to stay, so that implementations for clients and the MSP side have started.

(attachment:web-key-service-discovery.svg: see metadata in the SVG source-code for license of compilation and sub-graphics.)

The diagram shows the most important use-case:

• User A wants to send an email to B. The system will ask the MSP of B about the pubkey for the email address (1).
• After getting back one pubkey for users B (2), user A can now send an encrypted message (3)
• It gets transported via A's MSP to B's (4) and delivered to B (5).

The request URL looks like

https://example.org/.well-known/openpgpkey/hu/XXXX

With XXXX being a 32-char long string constructed of the mapped local part of the email, SHA-1 hashed and Z-Base-32 encoded.

Manage pubkey by email

The client will send an email to a submission email-address, may get a challenge email back to prove that it is in possession of the private key. Handling is done automatically without user interaction. This is called the web key service (WKS).

The draft spec 01 also shows how the pubkey publication can be managed between email client and provider.

2016-09-15: Some changes - mainly additions - are in discussion, like:

• About allowing an "auth-submit" option that will allow immediate publishing of te pubkey, because the submisson email came via an authenticated channel.
• How the email challenge will be done, so that the client may disregard emails automatically that are not coming as a result of his request.
• How to do rollover and deletion of the pubkey.

Fallback method

attachment:web-key-service-fallback.svg, see metadata in the SVG source-code for license of compilation and sub-graphics.

The diagram shows how encryption will be enabled in case the MSP does not offer the "web key service":

• When user B configured the email client for the provider, the client notices that its provider does not offer WKS. It now offers B to send the pubkey of B to the public key server (1), pointing out that the provider does not support WKS and the drawback of publishing his email address. (Unlike in the diagram above, the preparing step is made explicit here.)
• User A wants to send an email to B. There is no answer from B's MSP when asking for the pubkey, so A send the first email unencrypted, but signed (2) as with all emails. A may ask for B in the text to reply to start their email conversation.
• B replies by signed email (3), because signing is the default.
• Once A recieved B's email, her client will lookup B's pubkey for the signature from the key servers (4) and receives it (5).
• Now the next email of A to B will be encrypted (6).

Design goals and constraints

According to user story T1 in EasyGpg2016/VisionAndStories we want to find a single pubkey for a given email address even if there has not been a previous email exchange. It has to work without user interaction (to the extend possible). In T2 and T3 we think about what happens if the communication partner is not ready for this communication.

Assuming both partners have already been set up, we have to solve the following problems reliably without user interaction:

1. How can I get the pubkey for the email address? ("Discovery", "pubkey distribution")
2. How can I gain some basic level of trust that this is the current pubkey of the owner with the email address he wants me to use ("building trust" or "validation").

Within the EasyGpg-contract we are attempting a solution that is comparatively easy to implement to raise chances that it will actually widen the user base. It shall also be designed in a way that it can be introduced step by step together with existing and future solutions ("lean" introduction). The main design idea is to use the already existing relation of an email address owner to his or her mail service provider (MSP). It is an intrinsic common interest of both parties to actually provide a verified relationship so that only the owners can connect to their email storage.

The drawback of this design idea is that an man-in-the-middle attack by the MSP is a lot easier compared to the web of trust approach where several external parties are involved and users must explicitly argument about trust relations. The idea is that users with a higher security interest (like Annika or Bob) will build additional trust using advanced methods, like a mutual check of the fingerprints of their public keys. The tracked communication history also allows to detect indications of an MSP trying to attack its users.

(Old) discussion

TODO extract current proposal and move alternatives to section below.

Mail Service Provider

An email account owner has already trusted their email service provider to accept, deliver and possibly store her communication. Technically the connection will be done with TLS secured protocol variants of something like IMAP (receiving, access to storage), SMTP (outbound sending) or HTTP in case of a web application. In each case the user will have to authenticate to use the service.

All we need to use this trust relation is to find

1. a protocol to ask the MSP for the pubkey of a specific email address
2. a way for email applications to give, update, request deletion of their pubkey from the MSP. [ There is no need for deletion - if the mail address is taken out of service the key will vanish anyway. -- Werner Koch 2016-05-04 16:12:19 ] [ What if someone does not want to get encrypted emails anymore? Usually action should be undoable, this holds for activating this service. -- bernhard 2016-05-06 15:25:43 ]

Some design criteria:

• they are easy to implement for a mail service provider, especially the larger it gets. Because that raises the chance of adoption.
• In addition they should not reveal the existence of email-addresses [ You can't avoid that Werner Koch ] [ Why not? An encrypted HTTPS request for one email-address to a mail service provider will not publish this address as opposed if I go to a public cert/key server. -- bernhard 2016-05-06 15:25:43 ].
• They are better if they are revealing less about who want to communicate with whom.

Query

The first design idea in the 2011 STEED paper was to use DNS records and later profit from DNSSEC.

https://datatracker.ietf.org/doc/draft-ietf-dane-openpgpkey/?include_text=1

Drawbacks of DNS are:

• DNSSEC ist not fully deployed everywhere
• It is unclear if DNS can really handle millions of email addresses [ Nope: DNS can handle arbitrary amounts of data; the size of a data item is limited but not their numbers. -- Werner Koch 2016-05-04 16:12:19 ] [ In theory it is, but are DNS servers and caches ready for the request pattern with millions of emails and more frequent changes in practice? -- bernhard 2016-05-06 15:25:43 ]
• The decentral nature of DNS may leak more information than necessary. (Question could I protect email addresses with DNS?)

An advantage of DNS is that there already exist implementations (GnuPG 2.1) and some real world deployments.

The 2016 design idea is to use a well-known URL via an HTTPS request.

Advantages for HTTPS are:

• well understood how to scale high
• no time-out delays
• request details protected by TLS connection, only leaks the information that user may want to communicate with some user of this domain.
• can also be used on the potential fallback server.
• (expected:) adding one more URL to a webservice can be technically separated nicely from other service, DNS may be couple more deeply. So it may be easier to convince MSP to try the HTTPS for a while than to add a very dynamic system to their DNS service which will be mission critical.

Open question:

• If we use WKD to query with TLS, should be do an DNS request if the TLS request fails or at all? The problem is that if we do an DNS request all the time, it will lose the informaiton about the requested email address to people listening during transport.

Less attractive alternatives/documenting the design process

TODO documenting main disregarded alternatives and the design process

Only using email exchanges (and no lookup)

Possible advantages are

• Discovery of the pubkey to an email address is more difficult. It would depend on the user replying, thus the user can decide with whom to pick up encrypted communications.
• There is no need to choose an MSP that support WKS.

Neutral:

• Attack by the MSP as man-in-the middle there is no real advantage. Of course it is slightly easier to just send out a wrong pubkey via HTTPS, but this can also be done via email, as all emails goes through the MSP anyway. On the other hand it is a little harder to anonymously check that the MSP is serving the same pubkey for you that you also hold. By email, the MSP just filters this for a number of recipients, for HTTPS you could just go to a different network connection which is harder to detect by the MSP.

Considered less attractive because:

• Lot of (first) emails would go unencrypted that could easily be encrypted. Think of confirmation emails to web purchases or when replying to all that have received an email.
• Drawbacks for always attaching a pubkey (see below).

Attaching a pubkey to all emails

The idea is to always attach one pubkey to all emails the user sends out. This could help with

• No pubkey-server lookup necessary, in case of users without WKS supporting MSP.

Considered less attractive because:

• size, some pubkeys are quite large (1 Mebibyte) and we want to continue using the existing trust information in addition.
• a pubkeyserver lookup is strongly recommended anyway looking for revocations.

Managing by HTTPS

It would be possible to manage the uploading of pubkeys to the mailserver-provider via an authenticated HTTPS connection.

This is considered less attractive than handling by email because it is estimated to be harder to implement overall.

• Early examinations by expert hint towards that it is easier to get into the mail processing chain for MUAs than to be able to reuse or resent the authentication credentials for TLS connections. The send, receiving the raw-mail itself is a task of the MUA that will also need to have hooks for crypto to work on the mail-structure itself. The TLS credentials used for IMAPS and SMTPS may be hidden in a different module and probably cannot just be reused by extensions, to protect them against less carefully written add-on code. The experts are Werner Koch (e.g. has implemented significant code for mutt, GpgOL Outlook, gnus and Thunderbird) and Andre Heinecke (e.g. has implemented significant code for KMail and GpgOL Outlook).
• An email implementation makes it easier to support offline systems (carrying over the emails over by other means). Email is already asynchronous. Doing a challenge response method that allows disconnections with HTTPS is possible, but much harder.
• Email already has build-in queue handling on server and client side. A server module that only works one submission inbox is nicely separated from the rest of the system.
• On server side, the HTTPS server may not have direct access to the email credentials of the users.

The advantages of authenticated HTTPS would have been:

• faster turn-around time and better feedback until the pubkey is available from the MSP. Probably seconds as opposed to minutes.
• No need to intercept emails on client side.
• No need for email crypto processing on server side, if the authentication is considered sufficient. (Someone who can authenticate towards the MSP as user, is always able to request and confirm a different pubkey to be uploaded.)

Resorting to a central fallback server

In case if the MSP does not support a pubkey service, it was proposed to use a central fallback server.

The main advantage:

• Users can use crypto, starting with the first email

The main drawbacks:

• A single point of failure (or attack surface)
• Takes away some incentive for MSP to offer a pubkey service.

Overall an email exchange as fallback method is preferred because it is close to a natural way users build up trust in first email exchanges and comes without the drawbacks of the central fallback server.

See more discussion at /FallbackServer).

Using classic (pub)"key" servers only

Our unattended use for sending the email would not work in many cases. It would be by pure chance that only one public user id will be found on the public servers ("discovery") and that it carries a signature from a pubkey that I already "trust" to sign others ("validation"). Anyone can upload a pubkey with a specific email address. Instead a user could use the fallback method of exchanging signed emails before communicating more sensitive contents or using a more complex validation method, for example the web of trust.

However asking keys servers is a good idea to sometimes find revocation certificates or new signatures for pubkey I already know. This is important if an MSP service drops out completly. So a key server check is part of the proposed steps.

Related work

• https://keys.mailvelope.com announced in June 2016 as It's a very simple key server that borrows from Signal to allow TOFU m/automatic key lookup. No key transparency included.
• https://leap.se/en/docs/design/transitional-key-validation A key validation technique based on the TOFU model

Future other services?

It is possible that other pubkey distribution concepts will be proposed and implemented that solve the same problems in a better way. There already are some concepts of how a decentralized directory can be kept honest in public.

If an email service provider offers a different service for a very large fraction of users, GnuPG may for practical reasons adopt this method for this particular MSP.

There is a list of some known other approaches below that may become more relevant in the future. The goal of the EasyGpg2016 contract is to implement and deploy something simple in 2016 as proposed in this concept. The other approaches seems to adhere to an improved pubkey validation via the public and seem to require more players to join to be more useful than the model proposed here.

TODO , e.g.

• https://github.com/google/end-to-end/wiki/Key-Distribution
• https://wiki.gnupg.org/OpenPGPEmailSummit201512?highlight=Transparent%20Keyservers

• https://coniks.cs.princeton.edu/
• https://github.com/yahoo/coname
• https://www.certificate-transparency.org/

• A research project about the legal angle of pub-key distribution (2016-01 to 2017-12 in de_DE): https://www.uni-kassel.de/fb07/institute/iwr/personen-fachgebiete/rossnagel-prof-dr/forschung/provet/vertrauenswuerdige-verteilung-von-verschluesselungsschluesseln.html linked from http://www.heise.de/newsticker/meldung/Bund-gibt-Foerdermittel-fuer-Selbstdatenschutz-frei-3224514.html