Encryption

Encryption

User key management

Raft takes advantage of a webauthn extension called PRF that allow the web application to obtain a secure secret when the user logs in. Raft uses this secret to generate a public/private key pair that is used to unlock the user’s personal vault.

The personal vault securely stores the encryption keys for each workspace that the user has access to. The vault is locked again when the user logs out. At no time does the database server have access to the keys stored in the vault. The web application has temporary access to the vault while the user is logged in.

Most of the time, encryption allows you to solve the problem of confidentiality by creating a new problem of how to manage keys. The Webauthn PRF extension allows for automatic key management that is invisible to the user.

Workspace keys

Every workspace has its own encryption key. Nearly all content, and some metadata, is encrypted using this key while stored in the database. Workspace content is only readable by the web application when a member of the workspace logs in, unlocks their vault, and temporarily adds the workspace key in their session.

The objective with database encryption in Raft is to limit the potential for disclosure of sensitive information stored in the database, either because of an attack on the servers, negligence by the provider, a nefarious worker, or a court-ordered subpoena.

Per-user encryption at rest

How does this work? TL;DR edition:

When the user makes a request to the web application, the user’s personal vault is loaded, the workspace key is read from the vault, the database records are decrypted, and the values are returned to the user’s web browser.

Encryption and decryption takes place in the web application, not the browser or the database. These scheme could be called “per-user encryption at rest” because each user has their own vault and the data stored in the database (“at rest”) is encrypted so only the user can read it (and the web application while the user’s session is active).

There are six methods of database encryption:

Row encryption: individual database rows are encrypted using a random IV and the workspace key (IV is a random, unique seed necessary for block cipher encryption). Not all columns are encrypted, but most sensitive content is. Each row may be encrypted with a different key.
Searchable encryption: the searchable content is tokenized (separated into discrete words), shuffled in random order, and then each token is deterministically encrypted. To do this, the IV chosen is not random but derived from the digest of the token (and other key material). So, the token “cat” will always produce the same ciphertext for a given workspace. This undermines some of the security properties of the block cipher, but in our case this is an acceptable trade off (our goal is simply to protect the database from any data breaches). Searching these deterministically encrypted tokens is extremely fast, allowing for a user experience that feels like normal full text search (albeit without stemming or trigram support). Deterministic encryption like this is vulnerable to frequency analysis and chosen plaintext attacks.
S3 storage object encryption: attachments and files are stored in an object storage service compatible with the S3 protocol. Like the database, these objects are encrypted before being sent to the S3 service and decrypted when being read from the S3 service using a block cipher, random IV, and the workspace key.
App encryption: Some columns are not encrypted to the workspace key, but instead to a global, shared application key. The application key is primarily used for encrypting short-lived background jobs (e.g. generating a zip file for export).
Session encryption: All data in a user’s session, including their unlocked vault, is encrypted using a special session key. The session key is stored in the client’s session cookie. When a page load happens, the browser sends the cookie to the server, the server extracts the session id and key from the cookie, then loads and decrypts the session from the database, and finally erases the session key. This means that a user’s sensitive vault is available to the server only during page request, and not a moment after.
Link encryption: Users may create invites to join a workspace or links that grant access to a particular resource (such as a document or folder). These links contain the workspace key. To guard against key disclosure, the workspace key in the link is encrypted with the app key, and then encrypted again with another key that is deleted from the database on the day the link expires.

What is not encrypted at rest? The user’s login name and workspace names are not encrypted. Some indexed ordinal fields are not encrypted, such as dates used for calendar queries.

Workflow

User logs in
- The webauthn authenticator (e.g. security key) is given a seed value unique to the authenticator’s id.
- The authenticator responds with a deterministic digest of the seed, the “prf secret”.
Vault is unlocked
- This “prf secret” & salt are used to generate a public/private key pair.
- The private key is used to unlock the user’s personal vault.
Session is encrypted
- The web application generates a random session id and session secret.
- A new session record is created in the database, encrypted using the random session secret.
- The session record stores the newly unlocked personal vault.
- The web application responds to the browser with the new session cookie, containing only the session id and session secret.
The “prf secret”, private key, personal vault, and session secret are forgotten by the web app.

On every web request:

Load the session
- The browser sends the cookie to the server (which includes a session id & session secret)
- The session record is fetched from the database using the session id and decrypted using the session secret
Load the vault
- The personal vault is loaded from the decrypted session record.
Decrypt database records
- The workspace key is loaded from the personal vault
- Workspace records are loaded from the database and decrypted by the app using the workspace key
Send response
- The web app sends it’s HTML response back to the browser.
- The session, session secret, personal vault, and workspace keys are forgotten by the web app at the end of the request

Log out:

All session records associated with the user are destroyed.

Comparison with other approaches

For this discussion, we will limit our consideration primarily to the security property of confidentiality, or keeping data from being read by an unwanted party.

What about end-to-end encryption (e2ee)? Some web applications are advertised as supporting e2ee because the encryption/decryption takes place in the user’s browser using javascript.

When a webapp uses browser-based encryption, the encryption can be defeated by modifying the javascript sent to the browser (or adding additional javascript loaded by the browser). In order to do this, the attacker must modify the web application that runs on the server (or find an unknown injection vulnerability), which is also the same attack that server-based encrypted apps are potentially vulnerable to. In both cases, trust rests with the security of the web application and the web server.

In contrast, a true e2ee app is one that is installed as a local desktop or mobile app. In this model, the network and servers are entirely untrusted, with all trust placed on the local device.

App Type	Installed	Structural Weakpoint
Web app, traditional	Server	Data stored in cleartext on the server
Web app, encryption in the browser	Server	Server's app code can be modified to leak data
Web app, encryption on the server	Server	Server's app code can be modified to leak data, app's memory can be read by a privileged process
e2ee app	Client device	Data unprotected against a compromised client device

In summary:

A traditional web application provides no confidentiality promise, as all the data is readily accessible to the provider at all times.
A web application with encryption in the browser can be compromised when the javascript is loaded by the browser from the server, which can happen at any time the app is being used. This is how Hushmail added a backdoor into their browser-side encryption app.
Similarly, the confidentiality of a web application with encryption on the server can be compromised during a user’s page request by using a privileged account to access the application’s memory or modify the application’s code.
A true locally installed e2ee app places all trust on the local device. When written properly, its data is as confidential as the confidentiality of the device itself. Sadly, most devices can be readily cracked by law enforcement, although this is sometimes expensive.

The thesis that motivates the design of Raft is the following: It is much easier to properly harden the security of a single server than it is to harden the security of hundreds of end-user client devices. The structural weakpoint of encryption-on-the-server, that a compromised privileged user account can read memory or modify code, is a risk that can be effectively mitigated against, both in terms of best practices for server security and choosing providers with incentives that align with your own.

Advantages and Disadvantages

App Type	Advantages	Disadvantages
Web app, encryption in the browser	Nothing to install; Can sometimes be packaged as an installable e2ee app	Server must still be trusted; Very complex, even for basic apps
Web app, encryption on the server	Nothing to install; Very simple to write complicated apps	Server must still be trusted; More frequent opportunities to read app's memory
e2ee app	Trust only your client device	Requires installation and updates; Very complex, even for basic apps

Law Enforcement

When it comes to social movement threats from the US government, we must distinguish by type:

Mass surveillance: Large scale data gathering of the population or communities as a whole is largely outsource by governments to private companies, such as data brokers, or data aggregators like Palantir. In some cases, the NSA directly ingests and stores surveillance data from large providers, as is the case with SMS metadata. Mass surveillance is a serious threat to social movements, but is a long term on problem with how the information economy is structured and regulated. It is not something that can be addressed by different tool selection.
Administrative subpoena: Law enforcement can, and often do, issue their own subpoenas for stored communication. Lately, the large providers have begun to readily comply with these subpoenas, even though they are under no legal obligation to do so.
Court-ordered subpoena: When a judge signs off on a subpoena, any US entity served with this subpoena is required to furnish the requested stored information to law enforcement if they are able to do so. If a service provider is not able to do so, as in the case with stored data stored encrypted using a user-generated key, then the provider can simply reply that they don’t have the requested data.
Court-ordered wiretap: When a judge issues a wiretap order it covers capture and reporting of ongoing data transmission. If the provider is able to perform this capability, they are required to do so. If the provider’s technical architecture does not make this possible, the provider is not under any requirement to rewrite their software simply to facilitate live interception. This is written explicitly into the law governing voice communication wiretaps (CALEA), but is also current jurisprudence regarding digital interception generally.
Physical possession by law enforcement: When law enforcement has physical possession of a user’s device they will routinely use forensic tools to extract and archive all the information stored on the device. Except for newer devices from Apple or Google, nearly all mobile devices can be cracked by forensic tools without much effort. Because our devices hold the key to our entire digital lives, law enforcement is highly motivated to direct their effort toward obtaining these devices.

A locally installed e2ee app is particularly resistant to the first four threats, but typically vulnerable to the device being seized, an increasingly common occurrence. If you have a group of 100 people, all it takes is one person having their device taken for the group’s entire history to be exposed.

Web-based encryption applications are often (but not always) resistant to confidentiality compromise that originates with a device seizure. These applications (both client-side encryption and server-side encryption) also provide strong resistance to subpoenas, so long as the app encrypts user data in a manner where the provider does not have ready access to the key necessary to read that user data.

Web-based encryption applications are potentially weak against wiretap orders, in that it is technically possible to redesign a system to facilitate interception, although current US law does not require this. However, it is possible in the future that a US court may find a provider is under obligation to implement a feature that allows for real time collection of a web application’s active sessions.

Local storage encryption

Some e2ee apps mitigate against a loss of confidentiality caused by a compromised client device by encrypting their local storage using a key generated from a passphrase when the app is run (e.g. Briar, Anytype, etc), although many do not (e.g. Signal). This approach can greatly improve the confidentiality of local data, although these apps are usually still vulnerable to unlimited brute force attacks and thus place their security on the ability of the user to memorize a long unique random password (something humans have proven particularly incapable of doing, in aggregate). A potential solution to this problem, one not yet implemented by any app, is to split responsibility for generating the key between the local device and a remote service that can force rate limiting (Signal works this way for restoring backups).

Caching

Many applications loaded in a web browser will aggressively cache data to the user’s local device. Even many apps that use encryption in client-side javascript will store large amounts of data in the clear using the browser’s local storage facility.

When a web application operates in this manner, it becomes highly vulnerable to device compromise or seizure. In contrast, Raft is designed to store nothing on the client device.

Other mitigations

Although rarely done, an e2ee webapp that uses CORS can load javascript from one service and load data from another, allowing the user to trust the javascript provider but not the data provider. This approach, however, does not diminished in any way the degree to which the end-user must trust the javascript provider.