Simple Use
Smart Folders
Pachylet is a web service for storing and accessing your email. It retrieves your email from the Pachylet server itself, and from other POP3 or IMAP mail servers on which you have accounts. It stores your email in a database on the Pachylet server. You access your email from a web browser running elsewhere. Nothing gets stored on your local computer.

Pachylet has the traditional facilities for filing your email in folders, moving it to a trash folder, and eventually deleting it. However, it also has a very fast and powerful search mechanism, so you can largely ignore the folders and just use the “Drop” command to leave most of your email scattered around the single “Dropped” folder, then find appropriate messages by searching.

Additionally Pachylet provides “smart folders”, which let you use searches to automatically pre-sort and filter your incoming email. There’s more description of all this later in this document.

Pachylet tries to prevent malware intruding on your email through attachments or through active HTML. Nevertheless, you should never open an attachment unless you are certain that you trust the sender. You should also be aware that if you view an HTML message by clicking on the “View as Web Page” link, the images loaded there can disclose information about your email usage.

Pachylet works on all modern web browsers, including most smart phones. See the “Colophon” section of this page for details of the various versions.

Pachylet was written by me, Andrew Birrell, and is copyright © 2002-2014. All rights reserved. You may use Pachylet only on this server, and you do so entirely at your own risk. There is no warranty whatsoever.

Simple Use
While you can use Pachylet in the traditional manner (filing important messages carefully and systematically, and then deleting the rest) the user interface encourages a different style, which I recommend that you try:

The normal Javascript version of the user interface does most of its interactions with the Pachylet server asynchronously, so that you don’t have to wait for the server to respond. For example, after you’ve clicked to select a message, you can file that message somewhere, or select a different message, without waiting for the message contents to arrive on your screen.

The user interface is designed so that you can process your incoming messages with the central buttons “Next”, “Drop”, “Trash” and “Scan”. Occasionally, you’ll use “Move”, and as appropriate you’ll generate replies.

The “Find” button lets you do a general search (described below). “Scan” works its way through your unread messages that have arrived in your smart folders. “Folders” lets you view and manage your folders, be they normal or smart.

Messages that have been moved to the Trash folder can be permanently deleted: open the Trash folder, select the message, and click the “Delete” button.

In the version for smart phones, these buttons are organized differently. Also there, there are no features for creating or deleting folders or smart folders, or for permanently deleting messages.

The “Find” button lets you search for messages. A search usually produces its results very quickly, so it’s easy to try different search queries. The same queries get used in creating smart folders, described in the next section.

A search takes place within a folder (normal or smart), or across everything except the Trash folder. You describe the messages you want to find by giving some combination of words from the messages and date ranges. You can restrict the word search to the message header, or some of its fields.

By default, the “Find” dialog is set up to show the query you made most recently (or the folder you opened). If you want to ignore that and search elsewhere, click “Clear” in the “Find” dialog.

The text you search for in the “Find” command is actually a potentially elaborate expression in a search language (defined by the “Sphinx” search engine). The possibilities include:

quick fox … contains both “quick” and “fox”
quick | fox … contains either “quick” or “fox”, or both
quick -fox … contains “quick” but not “fox”
"quick brown" … contains “quick” immediately followed by “brown”
qui* … contains a word that begins with the letters “qui”
(quick brown) | fox … contains both “quick” and “brown”, or contains “fox”
quick brown | fox … contains “quick” and either “brown” or “fox”

A “word” is a sequence of letters, digits, and “_”; letter case is ignored. Many other characters have special meanings, and you should either avoid them or read the full documentation.

The results of a search are presented sorted by date, and positioned to the most recent. That means it’s usually fine to have the search criteria be quite loose, because you probably want the most recent messages. If you don’t want the most recent messages, then the date range options in “Find” should help you.

Smart Folders
Smart folders serve multiple purposes. They are the most powerful, and the most complex, feature of Pachylet. A smart folder is a real folder: you can move messages into or out of a smart folder just as you can with a normal folder. But a smart folder also has an associated query, that’s saved permanently with your Pachylet data.

The most common use of a smart folder is to pre-sort or filter your incoming messages. Whenever Pachylet receives new mail, the messages are matched against the queries of your smart folders. When a new message matches a smart folder’s query, the message is moved into that folder. The only messages that appear in your Inbox are those that match none of your smart folders’ queries.

This would be unhelpful without the “Scan” button on the main screen. This button searches your smart folders sequentially, checking for unread messages.

The net effect when new messages arrive is that they get sorted into your smart folders and your Inbox, then the “Scan” button lets you view them sequentially, in this sorted order. The user interface is arranged such that if you click “Scan” and get to a folder that you don’t want to read right now, you can ignore it (by clicking the button again, or by doing something else) without disturbing it.

A secondary use of smart folders is to filter your messages. When you create or edit a smart folder, there’s an option to mark matching incoming messages as having been read. This means they are ignored by “Scan”. For example, you can use this to move bcc copies of messages from yourself into a “Sent” folder that you’ll never see unless you explicitly open it.

The final use of smart folders has nothing to do with folders. When you use the “Folders” button to see your folders, in the smart folders section there’s a “Find” button. This finds messages matching the selected smart folder’s query, regardless of whether they are currently in the folder. It’s a way to keep a query that you use frequently, such as “messages dated within the last month”. (This feature isn’t implemented on iPhone.)

Composing Messages
The “Compose”, “Forward”, “To Author”, and “To All” buttons create a new draft message. You edit the draft to suit, then click “Send” or “Discard”.

The most recent state of an in-progress draft is held entirely in your web browser, but it can be saved to the Pachylet server. In the Javascript versions (including the iPhone version), this happens automatically within 30 seconds of an edit. In the non-script version, you need to manually click “Save” (which you can also do in the other versions, if you feel you need to).

Having saved the draft (automatically or manually), you can safely move away from the composition window (click “Done” or “Back” in the Javascript versions, or close the window in the non-script version). The draft remains in your “Unsent” folder, and you can resume composition by clicking on it there.

To add an attachment to a message in the non-iPhone javascript version, click the “Attach”, choose the file, then click “Attach” in the dialog. In the non-script version, jusy choose the file and click “save”. There’s no way to add an attachment in the iPhone version.

Pachylet includes a contacts list. You can manage this manually by clicking “Contacts” on the main screen, or you can add a name from an incoming message by clicking the “keep” link beside the name. (Neither of these is available on the iPhone version.)

You can add a recipient from your contacts list to a current draft by typing the recipient’s first or last name (or both), or the recipient’s nickname, into the “to” or “cc” fields. In the non-iPhone Javascript version you can see and search your contacts list in the side pane on the composition screen. Add a recipient by clicking on a contact in that list. In the iPhone version this is available by clicking the “Add” button. In the non-script version you can browse your contacts list by clicking “Contacts” in the composition window.

All versions of the Pachylet client use an encrypted (TLS) connection for all interactions with the server. This guarantees that all data is sent to and comes from the server, without it being visible to a third party outside of the client or server. However, it’s up to you to verify that you are actually using the correct server name with an HTTPS URL, by looking at your browser’s address bar.

You log in to Pachylet on its login screen by providing your user name and password. The server compares these to a salted hash of the password, stored in its database. If they match, your client is sent a “cookie”, which it uses to authenticate you on subsequent requests to the server. Generally, you should log out when you are finished using Pachylet. When you logout, the server makes the client destroy the cookie.

On desktop or laptop browsers, the authentication cookie is destroyed when you exit from the browser application. On mobile devices like an iPhone or iPad the cookie remains on the device for about a week, allowing you to use Pachylet again without typing your password again. This assumes that your mobile device is protected by a lock screen with a non-trivial PIN or password.

Your plain-text password is never stored in stable storage, only a secure salted hash of it.

If you care, you can read technical details of Pachylet’s security design.

Security: Technical Details
All communication between the browser and the Pachylet server occurs over TLS (using HTTPS). The Pachylet server is configured to prefer cipher suites that have "forward secrecy", if supported by your browser (true for all modern browsers). The forward secrecy property is effective after a week (when the server generates a new TLS session token key).

There are five values for each user, all intended to be secret:

At login, the client sends the plain-text password to the Pachylet server over a TLS connection. The server computes the derived key, and the hashes H1 and H2. The login is authenticated by comparing the computed H1 with the value stored in the file system. The computed H2 is presented to MySQL as the MySQL connection password. MySQL computes its hash of H2 and compares it to the stored value. On success, the server returns the derived key to the client as a cookie. The cookie is presented to the server on subsequent operations, and again used to authenticate the login and to connect to MySQL. When the user logs out, the server causes the client to erase the cookie.

Additionally, H2 is used to encrypt third-party passwords stored for the user in the database. These are the passwords that will be used to fetch the user’s email from various POP and IMAP servers. This encryption reduces the impact of an intruder who can read the database.

The plain-text password is never stored in stable storage, and is erased from the UI and discarded as soon as possible. The derived key is computed from the plain-text password by the PBKDF2 algorithm, using 100,000 iterations, a 128-bit random salt (from /dev/urandom), and HMAC-SHA256 to yield a 256-bit derived key. These parameters, including the chosen salt, are stored in the server’s file system. The derived key is never stored in stable storage on the server. H1 and H2 are computed using HMAC-SHA256 on separate (constant) strings, signed with the derived key. The hash value stored by MySQL for its login is “sha1(sha1(H2))”. The derived-key cookie is flagged “secure” and “HTTP only” and isn’t accessible to the client JavaScript. On desktop and laptop machines, the cookie is per-session and is destroyed when you exit from the web browser application. On mobile devices like iPhone, iPad, or Android tablets, the cookie persists for one week. This assumes that have reasonable PIN or password protection for access to your mobile device. The third-party passwords inside the database are encrypted using RC4 keyed by a random 128-bit IV hashed together with the derived key; the first 4096 bytes of the RC4 key stream are discarded.

If the user’s plain-text password has a decent level of entropy, such as 60 bits, then it is economically infeasible to acquire the plain-text password from the derived key by brute force. It is equally difficult to acquire the 256-bit derived key from H1, H2, or the MySQL hashed key; or to acquire H2 from the MySQL hashed key.

Aside from the obvious goal of allowing access only to authenticated users, there are secondary goals related to how badly a compromise of the system will hurt.

There are a few extreme possibilities. First, if the user’s device is compromised, then the user’s login identity including the plain-text password is unavoidably compromised. Second, if an intruder gains root access to the server, then all subsequent use of the server is compromised. Third, if an intruder can modify the scripts or programs used by the server, then all subsequent use of the server is compromised. Note, however, that the user’s plain-text password is compromised only if it is used while the device or server compromise is in effect, not earlier: we have perfect forward secrecy in that sense. (This is assuming the absence of attacks on the TLS connection, which might or might not have perfect forward secrecy.)

For lesser attacks, we have two secondary goals. First, an attack on the server must not allow the attacker to determine the user’s plain-text password, because it is most likely related to passwords for that user elsewhere. Assuming the integrity of the server’s programs, the derived key machinery accomplishes this. Second, an attack that permits read-only access to privileged areas of the server must not be permitted to escalate into read-write access. This is achieved because the hash stored by MySQL cannot be used to acquire H2 or the derived key; and because H1 cannot be used to aquire the derived key. Finally, observe that H2 is useful only within the server; access from the web requires the derived key. (There is no network access to the MySQL server.)

We shouldn’t allow server-side scripts free access to the H1 stored in the file system. Partly, this is because we would prefer to keep H1 secret. But more importantly, the Pachylet server-side scripts are no more privileged than any other server-side script: if we allow the Pachylet scripts to modify the stored H1 directly (e.g., to allow a user to change their password), then any other server-side script could also modify H1 and enable network login. This still would not enable access to the MySQL database, but this attack could be combined with some other attack to do so. Therefore we restrict all access to the stored H1 file to a more privileged user, and provide a setuid application that allows the requisite operations only to a script that knows the matching derived key.

However, we weaken this scheme in two ways.

First, we retain H2 in stable storage on the server. Reading this would allow an on-server program to acquire read-write access to the database. We do this to allow a periodic job to be run that incorporates new mail for the user while the user is offline. The stored H2 is in a file that is readable only to root, and is given only to a script protected as well as the server’s other application programs and scripts. Hence an intruder can read the stored H2 only if the server is already fully compromised. Note that even if H2 is revealed, it allows no access from the web, and the user’s plain-text password is not compromised.

Second, we maintain on the server a publicly readable file for each user, containing the number of unread messages for that user. This allows unauthenticated “new mail” indications, for example on some other web page.

This design doesn’t use a separate “session ID”. This is primarily because the server maintains no per-session state. Having a separate session ID would add complexity and gain us no useful functionality. Tangentially, note that we can revoke a derived key and H2 at any time by changing the user’s “salt” value and computing a new H1 and H2. This happens, for example, whenever the user informs the server of a new plain-text password, even if the new password is equal to the previous password.

Not all of the user’s data is stored in the database. Message parts (main body, alternative bodies, or attachments) larger than 10,000 bytes are stored in the file system, although the first 10,000 bytes are also stored in the database for use by the full-text indexing system. These files are accessible equally to any script running in the server. We encrypt the part files, using a key stored for the part in the database. Each part uses a different key, derived from the H2 hash and the part’s unique ID. The key is retained in the database for each part to avoid re-encryption when the user changes their password. The encryption is done with AES-256 in CBC mode with PKCS #7 block padding. There is a single fixed IV (which is OK, because each part file uses a different key). A MAC of the plain text is stored in the database. The MAC is computed with HMAC-SHA-256, keyed with the part’s key. The decryption function computes a MAC even if the block padding is incorrect, to avoid padding-oracle timing attacks. This design is intended to make the part files as secure as the database itself. The only additional vulnerability is denial of service by deleting a part file.

No security design would be complete without some arrangement for auditing what the system actually does. We do this by writing log entries whenever the client-side scripts do anything substantial (using the “syslog” machinery). The server sends a daily summary of these log entries to the system administrator.

Finally, note that the server must protect itself from many other attacks. SQL injection attacks are avoided if the server is careful to escape all arbitrary strings received from the client or from other servers; cross-site-scripting attacks need similar protection. To prevent cross-site request forgery attacks, the server insists that all incoming requests, including the login request, have an HTTP “referer” header line that references a page on the same machine and in the same directory as the server script, served by HTTPS.

The name “Pachylet” is reminiscent of “Pachyderm”, an animal rumoured to never forget your email. See also the Pachyderm web site or a Pachyderm talk, both from 1997.

There are three client-side versions:

The web pages generated by the V1 client are very simple, and don’t rely on any client-side scripting. They should work on any recent browser, even with Javascript disabled. They do require CSS support, and they won’t work on ancient browsers like IE4.

The V2 client and Pachylet Mobile use caching and asynchronous communication to remove a lot of the latency in communicating with the server, producing dramatically better performance. They also transfers much less data from the server (important on slow or expensive connections). In exchange, they needs a modern browser. V2 requires IE 10 or later, Safari 8 or later, or pretty much any version of Firefox or Chrome. V2 also works, a bit awkwardly, in the Kindle’s browser. Pachylet Mobile should be fine with iOS 8 or later, or any iPad or Android. I currently debug on IE10 on Windows 7, Chrome 44 on Android, Safari 8 on MacOS 10.10.4, Firefox 38 on MacOS 10.10.4, and iOS 8 on iPhone.

Source Code
Note that Pachylet is a copyright work. You may read the sources for educational purposes, but you are not allowed to use them elsewhere, nor to make derived works. Any use you make of these files is entirely at your own risk. There is no warranty whatsoever.

The V2 (AJAX) client (about 6200 lines):

Pachylet Mobile (about 2100 lines, layered on the regular V2 client):

The V1 (static HTML) client (about 3200 lines):

Client-independent code (about 5000 lines):