The Prayer Webmail System

Prayer is yet another Webmail interface for IMAP servers on Unix systems.

It exists because we weren't terribly happy about the characteristics of existing Webmail interfaces: in particular scalability problems with common open source Webmail packages and the lack of flexibility that commercial packages would give us. This doesn't mean that Prayer is trying to compete with existing Webmail packages. It just means that Prayer is better suited to our particular environment.

We've been running Prayer as our Webmail service since late 2001: initially as a pilot service and then as a full supported service since July 2002. We've had surprisingly few problems in that time: mostly obscure browser effects that we have had to work around.

The code has been released under the GNU public license and is available from our FTP server at: ftp://ftp.csx.cam.ac.uk/pub/software/email/prayer.

Why "Prayer"?

Malcolm Beattie, formerly of Oxford University Computing Services released a Webmail package named WING early in 1999. It was unusual at the time in that it used persistent IMAP connections and it was written using Apache/mod_perl rather than mod_php. We came to the conclusion that the WING code that was available to us in 1999 and 2000 was not sufficiently mature to use for our own Webmail system. Consequently (and with some reluctance) we decided to start writing our own package. Prayer was the in house development name for this package, a simple play on words to acknowledge the strong influence that WING had over Prayer development, especially in the early days. But at the end of the day, everything needs a name.

Performance/Scalability

• Uses persistent connections to IMAP server and support servers.
• Target folders remain SELECTed: not a simple-minded proxy.
• Full caching (including sort/thread cache) for each open folder.
• Up to five persistent IMAP connections (typically one or two in use):
  • INBOX and one other folder
  • Postponed message folder stream
  • Preferences stream
  • Folder transfer stream
  • Various optimisations/sharing to minimise actual IMAP connections
• Directory cache: single round trip to IMAP server for directory listing.
• Works well with UW IMAP server (even using Unix format mail folders).

• Uses 30% to 40% of the CPU and 800 MBytes of RAM on a PIII class system with 47,000 logins/day
  (3,700 logins/hour, 1800 concurrent logins peak)
• Currently supporting 130,000 logins/day on a fairly idle P4 system
  (7,700 logins/hour, 5000 concurrent logins peak, load average: 0.25).
• Aggressive HTTP/1.0 and 1.1 connection caching to reduce SSL overhead:
  quite happily handles 3.9 million page hits a day (2.8 million https) on the above hardware, where 65% are dynamically generated pages.
• Optional gzip compression of pages tunable by IP address range.
• Minimal use of fixed length buffers in C source code.

Small page sizes (typically less than 1 KByte compressed) give fast browser refresh rates even on dialup links. In a LAN environment, Prayer will serve pages as fast as the browser can render them, even under high concurrent load.

Simplicity

• No external database: user preferences stored on IMAP server.
• Single configuration file.

• Runs quite happily with Netscape, Mozilla, IE, Lynx, Opera.
• Text only mode for Lynx and w3m
• Use of nested tables restricted to avoid confusing Lynx.

• Single optional cookie used to hide session-ID from browser URL display.

User Interface

• Hierarchical listing of MIME BODYSTRUCTURE for display/download.
• User preferences to display TEXT/HTML inline with dangerous tags removed.
• Special handing for postponed message folder
• http:// and https:// in body converted into links.
• Message text wrapped to 80 columns (maintaining correct message quotation)
  • Colour codes for quoted message text

• Aggregate: save, forward, (un)delete, (un)read. Zoom mode.
• Sort and Reverse Sort folder listing (on one of 7 criteria).
• Change to/Save marked messages to favourite Folder.
• Supports Persistent and non-persistent message marking (user preference):
  • Non-persistent (normal Webmail behaviour using HTML checkboxes):
    • Fewer page refreshes, no way to maintain state if links used
  • Persistent:
    • Marks are HTML links, mark state recorded while stream open.

• Message Headers/Text, Delivery Date, Status, Size.
• Compound search on marked messages.

• Single level display with filters
• Favourite/subscribed folders for shortcuts on list screen.
• Transfer mailfolders to and from IMAP server using HTTP upload/download.

• Local address lookup and Gecos expansion.
• Recursive Addressbook Lookup (with loop detection)
• Spell Check with Personal dictionary.
• Adaptive (and hopefully quite intelligent) line wrap algorithm
  • breaks message into paragraph blocks and wraps blocks independently.
• Undo. Roles (identities) . Rich headers.
• Alt-addresses list and local domain list used when constructing Cc list.
• Arbitrary number of postponed messages using postponed messages folder

• Search. Import/Export local and remote pine .addressbook format files.
• Addressbook split into pages: can move to any page

• Change password and Gecos fields. Check disk quota.
• Mail redirection, filtering and vacation messages using Exim filter files or Sieve.