######################################################################## # # Presentation about Hermes to CUCS, 2003-01-22 # # $Cambridge: hermes/doc/talks/2003-01-cucs/talk.mgp,v 1.3 2003/04/07 23:11:55 fanf2 Exp $ # ######################################################################## # %deffont "standard" tfont "standard.ttf", size 5 %deffont "thick" tfont "thick.ttf" %deffont "typewriter" tfont "typewriter.ttf" # %default 1 area 90 90, leftfill, size 2, fore "white", back "black", font "thick" %default 2 size 7, vgap 10, prefix " " %default 3 size 2, bar "gray70", vgap 10 %default 4 fore "white", vgap 30, prefix " ", font "standard", size 5 # %tab 1 prefix " ", icon box "gray30" 50 # ######################################################################## %page %nodefault %center, fore "white", font "thick", size 8 Hermes and PPSW %size 6 Past, present, and future %font "standard", size 5 Tony Finch Mail Support University of Cambridge Computing Service ######################################################################## %page About me 1993-1994 Gap year 1994-1997 Trinity Comp. Sci. 1997-2000 Running web servers 2000-2001 Apache developer 2002-present again Computing Service 1996-present 1997-present 1999-present 2002-present # # CV expressed as a list of email addresses # Was a web guy for five years because I couldn't run an email server # in my room owing to the port 25 block, but I could run a web server # ######################################################################## %page Contents introduction Hermes functions PPSW functions old Hermes internals administrative server PPSW internals the need for change PPSW evolution future Hermes # # First section probably familiar content # Second section is behind the scenes # Finally changes we are making now and where we're aiming # ######################################################################## %page Introduction Hermes is the central email store 23,000 users 280GB storage daily stats: 200K emails, 500K POP, 300K IMAP, 20K webmail, 20K telnet concurrent: 3000 IMAP, 1000 webmail, 800 telnet, negligible POP PPSW is the central email relay more visible to COs than users relies on Hermes for its user interface also about 200K emails per day, although large overlap with Hermes # # numbers are not accurate # POP sessions are very short # overlap mainly because of lists, domains, and incoming @cam email # (in-system @cam is optimised and @hermes email doesn't go via ppsw) # Local software: MTA is Exim, webmail is "prayer" # inspired by Oxford's "wing" (web IMAP/NNTP gateway) # ######################################################################## %page Historical overview PPSW named after old JANET coloured-book email program PP Everything now runs Exim (of course) Hermes and PPSW used to be quite separate now growing together into one system Hermes was originally a quick-and-dirty clone of CUS still a fairly normal Unix system becoming a collection of appliances # # ppsw dates back to the time of phoenix, before hermes # X.400, RFC 1148, "perfectly painful" # early scaling problems # more about Unix and CUS later # ######################################################################## %page Hermes functions email storage accessible via IMAP, POP email user interface accessible via telnet, ssh email smarthost outgoing SMTP for users email filtering both canned and roll-your-own some file storage for attachments user interface for PPSW # # This should be familiar # ######################################################################## %page PPSW functions @cam forwarding @lists distribution managed mail domains central email hub MX for most email servers outgoing SMTP smarthost for servers # # Do managed mail domains need more explanation? # ######################################################################## %page Old/Current Hermes Internals Just a Unix system each user has an account (NIS) standard mailbox format + performance hacks although bigger than many Shared NFS filesystem: 2 x NetApp F740 four cloned front-end machines We don't trust our users restricted shell, strict quotas, security patches, audit scripts # # improved performance from indexes with IMAP metadata, .idx .ldx files # modified open source software # pine/pico/c-client exim openssh + cdb openssl perl ispell rcs + Solaris # Restricted user environment makes it easter to change the implementation # no data on cloned machines apart from exim spools # NetApp snapshots are a godsend # ######################################################################## %page administrative server NIS master user logins for password and friendly name changes Configuration and source code management periodic updates of PPSW configuration Install server Jumpstart (Solaris), Kickstart (Red Hat) Tape backups User admin scripts similar to CUS # # front-ends named after colours of the rainbow; admin box is "prism" # where all the messy odds and ends live # ######################################################################## %page PPSW internals table-driven Exim configuration uses Dan Bernstein's CDB routing for "hubbed" departmental email managed mail domains are just virtual /etc/aliases files doesn't depend on other machines being available one auxiliary program for mailing lists deals with moderation protocol and footers exceedingly simple almost the only hole in the port 25 block # # @cam optimization on Hermes etc. relies on Jackdaw # appliance-like system (only data is exim spools) # lists system has "special needs" # ######################################################################## %page The need for change Unix mailbox format has disastrous performance POP3 keep-mail-on-server mode makes shared mailboxes difficult too Centralized filesystem is single point of failure NetApps have high performance, are very reliable, but are very very expensive We need to increase quotas by at least a factor of ten # # cost of email is a tenner per user per year # number of users and volume of email mostly flat, but size growing # 10MB -> 100MB # # "Any problem in computer science can be solved with another layer of indirection." # -- David Wheeler # ######################################################################## %page PPSW changes Taking over some Hermes functions webmail was the first IMAP and POP proxies Central email filtering virus scanner (mandatory filter) spam tagging (optional filter) how to do statistical filtering? New lists system better user interface can the available software be clustered? # # Klez was a painful experience # MailScanner is current work-in-progress # new lists project comes after that # ######################################################################## %page Future Hermes Cyrus IMAP server on clustered PCs application-level replication modifications to replace NetApp snapshots proxy on PPSW provides indirection between imap.hermes.cam.ac.uk and the user's actual mailstore machine telnet environment will be just another user interface that talks IMAP just like webmail still some design issues to resolve # # mail store boxes not externally visible, like grey and black netapps # one file per message; no mailbox quotas # where to stage uploads & downloads? filtering databases? # timescales # ######################################################################## %page That's all, folks http://www.cus.cam.ac.uk/~fanf2/ ########################################################################