OpenIO at Oscon 2015, Portland

Part of the Open Messaging Project, OpenIO spoke at OSCON in Portland about the challenge of scaling email platforms.
| @openio

Capacity and random I/O performance combined is the worst case scenario for storage infrastructures – email + attachments attacks on both fronts. Given storage growth of 20-40% every year, traditional block or file storage is typically inconvenient and expensive, especially for managing email and archives.

With so many concurrent users randomly accessing tiny files, the reality is that not one, but two competing solutions are needed to build an efficient storage infrastructure for email. And the solutions need to be spread across both edges of the IOPS spectrum.

SSDs today are very good for delivering cheap high performance random IOPS on a restricted amount of space. This is efficient for mailbox metadata that represents only 5% of the capacity but 95% of the I/Os.

For the remaining 95%, what is the simplest and best way for the mailbox server talk to thousands of hard drives? Since email messages are immutable, Object Storage appears to be the perfect fit. It handles continuous capacity growth, with the most efficient infrastructure, so it is “scale-out” by design.

But now we need to make mailbox servers equally “scale-out” and fully integrated with Object Storage solutions.