Skip to main content

Posts

Showing posts from September, 2012

Improving BDB JE Storage for Voldemort

I am writing this blogpost, mainly to share my experiences with improving the BDB storage engine for use in Voldemort and also throw light on how this relates our VaaS goal. As you drill into the details, I hope you get a clear idea of the efforts that have gone into making BDB JE work. I have tried to include pointers whenever possible. Note : All of this is written in the context of SSDs. Hence, you will often find me ignoring IOPS completely, focusing on memory. BDB GC Tuning on SSD  The post on engineering blog delves to great details about GC issues we faced, when migrating to SSDs. The article mentions the pushing data off the heap, to play nicely with GC.  This is done by setting EVICT_LN cache mode for online traffic and EVICT_BIN cache mode for cursors.  However, to achieve this, we need a higher version of JE > 4.0.117 . We first evaluated BDB 5, which requires a non backwards compatible data conversion to be done on existing data. But we hit issues in the