Skip to main content

Posts

Showing posts from 2013

Setting up a Play Framework application on RedHat's Openshift

Play framework is an interesting web development option, using Netty directly as a web server, providing a basic MVC framework to build web applications on. A Here's how you marry them both.. The documentation out there ( git quick starts  ) seem to be outdated. But, its simple really and truly DIY.. So, thought I would let people know. 1. Install Play  2.0  Spin up the play server and make sure something renders on the localhost. 2. Open an account with openshift.com and create a DIY application When you checkout the git projects, by default, you get the following directory structure with a Ruby script to serve up a index.html page. $ ls -a   .  ..  .git  .openshift  README  diy  misc $ ls diy   logs   index.html   testrubyserver.rb $ ls .openshift/action_hooks   build   deploy   post_deploy   pre_build   start   stop 3. Package your Play app $ play stage $ target/start Now, copy target folder to $ cp -rf target $OPENSHIFT_PROJECT_GIT_REPO/diy/ Thro

Thoughts On Adding Spatial Indexing to Voldemort

This weekend, I set out to explore something that has always been a daemon running at the back of my head. What would it mean to add Spatial Indexing support to Voldemort , given that Voldemort supports a pluggable storage layer.. Would it fit well with the existing Voldemort server architecture? Or would it create a frankenstein freak show where two systems essentially exist side by side under one codebase... Let's explore.. Basic Idea The 50000 ft blueprint goes like this. Implement a new Storage Engine on top Postgres sql (Sorry innoDB, you don't have true spatial indexes yet and Postgres is kick ass) Implement a new smart partitioning layer that maps a given geolocation to a subset of servers in the cluster (There are a few ways to do this. But this needs to be done to get an efficient solution. I don't believe in naive spraying of results to all servers) Support "geolocation" as a new standard key serializer type in Voldemort. The values will sti

A simple non blocking event counter

I am writing share something I implemented recently, to monitor streaming operations into the Voldemort server. What I needed was a simple avg and event rate counter The idea is very simple. Have atomic longs for the event value sum and event count, to be updated by the application threads. Whats the big deal right?  The deal (not sure how big, depends on who you are) is to have this counter reset itself, every so often. This is a very common usage pattern in data systems, where we would like to monitor performance over small periods of time (so we get enough samples to smoothen the outliers) and reset every so often. Also, one should make sure the "monitoring" thread always reports statistics for a complete interval. The trick is to avoid resetting the atomic long every interval, since doing so would mean missing some concurrent updates from other application threads in the meantime. Also, since we are maintaining state about the previous interval, it becomes necessary