@mattdahse can speak to map/reduce - he recently built an M/R that would chew through 4m records per hour