Has anyone else experienced insanely long completion times for the getInputData stage of a map/reduce script? The search varies in result size 50,000 - 200,000 rows

if its a consistent thing, you probably have too much data in your search and your server is struggling to serialize it

I have seen repeated cases where once the # of GID results exceeds 30-40 thousand, the time that stage takes before map start scales very poorly, such that 60,000 results will take much, much more than twice the time of 30,000 results, and 120,000 results will take far more than twice as long as 60,000 results. I have found things run faster when I limit the GID to 30,000 results, and simply run the M/R multiple times as needed to process all of the data. The total run time ends up being faster. I have seen this effect even when my search or query only returns 1 or 2 small columns, so I don’t think reducing the columns will do much once you have that many results/rows. There could be factors affecting just our account, such as our service tier, but I would experiment with # of results in increments of 10,000 to find the point where performance starts to degrade for your particular script.

Okay that's interesting, I'll give that a go. Have either of you seen any improvements to speed in this stage by using SuiteQL?