OK so you're looking at an initial load of 5mill. That kind of volume you're on a hiding to nothing I'd say. You should probably look to break up your data set using multiple searches. If you have multiple threads you could have multiple deployments each with different searches on them and have them run concurrently. You'd end up with several output files but you could always process those afterwards. I would watch that file size though there is a limit to the size you can store and I think, transmit