Yeah I have to assume based on its intended use cases that map/reduce can handle large data sets

It should, each result of the search should be one thread in the script.