Hello People, I am working with Map/Reduce script and everything is working as it should. The getInputData & map functions are working fine but it is in reduce that I am facing some issues. I am passing context.values from map to reduce in a specific sort order but I am not receiving data in that same order. I tried to find the issue and read this in documentation. It says that values are listed in lexicographical order. Is there any way to stop that or pass data in our own sort order?

Conde sample, intended outcome and why?

yeah we'd need more details to help you, without that the short answer is no you can't control how that data comes into your reduce stage... technically YOU aren't passing anything from Map to Reduce, because there's a SHUFFLE stage you have no access to between the Map and the Reduce. add you sorting log to the start of your reduce stage. add some additional properties to your payload that makes said sorting very simple. (usually I'd concat something to the key if its simple enough, but you can also just add another data element in the value)

I am trying to merge pdf of multiple transactions of a single customer for 700-800 customers. In getInputData(), I am creating a saved search for those transactions and passing the results in a sorted manner which I want of the merged pdf. In map stage, I am creating pdf files for those transactions and storing it in a file cabinet. From map to reduce, I am passing the customer internal id as key and array of pdf files as values. In reduce, I am trying to merge all those pdfs together and store it in a separate folder. Since the file ids are getting shuffled, the order of those files is getting shuffled too.

yeah exactly something like this, if its transactions i figured you just use trandate? but if you're doing the work already to get them in a specific order and you can just give them numbers like 1 2 3 4 or w/e then sure that works too

You are creating the PDF in the order you need them, and stored them in the file cabinet. The Internal IDs of created records are incremental, in your grouped data in reduce simply sort your PDFs by internal IDs and you are done.

if he's using concurrency multiple maps can be running at once PDF 1 might actually get created after PDF2 if there's less data to process etc. and after working through 100s/1000s of PDFs that is almost guaranteed to happen at some point... I wouldn't trust intids

not using multiple concurrency, and @alien4u I didn't think about that. Interesting solution.

Then build and add a sequential ID to the data in the getInputData and make it flow all the way to reduce and sort by that.