TL;DR; - Can a MR script run a map stage without concurrency? (run sequentially) Hi All, I have a requirement to call a API repeatedly to get data Per Customer and per location (a customer may have more than 1 location) Due to many customer location combinations I feel a Scheduled script might not have enough points. Can I use a MR but not SPAM the API? Or any ideas whats the best approch

@battk for the purposes of this question (not to spam an API) a concurrency limit of 1 would work, but do you know if this is guaranteed to run sequentially? Or map invocations could be in a random order?

I shall give that a try, the order is not too much of a issue as long as we don't call the API at the same time

probably sequential, map and reduce stages grab keys from whatever queue the shuffle stage generates

I'd probably go with a scheduled script in this case because it's a simpler model.

Yeah SS was my go to option but I don't know why they removed the option to rescheduled like we can in 1.0

You can still reschedule, you just can't yield.

yield

was always flakey, so good riddance. So yes @reza.seedin the standard pattern I apply here is to have the script check for governance and automatically reschedule itself. For most scripts the entire thing (except for the main logic) looks like this (with NFT, since that's how I roll): nevermind. learn NFT if you want a better life.

In the past Yield never failed me 😄 the days before Map Reduce. some script which are 5 yrs old are still running with auto yielding working as far as I can tell

yield is scary when things go wrong

oh has it happened to you?

yes

I never liked the black magic of yield - it also had limitation around what it would actually preserve. IIRC it had issues with javascript scoping/closures. In my opinion, using

yield

was a code smell - better to design the process to intelligently know when to stop/reschedule itself.