tiger
04/17/2025, 3:51 AMCeligo AI
04/17/2025, 3:51 AMAI generated answer (using internal Slack threads):Based on the conversations, it is not recommended to call a Restlet in every map phase of a Map/Reduce script to increase the governance limit. The governance limit is higher in the reduce stage (5000 units) compared to the map stage (1000 units). However, if you're hitting the governance limit, it's likely that you're not utilizing the structure of Map/Reduce properly. It's almost never a good practice to come close to utilizing that much governance in one entry of either map or reduce. If you're running out of governance, it's suggested to look at how you're getting the data and if you can pare it down in a map phase. You can also try to optimize your reduce data set in the map phase. Calling a Restlet would give you an additional 5000 units, but this is not scalable if you have even more data to process. Another option is to implement the data fetching and grouping yourself in the getInputData phase and free up the map for processing. Remember, the goal of Map/Reduce is to process each result individually in the map entry, so in almost every case, the governance limit doesn't matter in Map/Reduce. ****** Disclaimer: This response was generated by an artificial intelligence assistant based on previous conversations from multiple Slack channels, and may contain inaccuracies or mistakes. ******
Celigo AI
04/17/2025, 3:51 AMSlack thread 1: here (available only if the original thread was posted in the last 90 days)
Similarity score: 0.808820486
Slack thread 2: here (available only if the original thread was posted in the last 90 days)
Similarity score: 0.808338284
Slack thread 3: here (available only if the original thread was posted in the last 90 days)
Similarity score: 0.798293173
Celigo AI
04/17/2025, 3:51 AM