what is buffer size in map/reduce exactly? like i ...
# suitescript
r
what is buffer size in map/reduce exactly? like i read the doc it says that keep the size 1 but why ? why cant we increase it to 3 or more whatever size? it had explanation, did not understand.
e
This is how it was explained to me by someone working at Oracle/Netsuite several years ago when I was still learning to craft map/reduce scripts. Buffer Size – This isn’t the amount of memory buffer but it’s connected. It’s the number of map or reduce iterations to go through before you write DB I/O back using the context object to save your results of that map or reduce iteration. So set to 1, it writes back to the DB (using the context object) the result of map right then. Set to 64, it will run map 64 times and then write the 64 results of the context object back to the DB. If your code is written in a way that incremental saves are best, then use lower numbers. However, if you want best performance (sacrificing reliability) pick 64. I usually write my map/reduce runs for speed so I usually use 64. If I get errors, I gracefully error out and continue with processing.
b
its the number of keys you are willing to have errors on if the map/reduce process dies out in the middle
the buffer size determines how many keys to process before updating the database that is used to persist data in map/reduce scripts
if there is a server restart, then the server wont know which keys in the buffer were processed or not, so it restarts them all
potentially creating a situation where you process a key twice
Be sure to Add Logic to Handle Map/Reduce Restarts if data duplication is an issue