Hi we're working a map reduce script trying to process 2000 transactions (each has 1 line) and the performance I'm getting roughly around 2 records per second. Is this the fastest. We have SC and I’m having it run around 7 concurrency which I find this performance scaling very alarming. Anyone has any thoughts? This is via map/reduce.

Normally it does not get a lot better than that, NetSuite is not a traditional DB, every transaction line is inter-connected to a bunch of other tables and those table are also inter-connected, lot of things happening in the background. If you want to do an experiment, find or create a transaction, let say an Invoice with 1000 or 2000 lines, de-activate any scripts or workflows, EDIT and SAVE in the UI, it will take closer to 5 minutes if not more to save, without changing/updating a single thing.

Application Performance Management is the standard answer for performance monitoring to see what cost each script/workflow is taking

what does "process" mean are you loading and editing? run queries to get additional data for each transaction??? you've told us nothing. "around 7 concurrency" ??? all that said you're concerned that a process takes 15minutes?

Process- simply creating records. (Ex. Invoices) there's no data lookups. Yes concerned because I have lots of records, batches. At this pace I would only hit 3,600 transactions an every half hour. And if I have batches to process (say ten thousands, it's taking way too long

I'd check your scripted records for the transaction type and see what things you have running on beforeSubmit workflows / other User event scripts

If you are transforming SOs to Invoices and you have a large number of those, perhaps you need to look at the business process itself, there is more than one way to skin a cat... Per instance you could have a UE script that transform the SO to Invoice once a particular condition is met and you automate that portion without having to use a Map Reduce or having to handle thousands of SOs in a cumulative way.

I was able to save a few ms to like 2.2 records per second without any scripts/workflows. There's no transformation. Just simple standalone invoice or cash sale with a single line item

It will not get better than that, NetSuite is not a traditional relational database, you will never archive a throughput similar to MySQL, MariaDB, Postgres, never ever....

I'm wondering if the overhead of creating the MR stages is a factor given your processing is so simple, normally I'd just ignore that, but it might be worth testing. assuming you're doing 1 invoice per map stage you have 1k governance per map stage and you're only using 20 i think for a transaction save? so would it be feasible chunk your data into groups of say 40? that would still only use 800 governance but you'd have way fewer map stage instances. I have no idea if that will actually help but if the data chunking is easier enough for you to do then it should be worth a quick test.

My current saves are in reduce. I could try to do it in map, but the parallelism happen in either map/reduce doesn't if? So does it really matter if it happen in the map or reduce?

not at all, i just assumed the map cos that's simple... reduce is actually better you have 5k governance instead of 1k... so you could potentially do 200 per reduce stages

I tried chunking by batching multiple invoices in a reduce, like 20, but the whole concurrency broke and it didn't distribute the workload across concurrency wells

I don't think you have a clear understanding of NetSuite's concurrency model

Yes I understand. But when I batch. However it’s splitting the 1000 records across the different queue/concurrency. For whatever reason it just goes to 2. Vs the other 5. The distribution was weird when I batch 20 records per reduce.

it might have just been some other unrelated process triggered at the same time and stole some of the queues 😉

It took longer. I think it might be a bug. Because if I had single recorder per reduce. It distributed fine and to what you explained.

where/how is the data coming into NS?

I had testing very specific. It was an odd behavior. I looked at the status and 5 just stopped and finished while only 2 was running. Hmm let me double check and change to high priority and make sure I have no other map/reduce running. I could be overlooking that.

your loading the csv file lines into memory and then using a map reduce per line? to create invoices?

Yes.

k and you ruled out csv import task because... you want error handling /validation or something?

Correct.

... how many data elements are you setting? like how many fields are there per invoice/line combination? approximatey? 10? 100? 1000?

I'm going to test the csv tomorrow to see the performance there. But just isn't ideal without error handling/logging.

oh when you do the record create are you using dynamic mode?

Fields about 15 fields. No lookups. All single lines.

... so a little crazier option... create a new custom record - Invoice Import Map create your 15 fields on that record map reduce to create those custom records instead of your invoices and have a UE aftersubmit on your new custom record which takes those 15 data elements and creates the invoice.

i can do a quick test.

exactly

i haven't tested but can do a quick test as my mr import tool is a CONFIG so i just need to make a new config to load and can see metrics there

that's 146 per minute? i thought you said it was 2? or is this custom records?

2 per second

oh duh right

yea dynamic/no dynamic is neligible. in my case it behave a bit worse. probably an outlier

@Phi-My Tran Not sure if this is suggested already, but you can also try using the promise version for save. It should shave off the time the script waits for the response to return for the save.

thanks. i was going to try that next. I was able to double the concurrency from 7 to 15 , and performance did gain upward to 742 records per minute average to 9 records a second. so sounds like SC+ is the way to go.

Use promise version of the save as well.

isn't promise only on clientscripts only? https://docs.oracle.com/en/cloud/saas/netsuite/ns-online-help/section_4440842328.html

No, it is on server side as well. I have used it in my MR/Scheduled scripts

Scripts still wait for the promise before stopping. Your function might return faster, but the script engine still needs to run to wait.