Is it possible to use Map/Reduce to run a saved search and save the results as CSV file in the File Cabinet? I was able to write a scheduled script to run the search, and save it as csv in the File Cabinet, but the saved search is in the hundred thousand lines.

I don't see why not - you could use the N/file appendLine function to keep adding lines to an existing file. You would just need to make sure your MR script is using a single processor, and would definitely want to leverage the reduce function to avoid opening and then saving a file each time the map function is called (for each line), which could take forever.

makes sense

Also, take a look at task.SuiteQLTask

@JC You don't need a Map Reduce to do that, you can use the N/task module to do that and would be 100% native NetSuite background process without using processors as Map Reduces do...

I have used SuiteQL and it's awesome. Somehow accounting wants this to be exported to .csv and save to the cabinet.

I recommend using N/task module as well as it will completely be done in the background. If you create a csv on your own, you will have to handle comma that may come in some columns, etc. And some other peskly little corner cases.

If you use

task.SearchTask

your script would be simple and asynchronous with few lines and native NetSuite functionality, meaning you don't need to worry about parsing lines or columns.

I think the question is if there are limits on how much data it will export

right

If you can export it in the UI you can export it with N/task... is the same background process/functionality...

When working with suiteql via the query module, there are 100k row limits, unless you have SuiteAnalytics Connect feature purchased.

so the limitation could come in the actual csv file

Best to try exporting more than 100k rows via the UI and see if it works

true

That's a good sign then

Does the system queue those SearchTasks if it's submitted more than once (already running) or too many are running at once?

I agree that the search task is the way to go. Writing to the file cabinet is is also possible (currently doing this with a massive n-ary tree json object file), but you have to keep in mind that map and reduce stages happen in parallel, so loading and saving the file will run into issues there. For my use case I build a bare bones tree in GID and save it, pass item IDs to map, gather related record data in reduce, and then load the file and insert my results into the tree in summary when execution is sequential again.

Presumably you could just write a simple scheduled script that just submits the search Task if you literally just need the output writing to a csv?

so I found this based on @alien4u https://ceanatech.com/2019/02/27/suitescript-2-0-why-you-should-check-out-n-task-searchtask/

@JC There are some shenanigans/bugs with NetSuite FileCabinet files when you try to overwrite them, best bet, delete the file if already exist, create an empty one and make the searchTask do its thing.

That's what I thought

If the name and folder where you save the file is consistent, they can use the folder Id to find the file or a combination of folder ID + fileName

@alien4u thanks for all the help!

Alternatively, load the file, set the content to empty and save it before you run the task.