I have to import a customers csv file that has 139k lines / Customers I do file.getContents() but it's failing, too many lines How can I read the file ? Furthermore, I will have to create the customers. It will be slow, any tricks to make it fast ?

easiest to just split into files of ~10k records & CSV them in. Turn scripts off if you don't need them to run. Run in parallel if you have lots of queues

@Simon •

task.CsvImportTask

native NetSuite method to asynchronously import record data into NetSuite. CSV Imports are limited to 25k lines per Import.

I understand the solution but it will cause more problem down the road. Users will have to do that, once a week Having said that, is there's a way to read a csv file, line per line ?

@Simon How would that solution cause issues down the road?

sorry, not issue, a lot of work for the users. They will have to split the file over and over. Sorry for the confusion

@Simon No, they don't, you can write an script that split the file dynamically in smaller files with < 25K lines and automatically import the files with the

task.CsvImportTask

module.

I don't need to use CSV import, in fact, what I'm looking is to read the CSV line per line.

@Simon Using the native CSV Import module allow you to use pre-defined/saved mappings which allow you to quickly change fields mappings in the UI and it also natively handles errors for you. If you go by reading CSV line by line and creating each record via script these are some of the problems you need to face: • For any fields mapping change you will need to update/modify your code. • You need to properly parse the CSV lines in your code to consider any possible issue with that, spaces, tabs, quotes, double quotes, special characters etc. • You need to implement all errors handling in your code. • It would be potentially slower.

If it's a CSV file you want to parse in your script take a look at papaparse as well. Will come in way too handy. I would rather do CSV import though just like others are suggesting, way faster will not occupy queue..

papaparse ?? I'm interested, what is it ?

Usual way to read csv or any file for that matter in Netsuite is by using file.getContents Then for CSV one split it by /r/n for different rows. And then split each row by comma to get all the columns. The problem is if a comma comes in a csv cell itself it splits that cell into 2 as well. So if your CSV had 10 columns for that row it will have 11. There are many other use cases which you have to handle. Papaparse does all that for you and converts your CSV into a neat little dataset.

file.getContents cannot read more than 64k or so I need another solution I'm looking at papaparse. Try to understand, mainly what is needed to use it

Map/Reduce scripts support file streaming. If you return a

File

instance or File descriptor from

getInputData

, the next stage will receive the contents of the file line by line

Papaparse works post you get the filecontent will not work in your case then. The MR option is one of the good option remaining for you.

I'm testing with Map Reduce like Eric wrote and it's looking very good

Papaparse is excellent, btw