If you had to set up a new record type and import ...
# general
j
If you had to set up a new record type and import ~1 million records, how would you tackle that, given that CSV import only supports 25k at a time?
r
There used to be like an "Advanced Integration" tool that I believe let you queue up a bunch of files one after another, but I don't think that let you do anything to bypass the limit, you could just queue up your files one after another. You could see if that's still around. Tbh if this is a one-time thing, it's "only" ~40 files, which I realize would be painful to get set up, but might be just as quick as spending time to (maybe) figure out something better. Different story obviously if you're going to be doing this on a regular cadence.
j
To be honest I’m worried about how long it will take no matter what method we use. Like even if we already had 40 files queued up how long would it take NS to process them all.
hopefully just a couple days at most.
r
How many queues do you have? If you only have 1, that could definitely be painful, but it shouldn't be too terrible if you can run at least 5 files at a time. Though the import time per record seems to vary wildly in my experience. Your 25k per queue could take 90 minutes or 3 or 4 hours. That part is definitely frustrating.
j
How do I tell how many queues I have?
m
in the advanced options of your CSV import you can select what Queue to use for the import
j
Something to note, that I only learned relatively recently. The number of queues does NOT always equal the number of threads/simultaneous imports.
image.png
so like for example for one suitecloud plus license you have 5 queues, but you'll only ever be running two imports simultaneously.
j
hm
feels like there should be a way to know my queue count w/o trying to create a CSV import
I mean I can but I’m lazy
m
Look under billing info to see how many suitecloud plus’s licenses you have
1️⃣ 1
💯 1
m
Integrator.io (Celigo) data loader is available on free tier and can handle much larger CSVs.