<@U5H9HECTS> I think `getRange()` is the easiest. ...
# suitescripts
stalbert
06/05/2018, 2:12 PM@screnshaw I think
getRange()getRange()runPaged()s
screnshaw
06/05/2018, 2:14 PMYeah, I've been using getRange. It just takes forever to return even 20,000 results. If I get much more than that, it'll start timing out.
a
alien4u
06/05/2018, 2:17 PM20,000 results mean processing 20,000 search/result objects, they are not small...
s
screnshaw
06/05/2018, 2:18 PMshouldn't be that much. It's just a query on the db. sql queries can load a whole lot more than that almost instantly
screnshaw
06/05/2018, 2:18 PMalso only returning a small number of columns
s
stalbert
06/05/2018, 2:21 PMif you can use a lazy processing approach it doesn't matter if they are small or not since you'd only hold a single page of results in memory at any given time
stalbert
06/05/2018, 2:23 PMNFT-SS1 lazy search does that, and I'm about to add the same sort of behavior (via a different lib) to NFT-SS2 because it's so handy
r
rant
06/05/2018, 2:24 PMYou could also load specific lines (i.e filter out the unwanted lines) with formulas
rant
06/05/2018, 2:24 PMLoading 20k results, even if you only load specific columns seems like alot
s
stalbert
06/05/2018, 2:25 PMI think it's safe to assume that he needs all those results. Most often though I find i'm processing one result at a time. So if you keep even a max page (1000) you only have 1000 to worry about in memory.
stalbert
06/05/2018, 2:26 PMwith scheduled scripts, the most common and simple pattern we use to process 1 million results is something like
Copy code
`
lazy.loadSearch(1234)
.nsSearchResult2obj()
.takeWhile(governanceRemains)
.each( /*... do something here*. )s
screnshaw
06/05/2018, 2:27 PMNice. Doesn't the search.run() already lazy load though?
s
stalbert
06/05/2018, 2:28 PMit does in the sense that you have to ask for pages of data
stalbert
06/05/2018, 2:29 PMthe key is to abstract yourself from managing and asking for the pages - that code snippet above doesn't change whether there's 1 result or 1 million
stalbert
06/05/2018, 2:30 PMthough the snippet above is more lazy in that each successive operation chained is only called as each individual result is processed, unlike lodash that creates copies of intermediate results.
s
screnshaw
06/05/2018, 2:30 PMis that querying the db directly via nft or just calling the search.run() stuff?
s
stalbert
06/05/2018, 2:31 PMNFT calls search.run() and getRange() behind the scenes, and surfaces a truly lazy sequence that processes results one at a time
stalbert
06/05/2018, 2:34 PMfor example, if you used standard lodash.map() and passed it 1000 results, it will return 1000 new results. So you have in memory now 2000 objects. Each time you do an operation you get a new array of results - that's eager evaluation. Long story short is NFT uses lazy evaluation even in the javascript processing, not just database access.