Hi Celigo question here.
I have an inventory sync...
# integrationss
Stewart
08/09/2024, 12:44 PMHi Celigo question here.
I have an inventory sync flow that takes a saved search from Netsuite and compares with a 3rd party, and creates an inventory adjustment for the difference.
At the moment, it runs 'a record at a time' and runs the Netsuite search for every sku, which is taking much too long.
Is there a way to run the saved search once, map it into the data and then do the lookup in a webhook?
Many thanks, Stewart
t
Tyler Lamparter
08/09/2024, 5:29 PMThe export step from NetSuite would process in batch. You mean the lookup step to get 3rd party data processes per record?
s
Stewart
08/09/2024, 5:37 PMHi Tyler, yes - that’s right. It’s quite a big lookup and quite slow so I’d like to run it once and then do the lookup in a webhook.
t
Tyler Lamparter
08/09/2024, 5:53 PMIs the lookup to another netsuite instance or just some generic rest api?
s
Stewart
08/09/2024, 5:55 PMIt’s a 3PL with a Rest API. The data appears as a page of records.
t
Tyler Lamparter
08/09/2024, 5:56 PMDoes their api allow you to get all products at once? Does it allow you to get just 20 or so if passed which items to get in a single request? Determing the best response for you depending on how it operates
s
Stewart
08/09/2024, 5:59 PMCurrently I'm getting all products in 1 call, the response I'm seeing is:
{
"page_of_records": [
{
"record": {
"data": [
{
"timestamp": "2024-08-09T195847+02:00",
"skuId": "010.01730.423.02636.010",
"countryOfOrigin": "CN",
"inventoryType": "OK1",
"quantityTotal": 17,
"quantityLocked": 0
},
{
"timestamp": "2024-08-09T195847+02:00",
"skuId": "010.01730.423.02636.006",
"countryOfOrigin": "CN",
"inventoryType": "OK1",
"quantityTotal": 4,
"quantityLocked": 0
},
{
"timestamp": "2024-08-09T195847+02:00",
"skuId": "010.01730.423.02636.016",
"countryOfOrigin": "CN",
"inventoryType": "OK1",
"quantityTotal": 6,
"quantityLocked": 0
},
{
"timestamp": "2024-08-09T195847+02:00",
"skuId": "010.01730.423.02636.014",
"countryOfOrigin": "CN",
"inventoryType": "OK1",
"quantityTotal": 10,
"quantityLocked": 0
},
{
"timestamp": "2024-08-09T195847+02:00",
"skuId": "010.01730.423.02636.008",
"countryOfOrigin": "CN",
"inventoryType": "OK1",
"quantityTotal": 13,
"quantityLocked": 0
},
{
"timestamp": "2024-08-09T195847+02:00",
"skuId": "010.01730.423.02636.018",
"countryOfOrigin": "CN",
"inventoryType": "OK1",
"quantityTotal": 2,
"quantityLocked": 0
},
{
"timestamp": "2024-08-09T195847+02:00",
"skuId": "010.01730.423.02636.012",
"countryOfOrigin": "CN",
"inventoryType": "OK1",
"quantityTotal": 15,
"quantityLocked": 0
}
]
}
}
]
}
t
Tyler Lamparter
08/09/2024, 6:01 PMHow many products do you have?
s
Stewart
08/09/2024, 6:01 PMProbably about 10,000
Stewart
08/09/2024, 6:02 PMActually - maybe less with stock, a few thousand I think
t
Tyler Lamparter
08/09/2024, 6:15 PMOne option could be option 1 in this post: https://docs.celigo.com/hc/en-us/community/posts/24051709605147/comments/24330110634907. Essentially, it does a lookup per page of data instead of per record, but requires some scripting to then map the one lookup to each record in the page of data. This may get you into a 5MB page size limit though depending on how many records are returned and what your page size on your export is.
Tyler Lamparter
08/09/2024, 6:16 PMSecond option, could be to have a flow that brings this 3rd party data to a NetSuite custom record and then just link it to the item, have this run once a day, then pull that data into your NetSuite saved search itself
Tyler Lamparter
08/09/2024, 6:18 PMThird option, could be to run a virtual export within a preSave page script on your netsuite export and set the pafe size on your netsuite export to a higher limit. This would get the whole list in JS memory, then you would map the response to the data coming from netsuite, then move on. This is similar to the first method, but you would most likely not run into a size limit issue because JS functions have more buffer memory than a saved page of data
s
Stewart
08/09/2024, 6:40 PMThanks Tyler - I'll give those a shot
👍 1
t
Tyler Lamparter
08/09/2024, 6:41 PMI personally like option 3 best and it’s what I would do. Let me know if you need help
11 Views