https://netsuiteprofessionals.com logo
Join SlackCommunities
Powered by
is there a way I can prepare data objects I can re...
# suitescript
s
is there a way I can prepare data objects I can reference (something like a global parameter) during getInputData, whilst keeping returning a searchObject for the map stage?
d
As far as I know, no. All the stages in MapReduce are independent, so you'll have to pass all the objects to the map stage explicitly.
d
If it's read only data, or you're just running single concurrency, you might be able to write the data to a script parameter or custom record.
d
Hm. Yes, didn't think about a custom record. Good option
s
i thought about writing to json into the file cabinet for temporary processing, script parameter could do but I have a scaling issue.
I'm given the customer code but I need to search for fields with the given customer code, naturally if I have master data in one json as a constant i can feed it through that to get what I need, and I'm reluctant to run searches everytime to get that
I'm almost considering searching in inputData and then saving a JSON with the file module and load the module every time to do it, but even that seems like it's not very efficient
in my mind this would simulate like the cache module, but that being said, not sure this is the way i should take it
d
I've had to write files to the file cabinet with a MR (up to 1k or so files per execution) and it was far less slow than I'd expected.
s
I'd have to run some tests, but storing one file from input stage then loading in map, feels like a better solution than running that during every map stage. Idk, I was hoping I can work around that some how. The scenario is i'm given a customerId (not internalid) in netsuite but i need some other data from the customer record. I'm reluctant to do a search but thats all I can do. So I kinda would rather load it to somewhere I can store, rather than do this mapping again in each map stag.
e
If you're searching in getInputData and saving to JSON, why not just build up your objects in getInputData and pass to map?
s
because I want to pass the searchObj to loop through in map
suppose the searchObj exceed 4000 elements, I think it's best I still run through that search Obj, I just want to save my map or performance a bit because I dont want to rerun that part which might be a constant every map run???
searchObj as in it's the thing from search.create or search.load, and it's huge
e
fair enough, makes sense if the search is large
s
I just dont know if it's better to do N/file or N/cache if I could to do the same thing of storing this data
or if theres an alternative
b
choose N/cache if you only want to read data
answers that involve writing are various levels of wrong if you have to handle concurrency
w
I think each individual map instance will keep it's global variables between map iterations. So maybe you could run tge the search on the first run in that instance and than all subsequent runs can use the same data (if it's the same customer info on all) When the map yields it will refetch the data.
b
the global environment is erased every time the map yields
w
...as I wrote. 😉
But depending on how many units the script uses, the yield might not happen very often.
I use this method to convert scriptids on list values to internal ids in map. Usually store them in a global variable with a parameter that has a boolean for "isFetched". If it isn't fetched, I'll do the fetching again.
s
@Watz really interested, could you explain more about how the map instance will keep global variables? so in the Map Reduce I'll define a global variable, and call the variable in the map stage? What you mean is there is some kind of memory across the same map stage deployment?
w
Yes, each Map or Reduce(I have never have never utilized it in reduce) deployment/instance/10000units share global variables. Note that if you execute it on five processors, those five processors won't share the variables between them as those are separate instances.
This is an example of the function that is called in the start of each Map (typescript):
Copy code
export const PROCESSING_STATUSES = {
    fetched: false,
    ERROR: '',
    FINISHED: '',
    PROCESSING: '',
    TO_BE_PROCESSED: '',
}
export const getProcessingStatuses = () : void => {
    if(!PROCESSING_STATUSES.fetched) {
        const searchObj = search.create({
            type: 'customrecord_processing_statuses',
            columns: ['scriptid']
        })
        searchObj.run().each(result => {
            PROCESSING_STATUSES[result.getValue('scriptid') as string] = result.id
            return true
        })
        PROCESSING_STATUSES.fetched = true
        log.debug('PROCESSING_STATUSES has been fetched', PROCESSING_STATUSES)
    }
}
🙌 1
s
thanks! that's really interesting!!
Open in Slack
PreviousNext
Powered by