Hi, anyone want to share your best practices on ho...
# suitescript
w
Hi, anyone want to share your best practices on how long you would let a scheduled script run per instance? I.e. hogging processors.
c
You can split out the script amongst the queues (processors) old school style. Depending on what that script is doing, the best thing is to convert it to a map/reduce which automagically handles processor management
w
Yeah, but now I already have a script that runs on a single queue. So I'm thinking about limiting it to 10 minutes per instance or so.
c
You can't limit the script in terms of how long itll run. You can yeild map/reduce after X minutes so that other things can run though. With the scheduled script, you could maybe figure out a happy medium of how much it processes in 10 min and only get that much data to process. Can you elaborate a little more on what it is you're doing? There may be a better way to do what you're thinking.
s
^ what are you trying to accomplish that needs to run that long
w
It is an old script that exports all transaction lines in a specific format.
c
is it just running super slow?
s
what happens if you decide you dont want it to run more than 10 minutes, those lines just dont go any where? I dont understand the thought of just stopping the script
if whatever it is doing is not needed, then turn it off imo
w
No no, I just want to yield it so that other scheduled scripts can get a slot to process
So that's why I maybe want to yield it every 3/10 minutes
c
You're going to want to update it to a 2.0 Map/Reduce then. You'll be able to auto specify queues, yielding (which is what you are after here) and it'll be up to date. Probably not the answer you want but thats what I would do.
w
Yeah, it's a pretty big one. Probably don't want to rewrite it completely.
🙂
c
If the script flags the lines as being processed, then you could cut down the amount of data it gets each time then. Besides just migrating it, that may be your best option.
s
I have to disagree here - the scheduled script gives you more control over how/when it yields. As far as I know, the only thing you can control with M/R is the number of processors it uses and a suggestion as to how long it should run before yielding. @Watz I'd suggest you review what scripts are doing and perhaps better coordinate them over time (e.g 24 hour period). An M/R script uses the same processor pool that a scheduled script does, so I don't see how it gets you magical extra horsepower. Another thing to keep in mind is I think even a basic NS accounts gets 2 processors? Make sure you're leveraging both of them.
for a scheduled script, you can trivially make it yield after an amount of clock time simply my starting a stopwatch at the start of the script, and looking at elapsed time as you process - and bail if once you've cross the time threshold.
b
ill give the simple advice of picking a number that reduces the probability of a recurring script deployment scheduled run time being skipped since its waiting in the queue already. That should mean a number smaller than 15 minutes
c
You can control the processors with a MR and yield time. MR Script:
Schedule Script:
so you have more options with a map/reduce
You have it backwards @stalbert
If it were a MR, you could open it to all processors and if you only wanted it to run 15 min before it let other things run, just put in 15 min in the yield box. Problem solved
s
nope - that "Yield after minutes" is a suggestion (read the help on it). Also, it's limiting in that you're only option for yielding is 'minutes'. With a scheduled script you can invent any logic you want to decide it's time to quit.
M/R has very obviously less control over how it executes - that's one of the touted benefits, and sometimes it is.
c
You'll never convince me that a schedule script is even worth while these days
but have at it lol
s
no need, I accept that some folks use only a hammer to build things. 🙂
c
lol
w
Thank you all for your good input Yes. optimally I would rewrite the whole thing (that someone else created) as a M/R and use all available resources in the current account. It currently is only capable of running in a single deployment/processor, and works with a chunk of 1000 transaction lines each loop iteration. So my best bet is probably, as @battk says, to set a number that is lower then the fifteen minute scheduler interval so that other scheduled scripts has a chance of cutting in line. Or as stalbert says, even Basic accounts have 2 processors, then I could hog one of them for an hour and let other scripts share the other one. 🙂
s
If you are doing something and its taking 10 minutes to process 1000 lines, it really might be worth rebuilding.
w
no no, it's processing millions of lines.
I just want to yield it after 10 minutes.
a
@stalbert @creece As far as I know Scheduled Scripts will only use one processor and Map Reduce can use up to 5 if you have SuiteCloud Plus license, in the other hand Scheduled script have usage limitations compared to Map Reduce script which would never go over usage limit when they are well designed. The only time I would choose an Scheduled Script is when I have to do ONE task per result and I don't need grouping/reducing and do multiple operation per stages.
I can give you multiple examples where Scheduled Scripts would be SHORT short for the task.
s
I can give the same where M/R approach is inadequate for the task. There's good reason (imho) that NetSuite still provides both types of scripts - they are different with their own strength/weakness. I'm not sure why people think of M/R as some sort of replacement for Scheduled. I hope NS never deprecates scheduled scripts, unless they come up with a single script type that encompasses all the features of today's Scheduled and MR scripts.
c
@stalbert its a replacement for scheduled scripts because in 1.0, only scheduled scripts existed. People did some magic to get the data split up amongst the available queues which with the solution that floated around, wasn't always accurate anyways and queues would go unused. Map/Reduce came out in 2.0 and handles splitting the data for you automagically and takes full advantage of all of the features without having to do it yourself. So 99.99% of the scheduled scripts out there were acting as map/reduce scripts anyways.
If you want to use a scheduled script go for it, but anything you can do with a scheduled script you can do in a map/reduce but the reverse isn't true.
s
MR is not great if you want to keep a global variable to hold some information, or if the order in which you process things matters.
c
You can use N/cache for global variables. Order matters in terms of what? you get your search results from the input phase and you can process whatever order you want to process in either map or reduce phases.
I honestly don't care what people use. I just use Map/Reduce as I don't ever run across a use case for a scheduled script. There's more than 1 way to do things and im ok with that.
s
Yeah same for me, I seldom find a use case for scheduled.
w
Couldn't there be some cases when a M/R just doesn't have enough points for a complex map/reduce stage? Then a custom process queue record and multiple deployments could process more points-intense tasks since it can yield and get new gov.points.
s
The map/reduce has more points at each stage than the entire scheduled script has, so doubtful
c
Yeah if you write the script correctly, you should never run out of governance. Get Input Data - 10,000 total Reduce - 5000 per iteration Map - 1000 per iteration Summarize - 10,000 total
s
I only recommend M/R scripts when the solution is highly parallelizable and fits well into the getData/map/reduce pattern and actually has significant volume to process. Sometimes a 10 line scheduled script is a much cleaner solution, both to write and maintain.
👍 1
117 Views