Curious if anyone has seen a slowdown this week in...
# suitescript
e
Curious if anyone has seen a slowdown this week in any of their scripts. We have a restlet that runs a few queries and generally runs at a very stable
0.7s
. This Monday around 11:00 Eastern, there is a huge cliff where it leaps up to
4.0s
and has stayed there since, with spikes going up to
10.0s
. This is a Production account on 2023.1, and we only deploy code on Tuesday nights, so this not the result of any configuration or code change.
watching following 1
👀 1
c
Is it back to normal or still fluctuating wildly?
a
Some other integration had a massive increase in volume and is cascading to all your restlets?
Is the performance impacting other systems? You have an incident and have to do an RCA and all?
Any change in the test env? Or is this just in prod?
A 5x slowdown is pretty extreme, and since it was a cliff rather than gradual I’d look for changes around that time window
 just cos there’s no code deployment doesn’t mean something else wasn’t changed in confit
c
its a multi-tenant system so there's too much you don't know to make a lot of assumptions on performance. APM can help but if you're sharing w/ more people and they do a lot then it can slow it down too.
a
I’m not sure the account in question is multi tenant
e
No it's not back to normal, and none of our sandboxes are affected
a
Contact support probably won’t help but they may see things you can’t
e
Yeah we're working with support, but I wanted to see if this was unique to our account
a
Or ideally have someone else contact them so you don’t have to deal with them 😂
đŸ€­ 1
c
#C29GDT351 seems to be saying there's some slowdowns too so you probably aren't alone
e
Oh thanks, I'm not in that channel
a
#C29GDT351 are almost always just talking about the UI this is specific to just RESTlets from what Eric said
j
I've heard this a lot lately. Especially after 2023.2
s
Does this code make any calls to external systems?
there was this huge internet outage the last few days... My ISP still having issue with some destinations
e
It does, but we've ruled those out as well
s
ah, ok just a coincidence then
e
We have pretty detailed metrics on this particular request
s
ah, you're using the NFT logger then? 🙂
😆 1
a
Was this account recently migrated to OC(something) Oracle Cloud Infrastructure?
e
Not recently, we went through similar-ish pains during that time though
a
Oracle does not share NetSuite shared servers usages, from my old days in Networking the way they host NetSuite instances is very similar to the VPS concept(nowadays everything have a different name starting with “cloud”), the core concepts of this, which I doubt evolved too much in the past few years is that you share the resources of a single hardware(server), therefore, if there is an usage spike in others instances sharing the same server with you, your instance could be affected, it could be for a period of time until that load leverage or until they realize they overloaded the server and need to move some instances to another physical server, is very hard to know because they don’t share any details of their backend infrastructure.
Again, too many factors in place, maybe is simply a new Layer 7 firewall that is not fully configured yet or configuration changes that prioritize other traffic, specially if they are actively migrating old netsuite instances to the new OCI in that that datacenter, etc

e
To anyone following along, this issue disappeared immediately upon our upgrade to 2023.2.
blinkingguy 1
🚀 1
c
The ol turn it on and off again trick
e
This issue has reappeared suddenly :(
đŸ˜© 1