Would it be possible to keep an sFTP connection within the map phase of a map/reduce script? I'm using the

connection.list

as the input.

not really sure what you mean by keep, just make a new connection in your map

I would like to prevent the script to relog the sFTP server with every file. So it would be nice to keep the connection alive

nope

That would be even worse than a simple scheduled since a map phase can only use 1000 governance, right?

you could try

N/cache

but I doubt it would maintain the underlying connection.

if you are trying to minimize logins, then you could use reduce, which has 5000 points to work with

yeh what I thought, so a scheduled would be best with a restart

A single invocation of the

map

stage can only use 1000 governance

Thanks, that's what I knew yeh but when we can't keep the connection alive that wouldn't help either way

Is there a reason you need to avoid logins?

but now that I read SuiteAnswers 88609 it looks like the connection is reset every up/download. "After receiving the confirmation about the possibility of the connection, the connection itself is established for every SFTP request, in this case it's file upload. This behavior is due to the design of the sftp.createConnection(options) function."

How much overhead does it add? How often does this run? How will that overhead impact the business?

exactly so we'll just have to work with it