I know that when I sdf deploy some script files, sdf will only deploy the files that have changed. But that doesn't seem to be the case for custom objects? It seems like it's sdf is still updating all my custom objects...

<deploy>
  <files>
    <path>~/FileCabinet/SuiteScripts/myScripts/*</path>  <!-- will only update changed files -->
  </files>
  <objects>
    <path>~/Objects/myCustObjects/*</path> <!-- will update all objects all the time -->
  </objects>
</deploy>

will there ever be an update, so that only changed objects are update?

For an ACP, it will always update the custom object

Here's a post describing a workflow that will deploy only the files you've changed. It assumes you're using git for version control though: https://medium.com/tolaeon/using-npm-git-and-the-netsuite-sdf-cli-for-your-dev-workflow-91793c56bfbd?source=friends_link&sk=684ea53067b8280652738a584ad8bf21