I wrote a session manager[1] to handle all that. Linux/sway/kitty only though.
hop does take it a notch further and allows to `hop open` images (or any binary really) from the remote machine such that they are opened by your host's xdg-open (as well as pasting images from host to remote).
Edit: forgot the ssh part. Same setup on my mac, on my server, at work. So my UX is pretty much always the same. Most of the time it’s via ssh, yes, only rarely I use my mac.