ssh tunnel - How can I remotely connect via ssh and reuse this connection for other apps without asking for the original credentials?

There's a remote Ubuntu container that I can access via SSH using a one-time connection code (username-password). I'm using Windows, and don't have much rights on the container itself. So I'm trying to do this without changing any settings on the container. I can create a SSH connection just fine to port 2222 (specifically) on the remote host.

However, the app I'm trying to use (remote debugging in Visual Studio, not very important), is apparently not compatible with single-use passwords. Let's assume that to be true.

So what I had in mind was opening a SSH connection from my computer to the host using the one-time password, create some kind of tunnel, and then have the application re-use that connection without having to provide any username or password.

I tried using Putty to forward a local port (2345) to the remote port (2222), but after that I can't SSH to localhost:2345 without any creds (and end up on the remote server like I had hoped).

After some reading, I unfortunately still don't have a great understanding of SSH tunnels. Is there a way to accomplish what I'm trying to do?

Thanks for any help.