[Hampshire] confused ssh newbie

Hi folks.

I can ssh into my debian machine on port 22 (with that port forwarded on
my router) using

ssh testermike@???

However, when i change the port to 2222 (changing the router of course)
and repeat using

ssh -p 2222 testermike@???

I get an error that the connection was reset by peer and I cannot ssh in.

What am I not doing please?

TIA
Mike