See AWS Systems Manager Session Manager to learn how to achieve secure terminal access to your Linux and Windows OS instances without the need to establish and maintain Internet-accesible bastion hosts. In the Windows context, you’re provided with a Powershell terminal.
Session Manager can also be used to provide RDP access to your Windows instances. See Forwarding Traffic Between a Local and Remote Port for an overview of this solution and the re:Invent 2019 lab for a detailed example.