What Is SSH? How Does It Work? (System Design for Beginners)

SSH (Secure Shell) is a foundational protocol in system design and cloud engineering that establishes a secure, authenticated, and multiplexed tunnel over TCP. It enables remote shell access, secure command execution, and flexible network traffic forwarding between local environments, cloud virtual machines, and private data centers.

1. Why Do We Need SSH? (The Cloud Deployment Problem)

When you provision a virtual machine on a cloud provider (such as Google Cloud Platform, AWS, Azure, or DigitalOcean), the machine starts as a blank slate.

• You need to install runtimes and servers (e.g., Golang, ReactJS, PostgreSQL).
• You need to configure environment variables, start background processes, and manage files.
• You need a secure method to interact with the virtual machine's terminal CLI remotely—almost as if you were sitting directly in front of it.

SSH solves this exact problem by granting developers and system administrators secure remote shell access and encrypted command execution over unsecured networks.

2. Core Mental Model: The Secure Tunnel

Think of an SSH connection as an armored, encrypted pipeline built directly between your local machine (the client) and a remote server.

• Secure: All data, commands, and terminal outputs transmitted through the tunnel are encrypted.
• Authenticated: Both the client and server verify each other's cryptographic identities, preventing unauthorized access or man-in-the-middle attacks.
• Multiplexed: Multiple distinct communication channels or sessions can run concurrently over a single underlying TCP connection.

3. Four Standard Architectural Patterns of SSH

SSH is remarkably versatile and is not limited to simple remote logins. Depending on your system design requirements, SSH tunnels can be configured in four primary patterns:

Detailed Breakdown of Patterns

4. The SSH Handshake & Connection Walkthrough

Before any actual commands or terminal data flow through the tunnel, SSH executes a rigorous cryptographic handshake and key verification process.

Step-by-Step Handshake Sequence:

1. TCP Connection Setup: The client initiates a standard three-way handshake with the SSH server over port 22.
2. Protocol Version Negotiation: Both machines agree on the compatible SSH protocol version.
3. Algorithm Negotiation: The client and server negotiate encryption algorithms, hashing algorithms, and public-key types (e.g., RSA, ECDSA, Ed25519).
4. Key Pair Generation: Temporary or permanent cryptographic key pairs are established.
5. Public Key Transmission: The client transmits its public key to the server.
6. Session Key Calculation: Both parties independently derive a shared symmetric session key for encrypting subsequent application data.
7. Login Request: The client requests authentication using credentials or SSH keys.
8. Encryption Challenge: To verify that the client genuinely possesses the corresponding private key, the server encrypts a random number using the client's public key and sends it back.
9. Decryption Verification: The client decrypts the random number using its private key and sends the decrypted result back to the server.
10. Tunnel Activation: Once the server verifies the correct decryption, the cryptographic handshake is complete.
11. Encrypted Transaction: Actual terminal commands, data streams, and interactive sessions begin, encrypted and decrypted securely using the session key.

5. Summary and Key Takeaways

• Definition: SSH is a secure, authenticated, multiplexed tunnel running over TCP, primarily used for remote shell access and secure command execution.
• Cloud Infrastructure Management: It serves as the primary gateway for developers and system administrators to provision, configure, and maintain virtual machines hosted on cloud platforms like AWS, GCP, Azure, and DigitalOcean.
• Versatile Architectural Patterns: Beyond simple remote login, SSH supports reverse tunnels (for webhooks and local exposure) and bastion/jump hosts to isolate and protect core infrastructure.
• Availability & Security: Bastion patterns enhance system availability by ensuring that potential network attackers target an expendable jump host rather than critical backend servers.
• Cryptographic Handshake: The connection sequence involves TCP establishment, protocol and algorithm negotiation, public key exchange, and a challenge-response decryption test using random numbers.
• Symmetric Session Keys: Once identity is cryptographically verified, all live command execution and data exchange occur over an encrypted session key, preventing eavesdropping or tampering.