What is IPsec (Internet Protocol Security)? - PyNet Labs (2024)

Introduction

Nowadays, most organizations rely heavily on the Internet for daily business; as a result, there has been a surge in cyberattacks. As cybercrime becomes more common, businesses find it difficult to protect sensitive data without compromising productivity. That’s why IPsec is widely used by businesses to allow secure transmission and access of data. But what is IPsec? It is a framework that protects IP traffic at the network layer. IPsec can secure a company’s precious data with characteristics like anti-replay, authenticity, integrity, and confidentiality.

In this blog, we will mainly focus on the Internet Protocol Security protocol, its history, how it works, its mode, and various other crucial concepts. Let’s Begin!

What is IPsec?

IPsec stands for Internet Protocol Security. It is a collection of communication protocols that establish secure connections across a network. The Internet Protocol (IP) is a widely adopted standard that regulates data transmission across the Internet. It enhances the protocol’s security by integrating encryption and authentication techniques.

The IPsec protocol suite may be utilized by anybody, from individuals to massive enterprises, and it also acts as the primary protocol for various Virtual Private Networks (VPNs).

History of IPsec

In the early 1980s, when the Internet Protocol was being developed, security was not a top priority. But as the number of internet users increased, it became clear that more security was needed.

In response to this need, the National Security Agency provided sponsorship for developing security protocols as part of the Secure Data Network Systems initiative. As a result, the emergence of a Security Protocol at Layer 3 occurred, which subsequently led to Network Layer Security Protocol. Various engineers were involved in the 1990s in the development of IPsec.

It is well known today because it can be easily adapted without making any modifications to the associated peers.

Why is IPSec Important?

IPSec is important because networking methods are not encrypted by default. The Internet Protocol Security protocol protects from unwanted surveillance by encrypting important information. The server can also verify whether received data packets are authorized or not.

In other words, networking protocols like TCP/IP are only concerned with delivery and connection, and anyone in between can read the messages because shared messages are not secret. Therefore, it and other protocols encrypt data with an exchange key to secure information between networks.

Uses of IPsec

It can be used to perform the following functions:

• Protect data transmissions at the router level when using the public Internet.
• It helps in encrypting the application data.
• Quickly verify the authenticity of data if it is sourced from a trusted sender.
• Enhance network data security by establishing encrypted circuits, commonly referred to as IPsec tunnels, which effectively encrypt all transmitted data between two chosen endpoints.

Businesses use it as a safeguard against replay attacks. Replay attacks, also known as man-in-the-middle attacks, involve altering the content of a transmission in real-time by redirecting it via a third machine. Each data packet is given a unique identifier and checked for any indications of duplication by the Internet Protocol Security protocol.

Before getting into its working, let’s first understand IPsec encryption.

What is IPsec encryption?

IPsec encryption protects the data by encrypting and authenticating each packet. Encryption keys are used to encode data, while decryption keys are used to restore the original data.

It uses both symmetric and asymmetric encryption techniques to achieve this. Symmetric encryption uses the same key to encrypt and decrypt the data, while asymmetric encryption uses a pair of keys: public and private keys. The public key can be shared with anyone, but the owner keeps the private key secret. It uses symmetric encryption for the data payload and asymmetric encryption for the key exchange.

Let’s move on to the working of Internet Protocol security.

Working of Internet Protocol Security

With IPsec, a client may safely exchange data with another network. Remember that this kind of communication isn’t commonly used for communicating with other devices but rather for connecting a laptop to a private network across a public network. It may also be used to link two private networks together.

Further, the working of IPsec is classified into five steps. These are shown below with a diagram.

1 – Host Recognition

The host system is capable of recognizing when a data packet requires security measures and should be transmitted using Internet Protocol Security protocols. At this stage, the data packet undergoes encryption authentication processes, making it ready for transmission.

2 – Negotiation (IKE Phase 1)

The two host systems that interact via IPsec set up an understanding of the protocols to be utilized and together authenticate the identities to each other. Between the two of them, a secure connection is formed, through which negotiation may occur as to what algorithms and regulations will be in operation. Further, negotiations can be of two types, i.e., main (simple) and aggressive.

• Main negotiation: The host system proposes encryption and authentication algorithms, and subsequent systems negotiate their acceptance with them.
• Aggressive negotiation: The host system presents its preferred encryption and authentication methods without engaging in negotiation or adjusting its preferences. If the host system receives agreement from the other party, the process proceeds to the subsequent step. If the specified condition is not met, the subsequent process will not proceed.

3 – IPsec Circuit (IKE Phase 2)

After successfully establishing a secure connection, as mentioned above, an Internet Protocol Security circuit is then established. The host systems establish a mutual agreement on the encryption and decryption keys to be utilized. Additionally, they exchange cryptographic nonces, which are randomized numbers utilized for authentication purposes.

4 – Transmission

The transmission of encrypted IP packets occurs between the host systems. Upon arrival, the data packet undergoes encryption using the encryption keys that were previously shared.

5 – Termination

The IPsec connection is terminated when either all of the data has been transmitted, or the session duration limit has been reached. The private keys used in the transaction are deleted upon completion of the packet transfer.

What are IPsec Protocols?

IPsec protocols ensure the safety of data transmissions. A data packet is a predefined data structure used to prepare data for transmission across a network. It mainly consists of three elements. These are:

1. Header: A header is a segment that precedes and contains instructional information necessary for the proper routing of a data packet to its intended destination.
2. Payload: The term “payload” refers to the actual information that can be contained inside a data packet.
3. Trailer: The trailer is an extra piece of information that is attached to the end of the payload in a data packet, used to signify the end of the sent data.

Below we have explained some of the Internet Protocol Security protocols.

Authentication Header (AH)

The authentication header (AH) protocol adds a header with sender authentication information to prevent unauthorized parties from compromising a packet’s data. It warns the receiver of any modification that may have occurred with the original data packet. The computer checks the header and the result of the cryptographic hash computation performed on the data payload to ensure they are identical.

Encapsulating security payload (ESP)

Authentication Headers are utilized to verify the identities of both the sender and receiver and to detect any tampering with the packets. On the other hand, Encapsulating Security Payloads (ESP) provides an additional layer of authentication along with encryption.

In the transport mode implementation of IPsec, the Encapsulating Security Payload (ESP) wraps the data payload while leaving the header intact and accessible. In tunnel mode, ESP envelops the whole data packet and adds an additional header on the outside. The outer header is the only visible component of the system without the need for authentication and the encryption key.

Internet Key Exchange (IKE)

The Internet Key Exchange (IKE) protocol creates a secure connection between two devices operating on the Internet. Both devices establish a security association (SA) by engaging in the process of negotiating encryption keys and algorithms that will be used for the transmission and reception of future data packets.

IPsec Modes

IPsec is a network protocol that allows secure communication by offering two modes of operation: tunnel mode and transport mode. The modes are closely linked to the choice of protocols used, specifically AH or ESP.

Let’s understand both these Modes in detail.

• Tunnel Mode: Every packet is kept safe when operating in tunnel mode. IPsec encapsulates the data packet within a new packet, applies encryption to it, and adds a new IP header. It is widely utilized for Site-to-site VPN setups.
• Transport Mode: The IPsec transport mode encrypts the data packet’s payload and keeps the IP header in its original state. The unencrypted packet header helps routers identify the destination address for each data packet.

IPsec transport is frequently used within a secure and trusted network environment to safeguard a direct and uninterrupted communication link between two computer systems.

Note: When it comes to virtual private networks (VPNs), ESP with authentication in tunnel mode is the most typical IPsec setup. This setup allows for encrypted and anonymous data transfer over unsecured networks inside a VPN tunnel.

What is IPsec VPN?

An IPsec VPN, also known as a virtual private network, is a type of VPN that utilizes the Internet Protocol Security protocol to establish a secure and encrypted tunnel over the internet.

A Virtual Private Network (VPN) is a technology that securely routes network traffic through an encrypted tunnel, ensuring the protection of data from unauthorized access. An IPsec VPN utilizes the IPsec protocol to establish a secure connection and encrypt data packets during transmission. This technology is especially beneficial for businesses and large organizations that have employees working remotely and require remote access to resources.

By configuring an Internet Protocol Security VPN between a remote worker’s device and an internal server, an organization may provide the person with the same level of secure access to internal resources as if they were physically present in the office.

There are a variety of strategies for setting up an IPsec VPN. Some of these are Site-to-site, Client-to-site, and Client-to-client.

There’s always confusion over IPsec and SSL VPN, which one is better or which one is more secure. Below we have explained the basic difference between Internet Protocol Security VPN and SSL VPN.

IPsec vs SSL

Advantages of IPSec

Here are a few advantages of IPSec –

• It protects critical data by ensuring network confidentiality and integrity through its strong cryptographic security services.
• Due to open standard protocols, it is supported by vendors and works in diverse environments.
• It can be arranged to secure various network topologies such as point-to-point, site-to-site, and remote access connections.
• It secures large-scale networks and can scale up or down as needed.
• It reduces network congestion and improves network efficiency, thereby increasing network performance.

Disadvantages of IPSec

Some of the disadvantages of IPSec are –

• It requires specialized skills and knowledge as it is difficult to configure.
• It impacts network performance because of the overhead of encryption and decryption of IP packets.
• It causes compatibility issues and interoperability issues related to network devices and applications.
• Cryptographic keys are essential for encrypting and retrieving information, so they require effective key management.
• The IPsec protocol provides limited security because it secures IP traffic, and other protocols, like ICMP, DNS, and routing protocols, may still be vulnerable to attacks.

Frequently Asked Questions

Q1 – What is IPsec and why it is used?

IPsec is a set of protocols that provide security for Internet Protocol (IP) networks. It is used to encrypt and authenticate the data packets that are exchanged between two or more parties. IPsec can protect the confidentiality, integrity, and availability of network communications.

Q2 – What are the 3 protocols used in IPsec?

The three main protocols used in IPsec are:

• Authentication Header (AH): This protocol provides data integrity and authentication for IP packets but not encryption.
• Encapsulating Security Payload (ESP): This protocol provides data integrity, authentication, and encryption for IP packets.
• Internet Key Exchange (IKE): This protocol establishes and manages the cryptographic keys and parameters for AH and ESP.

Q3 – Why SSL is better than IPsec?

SSL allows for more specific permissions for users than IPsec does. Members of the remote team are given access to specific programs rather than being given complete network membership. This makes it easy to provide varying degrees of access to various users.

Q4 – What port is IPsec?

IP Security Virtual Private Network (IPsec VPN) uses IP Protocol 50 (Encapsulating Security Payload, or ESP) as its layer 3 communication protocol. Internet Key Exchange (IKE) may also be needed to maintain encryption keys, which can require UDP port 500.

Conclusion

IPsec consists of a standard set of protocols used to protect internet connections, ensuring authenticity, confidentiality, and integrity. It offers a transparent and secure route for upper-layer protocols without requiring any changes to these protocols or applications.

This Blog has covered all the basic concepts and different modes as well as different IPsec protocols. Still, if anyone wants to add something, feel free to comment below.