Contents Previous Next

# The IPSEC protocols

This section provides details of the IPSEC protocols which FreeS/WAN implements

The basic idea of IPSEC is to provide security functions, authentication and encryption , at the IP (Internet Protocol) level. This requires a higher-level protocol (IKE) to set things up for the IP-level services (ESP and AH).

Three protocols are used in an IPSEC implementation:

ESP, Encapsulating Security Payload
  Encrypts and/or authenticates data
AH, Authentication Header
  Provides a packet authentication service
IKE, Internet Key Exchange
  Negotiates connection parameters, including keys, for the other two

The term "IPSEC" is slightly ambiguous. In some contexts, it includes all three of the above but in other contexts it refers only to AH and ESP.

## Applying IPSEC

Authentication and encryption functions for network data can, of course, be provided at other levels. Many security protocols work at levels above IP.

- PGP encrypts and authenticates mail messages
- SSH authenticates remote logins and then encrypts the session
- SSL or TLS provides security at the sockets layer, e.g. for secure web browsing

and so on. Other techniques work at levels below IP. For example, data on a communications circuit or an entire network can be encrypted by specialised hardware. This is common practice in high-security applications.

### Advantages of IPSEC

There are, however, advantages to doing it at the IP level instead of, or as well as, at other levels.

IPSEC is the most general way to provide these services for the Internet.

- Higher-level services protect a single protocol; for example PGP protects mail.
- Lower level services protect a single medium; for example a pair of encryption boxes on the ends of a line make wiretaps on that line useless unless the attacker is capable of breaking the encryption.

IPSEC, however, can protect any protocol running above IP and any medium which IP runs over. More to the point, it can protect a mixture of application protocols running over a complex combination of media. This is the normal situation for Internet communication; IPSEC is the only general solution.

IPSEC can also provide some security services "in the background", with no visible impact on users. To use PGP encryption and signatures on mail, for example, the user must at least:

- remember his or her passphrase,
- keep it secure
- follow procedures to validate correspondents' keys

These systems can be designed so that the burden on users is not onerous, but any system will place some requirements on users. No such system can hope to be secure if users are sloppy about meeting those requirements. The author has seen username and password stuck on terminals with post-it notes in an allegedly secure environment, for example.

### Limitations of IPSEC

IPSEC is designed to secure IP links between machines. It does that well, but it is important to remember that there are many things it does not do. Some of the important limitations are:

IPSEC cannot be secure if your system isn't
  System security on IPSEC gateway machines is an essential requirement if IPSEC is to function as designed. No system can be trusted if the underlying machine has been subverted. See books on Unix security such as Garfinkel and Spafford or our web references for Linux security or more general computer security.

Of course, there is another side to this. IPSEC can be a powerful tool for improving system and network security. For example, requiring packet authentication makes various spoofing attacks harder and IPSEC tunnels can be extremely useful for secure remote administration of various things.

IPSEC is not end-to-end
  IPSEC cannot provide the same end-to-end security as systems working at higher levels. IPSEC encrypts an IP connection between two machines, which is quite a different thing than encrypting messages between users or between applications.

For example, if you need mail encrypted from the sender's desktop to the recipient's desktop and decryptable only by the recipient, use PGP or another such system. IPSEC can encrypt any or all of the links involved -- between the two mail servers, or between either server and its clients. It could even be used to secure a direct IP link from the sender's desktop machine to the recipient's, cutting out any sort of network snoop. What it cannot ensure is end-to-end user-to-user security. If only IPSEC is used to secure mail, then anyone with appropriate privileges on any machine where that mail is stored (at either end or on any store-and-forward servers in the path) can read it.

In another common setup, IPSEC encrypts packets at a security gateway machine as they leave the sender's site and decrypts them on arrival at the gateway to the recipient's site. This does not even come close to providing an end-to-end service. In particular, anyone with appropriate privileges on either site's LAN can intercept the message in unencrypted form.

IPSEC cannot do everything
  IPSEC also cannot provide all the functions of systems working at higher levels of the protocol stack. If you need a document electronically signed by a particular person, then you need his or her digital signature and a public key cryptosystem to verify it with.

Note, however, that IPSEC authentication of the underlying communication can make various attacks on higher-level protocols more difficult. In particular, authentication prevents man-in-the-middle attacks.

IPSEC authenticates machines, not users
  IPSEC uses strong authentication mechanisms to control which messages go to which machines, but it does not have the concept of user ID, which is vital to many other security mechansims and policies. This means some care must be taken in fitting the various security mechansims on a network together. For example, if you need to control which users access your database server, you need some non-IPSEC mechansim for that. IPSEC can control which machines connect to the server, and can ensure that data transfer to those machines is done securely, but that is all. Either the machines themselves must control user access or there must be some form of user authentication to the database, independent of IPSEC.
IPSEC does not stop denial of service attacks
  Denial of service attacks aim at causing a system to crash, overload, or become confused so that legitimate users cannot get whatever services the system is supposed to provide. These are quite different from attacks in which the attacker seeks either to use the service himself or to subvert the service into delivering incorrect results.

IPSEC shifts the ground for DoS attacks; the attacks possible against systems using IPSEC are different than those that might be used against other systems. It does not, however, eliminate the possibility of such attacks.

IPSEC does not stop traffic analysis
  Traffic analysis is the attempt to derive intelligence from messages without regard for their contents. In the case of IPSEC, it would mean analysis based on things visible in the unencrypted headers of encrypted packets -- source and destination gateway addresses, packet size, et cetera. Given the resources to acquire such data and some skill in analysing it (both of which any national intelligence agency should have), this can be a very powerful technique.

IPSEC is not designed to defend against this. Partial defenses are certainly possible, and some are described below, but it is not clear that any complete defense can be provided.

### IPSEC is a general mechanism for securing IP

While IPSEC does not provide all functions of a mail encryption package, it can encrypt your mail. In particular, it can ensure that all mail passing between a pair or a group of sites is encrypted. An attacker looking only at external traffic, without access to anything on or behind the IPSEC gateway, cannot read your mail. He or she is stymied by IPSEC just as he or she would be by PGP.

The advantage is that IPSEC can provide the same protection for anything transmitted over IP. In a corporate network example, PGP lets the branch offices exchange secure mail with head office. SSL and SSH allow them to securely view web pages, connect as terminals to machines, and so on. IPSEC can support all those applications, plus database queries, file sharing (NFS or Windows), other protocols encapsulated in IP (Netware, Appletalk, ...), phone-over-IP, video-over-IP, ... anything-over-IP. The only limitation is that IP Multicast is not yet supported, though there are Internet Draft documents for that.

IPSEC creates secure tunnels through untrusted networks . Sites connected by these tunnels form VPNs, Virtual Private Networks.

IPSEC gateways can be installed wherever they are required.

- One organisation might choose to install IPSEC only on firewalls between their LANs and the Internet. This would allow them to create a VPN linking several offices. It would provide protection against anyone outside their sites.
- Another might install IPSEC on departmental servers so everything on the corporate backbone net was encrypted. This would protect messages on that net from everyone except the sending and receiving department.
- Another might be less concerned with information secrecy and more with controlling access to certain resources. They might use IPSEC packet authentication as part of an access control mechanism, with or without also using the IPSEC encryption service.
- It is even possible (assuming adequate processing power and an IPSEC implementation in each node) to make every machine its own IPSEC gateway so that everything on a LAN is encrypted. This protects information from everyone outside the sending and receiving machine.
- These techniques can be combined in various ways. One might, for example, require authentication everywhere on a network while using encryption only for a few links.

Which of these, or of the many other possible variants, to use is up to you. IPSEC provides mechanisms; you provide the policy .

No end user action is required for IPSEC security to be used; they don't even have to know about it. The site administrators, of course, do have to know about it and to put some effort into making it work. Poor administration can compromise IPSEC as badly as the post-it notes mentioned above. It seems reasonable, though, for organisations to hope their system administrators are generally both more security-conscious than end users and more able to follow computer security procedures. If not, at least there are fewer of them to educate or replace.

IPSEC can be, and often should be, used with along with security protocols at other levels. If two sites communicate with each other via the Internet, then IPSEC is the obvious way to protect that communication. If two others have a direct link between them, either link encryption or IPSEC would make sense. Choose one or use both. Whatever you use at and below the IP level, use other things as required above that level. Whatever you use above the IP level, consider what can be done with IPSEC to make attacks on the higher levels harder. For example, man-in-the-middle attacks on various protocols become difficult if authentication at packet level is in use on the potential victims' communication channel.

### Using authentication without encryption

Where appropriate, IPSEC can provide authentication without encryption. One might do this, for example:

- where the data is public but one wants to be sure of getting the right data, for example on some web sites
- where encryption is judged unnecessary, for example on some company or department LANs
- where strong encryption is provided at link level, below IP
- where strong encryption is provided in other protocols, above IP
  Note that IPSEC authentication may make some attacks on those protocols harder.

Authentication has lower overheads than encryption.

The protocols provide four ways to build such connections, using either an AH-only connection or ESP using null encryption, and in either manually or automatically keyed mode. FreeS/WAN supports only one of these, manually keyed AH-only connections, and we do not recommend using that. Our reasons are discussed under Resisting traffic analysis a few sections further along.

### Encryption without authentication is dangerous

Originally, the IPSEC encryption protocol ESP didn't do integrity checking. It only did encryption. Steve Bellovin found many ways to attack ESP used without authentication. See his paper Problem areas for the IP Security Protocols. To make a secure connection, you had to add an AH Authentication Header as well as ESP. Rather than incur the overhead of several layers (and rather than provide an ESP layer that didn't actually protect the traffic), the IPSEC working group built integrity and replay checking directly into ESP.

Today, typical usage is one of:

- ESP for encryption and authentication
- AH for authentication alone

Other variants are allowed by the standard, but not much used:

ESP encryption without authentication
  Bellovin has demonstrated fatal flaws in this. Do not use.
ESP encryption with AH authentication
  This has higher overheads than using the authentication in ESP, and no obvious benefit in most cases. The exception might be a network where AH authentication was widely or universally used. If you're going to do AH to conform with network policy, why authenticate again in the ESP layer?
Authenticate twice, with AH and with ESP
  Why? Of course, some folk consider "belt and suspenders" the sensible approach to security. If you're among them, you might use both protocols here. You might also use both to satisfy different parts of a security policy. For example, an organisation might require AH authentication everywhere but two users within the organisation might use ESP as well.
ESP authentication without encryption
  The standard allows this, calling it "null encryption". FreeS/WAN does not support it. We recommend that you use AH instead if authentication is all you require. AH authenticates parts of the IP header, which ESP-null does not do.

Some of these variants cannot be used with FreeS/WAN because we do not support ESP-null and do not support automatic keying of AH-only connections.

There are fairly frequent suggestions that AH be dropped entirely from the IPSEC specifications since ESP and null encryption can handle that situation. It is not clear whether this will occur. My guess is that it is unlikely.

### Multiple layers of IPSEC processing are possible

The above describes combinations possible on a single IPSEC connection. In a complex network you may have several layers of IPSEC in play, with any of the above combinations at each layer.

For example, a connection from a desktop machine to a database server might require AH authentication. Working with other host, network and database security measures, AH might be just the thing for access control. You might decide not to use ESP encryption on such packets, since it uses resources and might complicate network debugging. Within the site where the server is, then, only AH would be used on those packets.

Users at another office, however, might have their whole connection (AH headers and all) passing over an IPSEC tunnel connecting their office to the one with the database server. Such a tunnel should use ESP encryption and authentication. You need authentication in this layer because without authentication the encryption is vulnerable and the gateway cannot verify the AH authentication. The AH is between client and database server; the gateways aren't party to it.

In this situation, some packets would get multiple layers of IPSEC applied to them, AH on an end-to-end client-to-server basis and ESP from one office's security gateway to the other.

### Resisting traffic analysis

Traffic analysis is the attempt to derive useful intelligence from encrypted traffic without breaking the encryption.

Is your CEO exchanging email with a venture capital firm? With bankruptcy trustees? With an executive recruiting agency? With the holder of some important patents? If an eavesdropper learns that, he has interesting intelligence on your company, whether or not he can read the messages themselves.

Except in the simplest cases, traffic analysis is hard to do well. It requires both considerable resources and considerable analytic skill. However, intelligence agencies of various nations have been doing it for centuries and many of them are likely quite good at it by now. Various commercial organisations, especially those working on "targeted marketing" may also be quite good at analysing certain types of traffic.

In general, defending against traffic analysis is also difficult. Inventing a really good defense could get you a PhD and some interesting job offers.

IPSEC is not designed to stop traffic analysis and we know of no plausible method of extending it to do so. That said, there are ways to make traffic analysis harder. This section describes them.

#### Using "unnecessary" encryption

One might choose to use encryption even where it appears unnecessary in order to make analysis more difficult. Consider two offices which pass a small volume of business data between them using IPSEC and also transfer large volumes of Usenet news. At first glance, it would seem silly to encrypt the newsfeed, except possibly for any newsgroups that are internal to the company. Why encrypt data that is all publicly available from many sites?

However, if we encrypt a lot of news and send it down the same connection as our business data, we make traffic analysis much harder. A snoop cannot now make inferences based on patterns in the volume, direction, sizes, sender, destination, or timing of our business messages. Those messages are hidden in a mass of news messages encapsulated in the same way.

If we're going to do this we need to ensure that keys change often enough to remain secure even with high volumes and with the adversary able to get plaintext of much of the data. We also need to look at other attacks this might open up. For example, can the adversary use a chosen plaintext attack, deliberately posting news articles which, when we receive and encrypt them, will help break our encryption? Or can he block our business data transmission by flooding us with silly news articles? Or ...

Also, note that this does not provide complete protection against traffic analysis. A clever adversary might still deduce useful intelligence from statistical analysis (perhaps comparing the input newsfeed to encrypted output, or comparing the streams we send to different branch offices), or by looking for small packets which might indicate establishment of TCP connections, or ...

As a general rule, though, one can improve resistance to traffic analysis by encrypting as much traffic as possible rather than only as much as seems necessary.

This also applies to using multiple layers of encryption. If you have an IPSEC tunnel between two branch offices, it might appear silly to send PGP-encrypted email through that tunnel. However, if you suspect someone is snooping your traffic, then it does make sense:

- it protects the mail headers; they cannot even see who is mailing who
- it protects against user bungles or software malfunctions that accidentally send messages in the clear
- it makes any attack on the mail encryption much harder; first they have to find the mail

Similar arguments apply for SSL-encrypted web traffic or SSH-encrypted remote login sessions, even for end-to-end IPSEC tunnels between systems in the two offices.

#### Using fewer tunnels

It may also help to use fewer tunnels. For example, if all you actually need encrypted is connections between:

- mail servers at branch and head offices
- a few branch office users and the head office database server

You might build one tunnel per mail server and one per remote database user, restricting traffic to those applications. This gives the traffic analyst some information, however. He or she can distinguish the tunnels by looking at information in the ESP header and, given that distinction and the patterns of tunnel usage, might be able to figure out something useful. Perhaps not, but why take the risk?

We suggest instead that you build one tunnel per branch office, encrypting everything passing from head office to branches. This has a number of advantages:

- it is easier to build and administer
- it resists traffic analysis somewhat better
- it provides security for whatever you forgot. For example, if some user at a remote office browses proprietary company data on some head office web page (that the security people may not even know about!), then that data is encrypted before it reaches the Internet.

Of course you might also want to add additional tunnels. For example, if some of the database data is confidential and should not be exposed even within the company, then you need protection from the user's desktop to the database server. We suggest you do that in whatever way seems appropriate -- IPSEC, SSH or SSL might fit -- but, whatever you choose, pass it between locations via a gateway-to-gateway IPSEC tunnel to provide some resistance to traffic analysis.

## Cryptographic components

IPSEC combines a number of cryptographic techniques, all of them well-known and well-analyzed. The overall design approach was conservative; no new or poorly-understood components were included.

This section gives a brief overview of each technique. It is intended only as an introduction. There is more information, and links to related topics, in our glossary. See also our bibliography and cryptography web links.

### Block ciphers

The encryption in the ESP encapsulation protocol is done with a block cipher.

We do not implement single DES. It is insecure. Our default, and currently only, block cipher is triple DES .

The Rijndael block cipher has just won the AES competition to choose a relacement for DES. It will almost certainly be added to FreeS/WAN and to other IPSEC implementations.

### Hash functions

#### The HMAC construct

IPSEC packet authentication is done with the HMAC construct. This is not just a hash of the packet data, but a more complex operation which uses both a hashing algorithm (MD5 or SHA) and a key. It therefore does more than a simple hash would. A simple hash would only tell you that the packet data was not changed in transit, or that whoever changed it also regenerated the hash. An HMAC also tells you that the sender knew the HMAC key.

For IPSEC HMAC, the output of the hash algorithm is truncated to 96 bits. This saves some space in the packets. More important, it prevents an attacker from seeing all the hash output bits and perhaps creating some sort of attack based on that knowledge.

### Diffie-Hellman key agreement

The Diffie-Hellman key agreement protocol allows two parties (A and B or Alice and Bob) to agree on a key in such a way that an eavesdropper who intercepts the entire conversation cannot learn the key.

The protocol is based on the discrete logarithm problem and is therefore thought to be secure. Mathematicians have been working on that problem for years and seem no closer to a solution, though there is no proof that an efficient solution is impossible.

### RSA authentication

The RSA algorithm (named for its inventors -- Rivest, Shamir and Adleman) is a very widely used public key cryptographic technique. It is used in IPSEC as one method of authenticating gateways for Diffie-Hellman key negotiation.

## Structure of IPSEC

There are three protocols used in an IPSEC implementation:

ESP, Encapsulating Security Payload
  Encrypts and/or authenticates data
AH, Authentication Header
  Provides a packet authentication service
IKE, Internet Key Exchange
  Negotiates connection parameters, including keys, for the other two

The term "IPSEC" is slightly ambiguous. In some contexts, it includes all three of the above but in other contexts it refers only to AH and ESP.

### IKE (Internet Key Exchange)

The IKE protocol sets up IPSEC (ESP or AH) connections after negotiating appropriate parameters (algorithms to be used, keys, connection lifetimes) for them. This is done by exchanging packets on UDP port 500 between the two gateways.

IKE (RFC 2409) was the outcome of a long, complex process in which quite a number of protocols were proposed and debated. Oversimplifying mildly, IKE combines:

ISAKMP (RFC 2408)
  The Internet Security A ssociation and Key Management Protocol manages negotiation of connections and defines SAs (Security Associations) as a means of describing connection properties.
IPSEC DOI for ISAKMP (RFC 2407)
  A Domain Of I nterpretation fills in the details necessary to turn the rather abstract ISAKMP protocol into a more tightly specified protocol, so it becomes applicable in a particular domain.
Oakley key determination protocol (RFC 2412)
  Oakley creates keys using the Diffie-Hellman key agreement protocol.

For all the details, you would need to read the four RFCs just mentioned (over 200 pages) and a number of others. We give a summary below, but it is far from complete.

#### Phases of IKE

IKE negotiations have two phases.

Phase one
  The two gateways negotiate and set up a two-way ISAKMP SA which they can then use to handle phase two negotiations. One such SA between a pair of gateways can handle negotiations for multiple tunnels.
Phase two
  Using the ISAKMP SA, the gateways negotiate IPSEC (ESP and/or AH) SAs as required. IPSEC SAs are unidirectional (a different key is used in each direction) and are always negotiated in pairs to handle two-way traffic. There may be more than one pair defined between two gateways.

Both of these phases use the UDP protocol and port 500 for their negotiations.

After both IKE phases are complete, you have IPSEC SAs to carry your encrypted data. These use the ESP or AH protocols. These protocols do not have ports; ports apply only to UDP or TCP.

The IKE protocol is designed to be extremely flexible. Among the things that can be negotiated (separately for each SA) are:

- SA lifetime before rekeying
- encryption algorithm used. We currently support only triple DES. Single DES is insecure. The RFCs say you MUST do DES, SHOULD do 3DES and MAY do various others. We do not do any of the others.
- authentication algorithms. We support MD5 and SHA. These are the two the RFCs require.
- choice of group for Diffie-Hellman key agreement. We currently support Groups 2 and 5 (which are defined modulo primes of various lengths) and do not support Group 1 (defined modulo a shorter prime, and therefore cryptographically weak) or groups 3 and 4 (defined using elliptic curves). The RFCs require only Group 1.

The protocol also allows implementations to add their own encryption algorithms, authentication algorithms or Diffie-Hellman groups. We do not support any such extensions.

There are a number of complications:

- The gateways must be able to authenticate each other's identities before they can create a secure connection. This host authentication is part of phase one negotiations, and is a required prerequisite for packet authentication used later. Host authentication can be done in a variety of ways. Those supported by FreeS/WAN are discussed in our configuration document.
- Phase one can be done in two ways.
  - Main Mode is required by the RFCs and supported in FreeS/WAN.
  - Aggressive Mode is somewhat faster but reveals more to an eavesdropper. This is optional in the RFCs, not currently supported by FreeS/WAN, and not likely to be.
- Phase two always uses Quick Mode, but there are two variants of that:
  - One variant provides Perfect Forward Secrecy (PFS). An attacker that obtains your long-term host authentication key does not immediately get any of your short-term packet encryption of packet authentication keys. He must conduct another successful attack each time you rekey to get the short-term keys. Having some short-term keys does not help him learn others. In particular, breaking your system today does not let him read messages he archived yestarday, assuming you've changed short-term keys in the meanwhile. We enable PFS as the default.
  - The other variant disables PFS and is therefore slightly faster. We do not recommend this since it is less secure, but FreeS/WAN does support it. You can enable it with a pfs=no statement in ipsec.conf(5).
- Several types of notification message may be sent by either side during either phase, or later. FreeS/WAN does not currently support these, but they are a likely addition in future releases.
- A new group exchange may take place after phase one but before phase two, defining up an additional group for use in the Diffie-Hellman key agreement part of phase two. FreeS/WAN does not currently support this.
- There is a commit flag which may optionally be set on some messages. The errata page for the RFCs includes two changes related to this, one to clarify the description of its use and one to block a denial of service attack which uses it. We currently do not implement this feature.

These complications can of course lead to problems, particularly when two different implementations attempt to interoperate. For example, we have seen problems such as:

- Some implememtations (often products crippled by export laws) have the insecure DES algorithm as their only supported encryption method. See our DES insecurity comments for the reasons we do not implement single DES, and our FAQ for a discussion of how to cope with crippled products.
- Windows 2000 IPSEC tries to negotiate with FreeS/WAN using Aggressive Mode, which we don't support. Later on, it uses the commit bit, which we also don't support.
- FreeS/WAN's interaction with PGPnet is complicated by their use of notification messages we do not yet support.

Despite this, we do interoperate successfully with many implementations, including both Windows 2000 and PGPnet. Details are in our interoperability document.

#### Structure of IKE messages

Here is our Pluto developer explaining some of this on the mailing list:

    When one IKE system (for example, Pluto) is negotiating with another
    to create an SA, the Initiator proposes a bunch of choices and the
    Responder replies with one that it has selected.

    The structure of the choices is fairly complicated.  An SA payload
    contains a list of lists of "Proposals".  The outer list is a set of
    choices: the selection must be from one element of this list.

    Each of these elements is a list of Proposals.  A selection must be
    made from each of the elements of the inner list.  In other words,
    *all* of them apply (that is how, for example, both AH and ESP can
    apply at once).

    Within each of these Proposals is a list of Transforms.  For each
    Proposal selected, one Transform must be selected (in other words,
    each Proposal provides a choice of Transforms).

    Each Transform is made up of a list of Attributes describing, well,
    attributes.  Such as lifetime of the SA.  Such as algorithm to be
    used.  All the Attributes apply to a Transform.

    You will have noticed a pattern here: layers alternate between being
    disjunctions ("or") and conjunctions ("and").

    For Phase 1 / Main Mode (negotiating an ISAKMP SA), this structure is
    cut back.  There must be exactly one Proposal.  So this degenerates to
    a list of Transforms, one of which must be chosen.

### IPSEC Services, AH and ESP

IPSEC offers two services, authentication and encryption . These can be used separately but are often used together.

Authentication
  Packet-level authentication allows you to be confident that a packet came from a particular machine and that its contents were not altered en route to you. No attempt is made to conceal or protect the contents, only to assure their integrity.

Packet authentication can be provided separately using an Authentication Header, described just below, or it can be included as part of the ESP (Encapsulated Security Payload) service, described in the following section. That service offers encryption as well as authentication.

Encryption
  Encryption allows you to conceal the contents of a message from eavesdroppers.

In IPSEC this is done using a block cipher (normally Triple DES for Linux). In the most used setup, keys are automatically negotiated, and periodically re-negotiated, using the IKE (Internet Key Exchange) protocol. In Linux FreeS/WAN this is handled by the Pluto Daemon.

The IPSEC protocol offering encryption is ESP, Encapsulated Security Payload. It can also include a packet authentication service.

Note that encryption should always be used with some packet authentication service. Unauthenticated encryption is vulnerable to man-in-the-middle attacks . Also note that encryption does not necessarily prevent traffic analysis.

### The Authentication Header (AH)

Packet authentication can be provided separately from encryption by adding an authentication header (AH) after the IP header but before the other headers on the packet. This is the subject of this section. Details are in RFC 2402.

Each of the several headers on a packet header contains a "next protocol" field telling the system what header to look for next. IP headers generally have either TCP or UDP in this field. When IPSEC authentication is used, the packet IP header has AH in this field, saying that an Authentication Header comes next. The AH header then has the next header type -- usually TCP, UDP or encapsulated IP.

IPSEC packet authentication can be added in transport mode, as a modification of standard IP transport. This is shown in this diagram from the RFC:

                      BEFORE APPLYING AH
                ----------------------------
          IPv4  |orig IP hdr  |     |      |
                |(any options)| TCP | Data |
                ----------------------------

                      AFTER APPLYING AH
                ---------------------------------
          IPv4  |orig IP hdr  |    |     |      |
                |(any options)| AH | TCP | Data |
                ---------------------------------
                ||
                     except for mutable fields

Athentication can also be used in tunnel mode, encapsulating the underlying IP packet beneath AH and an additional IP header.

                             ||
    IPv4  | new IP hdr* |    | orig IP hdr*  |    |      |
          |(any options)| AH | (any options) |TCP | Data |
          ------------------------------------------------
          ||
          |           in the new IP hdr                  |

This would normally be used in a gateway-to-gateway tunnel. The receiving gateway then strips the outer IP header and the AH header and forwards the inner IP packet.

The mutable fields referred to are things like the time-to-live field in the IP header. These cannot be included in authentication calculations because they change as the packet travels.

#### Keyed MD5 and Keyed SHA

The actual authentication data in the header is typically 96 bits and depends both on a secret shared between sender and receiver and on every byte of the data being authenticated.

The algorithms involved are the MD5 Message Digest Algorithm or SHA, the Secure Hash Algorithm. For details on their use in this application, see RFCs 2403 and 2404 respectively.

For descriptions of the algorithms themselves, see RFC 1321 for MD5 and FIPS (Federal Information Processing Standard) number 186 from NIST , the US National Institute of Standards and Technology for SHA. Applied Cryptography covers both in some detail, MD5 starting on page 436 and SHA on 442.

These algorithms are intended to make it nearly impossible for anyone to alter the authenticated data in transit. The sender calculates a digest or hash value from that data and includes the result in the authentication header. The recipient does the same calculation and compares results. For unchanged data, the results will be identical. The hash algorithms are designed to make it extremely difficult to change the data in any way and still get the correct hash.

Since the shared secret key is also used in both calculations, an interceptor cannot simply alter the authenticated data and change the hash value to match. Without the key, he or she (or even the dreaded They) cannot produce a usable hash.

#### Sequence numbers

The authentication header includes a sequence number field which the sender is required to increment for each packet. The receiver can ignore it or use it to check that packets are indeed arriving in the expected sequence.

This provides partial protection against replay attacks in which an attacker resends intercepted packets in an effort to confuse or subvert the receiver. Complete protection is not possible since it is necessary to handle legitmate packets which are lost, duplicated, or delivered out of order, but use of sequence numbers makes the attack much more difficult.

The RFCs require that sequence numbers never cycle, that a new key always be negotiated before the sequence number reaches 2^32-1. This protects both against replays attacks using packets from a previous cyclce and against birthday attacks on the the packet authentication algorithm.

In Linux FreeS/WAN, the sequence number is ignored for manually keyed connections and checked for automatically keyed ones. In automatic mode, we do that. In manual mode, there is no way to negotiate a new key, or to recover from a sequence number problem, so we don't use sequence numbers.

### Encapsulated Security Payload (ESP)

The ESP protocol is defined in RFC 2406. It provides one or both of encryption and packet authentication. It may be used with or without AH packet authentication.

Note that some form of packet authentication should always be used whenever data is encrypted. Without authentication, the encryption is vulnerable to active attacks which may allow an enemy to break the encryption. ESP should always either include its own authentication or be used with AH authentication.

The RFCs require support for only two mandatory encryption algorithms -- DES, and null encryption -- and for two authentication methods -- keyed MD5 and keyed SHA. Implementers may choose to support additional algorithms in either category.

The authentication algorithms are the same ones used in the IPSEC authentication header.

We do not implement single DES since DES is insecure. Instead we provide triple DES or 3DES. This is currently the only encryption algorithm supported.

We do not implement null encryption since it is obviously insecure.

## IPSEC modes

IPSEC can connect in two modes. Transport mode is a host-to-host connection involving only two machines. In tunnel mode, the IPSEC machines act as gateways and trafiic for any number of client machines may be carried.

### Tunnel mode

Security gateways are required to support tunnel mode connections. In this mode the gateways provide tunnels for use by client machines behind the gateways. The client machines need not do any IPSEC processing; all they have to do is route things to gateways.

### Transport mode

Host machines (as opposed to security gateways) with IPSEC implementations must also support transport mode. In this mode, the host does its own IPSEC processing and routes some packets via IPSEC.

## FreeS/WAN parts

### KLIPS: Kernel IPSEC Support

KLIPS is KerneL IP SEC Support, the modifications necessary to support IPSEC within the Linux kernel. KILPS does all the actual IPSEC packet-handling, including

- encryption
- packet authentication calculations
- creation of ESP and AH headers for outgoing packets
- interpretation of those headers on incoming packets

KLIPS also checks all non-IPSEC packets to ensure they are not bypassing IPSEC security policies.

### The Pluto daemon

Pluto(8) is a daemon which implements the IKE protocol. It

- handles all the Phase one ISAKMP SAs
- performs host authentication and negotiates with other gateways
- creates IPSEC SAs and passes the data required to run them to KLIPS
- adjust routing and firewall setup to meet IPSEC requirements. See our IPSEC and firewalling document for details.

Pluto is controlled mainly by the ipsec.conf(5) configuration file.

### The ipsec(8) command

The ipsec(8) command is a front end that allows control over IPSEC activity.

### Linux FreeS/WAN configuration file

The configuration file for Linux FreeS/WAN is

            /etc/ipsec.conf

For details see the ipsec.conf(5) manual page and our Configuration section.

## Key management

There are several ways IPSEC can manage keys. Not all are implemented in Linux FreeS/WAN.

### Currently Implemented Methods

#### Manual keying

IPSEC allows keys to be manually set. In Linux FreeS/WAN, such keys are stored with the connection definitions in /etc/ipsec.conf.

Manual keying is useful for debugging since it allows you to test the KLIPS kernel IPSEC code without the Pluto daemon doing key negotiation.

In general, however, automatic keying is preferred because it is more secure.

#### Automatic keying

In automatic keying, the Pluto daemon negotiates keys using the IKE Internet Key Exchange protocol. Connections are automatically re-keyed periodically.

This is considerably more secure than manual keying. In either case an attacker who acquires a key can read every message encrypted with that key, but automatic keys can be changed every few hours or even every few minutes without breaking the connection or requiring intervention by the system administrators. Manual keys can only be changed manually; you need to shut down the connection and have the two admins make changes. Moreover, they have to communicate the new keys securely, perhaps with PGP or SSH. This may be possible in some cases, but as a general solution it is expensive, bothersome and unreliable. Far better to let Pluto handle these chores; no doubt the administrators have enough to do.

Also, automatic keying is inherently more secure against an attacker who manages to subvert your gateway system. If manual keying is in use and an adversary acquires root privilege on your gateway, he reads your keys from /etc/ipsec.conf and then reads all messages encrypted with those keys.

If automatic keying is used, an adversary with the same privileges can read /etc/ipsec.secrets, but this does not contain any keys, only the secrets used to authenticate key exchanges. Having an adversary able to authenticate your key exchanges need not worry you overmuch. Just having the secrets does not give him any keys. You are still secure against passive attacks. This property of automatic keying is called perfect forward secrecy, abbreviated PFS.

Unfortunately, having the secrets does allow an active attack, specifically a man-in-the-middle attack. Losing these secrets to an attacker may not be quite as disastrous as losing the actual keys, but it is still a serious security breach. These secrets should be guarded as carefully as keys.

### Methods not yet implemented

#### Unauthenticated key exchange

It would be possible to exchange keys without authenticating the players. This would support opportunistic encryption -- allowing any two systems to encrypt their communications without requiring a shared PKI or a previously negotiated secret -- and would be secure against passive attacks. It would, however, be highly vulnerable to active man-in-the-middle attacks. RFC 2408 therefore specifies that all ISAKMP key management interactions must be authenticated.

There is room for debate here. Should we provide immediate security against passive attacks and encourage widespread use of encryption, at the expense of risking the more difficult active attacks? Or should we wait until we can implement a solution that can both be widespread and offer security against active attacks?

So far, we have chosen the second course, complying with the RFCs and waiting for secure DNS (see below) so that we can do opportunistic encryption right.

#### Key exchange using DNS

The IPSEC RFCs allow key exchange based on authentication services provided by Secure DNS. Once Secure DNS service becomes widely available, we expect to make this the primary key management method for Linux FreeS/WAN. It is the best way we know of to support opportunistic encryption, allowing two systems without a common PKI or previous negotiation to secure their communication.

As of FreeS/WAN 1.4, we have experimental code to acquire RSA keys from DNS but do not yet have code to validate Secure DNS signatures.

#### Key exchange using a PKI

The IPSEC RFCs allow key exchange based on authentication services provided by a PKI or Public Key Infrastructure. With many vendors selling such products and many large organisations building these infrastructures, this will clearly be an important application of IPSEC and one Linux FreeS/WAN will eventually support.

On the other hand, this is not as high a priority for Linux FreeS/WAN as solutions based on secure DNS . We do not expect any PKI to become as universal as DNS.

Some patches to handle authentication with X.509 certificates, which most PKIs use, are available.

#### Photuris

Photuris is another key management protocol, an alternative to IKE and ISAKMP, described in RFCs 2522 and 2523 which are labelled "experimental". Adding Photuris support to Linux FreeS/WAN might be a good project for a volunteer. The likely starting point would be the OpenBSD photurisd code.

#### SKIP

SKIP is yet another key management protocol, developed by Sun. At one point it was fairly widely used, but our current impression is that it is moribund, displaced by IKE. Sun now (as of Solaris 8.0) ship an IPSEC implementation using IKE. We have no plans to implement SKIP.

Contents Previous Next