hacks and glory await
This commit is contained in:
88
RFC/rfc-index-latest.txt
Normal file
88
RFC/rfc-index-latest.txt
Normal file
@@ -0,0 +1,88 @@
|
||||
|
||||
===========================================================================
|
||||
This is a list of the latest RFCs only.
|
||||
To get the full list of RFCs, please look up rfc-index.txt
|
||||
===========================================================================
|
||||
|
||||
8197 A SIP Response Code for Unwanted Calls. H. Schulzrinne. July 2017.
|
||||
(Format: TXT=19114 bytes) (Status: PROPOSED STANDARD) (DOI:
|
||||
10.17487/RFC8197)
|
||||
|
||||
8198 Aggressive Use of DNSSEC-Validated Cache. K. Fujiwara, A. Kato, W.
|
||||
Kumari. July 2017. (Format: TXT=27918 bytes) (Updates RFC4035)
|
||||
(Status: PROPOSED STANDARD) (DOI: 10.17487/RFC8198)
|
||||
|
||||
8199 YANG Module Classification. D. Bogdanovic, B. Claise, C. Moberg.
|
||||
July 2017. (Format: TXT=23080 bytes) (Status: INFORMATIONAL) (DOI:
|
||||
10.17487/RFC8199)
|
||||
|
||||
8200 Internet Protocol, Version 6 (IPv6) Specification. S. Deering, R.
|
||||
Hinden. July 2017. (Format: TXT=93658 bytes) (Obsoletes RFC2460)
|
||||
(Also STD0086) (Status: INTERNET STANDARD) (DOI: 10.17487/RFC8200)
|
||||
|
||||
8201 Path MTU Discovery for IP version 6. J. McCann, S. Deering, J.
|
||||
Mogul, R. Hinden, Ed.. July 2017. (Format: TXT=42751 bytes)
|
||||
(Obsoletes RFC1981) (Also STD0087) (Status: INTERNET STANDARD) (DOI:
|
||||
10.17487/RFC8201)
|
||||
|
||||
8202 IS-IS Multi-Instance. L. Ginsberg, S. Previdi, W. Henderickx. June
|
||||
2017. (Format: TXT=35114 bytes) (Obsoletes RFC6822) (Status:
|
||||
PROPOSED STANDARD) (DOI: 10.17487/RFC8202)
|
||||
|
||||
8203 BGP Administrative Shutdown Communication. J. Snijders, J. Heitz, J.
|
||||
Scudder. July 2017. (Format: TXT=12532 bytes) (Updates RFC4486)
|
||||
(Status: PROPOSED STANDARD) (DOI: 10.17487/RFC8203)
|
||||
|
||||
8212 Default External BGP (EBGP) Route Propagation Behavior without
|
||||
Policies. J. Mauch, J. Snijders, G. Hankins. July 2017. (Format:
|
||||
TXT=12552 bytes) (Updates RFC4271) (Status: PROPOSED STANDARD) (DOI:
|
||||
10.17487/RFC8212)
|
||||
|
||||
8213 Security of Messages Exchanged between Servers and Relay Agents. B.
|
||||
Volz, Y. Pal. August 2017. (Format: TXT=17657 bytes) (Status:
|
||||
PROPOSED STANDARD) (DOI: 10.17487/RFC8213)
|
||||
|
||||
8214 Virtual Private Wire Service Support in Ethernet VPN. S. Boutros, A.
|
||||
Sajassi, S. Salam, J. Drake, J. Rabadan. August 2017. (Format:
|
||||
TXT=34563 bytes) (Status: PROPOSED STANDARD) (DOI: 10.17487/RFC8214)
|
||||
|
||||
8215 Local-Use IPv4/IPv6 Translation Prefix. T. Anderson. August 2017.
|
||||
(Format: TXT=14846 bytes) (Status: PROPOSED STANDARD) (DOI:
|
||||
10.17487/RFC8215)
|
||||
|
||||
8217 Clarifications for When to Use the name-addr Production in SIP
|
||||
Messages. R. Sparks. August 2017. (Format: TXT=12829 bytes) (Updates
|
||||
RFC3261, RFC3325, RFC3515, RFC3892, RFC4508, RFC5002, RFC5318,
|
||||
RFC5360, RFC5502) (Status: PROPOSED STANDARD) (DOI:
|
||||
10.17487/RFC8217)
|
||||
|
||||
8218 Multipath Extension for the Optimized Link State Routing Protocol
|
||||
Version 2 (OLSRv2). J. Yi, B. Parrein. August 2017. (Format:
|
||||
TXT=56286 bytes) (Status: EXPERIMENTAL) (DOI: 10.17487/RFC8218)
|
||||
|
||||
8219 Benchmarking Methodology for IPv6 Transition Technologies. M.
|
||||
Georgescu, L. Pislaru, G. Lencse. August 2017. (Format: TXT=66085
|
||||
bytes) (Status: INFORMATIONAL) (DOI: 10.17487/RFC8219)
|
||||
|
||||
8227 MPLS-TP Shared-Ring Protection (MSRP) Mechanism for Ring Topology.
|
||||
W. Cheng, L. Wang, H. Li, H. van Helvoort, J. Dong. August 2017.
|
||||
(Format: TXT=128880 bytes) (Status: PROPOSED STANDARD) (DOI:
|
||||
10.17487/RFC8227)
|
||||
|
||||
8228 Guidance on Designing Label Generation Rulesets (LGRs) Supporting
|
||||
Variant Labels. A. Freytag. August 2017. (Format: TXT=50900 bytes)
|
||||
(Status: INFORMATIONAL) (DOI: 10.17487/RFC8228)
|
||||
|
||||
8229 TCP Encapsulation of IKE and IPsec Packets. T. Pauly, S. Touati, R.
|
||||
Mantha. August 2017. (Format: TXT=56294 bytes) (Status: PROPOSED
|
||||
STANDARD) (DOI: 10.17487/RFC8229)
|
||||
|
||||
8234 Updates to MPLS Transport Profile (MPLS-TP) Linear Protection in
|
||||
Automatic Protection Switching (APS) Mode. J. Ryoo, T. Cheung, H.
|
||||
van Helvoort, I. Busi, G. Wen. August 2017. (Format: TXT=16898
|
||||
bytes) (Updates RFC7271) (Status: PROPOSED STANDARD) (DOI:
|
||||
10.17487/RFC8234)
|
||||
|
||||
===========================================================================
|
||||
- This file last updated Wed Aug 30 02:30:01 PDT 2017
|
||||
|
||||
451
RFC/rfc1701.txt
Normal file
451
RFC/rfc1701.txt
Normal file
@@ -0,0 +1,451 @@
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Network Working Group S. Hanks
|
||||
Request for Comments: 1701 NetSmiths, Ltd.
|
||||
Category: Informational T. Li
|
||||
D. Farinacci
|
||||
P. Traina
|
||||
cisco Systems
|
||||
October 1994
|
||||
|
||||
|
||||
Generic Routing Encapsulation (GRE)
|
||||
|
||||
Status of this Memo
|
||||
|
||||
|
||||
This memo provides information for the Internet community. This memo
|
||||
does not specify an Internet standard of any kind. Distribution of
|
||||
this memo is unlimited.
|
||||
|
||||
Abstract
|
||||
|
||||
This document specifies a protocol for performing encapsulation of an
|
||||
arbitrary network layer protocol over another arbitrary network layer
|
||||
protocol.
|
||||
|
||||
Introduction
|
||||
|
||||
A number of different proposals [RFC 1234, RFC 1226] currently exist
|
||||
for the encapsulation of one protocol over another protocol. Other
|
||||
types of encapsulations [RFC 1241, SDRP, RFC 1479] have been proposed
|
||||
for transporting IP over IP for policy purposes. This memo describes
|
||||
a protocol which is very similar to, but is more general than, the
|
||||
above proposals. In attempting to be more general, many protocol
|
||||
specific nuances have been ignored. The result is that this proposal
|
||||
is may be less suitable for a situation where a specific "X over Y"
|
||||
encapsulation has been described. It is the attempt of this protocol
|
||||
to provide a simple, general purpose mechanism which is reduces the
|
||||
problem of encapsulation from its current O(n^2) problem to a more
|
||||
manageable state. This proposal also attempts to provide a
|
||||
lightweight encapsulation for use in policy based routing. This memo
|
||||
explicitly does not address the issue of when a packet should be
|
||||
encapsulated. This memo acknowledges, but does not address problems
|
||||
with mutual encapsulation [RFC 1326].
|
||||
|
||||
In the most general case, a system has a packet that needs to be
|
||||
encapsulated and routed. We will call this the payload packet. The
|
||||
payload is first encapsulated in a GRE packet, which possibly also
|
||||
includes a route. The resulting GRE packet can then be encapsulated
|
||||
in some other protocol and then forwarded. We will call this outer
|
||||
|
||||
|
||||
|
||||
Hanks, Li, Farinacci & Traina [Page 1]
|
||||
|
||||
RFC 1701 Generic Routing Encapsulation (GRE) October 1994
|
||||
|
||||
|
||||
protocol the delivery protocol. The algorithms for processing this
|
||||
packet are discussed later.
|
||||
|
||||
Overall packet
|
||||
|
||||
The entire encapsulated packet would then have the form:
|
||||
|
||||
---------------------------------
|
||||
| |
|
||||
| Delivery Header |
|
||||
| |
|
||||
---------------------------------
|
||||
| |
|
||||
| GRE Header |
|
||||
| |
|
||||
---------------------------------
|
||||
| |
|
||||
| Payload packet |
|
||||
| |
|
||||
---------------------------------
|
||||
|
||||
Packet header
|
||||
|
||||
The GRE packet header has the form:
|
||||
|
||||
0 1 2 3
|
||||
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
|
||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
||||
|C|R|K|S|s|Recur| Flags | Ver | Protocol Type |
|
||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
||||
| Checksum (optional) | Offset (optional) |
|
||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
||||
| Key (optional) |
|
||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
||||
| Sequence Number (optional) |
|
||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
||||
| Routing (optional)
|
||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
||||
|
||||
Flags and version (2 octets)
|
||||
|
||||
The GRE flags are encoded in the first two octets. Bit 0 is the
|
||||
most significant bit, bit 15 is the least significant bit. Bits
|
||||
13 through 15 are reserved for the Version field. Bits 5 through
|
||||
12 are reserved for future use and MUST be transmitted as zero.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Hanks, Li, Farinacci & Traina [Page 2]
|
||||
|
||||
RFC 1701 Generic Routing Encapsulation (GRE) October 1994
|
||||
|
||||
|
||||
Checksum Present (bit 0)
|
||||
|
||||
If the Checksum Present bit is set to 1, then the Checksum field
|
||||
is present and contains valid information.
|
||||
|
||||
If either the Checksum Present bit or the Routing Present bit are
|
||||
set, BOTH the Checksum and Offset fields are present in the GRE
|
||||
packet.
|
||||
|
||||
Routing Present (bit 1)
|
||||
|
||||
If the Routing Present bit is set to 1, then it indicates that the
|
||||
Offset and Routing fields are present and contain valid
|
||||
information.
|
||||
|
||||
If either the Checksum Present bit or the Routing Present bit are
|
||||
set, BOTH the Checksum and Offset fields are present in the GRE
|
||||
packet.
|
||||
|
||||
Key Present (bit 2)
|
||||
|
||||
If the Key Present bit is set to 1, then it indicates that the Key
|
||||
field is present in the GRE header. Otherwise, the Key field is
|
||||
not present in the GRE header.
|
||||
|
||||
Sequence Number Present (bit 3)
|
||||
|
||||
If the Sequence Number Present bit is set to 1, then it indicates
|
||||
that the Sequence Number field is present. Otherwise, the
|
||||
Sequence Number field is not present in the GRE header.
|
||||
|
||||
Strict Source Route (bit 4)
|
||||
|
||||
The meaning of the Strict Source route bit is defined in other
|
||||
documents. It is recommended that this bit only be set to 1 if
|
||||
all of the the Routing Information consists of Strict Source
|
||||
Routes.
|
||||
|
||||
Recursion Control (bits 5-7)
|
||||
|
||||
Recursion control contains a three bit unsigned integer which
|
||||
contains the number of additional encapsulations which are
|
||||
permissible. This SHOULD default to zero.
|
||||
|
||||
Version Number (bits 13-15)
|
||||
|
||||
The Version Number field MUST contain the value 0. Other values
|
||||
are outside of the scope of this document.
|
||||
|
||||
|
||||
|
||||
Hanks, Li, Farinacci & Traina [Page 3]
|
||||
|
||||
RFC 1701 Generic Routing Encapsulation (GRE) October 1994
|
||||
|
||||
|
||||
Protocol Type (2 octets)
|
||||
|
||||
The Protocol Type field contains the protocol type of the payload
|
||||
packet. In general, the value will be the Ethernet protocol type
|
||||
field for the packet. Currently defined protocol types are listed
|
||||
below. Additional values may be defined in other documents.
|
||||
|
||||
Offset (2 octets)
|
||||
|
||||
The offset field indicates the octet offset from the start of the
|
||||
Routing field to the first octet of the active Source Route Entry
|
||||
to be examined. This field is present if the Routing Present or
|
||||
the Checksum Present bit is set to 1, and contains valid
|
||||
information only if the Routing Present bit is set to 1.
|
||||
|
||||
Checksum (2 octets)
|
||||
|
||||
The Checksum field contains the IP (one's complement) checksum of
|
||||
the GRE header and the payload packet. This field is present if
|
||||
the Routing Present or the Checksum Present bit is set to 1, and
|
||||
contains valid information only if the Checksum Present bit is set
|
||||
to 1.
|
||||
|
||||
Key (4 octets)
|
||||
|
||||
The Key field contains a four octet number which was inserted by
|
||||
the encapsulator. It may be used by the receiver to authenticate
|
||||
the source of the packet. The techniques for determining
|
||||
authenticity are outside of the scope of this document. The Key
|
||||
field is only present if the Key Present field is set to 1.
|
||||
|
||||
Sequence Number (4 octets)
|
||||
|
||||
The Sequence Number field contains an unsigned 32 bit integer
|
||||
which is inserted by the encapsulator. It may be used by the
|
||||
receiver to establish the order in which packets have been
|
||||
transmitted from the encapsulator to the receiver. The exact
|
||||
algorithms for the generation of the Sequence Number and the
|
||||
semantics of their reception is outside of the scope of this
|
||||
document.
|
||||
|
||||
Routing (variable)
|
||||
|
||||
The Routing field is optional and is present only if the Routing
|
||||
Present bit is set to 1.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Hanks, Li, Farinacci & Traina [Page 4]
|
||||
|
||||
RFC 1701 Generic Routing Encapsulation (GRE) October 1994
|
||||
|
||||
|
||||
The Routing field is a list of Source Route Entries (SREs). Each
|
||||
SRE has the form:
|
||||
|
||||
0 1 2 3
|
||||
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
|
||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
||||
| Address Family | SRE Offset | SRE Length |
|
||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
||||
| Routing Information ...
|
||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
||||
|
||||
The routing field is terminated with a "NULL" SRE containing an
|
||||
address family of type 0x0000 and a length of 0.
|
||||
|
||||
Address Family (2 octets)
|
||||
|
||||
The Address Family field contains a two octet value which indicates
|
||||
the syntax and semantics of the Routing Information field. The
|
||||
values for this field and the corresponding syntax and semantics for
|
||||
Routing Information are defined in other documents.
|
||||
|
||||
SRE Offset (1 octet)
|
||||
|
||||
The SRE Offset field indicates the octet offset from the start of the
|
||||
Routing Information field to the first octet of the active entry in
|
||||
Source Route Entry to be examined.
|
||||
|
||||
SRE Length (1 octet)
|
||||
|
||||
The SRE Length field contains the number of octets in the SRE. If
|
||||
the SRE Length is 0, this indicates this is the last SRE in the
|
||||
Routing field.
|
||||
|
||||
Routing Information (variable)
|
||||
|
||||
The Routing Information field contains data which may be used in
|
||||
routing this packet. The exact semantics of this field is defined in
|
||||
other documents.
|
||||
|
||||
Forwarding of GRE packets
|
||||
|
||||
Normally, a system which is forwarding delivery layer packets will
|
||||
not differentiate GRE packets from other packets in any way.
|
||||
However, a GRE packet may be received by a system. In this case, the
|
||||
system should use some delivery-specific means to determine that this
|
||||
is a GRE packet. Once this is determined, the Key, Sequence Number
|
||||
and Checksum fields if they contain valid information as indicated by
|
||||
the corresponding flags may be checked. If the Routing Present bit
|
||||
|
||||
|
||||
|
||||
Hanks, Li, Farinacci & Traina [Page 5]
|
||||
|
||||
RFC 1701 Generic Routing Encapsulation (GRE) October 1994
|
||||
|
||||
|
||||
is set to 1, then the Address Family field should be checked to
|
||||
determine the semantics and use of the SRE Length, SRE Offset and
|
||||
Routing Information fields. The exact semantics for processing a SRE
|
||||
for each Address Family is defined in other documents.
|
||||
|
||||
Once all SREs have been processed, then the source route is complete,
|
||||
the GRE header should be removed, the payload's TTL MUST be
|
||||
decremented (if one exists) and the payload packet should be
|
||||
forwarded as a normal packet. The exact forwarding method depends on
|
||||
the Protocol Type field.
|
||||
|
||||
Current List of Protocol Types
|
||||
|
||||
The following are currently assigned protocol types for GRE. Future
|
||||
protocol types must be taken from DIX ethernet encoding. For
|
||||
historical reasons, a number of other values have been used for some
|
||||
protocols. The following table of values MUST be used to identify
|
||||
the following protocols:
|
||||
|
||||
Protocol Family PTYPE
|
||||
--------------- -----
|
||||
Reserved 0000
|
||||
SNA 0004
|
||||
OSI network layer 00FE
|
||||
PUP 0200
|
||||
XNS 0600
|
||||
IP 0800
|
||||
Chaos 0804
|
||||
RFC 826 ARP 0806
|
||||
Frame Relay ARP 0808
|
||||
VINES 0BAD
|
||||
VINES Echo 0BAE
|
||||
VINES Loopback 0BAF
|
||||
DECnet (Phase IV) 6003
|
||||
Transparent Ethernet Bridging 6558
|
||||
Raw Frame Relay 6559
|
||||
Apollo Domain 8019
|
||||
Ethertalk (Appletalk) 809B
|
||||
Novell IPX 8137
|
||||
RFC 1144 TCP/IP compression 876B
|
||||
IP Autonomous Systems 876C
|
||||
Secure Data 876D
|
||||
Reserved FFFF
|
||||
|
||||
See the IANA list of Ether Types for the complete list of these
|
||||
values.
|
||||
|
||||
URL = ftp://ftp.isi.edu/in-notes/iana/assignments/ethernet-numbers.
|
||||
|
||||
|
||||
|
||||
Hanks, Li, Farinacci & Traina [Page 6]
|
||||
|
||||
RFC 1701 Generic Routing Encapsulation (GRE) October 1994
|
||||
|
||||
|
||||
References
|
||||
|
||||
RFC 1479
|
||||
Steenstrup, M. "Inter-Domain Policy Routing Protocol
|
||||
Specification: Version 1", RFC1479, BBN Systems and Technologies,
|
||||
July 1993.
|
||||
|
||||
RFC 1226
|
||||
Kantor, B. "Internet Protocol Encapsulation of AX.25 Frames", RFC
|
||||
1226, University of California, San Diego, May 1991.
|
||||
|
||||
RFC 1234
|
||||
Provan, D. "Tunneling IPX Traffic through IP Networks", RFC 1234,
|
||||
Novell, Inc., June 1991.
|
||||
|
||||
RFC 1241
|
||||
Woodburn, R., and D. Mills, "Scheme for an Internet Encapsulation
|
||||
Protocol: Version 1", RFC 1241, SAIC, University of Delaware, July
|
||||
1991.
|
||||
|
||||
RFC 1326
|
||||
Tsuchiya, P., "Mutual Encapsulation Considered Dangerous", RFC
|
||||
1326, Bellcore, May 1992.
|
||||
|
||||
SDRP
|
||||
Estrin, D., Li, T., and Y. Rekhter, "Source Demand Routing
|
||||
Protocol Specification (Version 1)", Work in Progress.
|
||||
|
||||
RFC 1702
|
||||
Hanks, S., Li, T., Farinacci, D., and P. Traina, "Generic Routing
|
||||
Encapsulation over IPv4 networks", RFC 1702, NetSmiths, Ltd.,
|
||||
cisco Systems, October 1994.
|
||||
|
||||
Security Considerations
|
||||
|
||||
Security issues are not discussed in this memo.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Hanks, Li, Farinacci & Traina [Page 7]
|
||||
|
||||
RFC 1701 Generic Routing Encapsulation (GRE) October 1994
|
||||
|
||||
|
||||
Acknowledgements
|
||||
|
||||
The authors would like to acknowledge Yakov Rekhter (IBM) and Deborah
|
||||
Estrin (USC) for their advice, encouragement and insightful comments.
|
||||
|
||||
Authors' Addresses
|
||||
|
||||
Stan Hanks
|
||||
NetSmiths, Ltd.
|
||||
2025 Lincoln Highway
|
||||
Edison NJ, 08817
|
||||
|
||||
EMail: stan@netsmiths.com
|
||||
|
||||
|
||||
Tony Li
|
||||
cisco Systems, Inc.
|
||||
1525 O'Brien Drive
|
||||
Menlo Park, CA 94025
|
||||
|
||||
EMail: tli@cisco.com
|
||||
|
||||
|
||||
Dino Farinacci
|
||||
cisco Systems, Inc.
|
||||
1525 O'Brien Drive
|
||||
Menlo Park, CA 94025
|
||||
|
||||
EMail: dino@cisco.com
|
||||
|
||||
|
||||
Paul Traina
|
||||
cisco Systems, Inc.
|
||||
1525 O'Brien Drive
|
||||
Menlo Park, CA 94025
|
||||
|
||||
EMail: pst@cisco.com
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Hanks, Li, Farinacci & Traina [Page 8]
|
||||
|
||||
227
RFC/rfc1702.txt
Normal file
227
RFC/rfc1702.txt
Normal file
@@ -0,0 +1,227 @@
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Network Working Group S. Hanks
|
||||
Request for Comments: 1702 NetSmiths, Ltd.
|
||||
Category: Informational T. Li
|
||||
D. Farinacci
|
||||
P. Traina
|
||||
cisco Systems
|
||||
October 1994
|
||||
|
||||
|
||||
Generic Routing Encapsulation over IPv4 networks
|
||||
|
||||
Status of this Memo
|
||||
|
||||
This memo provides information for the Internet community. This memo
|
||||
does not specify an Internet standard of any kind. Distribution of
|
||||
this memo is unlimited.
|
||||
|
||||
Introduction
|
||||
|
||||
In an earlier memo [RFC 1701], we described GRE, a mechanism for
|
||||
encapsulating arbitrary packets within an arbitrary transport
|
||||
protocol. This is a companion memo which describes the use of GRE
|
||||
with IP. This memo addresses the case of using IP as the delivery
|
||||
protocol or the payload protocol and the special case of IP as both
|
||||
the delivery and payload. This memo also describes using IP
|
||||
addresses and autonomous system numbers as part of a GRE source
|
||||
route.
|
||||
|
||||
IP as a delivery protocol
|
||||
|
||||
GRE packets which are encapsulated within IP will use IP protocol
|
||||
type 47.
|
||||
|
||||
IP as a payload protocol
|
||||
|
||||
IP packets will be encapsulated with a Protocol Type field of 0x800.
|
||||
|
||||
For the Address Family value of 0x800, the Routing Information field
|
||||
will consist of a list of IP addresses and indicates an IP source
|
||||
route. The first octet of the Routing Information field constitute a
|
||||
8 bit integer offset from the start of the Source Route Entry (SRE),
|
||||
called the SRE Offset. The SRE Offset indicates the first octet of
|
||||
the next IP address. The SRE Length field consists of the total
|
||||
length of the IP Address List in octets.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Hanks, Li, Farinacci & Traina [Page 1]
|
||||
|
||||
RFC 1702 GRE over IPv4 networks October 1994
|
||||
|
||||
|
||||
This has the form:
|
||||
|
||||
0 1 2 3
|
||||
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
|
||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
||||
| Address Family | SRE Offset | SRE Length |
|
||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
||||
| IP Address List ...
|
||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
||||
|
||||
For the Address Family value of 0xfffe, the Routing Information field
|
||||
will consist of a list of Autonomous System numbers and indicates an
|
||||
AS source route. The third octet of the Routing Information field
|
||||
contains an 8 bit unsigned integer offset from the start of the
|
||||
Source Route Entry (SRE), called the SRE Offset. The SRE Offset
|
||||
indicates the first octet of the next AS number. THe SRE Length
|
||||
field consists of the total length of the AS Number list in octets.
|
||||
|
||||
0 1 2 3
|
||||
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
|
||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
||||
| Address Family | SRE Offset | SRE Length |
|
||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
||||
| AS Number List ...
|
||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
||||
|
||||
IP as both delivery and payload protocol
|
||||
|
||||
When IP is encapsulated in IP, the TTL, TOS, and IP security options
|
||||
MAY be copied from the payload packet into the same fields in the
|
||||
delivery packet. The payload packet's TTL MUST be decremented when
|
||||
the packet is decapsulated to insure that no packet lives forever.
|
||||
|
||||
IP source routes
|
||||
|
||||
When a system is processing a SRE with an Address Family indicating
|
||||
an IP source route, it MUST use the SRE Offset to determine the next
|
||||
destination IP address. If the next IP destination is this system,
|
||||
the SRE Offset field should be increased by four (the size of an IP
|
||||
address). If the SRE Offset is equal to the SRE Length in this SRE,
|
||||
then the Offset field in the GRE header should be adjusted to point
|
||||
to the next SRE (if any). This should be repeated until the next IP
|
||||
destination is not this system or until the entire SRE has been
|
||||
processed.
|
||||
|
||||
If the source route is incomplete, then the Strict Source Route bit
|
||||
is checked. If the source route is a strict source route and the
|
||||
next IP destination is NOT an adjacent system, the packet MUST be
|
||||
|
||||
|
||||
|
||||
Hanks, Li, Farinacci & Traina [Page 2]
|
||||
|
||||
RFC 1702 GRE over IPv4 networks October 1994
|
||||
|
||||
|
||||
dropped. Otherwise, the system should use the IP address indicated
|
||||
by the Offset field to replace the destination address in the
|
||||
delivery header and forward the packet.
|
||||
|
||||
Autonomous system source routes
|
||||
|
||||
When a system is processing a SRE with an Address Family indicating
|
||||
an AS source route, it MUST use the SRE Offset field to determine the
|
||||
next autonomous system. If the next autonomous system is the local
|
||||
autonomous system, the SRE Offset field should be increased by two
|
||||
(the size of an autonomous system number). If the SRE Offset is
|
||||
equal to the SRE Length in this SRE, then the Offset field in the GRE
|
||||
header should be adjusted to point to the next SRE (if any). This
|
||||
should be repeated until the next autonomous system number is not
|
||||
equal to the local autonomous system number or until the entire SRE
|
||||
has been processed.
|
||||
|
||||
If the source route is incomplete, then the Strict Source Route bit
|
||||
is checked. If the source route is a strict source route and the
|
||||
next autonomous system is NOT an adjacent autonomous system, the
|
||||
packet should be dropped. Otherwise, the system should use the
|
||||
autonomous system number indicated by the SRE Offset field to replace
|
||||
the destination address in the delivery header and forward the
|
||||
packet. The exact mechanism for determining the next delivery
|
||||
destination address given the AS number is outside of the scope of
|
||||
this document.
|
||||
|
||||
Security Considerations
|
||||
|
||||
Security issues are not discussed in this memo.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Hanks, Li, Farinacci & Traina [Page 3]
|
||||
|
||||
RFC 1702 GRE over IPv4 networks October 1994
|
||||
|
||||
|
||||
Authors' Addresses
|
||||
|
||||
Stan Hanks
|
||||
NetSmiths, Ltd.
|
||||
2025 Lincoln Highway
|
||||
Edison, NJ 08817
|
||||
|
||||
EMail: stan@netsmiths.com
|
||||
|
||||
|
||||
Tony Li
|
||||
cisco Systems, Inc.
|
||||
1525 O'Brien Drive
|
||||
Menlo Park, CA 94025
|
||||
|
||||
EMail: tli@cisco.com
|
||||
|
||||
|
||||
Dino Farinacci
|
||||
cisco Systems, Inc.
|
||||
1525 O'Brien Drive
|
||||
Menlo Park, CA 94025
|
||||
|
||||
EMail: dino@cisco.com
|
||||
|
||||
|
||||
Paul Traina
|
||||
cisco Systems, Inc.
|
||||
1525 O'Brien Drive
|
||||
Menlo Park, CA 94025
|
||||
|
||||
EMail: pst@cisco.com
|
||||
|
||||
References
|
||||
|
||||
RFC 1701
|
||||
Hanks, S., Li, T, Farinacci, D., and P. Traina, "Generic Routing
|
||||
Encapsulation", RFC 1701, NetSmiths, Ltd., and cisco Systems,
|
||||
October 1994.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Hanks, Li, Farinacci & Traina [Page 4]
|
||||
|
||||
171
RFC/rfc2119.txt
Normal file
171
RFC/rfc2119.txt
Normal file
@@ -0,0 +1,171 @@
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Network Working Group S. Bradner
|
||||
Request for Comments: 2119 Harvard University
|
||||
BCP: 14 March 1997
|
||||
Category: Best Current Practice
|
||||
|
||||
|
||||
Key words for use in RFCs to Indicate Requirement Levels
|
||||
|
||||
Status of this Memo
|
||||
|
||||
This document specifies an Internet Best Current Practices for the
|
||||
Internet Community, and requests discussion and suggestions for
|
||||
improvements. Distribution of this memo is unlimited.
|
||||
|
||||
Abstract
|
||||
|
||||
In many standards track documents several words are used to signify
|
||||
the requirements in the specification. These words are often
|
||||
capitalized. This document defines these words as they should be
|
||||
interpreted in IETF documents. Authors who follow these guidelines
|
||||
should incorporate this phrase near the beginning of their document:
|
||||
|
||||
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
|
||||
NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and
|
||||
"OPTIONAL" in this document are to be interpreted as described in
|
||||
RFC 2119.
|
||||
|
||||
Note that the force of these words is modified by the requirement
|
||||
level of the document in which they are used.
|
||||
|
||||
1. MUST This word, or the terms "REQUIRED" or "SHALL", mean that the
|
||||
definition is an absolute requirement of the specification.
|
||||
|
||||
2. MUST NOT This phrase, or the phrase "SHALL NOT", mean that the
|
||||
definition is an absolute prohibition of the specification.
|
||||
|
||||
3. SHOULD This word, or the adjective "RECOMMENDED", mean that there
|
||||
may exist valid reasons in particular circumstances to ignore a
|
||||
particular item, but the full implications must be understood and
|
||||
carefully weighed before choosing a different course.
|
||||
|
||||
4. SHOULD NOT This phrase, or the phrase "NOT RECOMMENDED" mean that
|
||||
there may exist valid reasons in particular circumstances when the
|
||||
particular behavior is acceptable or even useful, but the full
|
||||
implications should be understood and the case carefully weighed
|
||||
before implementing any behavior described with this label.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Bradner Best Current Practice [Page 1]
|
||||
|
||||
RFC 2119 RFC Key Words March 1997
|
||||
|
||||
|
||||
5. MAY This word, or the adjective "OPTIONAL", mean that an item is
|
||||
truly optional. One vendor may choose to include the item because a
|
||||
particular marketplace requires it or because the vendor feels that
|
||||
it enhances the product while another vendor may omit the same item.
|
||||
An implementation which does not include a particular option MUST be
|
||||
prepared to interoperate with another implementation which does
|
||||
include the option, though perhaps with reduced functionality. In the
|
||||
same vein an implementation which does include a particular option
|
||||
MUST be prepared to interoperate with another implementation which
|
||||
does not include the option (except, of course, for the feature the
|
||||
option provides.)
|
||||
|
||||
6. Guidance in the use of these Imperatives
|
||||
|
||||
Imperatives of the type defined in this memo must be used with care
|
||||
and sparingly. In particular, they MUST only be used where it is
|
||||
actually required for interoperation or to limit behavior which has
|
||||
potential for causing harm (e.g., limiting retransmisssions) For
|
||||
example, they must not be used to try to impose a particular method
|
||||
on implementors where the method is not required for
|
||||
interoperability.
|
||||
|
||||
7. Security Considerations
|
||||
|
||||
These terms are frequently used to specify behavior with security
|
||||
implications. The effects on security of not implementing a MUST or
|
||||
SHOULD, or doing something the specification says MUST NOT or SHOULD
|
||||
NOT be done may be very subtle. Document authors should take the time
|
||||
to elaborate the security implications of not following
|
||||
recommendations or requirements as most implementors will not have
|
||||
had the benefit of the experience and discussion that produced the
|
||||
specification.
|
||||
|
||||
8. Acknowledgments
|
||||
|
||||
The definitions of these terms are an amalgam of definitions taken
|
||||
from a number of RFCs. In addition, suggestions have been
|
||||
incorporated from a number of people including Robert Ullmann, Thomas
|
||||
Narten, Neal McBurnett, and Robert Elz.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Bradner Best Current Practice [Page 2]
|
||||
|
||||
RFC 2119 RFC Key Words March 1997
|
||||
|
||||
|
||||
9. Author's Address
|
||||
|
||||
Scott Bradner
|
||||
Harvard University
|
||||
1350 Mass. Ave.
|
||||
Cambridge, MA 02138
|
||||
|
||||
phone - +1 617 495 3864
|
||||
|
||||
email - sob@harvard.edu
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Bradner Best Current Practice [Page 3]
|
||||
|
||||
507
RFC/rfc2784.txt
Normal file
507
RFC/rfc2784.txt
Normal file
@@ -0,0 +1,507 @@
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Network Working Group D. Farinacci
|
||||
Request for Comments: 2784 T. Li
|
||||
Category: Standards Track Procket Networks
|
||||
S. Hanks
|
||||
Enron Communications
|
||||
D. Meyer
|
||||
Cisco Systems
|
||||
P. Traina
|
||||
Juniper Networks
|
||||
March 2000
|
||||
|
||||
|
||||
Generic Routing Encapsulation (GRE)
|
||||
|
||||
Status of this Memo
|
||||
|
||||
This document specifies an Internet standards track protocol for the
|
||||
Internet community, and requests discussion and suggestions for
|
||||
improvements. Please refer to the current edition of the "Internet
|
||||
Official Protocol Standards" (STD 1) for the standardization state
|
||||
and status of this protocol. Distribution of this memo is unlimited.
|
||||
|
||||
Copyright Notice
|
||||
|
||||
Copyright (C) The Internet Society (2000). All Rights Reserved.
|
||||
|
||||
Abstract
|
||||
|
||||
This document specifies a protocol for encapsulation of an arbitrary
|
||||
network layer protocol over another arbitrary network layer protocol.
|
||||
|
||||
1. Introduction
|
||||
|
||||
A number of different proposals [RFC1234, RFC1226] currently exist
|
||||
for the encapsulation of one protocol over another protocol. Other
|
||||
types of encapsulations [RFC1241, RFC1479] have been proposed for
|
||||
transporting IP over IP for policy purposes. This memo describes a
|
||||
protocol which is very similar to, but is more general than, the
|
||||
above proposals. In attempting to be more general, many protocol
|
||||
specific nuances have been ignored. The result is that this proposal
|
||||
may be less suitable for a situation where a specific "X over Y"
|
||||
encapsulation has been described. It is the attempt of this protocol
|
||||
to provide a simple, general purpose mechanism which reduces the
|
||||
problem of encapsulation from its current O(n^2) size to a more
|
||||
manageable size. This memo purposely does not address the issue of
|
||||
when a packet should be encapsulated. This memo acknowledges, but
|
||||
does not address problems such as mutual encapsulation [RFC1326].
|
||||
|
||||
|
||||
|
||||
|
||||
Farinacci, et al. Standards Track [Page 1]
|
||||
|
||||
RFC 2784 Generic Routing Encapsulation March 2000
|
||||
|
||||
|
||||
In the most general case, a system has a packet that needs to be
|
||||
encapsulated and delivered to some destination. We will call this
|
||||
the payload packet. The payload is first encapsulated in a GRE
|
||||
packet. The resulting GRE packet can then be encapsulated in some
|
||||
other protocol and then forwarded. We will call this outer protocol
|
||||
the delivery protocol. The algorithms for processing this packet are
|
||||
discussed later.
|
||||
|
||||
Finally this specification describes the intersection of GRE
|
||||
currently deployed by multiple vendors.
|
||||
|
||||
The keywords MUST, MUST NOT, MAY, OPTIONAL, REQUIRED, RECOMMENDED,
|
||||
SHALL, SHALL NOT, SHOULD, SHOULD NOT are to be interpreted as defined
|
||||
in RFC 2119 [RFC2119].
|
||||
|
||||
2. Structure of a GRE Encapsulated Packet
|
||||
|
||||
A GRE encapsulated packet has the form:
|
||||
|
||||
---------------------------------
|
||||
| |
|
||||
| Delivery Header |
|
||||
| |
|
||||
---------------------------------
|
||||
| |
|
||||
| GRE Header |
|
||||
| |
|
||||
---------------------------------
|
||||
| |
|
||||
| Payload packet |
|
||||
| |
|
||||
---------------------------------
|
||||
|
||||
This specification is generally concerned with the structure of the
|
||||
GRE header, although special consideration is given to some of the
|
||||
issues surrounding IPv4 payloads.
|
||||
|
||||
2.1. GRE Header
|
||||
|
||||
The GRE packet header has the form:
|
||||
|
||||
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
|
||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
||||
|C| Reserved0 | Ver | Protocol Type |
|
||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
||||
| Checksum (optional) | Reserved1 (Optional) |
|
||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
||||
|
||||
|
||||
|
||||
|
||||
Farinacci, et al. Standards Track [Page 2]
|
||||
|
||||
RFC 2784 Generic Routing Encapsulation March 2000
|
||||
|
||||
|
||||
2.2. Checksum Present (bit 0)
|
||||
|
||||
If the Checksum Present bit is set to one, then the Checksum and the
|
||||
Reserved1 fields are present and the Checksum field contains valid
|
||||
information. Note that a compliant implementation MUST accept and
|
||||
process this field.
|
||||
|
||||
2.3. Reserved0 (bits 1-12)
|
||||
|
||||
A receiver MUST discard a packet where any of bits 1-5 are non-zero,
|
||||
unless that receiver implements RFC 1701. Bits 6-12 are reserved for
|
||||
future use. These bits MUST be sent as zero and MUST be ignored on
|
||||
receipt.
|
||||
|
||||
2.3.1. Version Number (bits 13-15)
|
||||
|
||||
The Version Number field MUST contain the value zero.
|
||||
|
||||
2.4. Protocol Type (2 octets)
|
||||
|
||||
The Protocol Type field contains the protocol type of the payload
|
||||
packet. These Protocol Types are defined in [RFC1700] as "ETHER
|
||||
TYPES" and in [ETYPES]. An implementation receiving a packet
|
||||
containing a Protocol Type which is not listed in [RFC1700] or
|
||||
[ETYPES] SHOULD discard the packet.
|
||||
|
||||
2.5. Checksum (2 octets)
|
||||
|
||||
The Checksum field contains the IP (one's complement) checksum sum of
|
||||
the all the 16 bit words in the GRE header and the payload packet.
|
||||
For purposes of computing the checksum, the value of the checksum
|
||||
field is zero. This field is present only if the Checksum Present bit
|
||||
is set to one.
|
||||
|
||||
2.6. Reserved1 (2 octets)
|
||||
|
||||
The Reserved1 field is reserved for future use, and if present, MUST
|
||||
be transmitted as zero. The Reserved1 field is present only when the
|
||||
Checksum field is present (that is, Checksum Present bit is set to
|
||||
one).
|
||||
|
||||
3. IPv4 as a Payload
|
||||
|
||||
When IPv4 is being carried as the GRE payload, the Protocol Type
|
||||
field MUST be set to 0x800.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Farinacci, et al. Standards Track [Page 3]
|
||||
|
||||
RFC 2784 Generic Routing Encapsulation March 2000
|
||||
|
||||
|
||||
3.1. Forwarding Decapsulated IPv4 Payload Packets
|
||||
|
||||
When a tunnel endpoint decapsulates a GRE packet which has an IPv4
|
||||
packet as the payload, the destination address in the IPv4 payload
|
||||
packet header MUST be used to forward the packet and the TTL of the
|
||||
payload packet MUST be decremented. Care should be taken when
|
||||
forwarding such a packet, since if the destination address of the
|
||||
payload packet is the encapsulator of the packet (i.e., the other end
|
||||
of the tunnel), looping can occur. In this case, the packet MUST be
|
||||
discarded.
|
||||
|
||||
4. IPv4 as a Delivery Protocol
|
||||
|
||||
The IPv4 protocol 47 [RFC1700] is used when GRE packets are
|
||||
enapsulated in IPv4. See [RFC1122] for requirements relating to the
|
||||
delivery of packets over IPv4 networks.
|
||||
|
||||
5. Interoperation with RFC 1701 Compliant Implementations
|
||||
|
||||
In RFC 1701, the field described here as Reserved0 contained a number
|
||||
of flag bits which this specification deprecates. In particular, the
|
||||
Routing Present, Key Present, Sequence Number Present, and Strict
|
||||
Source Route bits have been deprecated, along with the Recursion
|
||||
Control field. As a result, the GRE header will never contain the
|
||||
Key, Sequence Number or Routing fields specified in RFC 1701.
|
||||
|
||||
There are, however, existing implementations of RFC 1701. The
|
||||
following sections describe correct interoperation with such
|
||||
implementations.
|
||||
|
||||
5.1. RFC 1701 Compliant Receiver
|
||||
|
||||
An implementation complying to this specification will transmit the
|
||||
Reserved0 field set to zero. An RFC 1701 compliant receiver will
|
||||
interpret this as having the Routing Present, Key Present, Sequence
|
||||
Number Present, and Strict Source Route bits set to zero, and will
|
||||
not expect the RFC 1701 Key, Sequence Number or Routing fields to be
|
||||
present.
|
||||
|
||||
5.2. RFC 1701 Compliant Transmitter
|
||||
|
||||
An RFC 1701 transmitter may set any of the Routing Present, Key
|
||||
Present, Sequence Number Present, and Strict Source Route bits set to
|
||||
one, and thus may transmit the RFC 1701 Key, Sequence Number or
|
||||
Routing fields in the GRE header. As stated in Section 5.3, a packet
|
||||
with non-zero bits in any of bits 1-5 MUST be discarded unless the
|
||||
receiver implements RFC 1701.
|
||||
|
||||
|
||||
|
||||
|
||||
Farinacci, et al. Standards Track [Page 4]
|
||||
|
||||
RFC 2784 Generic Routing Encapsulation March 2000
|
||||
|
||||
|
||||
6. Security Considerations
|
||||
|
||||
Security in a network using GRE should be relatively similar to
|
||||
security in a normal IPv4 network, as routing using GRE follows the
|
||||
same routing that IPv4 uses natively. Route filtering will remain
|
||||
unchanged. However packet filtering requires either that a firewall
|
||||
look inside the GRE packet or that the filtering is done on the GRE
|
||||
tunnel endpoints. In those environments in which this is considered
|
||||
to be a security issue it may be desirable to terminate the tunnel at
|
||||
the firewall.
|
||||
|
||||
7. IANA Considerations
|
||||
|
||||
This section considers the assignment of additional GRE Version
|
||||
Numbers and Protocol Types.
|
||||
|
||||
7.1. GRE Version Numbers
|
||||
|
||||
This document specifies GRE version number 0. GRE version number 1 is
|
||||
used by PPTP [RFC2637]. Additional GRE version numbers are assigned
|
||||
by IETF Consensus as defined in RFC 2434 [RFC2434].
|
||||
|
||||
7.2. Protocol Types
|
||||
|
||||
GRE uses an ETHER Type for the Protocol Type. New ETHER TYPES are
|
||||
assigned by Xerox Systems Institute [RFC1700].
|
||||
|
||||
8. Acknowledgments
|
||||
|
||||
This document is derived from the original ideas of the authors of
|
||||
RFC 1701 and RFC 1702. Hitoshi Asaeda, Scott Bradner, Randy Bush,
|
||||
Brian Carpenter, Bill Fenner, Andy Malis, Thomas Narten, Dave Thaler,
|
||||
Tim Gleeson and others provided many constructive and insightful
|
||||
comments.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Farinacci, et al. Standards Track [Page 5]
|
||||
|
||||
RFC 2784 Generic Routing Encapsulation March 2000
|
||||
|
||||
|
||||
9. Appendix -- Known Issues
|
||||
|
||||
This document specifies the behavior of currently deployed GRE
|
||||
implementations. As such, it does not attempt to address the
|
||||
following known issues:
|
||||
|
||||
o Interaction Path MTU Discovery (PMTU) [RFC1191]
|
||||
|
||||
Existing implementations of GRE, when using IPv4 as the Delivery
|
||||
Header, do not implement Path MTU discovery and do not set the
|
||||
Don't Fragment bit in the Delivery Header. This can cause large
|
||||
packets to become fragmented within the tunnel and reassembled at
|
||||
the tunnel exit (independent of whether the payload packet is using
|
||||
PMTU). If a tunnel entry point were to use Path MTU discovery,
|
||||
however, that tunnel entry point would also need to relay ICMP
|
||||
unreachable error messages (in particular the "fragmentation needed
|
||||
and DF set" code) back to the originator of the packet, which is
|
||||
not a requirement in this specification. Failure to properly relay
|
||||
Path MTU information to an originator can result in the following
|
||||
behavior: the originator sets the don't fragment bit, the packet
|
||||
gets dropped within the tunnel, but since the originator doesn't
|
||||
receive proper feedback, it retransmits with the same PMTU, causing
|
||||
subsequently transmitted packets to be dropped.
|
||||
|
||||
o IPv6 as Delivery and/or Payload Protocol
|
||||
|
||||
This specification describes the intersection of GRE currently
|
||||
deployed by multiple vendors. IPv6 as delivery and/or payload
|
||||
protocol is not included in the currently deployed versions of GRE.
|
||||
|
||||
o Interaction with ICMP
|
||||
|
||||
o Interaction with the Differentiated Services Architecture
|
||||
|
||||
o Multiple and Looping Encapsulations
|
||||
|
||||
10. REFERENCES
|
||||
|
||||
[ETYPES] ftp://ftp.isi.edu/in-notes/iana/assignments/ethernet-
|
||||
numbers
|
||||
|
||||
[RFC1122] Braden, R., "Requirements for Internet hosts -
|
||||
communication layers", STD 3, RFC 1122, October 1989.
|
||||
|
||||
[RFC1191] Mogul, J. and S. Deering, "Path MTU Discovery", RFC 1191,
|
||||
November 1990.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Farinacci, et al. Standards Track [Page 6]
|
||||
|
||||
RFC 2784 Generic Routing Encapsulation March 2000
|
||||
|
||||
|
||||
[RFC1226] Kantor, B., "Internet Protocol Encapsulation of AX.25
|
||||
Frames", RFC 1226, May 1991.
|
||||
|
||||
[RFC1234] Provan, D., "Tunneling IPX Traffic through IP Networks",
|
||||
RFC 1234, June 1991.
|
||||
|
||||
[RFC1241] Woodburn, R. and D. Mills, "Scheme for an Internet
|
||||
Encapsulation Protocol: Version 1", RFC 1241, July 1991.
|
||||
|
||||
[RFC1326] Tsuchiya, P., "Mutual Encapsulation Considered Dangerous",
|
||||
RFC 1326, May 1992.
|
||||
|
||||
[RFC1479] Steenstrup, M., "Inter-Domain Policy Routing Protocol
|
||||
Specification: Version 1", RFC 1479, July 1993.
|
||||
|
||||
[RFC1700] Reynolds, J. and J. Postel, "Assigned Numbers", STD 2, RFC
|
||||
1700, October 1994.
|
||||
|
||||
[RFC1701] Hanks, S., Li, T., Farinacci, D. and P. Traina, "Generic
|
||||
Routing Encapsulation", RFC 1701, October 1994.
|
||||
|
||||
[RFC1702] Hanks, S., Li, T., Farinacci, D. and P. Traina, "Generic
|
||||
Routing Encapsulation over IPv4 networks", RFC 1702,
|
||||
October 1994.
|
||||
|
||||
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
|
||||
Requirement Levels", BCP 14, RFC 2119, March, 1997.
|
||||
|
||||
[RFC2408] Maughan, D., Schertler, M., Schneider, M. and J. Turner,
|
||||
"Internet Security Association and Key Management Protocol
|
||||
(ISAKMP)", RFC 2408, November 1998.
|
||||
|
||||
[RFC2434] Narten, T. and H. Alvestrand, "Guidelines for Writing an
|
||||
IANA Considerations Section in RFCs", BCP 26, RFC 2434,
|
||||
October, 1998.
|
||||
|
||||
[RFC2637] Hamzeh, K., et al., "Point-to-Point Tunneling Protocol
|
||||
(PPTP)", RFC 2637, July, 1999.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Farinacci, et al. Standards Track [Page 7]
|
||||
|
||||
RFC 2784 Generic Routing Encapsulation March 2000
|
||||
|
||||
|
||||
11. Authors' Addresses
|
||||
|
||||
Dino Farinacci
|
||||
Procket Networks
|
||||
3850 No. First St., Ste. C
|
||||
San Jose, CA 95134
|
||||
|
||||
EMail: dino@procket.com
|
||||
|
||||
|
||||
Tony Li
|
||||
Procket Networks
|
||||
3850 No. First St., Ste. C
|
||||
San Jose, CA 95134
|
||||
|
||||
Phone: +1 408 954 7903
|
||||
Fax: +1 408 987 6166
|
||||
EMail: tony1@home.net
|
||||
|
||||
|
||||
Stan Hanks
|
||||
Enron Communications
|
||||
|
||||
EMail: stan_hanks@enron.net
|
||||
|
||||
|
||||
David Meyer
|
||||
Cisco Systems, Inc.
|
||||
170 Tasman Drive
|
||||
San Jose, CA, 95134
|
||||
|
||||
EMail: dmm@cisco.com
|
||||
|
||||
|
||||
Paul Traina
|
||||
Juniper Networks
|
||||
EMail: pst@juniper.net
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Farinacci, et al. Standards Track [Page 8]
|
||||
|
||||
RFC 2784 Generic Routing Encapsulation March 2000
|
||||
|
||||
|
||||
12. Full Copyright Statement
|
||||
|
||||
Copyright (C) The Internet Society (2000). All Rights Reserved.
|
||||
|
||||
This document and translations of it may be copied and furnished to
|
||||
others, and derivative works that comment on or otherwise explain it
|
||||
or assist in its implementation may be prepared, copied, published
|
||||
and distributed, in whole or in part, without restriction of any
|
||||
kind, provided that the above copyright notice and this paragraph are
|
||||
included on all such copies and derivative works. However, this
|
||||
document itself may not be modified in any way, such as by removing
|
||||
the copyright notice or references to the Internet Society or other
|
||||
Internet organizations, except as needed for the purpose of
|
||||
developing Internet standards in which case the procedures for
|
||||
copyrights defined in the Internet Standards process must be
|
||||
followed, or as required to translate it into languages other than
|
||||
English.
|
||||
|
||||
The limited permissions granted above are perpetual and will not be
|
||||
revoked by the Internet Society or its successors or assigns.
|
||||
|
||||
This document and the information contained herein is provided on an
|
||||
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
|
||||
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
|
||||
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
|
||||
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
|
||||
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
|
||||
|
||||
Acknowledgement
|
||||
|
||||
Funding for the RFC Editor function is currently provided by the
|
||||
Internet Society.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Farinacci, et al. Standards Track [Page 9]
|
||||
|
||||
7507
RFC/rfc2960.txt
Normal file
7507
RFC/rfc2960.txt
Normal file
File diff suppressed because it is too large
Load Diff
507
RFC/rfc2991.txt
Normal file
507
RFC/rfc2991.txt
Normal file
@@ -0,0 +1,507 @@
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Network Working Group D. Thaler
|
||||
Request for Comments: 2991 Microsoft
|
||||
Category: Informational C. Hopps
|
||||
NextHop Technologies
|
||||
November 2000
|
||||
|
||||
|
||||
Multipath Issues in Unicast and Multicast Next-Hop Selection
|
||||
|
||||
Status of this Memo
|
||||
|
||||
This memo provides information for the Internet community. It does
|
||||
not specify an Internet standard of any kind. Distribution of this
|
||||
memo is unlimited.
|
||||
|
||||
Copyright Notice
|
||||
|
||||
Copyright (C) The Internet Society (2000). All Rights Reserved.
|
||||
|
||||
Abstract
|
||||
|
||||
Various routing protocols, including Open Shortest Path First (OSPF)
|
||||
and Intermediate System to Intermediate System (ISIS), explicitly
|
||||
allow "Equal-Cost Multipath" (ECMP) routing. Some router
|
||||
implementations also allow equal-cost multipath usage with RIP and
|
||||
other routing protocols. The effect of multipath routing on a
|
||||
forwarder is that the forwarder potentially has several next-hops for
|
||||
any given destination and must use some method to choose which next-
|
||||
hop should be used for a given data packet.
|
||||
|
||||
1. Introduction
|
||||
|
||||
Various routing protocols, including OSPF and ISIS, explicitly allow
|
||||
"Equal-Cost Multipath" routing. Some router implementations also
|
||||
allow equal-cost multipath usage with RIP and other routing
|
||||
protocols. Using equal-cost multipath means that if multiple equal-
|
||||
cost routes to the same destination exist, they can be discovered and
|
||||
used to provide load balancing among redundant paths.
|
||||
|
||||
The effect of multipath routing on a forwarder is that the forwarder
|
||||
potentially has several next-hops for any given destination and must
|
||||
use some method to choose which next-hop should be used for a given
|
||||
data packet. This memo summarizes current practices, problems, and
|
||||
solutions.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Thaler & Hopps Informational [Page 1]
|
||||
|
||||
RFC 2991 Multipath Issues November 2000
|
||||
|
||||
|
||||
2. Concerns
|
||||
|
||||
Several router implementations allow multipath forwarding. This is
|
||||
sometimes done naively via round-robin, where each packet matching a
|
||||
given destination route is forwarded using the subsequent next-hop,
|
||||
in a round-robin fashion. This does provide a form of load
|
||||
balancing, but there are several problems with approaches such as
|
||||
round-robin or random:
|
||||
|
||||
Variable Path MTU
|
||||
Since each of the redundant paths may have a different MTU,
|
||||
this means that the overall path MTU can change on a packet-
|
||||
by-packet basis, negating the usefulness of path MTU discovery.
|
||||
|
||||
Variable Latencies
|
||||
Since each of the redundant paths may have a different latency
|
||||
involved, having packets take separate paths can cause packets
|
||||
to always arrive out of order, increasing delivery latency and
|
||||
buffering requirements.
|
||||
|
||||
Packet reordering causes TCP to believe that loss has taken
|
||||
place when packets with higher sequence numbers arrive before
|
||||
an earlier one. When three or more packets are received before
|
||||
a "late" packet, TCP enters a mode called "fast-retransmit" [6]
|
||||
which consumes extra bandwidth (which could potentially cause
|
||||
more loss, decreasing throughput) as it attempts to
|
||||
unnecessarily retransmit the delayed packet(s). Hence,
|
||||
reordering can be detrimental to network performance.
|
||||
|
||||
Debugging
|
||||
Common debugging utilities such as ping and traceroute are much
|
||||
less reliable in the presence of multiple paths and may even
|
||||
present completely wrong results.
|
||||
|
||||
In multicast routing, the problem with multiple paths is that
|
||||
multicast routing protocols prevent loops and duplicates by
|
||||
constructing a single tree to all receivers of the same group
|
||||
address. Multicast routing protocols deployed today (DVMRP, PIM-DM,
|
||||
PIM-SM) [2] construct shortest-path trees rooted at either the
|
||||
source, or another router known as a Core or Rendezvous Point.
|
||||
Hence, the way they ensure that duplicates will not arise is that a
|
||||
given tree must use only a single next-hop towards the root of the
|
||||
tree.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Thaler & Hopps Informational [Page 2]
|
||||
|
||||
RFC 2991 Multipath Issues November 2000
|
||||
|
||||
|
||||
3. Requirements
|
||||
|
||||
In the remainder of this document, we will use the term "flow" to
|
||||
represent the granularity at which the router keeps state (if at all)
|
||||
for classes of traffic. The exact definition of a flow may depend on
|
||||
the actual implementation. For example, a flow might be identified
|
||||
solely by destination address, or it might be identified by (source
|
||||
address, destination address, protocol id) triplet. Hence "flow" is
|
||||
not necessarily synonymous with the term "microflow" as used in RFC
|
||||
2474 [7], which also includes port numbers. Indeed, including
|
||||
transport-layer information in the next-hop selection process can
|
||||
actually be problematic. For example, if packets are fragmented, the
|
||||
transport-layer information may not be available in every packet.
|
||||
Furthermore, having the choice of path depend on transport-layer
|
||||
fields may negate the benefit of caching information such as MTU for
|
||||
use in subsequent connections between the same endpoints.
|
||||
|
||||
All of the problems outlined in the previous section arise when
|
||||
packets in the same unicast or multicast "flow" are split among
|
||||
multiple paths. The natural solution is therefore to ensure that
|
||||
packets for the same flow always use the same path.
|
||||
|
||||
Two additional features are desirable:
|
||||
|
||||
Minimal disruption
|
||||
When multipath is used, meaning that multiple routes contribute
|
||||
valid next-hops, the chances are higher of routes being added
|
||||
and deleted from consideration than when only the "best" route
|
||||
is used (in which case metric changes in alternate routes have
|
||||
no effect on traffic paths). Since a higher number of routes
|
||||
may actually be used for forwarding when multipath is in use,
|
||||
the potential for packet reordering and packet loss due to
|
||||
route flaps can be much greater than when not using multipath.
|
||||
Hence, it is desirable to minimize the number of active flows
|
||||
affected by the addition or deletion of another next-hop.
|
||||
|
||||
Fast implementation
|
||||
The amount of additional computation required to forward a
|
||||
packet should be small. For example, when doing round-robin,
|
||||
this computation might consist of incrementing (modulo the
|
||||
number of next-hops) a next-hop index.
|
||||
|
||||
4. Solutions
|
||||
|
||||
We now provide three possible methods for improving the performance
|
||||
of multipath and then discuss their applicability to unicast and
|
||||
multicast forwarding.
|
||||
|
||||
|
||||
|
||||
|
||||
Thaler & Hopps Informational [Page 3]
|
||||
|
||||
RFC 2991 Multipath Issues November 2000
|
||||
|
||||
|
||||
Modulo-N Hash
|
||||
To select a next-hop from the list of N next-hops, the router
|
||||
performs a modulo-N hash over the packet header fields that
|
||||
identify a flow. This has the advantage of being fast, at the
|
||||
expense of (N-1)/N of all flows changing paths whenever a
|
||||
next-hop is added or removed.
|
||||
|
||||
Hash-Threshold
|
||||
The router first selects a key by performing a hash over the
|
||||
packet header fields that identify the flow. The N next-hops
|
||||
have been assigned unique regions in the hash function's output
|
||||
space. By comparing the hash value against region boundaries
|
||||
the router can determine which region the hash value belongs to
|
||||
and thus which next-hop to use. This method has the advantage
|
||||
of only affecting flows near the region boundaries (or
|
||||
thresholds) when next-hops are added or removed. For ECMP
|
||||
hash-threshold's lookup can be done with a simple division
|
||||
(hash_value / fixed_region_size). When a next-hop is added or
|
||||
removed, between 1/4 and 1/2 of all flows change paths. An
|
||||
analysis of this method can be found in [3].
|
||||
|
||||
Highest Random Weight (HRW)
|
||||
The router computes a key for EACH next-hop by performing a
|
||||
hash over the packet header fields that identify the flow, as
|
||||
well as over the address of the next-hop. The router then
|
||||
chooses the next-hop with the highest resulting key value [4].
|
||||
This has the advantage of minimizing the number of flows
|
||||
affected by a next-hop addition or deletion (only 1/N of them),
|
||||
but is approximately N times as expensive as a modulo-N hash.
|
||||
|
||||
The applicability of these three alternatives depends on (at least)
|
||||
two factors: whether the forwarder maintains per-flow state, and how
|
||||
precious CPU is to a multipath forwarder.
|
||||
|
||||
Some routers may maintain per-flow state for reasons other than for
|
||||
supporting multipath. For example, routers typically keep per-flow
|
||||
state for multicast flows so that they can maintain the list of
|
||||
interfaces to which packets in the flow should be copied.
|
||||
|
||||
If per-flow state is maintained in a multipath forwarder, then
|
||||
computation of the next-hop can be done by the router at state
|
||||
creation time. This entails no additional computations at packet
|
||||
forwarding time compared with normal forwarding to a single next-hop,
|
||||
since the next-hop is precomputed. In this case, any method can be
|
||||
used, including round-robin, random, modulo-N, hash-threshold or HRW.
|
||||
Hash functions such as modulo-N, hash-threshold and HRW are better if
|
||||
the forwarder state may be deleted for any reason during the lifetime
|
||||
of a flow since subsequent next-hop computations by the router will
|
||||
|
||||
|
||||
|
||||
Thaler & Hopps Informational [Page 4]
|
||||
|
||||
RFC 2991 Multipath Issues November 2000
|
||||
|
||||
|
||||
always select the same path. This also improves the usefulness of
|
||||
debugging utilities such as traceroute. Finally, to maximize the
|
||||
stability of paths (and hence the usefulness of traceroute, etc.),
|
||||
the use of HRW is recommended over the other methods mentioned
|
||||
herein.
|
||||
|
||||
If per-flow state is not maintained by the forwarder, then using
|
||||
multiple next-hops requires that the next-hop be calculated at packet
|
||||
arrival time. When CPU is more precious than stability of flow
|
||||
paths, hash-threshold is recommended over the other methods mentioned
|
||||
herein.
|
||||
|
||||
4.1. Unicast Forwarding
|
||||
|
||||
Depending on the implementation, unicast forwarding may or may not
|
||||
keep per-flow state. We recommend that where forwarder
|
||||
implementations keep flow state, routers should use HRW at state
|
||||
creation time (and next-hop deletion time) to select the next-hop,
|
||||
and that forwarders without per-flow state use hash-threshold.
|
||||
|
||||
4.2. Multicast Forwarding
|
||||
|
||||
Today's multicast forwarding engines use a cache of forwarding
|
||||
entries indexed by group (or group prefix) and source (or source
|
||||
prefix). This means that today's multicast forwarder's always keep
|
||||
per-flow state, although for some multicast routing protocols, the
|
||||
"flow" may be fairly coarse (e.g., traffic from all sources to the
|
||||
same destination). Since per-flow state is kept by the forwarder, it
|
||||
is recommended that the router always use HRW to select the next-hop.
|
||||
|
||||
Routers using explicit-joining protocols such as PIM-SM [5] should
|
||||
thus use the multipath information when determining to which neighbor
|
||||
a join message should be sent. For example, when multiple next-hops
|
||||
exist for a given Rendezvous Point (RP) toward which a (*,G) Join
|
||||
should be sent, it is recommended that HRW be used to select the
|
||||
next-hop to use for each group.
|
||||
|
||||
5. Applicability
|
||||
|
||||
The algorithms discussed above (except round-robin) all rely on some
|
||||
form of hash function. Equal flow distribution is achieved when the
|
||||
hash function is uniformly distributed. Since the commonly used hash
|
||||
functions only become uniformly distributed when the number of inputs
|
||||
is relatively large, these algorithms are more applicable to routers
|
||||
used to route many flows, than in, for example, a small business
|
||||
setting.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Thaler & Hopps Informational [Page 5]
|
||||
|
||||
RFC 2991 Multipath Issues November 2000
|
||||
|
||||
|
||||
6. Redundant Parallel Links
|
||||
|
||||
A related problem occurs when multiple parallel links are used
|
||||
between the same pair of routers. A common solution is to bundle the
|
||||
two links together into a "super"-link when is then used for routing.
|
||||
For multicast forwarding, this results in the two links being reduced
|
||||
to a single next-hop (over the combined link) which can be used to
|
||||
prevent duplicates. When a unicast or multicast packet is queued to
|
||||
the combined link, some method, such as those discussed earlier, is
|
||||
still required to determine the physical link on which to transmit
|
||||
the packet. If the parallel links are identical, then most of the
|
||||
concerns discussed in this document are avoided with the combined
|
||||
link. The exception is packet reordering, which can still occur with
|
||||
round-robin, adversely affecting TCP.
|
||||
|
||||
7. Security Considerations
|
||||
|
||||
This document discusses issues with various methods of choosing a
|
||||
next-hop from among multiple valid next-hops. As such, it does not
|
||||
directly impact the security of the Internet infrastructure or its
|
||||
applications.
|
||||
|
||||
One issue that is worth mentioning, however, is that when next-hop
|
||||
selection is predictable, an attacker can synthesize traffic that
|
||||
will all hash the same, making it possible to launch a denial-of-
|
||||
service attack that overloads a particular path. Since a special
|
||||
case of this is when the same (single) next-hop is always selected,
|
||||
such an attack is easiest when multipath is not being used.
|
||||
Introducing multipath routing can make such an attack more difficult;
|
||||
the more unpredictable the hash is, the harder it becomes to conduct
|
||||
a denial-of-service attack against any single link.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Thaler & Hopps Informational [Page 6]
|
||||
|
||||
RFC 2991 Multipath Issues November 2000
|
||||
|
||||
|
||||
8. References
|
||||
|
||||
[1] Moy, J., "OSPF Version 2", STD 54, RFC 2328, April 1998.
|
||||
|
||||
[2] Maufer, T., "Deploying IP Multicast in the Enterprise",
|
||||
Prentice-Hall, 1998.
|
||||
|
||||
[3] Hopps, C., "Analysis of an Equal-Cost Multi-Path Algorithm", RFC
|
||||
2992, November 2000.
|
||||
|
||||
[4] Thaler, D., and C.V. Ravishankar, "Using Name-Based Mappings to
|
||||
Increase Hit Rates", IEEE/ACM Transactions on Networking,
|
||||
February 1998.
|
||||
|
||||
[5] Estrin, D., Farinacci, D., Helmy, A., Thaler, D., Deering, S.,
|
||||
Handley, M., Jacobson, V., Liu, C., Sharma, P. and L. Wei,
|
||||
"Protocol Independent Multicast-Sparse Mode (PIM-SM): Protocol
|
||||
Specification", RFC 2362, June 1998.
|
||||
|
||||
[6] Allman, M., Paxson, V. and W. Stevens, "TCP Congestion Control",
|
||||
RFC 2581, April 1999.
|
||||
|
||||
[7] Nichols, K., Blake, S., Baker, F. and D. Black., "Definition of
|
||||
the Differentiated Services Field (DS Field) in the IPv4 and
|
||||
IPv6 Headers", RFC 2474, December 1998.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Thaler & Hopps Informational [Page 7]
|
||||
|
||||
RFC 2991 Multipath Issues November 2000
|
||||
|
||||
|
||||
9. Authors' Addresses
|
||||
|
||||
Dave Thaler
|
||||
Microsoft
|
||||
One Microsoft Way
|
||||
Redmond, WA 98052
|
||||
|
||||
Phone: +1 425 703 8835
|
||||
EMail: dthaler@dthaler.microsoft.com
|
||||
|
||||
|
||||
Christian E. Hopps
|
||||
NextHop Technologies, Inc.
|
||||
517 W. William Street
|
||||
Ann Arbor, MI 48103-4943
|
||||
U.S.A
|
||||
|
||||
Phone: +1 734 936 0291
|
||||
EMail: chopps@nexthop.com
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Thaler & Hopps Informational [Page 8]
|
||||
|
||||
RFC 2991 Multipath Issues November 2000
|
||||
|
||||
|
||||
10. Full Copyright Statement
|
||||
|
||||
Copyright (C) The Internet Society (2000). All Rights Reserved.
|
||||
|
||||
This document and translations of it may be copied and furnished to
|
||||
others, and derivative works that comment on or otherwise explain it
|
||||
or assist in its implementation may be prepared, copied, published
|
||||
and distributed, in whole or in part, without restriction of any
|
||||
kind, provided that the above copyright notice and this paragraph are
|
||||
included on all such copies and derivative works. However, this
|
||||
document itself may not be modified in any way, such as by removing
|
||||
the copyright notice or references to the Internet Society or other
|
||||
Internet organizations, except as needed for the purpose of
|
||||
developing Internet standards in which case the procedures for
|
||||
copyrights defined in the Internet Standards process must be
|
||||
followed, or as required to translate it into languages other than
|
||||
English.
|
||||
|
||||
The limited permissions granted above are perpetual and will not be
|
||||
revoked by the Internet Society or its successors or assigns.
|
||||
|
||||
This document and the information contained herein is provided on an
|
||||
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
|
||||
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
|
||||
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
|
||||
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
|
||||
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
|
||||
|
||||
Acknowledgement
|
||||
|
||||
Funding for the RFC Editor function is currently provided by the
|
||||
Internet Society.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Thaler & Hopps Informational [Page 9]
|
||||
|
||||
3419
RFC/rfc3986.txt
Normal file
3419
RFC/rfc3986.txt
Normal file
File diff suppressed because it is too large
Load Diff
2579
RFC/rfc3987.txt
Normal file
2579
RFC/rfc3987.txt
Normal file
File diff suppressed because it is too large
Load Diff
1011
RFC/rfc4088.txt
Normal file
1011
RFC/rfc4088.txt
Normal file
File diff suppressed because it is too large
Load Diff
5827
RFC/rfc4271.txt
Normal file
5827
RFC/rfc4271.txt
Normal file
File diff suppressed because it is too large
Load Diff
1963
RFC/rfc4838.txt
Normal file
1963
RFC/rfc4838.txt
Normal file
File diff suppressed because it is too large
Load Diff
2803
RFC/rfc5050.txt
Normal file
2803
RFC/rfc5050.txt
Normal file
File diff suppressed because it is too large
Load Diff
731
RFC/rfc7098.txt
Normal file
731
RFC/rfc7098.txt
Normal file
@@ -0,0 +1,731 @@
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Internet Engineering Task Force (IETF) B. Carpenter
|
||||
Request for Comments: 7098 Univ. of Auckland
|
||||
Category: Informational S. Jiang
|
||||
ISSN: 2070-1721 Huawei Technologies Co., Ltd
|
||||
W. Tarreau
|
||||
HAProxy Technologies, Inc.
|
||||
January 2014
|
||||
|
||||
|
||||
Using the IPv6 Flow Label for Load Balancing in Server Farms
|
||||
|
||||
Abstract
|
||||
|
||||
This document describes how the currently specified IPv6 flow label
|
||||
can be used to enhance layer 3/4 (L3/4) load distribution and
|
||||
balancing for large server farms.
|
||||
|
||||
Status of This Memo
|
||||
|
||||
This document is not an Internet Standards Track specification; it is
|
||||
published for informational purposes.
|
||||
|
||||
This document is a product of the Internet Engineering Task Force
|
||||
(IETF). It represents the consensus of the IETF community. It has
|
||||
received public review and has been approved for publication by the
|
||||
Internet Engineering Steering Group (IESG). Not all documents
|
||||
approved by the IESG are a candidate for any level of Internet
|
||||
Standard; see Section 2 of RFC 5741.
|
||||
|
||||
Information about the current status of this document, any errata,
|
||||
and how to provide feedback on it may be obtained at
|
||||
http://www.rfc-editor.org/info/rfc7098.
|
||||
|
||||
Copyright Notice
|
||||
|
||||
Copyright (c) 2014 IETF Trust and the persons identified as the
|
||||
document authors. All rights reserved.
|
||||
|
||||
This document is subject to BCP 78 and the IETF Trust's Legal
|
||||
Provisions Relating to IETF Documents
|
||||
(http://trustee.ietf.org/license-info) in effect on the date of
|
||||
publication of this document. Please review these documents
|
||||
carefully, as they describe your rights and restrictions with respect
|
||||
to this document. Code Components extracted from this document must
|
||||
include Simplified BSD License text as described in Section 4.e of
|
||||
the Trust Legal Provisions and are provided without warranty as
|
||||
described in the Simplified BSD License.
|
||||
|
||||
|
||||
|
||||
|
||||
Carpenter, et al. Informational [Page 1]
|
||||
|
||||
RFC 7098 Flow Label for Server Load Balancing January 2014
|
||||
|
||||
|
||||
Table of Contents
|
||||
|
||||
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
|
||||
2. Summary of Flow Label Specification . . . . . . . . . . . . . 2
|
||||
3. Summary of Server Farm Load-Balancing Techniques . . . . . . 4
|
||||
4. Applying the Flow Label to Layer 3/4 Load Balancing . . . . . 8
|
||||
5. Security Considerations . . . . . . . . . . . . . . . . . . . 10
|
||||
6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 11
|
||||
7. References . . . . . . . . . . . . . . . . . . . . . . . . . 12
|
||||
7.1. Normative References . . . . . . . . . . . . . . . . . . 12
|
||||
7.2. Informative References . . . . . . . . . . . . . . . . . 12
|
||||
|
||||
1. Introduction
|
||||
|
||||
The IPv6 flow label has been redefined [RFC6437] and is now a
|
||||
recommended IPv6 node requirement [RFC6434]. Its use for load
|
||||
sharing in multipath routing has been specified [RFC6438]. Another
|
||||
scenario in which the flow label could be used is in load
|
||||
distribution for large server farms. Load distribution is a slightly
|
||||
more general term than load balancing, but the latter is more
|
||||
commonly used. In the context of a server farm, both terms refer to
|
||||
mechanisms that distribute the workload of a server farm among
|
||||
different servers in order to optimize performance. Server load
|
||||
balancing commonly applies to HTTP traffic, but most of the
|
||||
techniques described would apply to other upper-layer applications as
|
||||
well. This document starts with brief introductions to the flow
|
||||
label and to server load-balancing techniques, and then describes how
|
||||
the flow label can be used to enhance load balancers operating on IP
|
||||
packets and TCP sessions, commonly known as layer 3/4 load balancers.
|
||||
|
||||
The motivation for this approach is to improve the performance of
|
||||
most types of layer 3/4 load balancers, especially for traffic
|
||||
including multiple IPv6 extension headers and in particular for
|
||||
fragmented packets. Fragmented packets, often the result of
|
||||
customers reaching the load balancer via a VPN with a limited MTU,
|
||||
are a common performance problem.
|
||||
|
||||
2. Summary of Flow Label Specification
|
||||
|
||||
The IPv6 flow label [RFC6437] is a 20-bit field included in every
|
||||
IPv6 header [RFC2460]. It is recommended to be supported in all IPv6
|
||||
nodes by [RFC6434]. There is additional background material in
|
||||
[RFC6436] and [RFC6294]. According to its definition, the flow label
|
||||
should be set to a constant value for a given traffic flow (such as
|
||||
an HTTP connection), and that value will belong to a uniform
|
||||
statistical distribution, making it potentially valuable for load-
|
||||
balancing purposes.
|
||||
|
||||
|
||||
|
||||
|
||||
Carpenter, et al. Informational [Page 2]
|
||||
|
||||
RFC 7098 Flow Label for Server Load Balancing January 2014
|
||||
|
||||
|
||||
Any device that has access to the IPv6 header has access to the flow
|
||||
label, and it is at a fixed position in every IPv6 packet. In
|
||||
contrast, transport-layer information, such as the port numbers, is
|
||||
not always in a fixed position, since it follows any IPv6 extension
|
||||
headers that may be present. In fact, the logic of finding the
|
||||
transport header is always more complex for IPv6 than for IPv4, due
|
||||
to the absence of an Internet Header Length field in IPv6.
|
||||
Additionally, if packets are fragmented, the flow label will be
|
||||
present in all fragments, but the transport header will only be in
|
||||
one packet. Therefore, within the lifetime of a given transport-
|
||||
layer connection, the flow label can be a more convenient "handle"
|
||||
than the port number for identifying that particular connection.
|
||||
|
||||
According to RFC 6437, source hosts should set the flow label;
|
||||
however, if they do not (i.e., its value is zero), forwarding nodes
|
||||
(such as the first-hop router) may set it instead. In both cases,
|
||||
the flow label value must be constant for a given transport session,
|
||||
normally identified by the IPv6 and Transport header 5-tuple. By
|
||||
default, the flow label value should be calculated by a stateless
|
||||
algorithm. The resulting value should form part of a statistically
|
||||
uniform distribution, regardless of which node sets it.
|
||||
|
||||
It is recognized that at the time of writing, very few traffic flows
|
||||
include a non-zero flow label value. The mechanism described below
|
||||
is one that can be added to existing load-balancing mechanisms, so
|
||||
that it will become effective as more and more flows contain a non-
|
||||
zero label. Even if the flow label is chosen from an imperfectly
|
||||
uniform distribution, it will nevertheless increase the information
|
||||
entropy of the IPv6 header as a whole. This allows for progressive
|
||||
introduction of load balancing based on the flow label.
|
||||
|
||||
If the recommendations in Section 3 of RFC 6437 are followed for
|
||||
traffic from a given source accessing a well-known TCP port at a
|
||||
given destination, the flow label can act as a substitute for the
|
||||
port numbers as far as a load balancer is concerned, and it can be
|
||||
found at a fixed position in the layer 3 header even if extension
|
||||
headers are present.
|
||||
|
||||
The flow label is defined as an end-to-end component of the IPv6
|
||||
header, but there are three qualifications to this:
|
||||
|
||||
1. Until the IPv6 flow label specification in RFC 6437 is widely
|
||||
implemented as recommended by RFC 6434, the flow label will often
|
||||
be set to the default value of zero.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Carpenter, et al. Informational [Page 3]
|
||||
|
||||
RFC 7098 Flow Label for Server Load Balancing January 2014
|
||||
|
||||
|
||||
2. Because of the recommendation to use a stateless algorithm to
|
||||
calculate the label, there is a low (but non-zero) probability
|
||||
that two simultaneous flows from the same source to the same
|
||||
destination have the same flow label value despite having
|
||||
different transport-protocol port numbers.
|
||||
|
||||
3. The Flow Label field is in an unprotected part of the IPv6
|
||||
header, which means that intentional or unintentional changes to
|
||||
its value cannot be easily detected by a receiver.
|
||||
|
||||
The first two points are addressed below in Section 4 and the third
|
||||
in Section 5.
|
||||
|
||||
3. Summary of Server Farm Load-Balancing Techniques
|
||||
|
||||
Load balancing for server farms is achieved by a variety of methods,
|
||||
often used in combination [Tarreau]. This section gives a general
|
||||
overview of common methods, although the flow label is not relevant
|
||||
to all of them. The actual load-balancing algorithm (the choice of
|
||||
which server to use for a new client session) is irrelevant to this
|
||||
discussion. We give examples for HTTP, but analogous techniques may
|
||||
be used for other application protocols.
|
||||
|
||||
o The simplest method is using the DNS to return different server
|
||||
addresses for a single name such as www.example.com to different
|
||||
users. This is typically done by rotating the order in which
|
||||
different addresses within the server site are listed by the
|
||||
relevant authoritative DNS server, on the assumption that the
|
||||
client will pick the first one. Routing may be configured such
|
||||
that the different addresses are handled by different ingress
|
||||
routers. Several variants of this load-balancing mechanism exist,
|
||||
such as expecting some clients to use all the advertised addresses
|
||||
when multiple connections are involved, or directing the traffic
|
||||
to multiple sites, also known as global load balancing. None of
|
||||
these mechanisms are in the scope of this document, and the
|
||||
proposal in this document does not affect their usability nor aim
|
||||
to replace them, so they will not be discussed further.
|
||||
|
||||
o Another method, for HTTP servers, is to operate a layer 7 reverse
|
||||
proxy in front of the server farm. The reverse proxy will present
|
||||
a single IP address to the world, communicated to clients by a
|
||||
single AAAA record. For each new client session (an incoming TCP
|
||||
connection and HTTP request), it will pick a particular server and
|
||||
proxy the session to it. The act of proxying should be more
|
||||
efficient and less resource-intensive than the act of serving the
|
||||
required content. The proxy must retain TCP state and proxy state
|
||||
for the duration of the session. This TCP state could,
|
||||
potentially, include the incoming flow label value.
|
||||
|
||||
|
||||
|
||||
Carpenter, et al. Informational [Page 4]
|
||||
|
||||
RFC 7098 Flow Label for Server Load Balancing January 2014
|
||||
|
||||
|
||||
o A component of some load-balancing systems is an SSL reverse proxy
|
||||
farm. The individual SSL proxies handle all cryptographic aspects
|
||||
and exchange unencrypted HTTP with the actual servers. Thus, from
|
||||
the load-balancing point of view, this really looks just like a
|
||||
server farm, except that it's specialized for HTTPS. Each proxy
|
||||
will retain SSL and TCP and maybe HTTP state for the duration of
|
||||
the session, and the TCP state could potentially include the flow
|
||||
label.
|
||||
|
||||
o Finally the "front end" of many load-balancing systems is a layer
|
||||
3/4 load balancer. While it can be a dedicated device, it is also
|
||||
a standard function of some network switches or routers (e.g.
|
||||
using Equal-Cost Multipath Routing (ECMP) [RFC2991]). In this
|
||||
case, it is the layer 3/4 load balancer whose IP address is
|
||||
published as the primary AAAA record for the service. All client
|
||||
sessions will pass through this device. Depending on the specific
|
||||
scenario, the balancer will assign new sessions among the actual
|
||||
application servers, across an SSL proxy farm, or among a set of
|
||||
layer 7 proxies. In all cases, the layer 3/4 load balancer has to
|
||||
classify incoming packets very quickly and choose the target
|
||||
server or proxy so as to ensure persistence. 'Persistence' is
|
||||
defined as the guarantee that a given client session will run to
|
||||
completion on a single server. The layer 3/4 load balancer
|
||||
therefore needs to inspect each incoming packet to classify it.
|
||||
There are two common types of layer 3/4 load balancers, the
|
||||
totally stateless ones which only act on single packets, generally
|
||||
involving a per-packet hashing of easy-to-find information such as
|
||||
the source address and/or port into a server number, and the
|
||||
stateful ones that take the routing decision on the very first
|
||||
packets of a session and maintain the same direction for all
|
||||
packets belonging to the same session. Clearly, both types of
|
||||
layer 3/4 balancers could inspect and make use of the flow label
|
||||
value.
|
||||
|
||||
Our focus is on how the balancer identifies a particular flow.
|
||||
For clarity, note that two aspects of layer 3/4 load balancers are
|
||||
not affected by use of the flow label to identify sessions:
|
||||
|
||||
1. Balancers use various techniques to redirect traffic to a
|
||||
specific target server.
|
||||
|
||||
+ All servers are configured with the same IP address, they
|
||||
are all on the same LAN, and the load balancer sends
|
||||
directly to their individual MAC addresses. In this case,
|
||||
return packets from the server to the client are sent back
|
||||
without passing through the balancer, a technique known as
|
||||
direct server return, but we are not concerned here with
|
||||
the return packets.
|
||||
|
||||
|
||||
|
||||
Carpenter, et al. Informational [Page 5]
|
||||
|
||||
RFC 7098 Flow Label for Server Load Balancing January 2014
|
||||
|
||||
|
||||
+ All servers are configured with the same IP address,
|
||||
treated locally as an anycast address by layer 3 ECMP
|
||||
routing.
|
||||
|
||||
+ Each server has its own IP address, and the balancer uses
|
||||
an IP-in-IP tunnel to reach it.
|
||||
|
||||
+ Each server has its own IP address, and the balancer
|
||||
performs NAPT (Network Address and Port Translation) to
|
||||
deliver the client's packets to that address.
|
||||
|
||||
+ The choice between these methods is not affected by use of
|
||||
the flow label.
|
||||
|
||||
2. A layer 3/4 balancer must correctly handle Path MTU Discovery
|
||||
by forwarding relevant ICMPv6 packets in both directions.
|
||||
This too is not directly affected by use of the flow label.
|
||||
It should be noted that there may be difficulty correlating an
|
||||
ICMPv6 "Packet too big" response with the session it refers
|
||||
to, but that is out of the scope of the present document.
|
||||
|
||||
The following diagram, inspired by [Tarreau], shows a layout with
|
||||
various methods in use together. (Below, "ASIC" stands for
|
||||
"Application-Specific Integrated Circuit".)
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Carpenter, et al. Informational [Page 6]
|
||||
|
||||
RFC 7098 Flow Label for Server Load Balancing January 2014
|
||||
|
||||
|
||||
___________________________________________
|
||||
( )
|
||||
( Clients in the Internet )
|
||||
(___________________________________________)
|
||||
| |
|
||||
------------ DNS-based ------------
|
||||
| Ingress | load splitting | Ingress |
|
||||
| router | affects | router |
|
||||
------------ routing ------------
|
||||
___|____________________________|___
|
||||
| |
|
||||
| |
|
||||
| |
|
||||
------------ ------------
|
||||
| L3/4 ASIC| | L3/4 ASIC|
|
||||
| balancer | | balancer |
|
||||
------------ ------------
|
||||
| load |
|
||||
| spreading |
|
||||
__________|________________________|___________
|
||||
| | | |
|
||||
------------ ------------ -------- --------
|
||||
|HTTP proxy|...|HTTP proxy| | SSL |...| SSL |
|
||||
| balancer | | balancer | | proxy| | proxy|
|
||||
------------ ------------ -------- --------
|
||||
____|_____________|_____________|_________|_____
|
||||
| | | | |
|
||||
-------- -------- -------- -------- --------
|
||||
|HTTP | |HTTP | |HTTP | |HTTP | |HTTP |
|
||||
|server| |server| |server| |server| |server|
|
||||
-------- -------- -------- -------- --------
|
||||
|
||||
From the previous paragraphs, we can identify several points in this
|
||||
diagram where the flow label might be relevant:
|
||||
|
||||
1. Layer 3/4 load balancers.
|
||||
|
||||
2. SSL proxies.
|
||||
|
||||
3. HTTP proxies.
|
||||
|
||||
However, usage by the proxies seems unlikely to affect performance,
|
||||
because they must in any case process the application-layer header,
|
||||
so in this document we focus only on layer 3/4 balancers.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Carpenter, et al. Informational [Page 7]
|
||||
|
||||
RFC 7098 Flow Label for Server Load Balancing January 2014
|
||||
|
||||
|
||||
4. Applying the Flow Label to Layer 3/4 Load Balancing
|
||||
|
||||
The suggested model for using the flow label to enhance an layer 3/4
|
||||
load-balancing mechanism is as follows:
|
||||
|
||||
o We are only concerned with IPv6 traffic in which the flow label
|
||||
value has been set according to [RFC6437]. If the flow label of
|
||||
an incoming packet is zero, load balancers will continue to use
|
||||
the transport header in the traditional way. As the use of the
|
||||
flow label becomes more prevalent according to RFC 6434, load
|
||||
balancers, and therefore users, will reap a growing performance
|
||||
benefit.
|
||||
|
||||
o If the flow label of an incoming packet is non-zero, layer 3/4
|
||||
load balancers can use the 2-tuple {source address, flow label} as
|
||||
the session key for whatever load distribution algorithm they
|
||||
support. Alternatively, they might use the 3-tuple {dest address,
|
||||
source address, flow label}, especially if the server farm
|
||||
supports multiple server IP addresses, but using the 3-tuple will
|
||||
be significantly quicker than searching for the transport port
|
||||
numbers later in the packet. Moreover, the transport-layer
|
||||
information such as the source port is not repeated in fragments,
|
||||
which generally prevents stateless load balancers from supporting
|
||||
fragmented traffic since they generally cannot reassemble
|
||||
fragments.
|
||||
|
||||
A stateless layer 3/4 load balancer would simply apply a hash
|
||||
algorithm to the 2-tuple or 3-tuple on all packets in order to
|
||||
select the same target server consistently for a given flow.
|
||||
Needless to say, the hash algorithm has to be well chosen for its
|
||||
purpose, but this problem is common to several forms of stateless
|
||||
load balancing. The discussion in [RFC6438] applies.
|
||||
|
||||
A stateful layer 3/4 load balancer would apply its usual load
|
||||
distribution algorithm to the first packet of a session, and store
|
||||
the {tuple, server} association in a table so that subsequent
|
||||
packets belonging to the same session are forwarded to the same
|
||||
server. Thus, for all subsequent packets of the session, it can
|
||||
ignore all IPv6 extension headers, which should lead to a
|
||||
performance benefit. Whether this benefit is valuable will depend
|
||||
on engineering details of the specific load balancer.
|
||||
|
||||
Note that such a balancer will not identify new transport sessions
|
||||
from the same source that use the same flow label; they will be
|
||||
delivered to the same server. This is like the behavior of
|
||||
existing hash-based layer 4 balancers that always send similarly
|
||||
hashed packets to the same destination. However, a global state
|
||||
table in a flow label balancer cannot be shared between multiple
|
||||
|
||||
|
||||
|
||||
Carpenter, et al. Informational [Page 8]
|
||||
|
||||
RFC 7098 Flow Label for Server Load Balancing January 2014
|
||||
|
||||
|
||||
services if these services rely on transport-layer information,
|
||||
since the goal of using the flow label is to avoid looking up that
|
||||
information.
|
||||
|
||||
A related issue is that the balancer will not detect FIN/ACK
|
||||
sequences at the end of sessions. Therefore, it will rely on
|
||||
inactivity timers to delete session state. However, all existing
|
||||
balancers must maintain such timers to deal with hung sessions,
|
||||
and the practical impact on memory utilization is unlikely to be
|
||||
significant.
|
||||
|
||||
o Layer 3/4 balancers that redirect the incoming packets by NAPT are
|
||||
not expected to obtain any saving of time by using the flow label,
|
||||
because they have no choice but to follow the extension header
|
||||
chain in order to locate and modify the port number and transport
|
||||
checksum. The same would apply to balancers that perform TCP
|
||||
state tracking for any reason.
|
||||
|
||||
o Note that correct handling of ICMPv6 for Path MTU Discovery
|
||||
requires the layer 3/4 balancer to keep state for the client
|
||||
source address, independently of either the port numbers or the
|
||||
flow label.
|
||||
|
||||
o SSL and HTTP proxies, if present, should forward the flow label
|
||||
value towards the server. This usually has no performance
|
||||
benefit, but it is consistent with the general model for the flow
|
||||
label described in RFC 6437.
|
||||
|
||||
It should be noted that the performance benefit, if any, depends
|
||||
entirely on engineering trade-offs in the design of the layer 3/4
|
||||
balancer. An extra test is needed to check if the label is non-zero,
|
||||
but if there is a non-zero label, all logic for handling extension
|
||||
headers can be skipped except for the first packet of a new flow.
|
||||
Since the identifying state to be stored is only the tuple and the
|
||||
server identifier, storage requirements will be reduced.
|
||||
Additionally, the method will work for fragmented traffic and for
|
||||
flows where the transport information is missing (unknown transport
|
||||
protocol) or obfuscated (e.g., IPsec). Traffic reaching the load
|
||||
balancer via a VPN is particularly prone to the fragmentation issue,
|
||||
due to MTU size issues. For some load-balancer designs, these are
|
||||
very significant advantages.
|
||||
|
||||
In the unlikely event of two simultaneous flows from the same source
|
||||
address having the same flow label value, the two flows would end up
|
||||
assigned to the same server, where they would be distinguished as
|
||||
normal by their port numbers. There are approximately one million
|
||||
possible flow label values, and if the rules for flow label
|
||||
generation [RFC6437] are followed, this would be a statistically rare
|
||||
|
||||
|
||||
|
||||
Carpenter, et al. Informational [Page 9]
|
||||
|
||||
RFC 7098 Flow Label for Server Load Balancing January 2014
|
||||
|
||||
|
||||
event, and would not damage the overall load-balancing effect.
|
||||
Moreover, with a million possible label values, it is very likely
|
||||
that there will be many more flow label values than servers at most
|
||||
sites, so it is already expected that multiple flow label values will
|
||||
end up on the same server for a given client IP address.
|
||||
|
||||
In the case that many thousands of clients are hidden behind the same
|
||||
large-scale NAPT with a single shared IP address, the assumption of
|
||||
low probability of conflicts might become incorrect, unless flow
|
||||
label values are random enough to avoid following similar sequences
|
||||
for all clients. This is not expected to be a factor for IPv6
|
||||
anyway, since there is no need to implement large-scale NAPT with
|
||||
address sharing [RFC4864]. The probability of conflicts is low for
|
||||
sites that implement network prefix translation [RFC6296], since this
|
||||
technique provides a different address for each client.
|
||||
|
||||
5. Security Considerations
|
||||
|
||||
Security aspects of the flow label are discussed in [RFC6437]. As
|
||||
noted there, a malicious source or man-in-the-middle could disturb
|
||||
load balancing by manipulating flow labels. This risk already exists
|
||||
today where the source address and port are used as a hashing key in
|
||||
layer 3/4 load balancers, as well as where a persistence cookie is
|
||||
used in HTTP to designate a server. It even exists on layer 3
|
||||
components that only rely on the source address to select a
|
||||
destination, making them more DDoS-prone. Nevertheless, all these
|
||||
methods are currently used because the benefits for load balancing
|
||||
and persistence hugely outweigh the risks. The flow label does not
|
||||
significantly alter this situation.
|
||||
|
||||
Specifically, the IPv6 flow label specification [RFC6437] states that
|
||||
"stateless classifiers should not use the flow label alone to control
|
||||
load distribution, and stateful classifiers should include explicit
|
||||
methods to detect and ignore suspect flow label values." The former
|
||||
point is answered by also using the source address. The latter point
|
||||
is more complex. If the risk is considered serious, the site ingress
|
||||
router or the layer 3/4 balancer should use a suitable heuristic to
|
||||
verify incoming flows with non-zero flow label values. If a flow
|
||||
from a given source address and port number does not have a constant
|
||||
flow label value, it is suspect and should be dropped. This would
|
||||
deal with both intentional and accidental changes to the flow label.
|
||||
|
||||
A malicious source or man-in-the-middle could generate a flow in
|
||||
which the flow label is constant but the transport port numbers in
|
||||
some packets are invalid. Such packets, if load-balanced only on the
|
||||
basis of the flow label, could reach the target server and create a
|
||||
single-source DoS attack on its TCP engine.
|
||||
|
||||
|
||||
|
||||
|
||||
Carpenter, et al. Informational [Page 10]
|
||||
|
||||
RFC 7098 Flow Label for Server Load Balancing January 2014
|
||||
|
||||
|
||||
RFC 6437 notes in its Security Considerations that if the covert
|
||||
channel risk is considered significant, a firewall might rewrite non-
|
||||
zero flow labels. As long as this is done as described in RFC 6437,
|
||||
it will not invalidate the mechanisms described above.
|
||||
|
||||
The flow label may be of use in protecting against DDoS attacks
|
||||
against servers. As noted in RFC 6437, a source should generate flow
|
||||
label values that are hard to predict, most likely by including a
|
||||
secret nonce in the hash used to generate each label. The attacker
|
||||
does not know the nonce and therefore has no way to invent flow
|
||||
labels that will all target the same server, even with knowledge of
|
||||
both the hash algorithm and the load-balancing algorithm. Still, it
|
||||
is important to understand that it is always trivial to force a load
|
||||
balancer to stick to the same server during an attack, so the
|
||||
security of the whole solution must not rely on the unpredictability
|
||||
of the flow label values alone, but should include defensive measures
|
||||
like most load balancers already have against abnormal use of source
|
||||
addresses or session cookies.
|
||||
|
||||
New flows are assigned to a server according to any of the usual
|
||||
algorithms available on the load balancer (e.g., least connections,
|
||||
round robin, etc.). The association between the 2-tuple {source
|
||||
address, flow label} and the server is stored in a table (often
|
||||
called stick table) so that future traffic from the same source using
|
||||
the same flow label can be sent to the same server. This method is
|
||||
more robust against a loss of server and also makes it harder for an
|
||||
attacker to target a specific server, because the association between
|
||||
a flow label value and a server is not known externally.
|
||||
|
||||
In the case that a stateless hash function is used to assign client
|
||||
packets to specific servers, it may be advisable to use a
|
||||
cryptographic hash function of some kind, to ensure that an attacker
|
||||
cannot predict the behavior of the load balancer.
|
||||
|
||||
6. Acknowledgements
|
||||
|
||||
Valuable comments and contributions were made by Fred Baker, Olivier
|
||||
Bonaventure, Ben Campbell, Lorenzo Colitti, Linda Dunbar, Donald
|
||||
Eastlake, Joel Jaeggli, Gurudeep Kamat, Warren Kumari, Julia
|
||||
Renouard, Julius Volz, and others.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Carpenter, et al. Informational [Page 11]
|
||||
|
||||
RFC 7098 Flow Label for Server Load Balancing January 2014
|
||||
|
||||
|
||||
7. References
|
||||
|
||||
7.1. Normative References
|
||||
|
||||
[RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6
|
||||
(IPv6) Specification", RFC 2460, December 1998.
|
||||
|
||||
[RFC6434] Jankiewicz, E., Loughney, J., and T. Narten, "IPv6 Node
|
||||
Requirements", RFC 6434, December 2011.
|
||||
|
||||
[RFC6437] Amante, S., Carpenter, B., Jiang, S., and J. Rajahalme,
|
||||
"IPv6 Flow Label Specification", RFC 6437, November 2011.
|
||||
|
||||
7.2. Informative References
|
||||
|
||||
[RFC2991] Thaler, D. and C. Hopps, "Multipath Issues in Unicast and
|
||||
Multicast Next-Hop Selection", RFC 2991, November 2000.
|
||||
|
||||
[RFC4864] Van de Velde, G., Hain, T., Droms, R., Carpenter, B., and
|
||||
E. Klein, "Local Network Protection for IPv6", RFC 4864,
|
||||
May 2007.
|
||||
|
||||
[RFC6294] Hu, Q. and B. Carpenter, "Survey of Proposed Use Cases for
|
||||
the IPv6 Flow Label", RFC 6294, June 2011.
|
||||
|
||||
[RFC6296] Wasserman, M. and F. Baker, "IPv6-to-IPv6 Network Prefix
|
||||
Translation", RFC 6296, June 2011.
|
||||
|
||||
[RFC6436] Amante, S., Carpenter, B., and S. Jiang, "Rationale for
|
||||
Update to the IPv6 Flow Label Specification", RFC 6436,
|
||||
November 2011.
|
||||
|
||||
[RFC6438] Carpenter, B. and S. Amante, "Using the IPv6 Flow Label
|
||||
for Equal Cost Multipath Routing and Link Aggregation in
|
||||
Tunnels", RFC 6438, November 2011.
|
||||
|
||||
[Tarreau] Tarreau, W., "Making applications scalable with load
|
||||
balancing", 2006, <http://1wt.eu/articles/2006_lb/>.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Carpenter, et al. Informational [Page 12]
|
||||
|
||||
RFC 7098 Flow Label for Server Load Balancing January 2014
|
||||
|
||||
|
||||
Authors' Addresses
|
||||
|
||||
Brian Carpenter
|
||||
Department of Computer Science
|
||||
University of Auckland
|
||||
PB 92019
|
||||
Auckland 1142
|
||||
New Zealand
|
||||
|
||||
EMail: brian.e.carpenter@gmail.com
|
||||
|
||||
|
||||
Sheng Jiang
|
||||
Huawei Technologies Co., Ltd
|
||||
Q14, Huawei Campus
|
||||
No.156 Beiqing Road
|
||||
Hai-Dian District, Beijing 100095
|
||||
P.R. China
|
||||
|
||||
EMail: jiangsheng@huawei.com
|
||||
|
||||
|
||||
Willy Tarreau
|
||||
HAProxy Technologies, Inc.
|
||||
R&D Network Products
|
||||
3 rue du petit Robinson
|
||||
78350 Jouy-en-Josas
|
||||
France
|
||||
|
||||
EMail: willy@haproxy.com
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Carpenter, et al. Informational [Page 13]
|
||||
|
||||
5185
RFC/rfc761.txt
Normal file
5185
RFC/rfc761.txt
Normal file
File diff suppressed because it is too large
Load Diff
174
RFC/rfc768.txt
Normal file
174
RFC/rfc768.txt
Normal file
@@ -0,0 +1,174 @@
|
||||
|
||||
|
||||
RFC 768 J. Postel
|
||||
ISI
|
||||
28 August 1980
|
||||
|
||||
|
||||
|
||||
User Datagram Protocol
|
||||
----------------------
|
||||
|
||||
Introduction
|
||||
------------
|
||||
|
||||
This User Datagram Protocol (UDP) is defined to make available a
|
||||
datagram mode of packet-switched computer communication in the
|
||||
environment of an interconnected set of computer networks. This
|
||||
protocol assumes that the Internet Protocol (IP) [1] is used as the
|
||||
underlying protocol.
|
||||
|
||||
This protocol provides a procedure for application programs to send
|
||||
messages to other programs with a minimum of protocol mechanism. The
|
||||
protocol is transaction oriented, and delivery and duplicate protection
|
||||
are not guaranteed. Applications requiring ordered reliable delivery of
|
||||
streams of data should use the Transmission Control Protocol (TCP) [2].
|
||||
|
||||
Format
|
||||
------
|
||||
|
||||
|
||||
0 7 8 15 16 23 24 31
|
||||
+--------+--------+--------+--------+
|
||||
| Source | Destination |
|
||||
| Port | Port |
|
||||
+--------+--------+--------+--------+
|
||||
| | |
|
||||
| Length | Checksum |
|
||||
+--------+--------+--------+--------+
|
||||
|
|
||||
| data octets ...
|
||||
+---------------- ...
|
||||
|
||||
User Datagram Header Format
|
||||
|
||||
Fields
|
||||
------
|
||||
|
||||
Source Port is an optional field, when meaningful, it indicates the port
|
||||
of the sending process, and may be assumed to be the port to which a
|
||||
reply should be addressed in the absence of any other information. If
|
||||
not used, a value of zero is inserted.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Postel [page 1]
|
||||
|
||||
|
||||
28 Aug 1980
|
||||
User Datagram Protocol RFC 768
|
||||
Fields
|
||||
|
||||
|
||||
|
||||
Destination Port has a meaning within the context of a particular
|
||||
internet destination address.
|
||||
|
||||
Length is the length in octets of this user datagram including this
|
||||
header and the data. (This means the minimum value of the length is
|
||||
eight.)
|
||||
|
||||
Checksum is the 16-bit one's complement of the one's complement sum of a
|
||||
pseudo header of information from the IP header, the UDP header, and the
|
||||
data, padded with zero octets at the end (if necessary) to make a
|
||||
multiple of two octets.
|
||||
|
||||
The pseudo header conceptually prefixed to the UDP header contains the
|
||||
source address, the destination address, the protocol, and the UDP
|
||||
length. This information gives protection against misrouted datagrams.
|
||||
This checksum procedure is the same as is used in TCP.
|
||||
|
||||
0 7 8 15 16 23 24 31
|
||||
+--------+--------+--------+--------+
|
||||
| source address |
|
||||
+--------+--------+--------+--------+
|
||||
| destination address |
|
||||
+--------+--------+--------+--------+
|
||||
| zero |protocol| UDP length |
|
||||
+--------+--------+--------+--------+
|
||||
|
||||
If the computed checksum is zero, it is transmitted as all ones (the
|
||||
equivalent in one's complement arithmetic). An all zero transmitted
|
||||
checksum value means that the transmitter generated no checksum (for
|
||||
debugging or for higher level protocols that don't care).
|
||||
|
||||
User Interface
|
||||
--------------
|
||||
|
||||
A user interface should allow
|
||||
|
||||
the creation of new receive ports,
|
||||
|
||||
receive operations on the receive ports that return the data octets
|
||||
and an indication of source port and source address,
|
||||
|
||||
and an operation that allows a datagram to be sent, specifying the
|
||||
data, source and destination ports and addresses to be sent.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
[page 2] Postel
|
||||
|
||||
|
||||
28 Aug 1980
|
||||
RFC 768 User Datagram Protocol
|
||||
IP Interface
|
||||
|
||||
|
||||
|
||||
IP Interface
|
||||
-------------
|
||||
|
||||
The UDP module must be able to determine the source and destination
|
||||
internet addresses and the protocol field from the internet header. One
|
||||
possible UDP/IP interface would return the whole internet datagram
|
||||
including all of the internet header in response to a receive operation.
|
||||
Such an interface would also allow the UDP to pass a full internet
|
||||
datagram complete with header to the IP to send. The IP would verify
|
||||
certain fields for consistency and compute the internet header checksum.
|
||||
|
||||
Protocol Application
|
||||
--------------------
|
||||
|
||||
The major uses of this protocol is the Internet Name Server [3], and the
|
||||
Trivial File Transfer [4].
|
||||
|
||||
Protocol Number
|
||||
---------------
|
||||
|
||||
This is protocol 17 (21 octal) when used in the Internet Protocol.
|
||||
Other protocol numbers are listed in [5].
|
||||
|
||||
References
|
||||
----------
|
||||
|
||||
[1] Postel, J., "Internet Protocol," RFC 760, USC/Information
|
||||
Sciences Institute, January 1980.
|
||||
|
||||
[2] Postel, J., "Transmission Control Protocol," RFC 761,
|
||||
USC/Information Sciences Institute, January 1980.
|
||||
|
||||
[3] Postel, J., "Internet Name Server," USC/Information Sciences
|
||||
Institute, IEN 116, August 1979.
|
||||
|
||||
[4] Sollins, K., "The TFTP Protocol," Massachusetts Institute of
|
||||
Technology, IEN 133, January 1980.
|
||||
|
||||
[5] Postel, J., "Assigned Numbers," USC/Information Sciences
|
||||
Institute, RFC 762, January 1980.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Postel [page 3]
|
||||
|
||||
1218
RFC/rfc792.txt
Normal file
1218
RFC/rfc792.txt
Normal file
File diff suppressed because it is too large
Load Diff
171
RFC/rfc894.txt
Normal file
171
RFC/rfc894.txt
Normal file
@@ -0,0 +1,171 @@
|
||||
|
||||
|
||||
Network Working Group Charles Hornig
|
||||
Request for Comments: 894 Symbolics Cambridge Research Center
|
||||
April 1984
|
||||
|
||||
A Standard for the Transmission of IP Datagrams over Ethernet Networks
|
||||
|
||||
|
||||
Status of this Memo
|
||||
|
||||
This RFC specifies a standard method of encapsulating Internet
|
||||
Protocol (IP) [1] datagrams on an Ethernet [2]. This RFC specifies a
|
||||
standard protocol for the ARPA-Internet community.
|
||||
|
||||
Introduction
|
||||
|
||||
This memo applies to the Ethernet (10-megabit/second, 48-bit
|
||||
addresses). The procedure for transmission of IP datagrams on the
|
||||
Experimental Ethernet (3-megabit/second, 8-bit addresses) is
|
||||
described in [3].
|
||||
|
||||
Frame Format
|
||||
|
||||
IP datagrams are transmitted in standard Ethernet frames. The type
|
||||
field of the Ethernet frame must contain the value hexadecimal 0800.
|
||||
The data field contains the IP header followed immediately by the IP
|
||||
data.
|
||||
|
||||
The minimum length of the data field of a packet sent over an
|
||||
Ethernet is 46 octets. If necessary, the data field should be padded
|
||||
(with octets of zero) to meet the Ethernet minimum frame size. This
|
||||
padding is not part of the IP packet and is not included in the total
|
||||
length field of the IP header.
|
||||
|
||||
The minimum length of the data field of a packet sent over an
|
||||
Ethernet is 1500 octets, thus the maximum length of an IP datagram
|
||||
sent over an Ethernet is 1500 octets. Implementations are encouraged
|
||||
to support full-length packets. Gateway implementations MUST be
|
||||
prepared to accept full-length packets and fragment them if
|
||||
necessary. If a system cannot receive full-length packets, it should
|
||||
take steps to discourage others from sending them, such as using the
|
||||
TCP Maximum Segment Size option [4].
|
||||
|
||||
Note: Datagrams on the Ethernet may be longer than the general
|
||||
Internet default maximum packet size of 576 octets. Hosts connected
|
||||
to an Ethernet should keep this in mind when sending datagrams to
|
||||
hosts not on the same Ethernet. It may be appropriate to send
|
||||
smaller datagrams to avoid unnecessary fragmentation at intermediate
|
||||
gateways. Please see [4] for further information on this point.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Hornig [Page 1]
|
||||
|
||||
|
||||
|
||||
RFC 894 April 1984
|
||||
|
||||
|
||||
Address Mappings
|
||||
|
||||
The mapping of 32-bit Internet addresses to 48-bit Ethernet addresses
|
||||
can be done several ways. A static table could be used, or a dynamic
|
||||
discovery procedure could be used.
|
||||
|
||||
Static Table
|
||||
|
||||
Each host could be provided with a table of all other hosts on the
|
||||
local network with both their Ethernet and Internet addresses.
|
||||
|
||||
Dynamic Discovery
|
||||
|
||||
Mappings between 32-bit Internet addresses and 48-bit Ethernet
|
||||
addresses could be accomplished through the Address Resolution
|
||||
Protocol (ARP) [5]. Internet addresses are assigned arbitrarily
|
||||
on some Internet network. Each host's implementation must know
|
||||
its own Internet address and respond to Ethernet Address
|
||||
Resolution packets appropriately. It should also use ARP to
|
||||
translate Internet addresses to Ethernet addresses when needed.
|
||||
|
||||
Broadcast Address
|
||||
|
||||
The broadcast Internet address (the address on that network with a
|
||||
host part of all binary ones) should be mapped to the broadcast
|
||||
Ethernet address (of all binary ones, FF-FF-FF-FF-FF-FF hex).
|
||||
|
||||
The use of the ARP dynamic discovery procedure is strongly
|
||||
recommended.
|
||||
|
||||
Trailer Formats
|
||||
|
||||
Some versions of Unix 4.2bsd use a different encapsulation method in
|
||||
order to get better network performance with the VAX virtual memory
|
||||
architecture. Consenting systems on the same Ethernet may use this
|
||||
format between themselves.
|
||||
|
||||
No host is required to implement it, and no datagrams in this format
|
||||
should be sent to any host unless the sender has positive knowledge
|
||||
that the recipient will be able to interpret them. Details of the
|
||||
trailer encapsulation may be found in [6].
|
||||
|
||||
(Note: At the present time Unix 4.2bsd will either always use
|
||||
trailers or never use them (per interface), depending on a boot-time
|
||||
option. This is expected to be changed in the future. Unix 4.2bsd
|
||||
also uses a non-standard Internet broadcast address with a host part
|
||||
of all zeroes, this may also be changed in the future.)
|
||||
|
||||
|
||||
|
||||
Hornig [Page 2]
|
||||
|
||||
|
||||
|
||||
RFC 894 April 1984
|
||||
|
||||
|
||||
Byte Order
|
||||
|
||||
As described in Appendix B of the Internet Protocol
|
||||
specification [1], the IP datagram is transmitted over the Ethernet
|
||||
as a series of 8-bit bytes.
|
||||
|
||||
References
|
||||
|
||||
[1] Postel, J., "Internet Protocol", RFC-791, USC/Information
|
||||
Sciences Institute, September 1981.
|
||||
|
||||
[2] "The Ethernet - A Local Area Network", Version 1.0, Digital
|
||||
Equipment Corporation, Intel Corporation, Xerox Corporation,
|
||||
September 1980.
|
||||
|
||||
[3] Postel, J., "A Standard for the Transmission of IP Datagrams
|
||||
over Experimental Ethernet Networks", RFC-895, USC/Information
|
||||
Sciences Institute, April 1984.
|
||||
|
||||
[4] Postel, J., "The TCP Maximum Segment Size Option and Related
|
||||
Topics", RFC-879, USC/Information Sciences Institute, November 1983.
|
||||
|
||||
[5] Plummer, D., "An Ethernet Address Resolution Protocol", RFC-826,
|
||||
Symbolics Cambridge Research Center, November 1982.
|
||||
|
||||
[6] Leffler, S., and M. Karels, "Trailer Encapsulations", RFC-893,
|
||||
University of California at Berkeley, April 1984.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Hornig [Page 3]
|
||||
|
||||
Reference in New Issue
Block a user