problem-old.txt   ram-problem.txt 
Network Working Group T. Narten Network Working Group T. Narten
Internet-Draft IBM Internet-Draft IBM
Intended status: Informational September 25, 2007 Intended status: Informational October 9, 2007
Expires: March 28, 2008 Expires: April 11, 2008
Routing and Addressing Problem Statement Routing and Addressing Problem Statement
draft-narten-radir-problem-statement-01.txt draft-narten-radir-problem-statement-01.txt
Status of this Memo Status of this Memo
By submitting this Internet-Draft, each author represents that any By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79. aware will be disclosed, in accordance with Section 6 of BCP 79.
skipping to change at page 1, line 33 skipping to change at page 1, line 34
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt. http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
This Internet-Draft will expire on March 28, 2008. This Internet-Draft will expire on April 11, 2008.
Copyright Notice Copyright Notice
Copyright (C) The IETF Trust (2007). Copyright (C) The IETF Trust (2007).
Abstract Abstract
Problem statement for the route scaling problem. There has been much discussion over the last years about the overall
scalability of the Internet routing system. This document attempts
to describe what the actual problem is and the various demands being
placed on the routing system that have made finding a straightforward
solution difficult.
Comments should be sent to ram@iab.org or to radir@ietf.org. Comments should be sent to rrg@psg.com or to radir@ietf.org.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Terms and Definitions . . . . . . . . . . . . . . . . . . . . 4 2. Terms and Definitions . . . . . . . . . . . . . . . . . . . . 4
3. Background . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3. Background . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.1. Technical Aspects . . . . . . . . . . . . . . . . . . . . 6 3.1. Technical Aspects . . . . . . . . . . . . . . . . . . . . 6
3.2. Business Aspects . . . . . . . . . . . . . . . . . . . . . 7 3.2. Business Aspects . . . . . . . . . . . . . . . . . . . . . 7
3.3. Alignment of Incentives . . . . . . . . . . . . . . . . . 8 3.3. Alignment of Incentives . . . . . . . . . . . . . . . . . 8
3.4. Table Growth Targets . . . . . . . . . . . . . . . . . . . 8 3.4. Table Growth Targets . . . . . . . . . . . . . . . . . . . 8
skipping to change at page 2, line 40 skipping to change at page 2, line 44
5.1. Interconnection Richness . . . . . . . . . . . . . . . . . 15 5.1. Interconnection Richness . . . . . . . . . . . . . . . . . 15
5.2. Multihoming . . . . . . . . . . . . . . . . . . . . . . . 15 5.2. Multihoming . . . . . . . . . . . . . . . . . . . . . . . 15
5.3. Traffic Engineering . . . . . . . . . . . . . . . . . . . 15 5.3. Traffic Engineering . . . . . . . . . . . . . . . . . . . 15
5.4. Questionable Operational Practices? . . . . . . . . . . . 16 5.4. Questionable Operational Practices? . . . . . . . . . . . 16
5.4.1. Rapid shuffling of prefixes . . . . . . . . . . . . . 16 5.4.1. Rapid shuffling of prefixes . . . . . . . . . . . . . 16
5.4.2. Anti-Route Hijacking . . . . . . . . . . . . . . . . . 16 5.4.2. Anti-Route Hijacking . . . . . . . . . . . . . . . . . 16
5.4.3. Operational Ignorance . . . . . . . . . . . . . . . . 16 5.4.3. Operational Ignorance . . . . . . . . . . . . . . . . 16
5.5. RIR Policy . . . . . . . . . . . . . . . . . . . . . . . . 17 5.5. RIR Policy . . . . . . . . . . . . . . . . . . . . . . . . 17
6. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 6. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
7. Security Considerations . . . . . . . . . . . . . . . . . . . 20 7. Security Considerations . . . . . . . . . . . . . . . . . . . 20
8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 21 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21
9. Informative References . . . . . . . . . . . . . . . . . . . . 22 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 22
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 23 10. Informative References . . . . . . . . . . . . . . . . . . . . 23
Intellectual Property and Copyright Statements . . . . . . . . . . 24 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 24
Intellectual Property and Copyright Statements . . . . . . . . . . 25
1. Introduction 1. Introduction
Prompted in part by the recent IAB workshop on Routing & Addressing Prompted in part by the recent IAB workshop on Routing & Addressing
[RAWS-REPORT], there has been a renewed focus on the problem of [RFC4984], there has been a renewed focus on the problem of routing
routing scalability within the Internet. The issue itself is not scalability within the Internet. The issue itself is not new, with
new, with discussions dating back at least 10-15 years [GSE, discussions dating back at least 10-15 years [GSE, ROAD,...].
ROAD,...].
This document attempts to define the "problem", with the aim of This document attempts to define the "problem", with the aim of
describing the essential aspects so that the community has a way of describing the essential aspects so that the community has a way of
evaluating whether proposed solutions actually address or impact the evaluating whether proposed solutions actually address or impact the
underlying problem or "pain points" in a significant manner. underlying problem or "pain points" in a significant manner.
2. Terms and Definitions 2. Terms and Definitions
Control Plane: The routing setup protocols and their associated Control Plane: The routing setup protocols and their associated
state that are needed to create and maintain the data structures state that are needed to create and maintain the data structures
skipping to change at page 4, line 22 skipping to change at page 4, line 22
Control Plane Load: The actual load associated with operating the Control Plane Load: The actual load associated with operating the
Control Plane. The higher the control plane load, the higher the Control Plane. The higher the control plane load, the higher the
cost of operating the control plane (in terms of hardware and cost of operating the control plane (in terms of hardware and
bandwidth). bandwidth).
Control Plane Cost: The overall cost associated with operating the Control Plane Cost: The overall cost associated with operating the
Control Plane. The cost consists of capital costs (for hardware), Control Plane. The cost consists of capital costs (for hardware),
bandwidth costs (for the control plane signalling) and any other bandwidth costs (for the control plane signalling) and any other
operational cost associated with operating and maintaining the operational cost associated with operating and maintaining the
control plane. In the present Internet, the primary current cost control plane.
concern is on the capital hardware.
Default Free Zone (DFZ): That part of the Internet where routers Default Free Zone (DFZ): That part of the Internet where routers
maintain full routing tables. Many routers maintain only partial maintain full routing tables. Many routers maintain only partial
routes, having explicit routes for "local" destinations (i.e., routes, having explicit routes for "local" destinations (i.e.,
prefixes) plus a "default" for everything else. For such routers, prefixes) plus a "default" for everything else. For such routers,
building and maintaining routing tables is relatively simple building and maintaining routing tables is relatively simple
because the amount of information learned and maintained can be because the amount of information learned and maintained can be
small. In contrast, routers in the DFZ maintain complete small. In contrast, routers in the DFZ maintain complete
information about all reachable destinations, which currently information about all reachable destinations, which currently
number in the hundreds of thousands of entries. number in the hundreds of thousands of entries.
skipping to change at page 5, line 9 skipping to change at page 5, line 8
mapping a packet's destination address to an outgoing interface mapping a packet's destination address to an outgoing interface
and next-hop. The FIB only stores information about paths and next-hop. The FIB only stores information about paths
actually used for forwarding; it typically does not store actually used for forwarding; it typically does not store
information about backup paths. The FIB is typically constructed information about backup paths. The FIB is typically constructed
from specialized hardware components, which have different (and from specialized hardware components, which have different (and
higher) cost properties than the hardware typically used to higher) cost properties than the hardware typically used to
maintain the RIB. maintain the RIB.
Traffic Engineering (TE): In this document, "traffic engineering" Traffic Engineering (TE): In this document, "traffic engineering"
refers to the current practice of inbound, inter-AS traffic refers to the current practice of inbound, inter-AS traffic
engineering. TE is accomplished by placing more specific routes engineering. TE is accomplished by placing more-specific routes
in the FIB and/ or increasing the frequency of routing updates in in the FIB and/ or increasing the frequency of routing updates in
order to control inbound traffic at the boundary of an Autonomous order to control inbound traffic at the boundary of an Autonomous
system (AS). system (AS).
Provider Aggregatable (PA) address space: Address space that an end Provider Aggregatable (PA) address space: Address space that an end
site obtains from an ISP's address block. The main benefit of PA site obtains from an upstream ISP's address block. The main
address space is that reachability to all of a provider's benefit of PA address space is that reachability to all of a
customers can be achieved by advertising a single "provider provider's customers can be achieved by advertising a single
aggregate" address prefix into the DFZ, rather than needing to "provider aggregate" address prefix into the DFZ, rather than
announce individual prefixes for each customer. An important needing to announce individual prefixes for each customer. An
disadvantage is a requirement that the customer return those important disadvantage is a requirement that the customer return
addresses (and renumber) when changing providers. those addresses (and renumber) when changing providers.
Provider Independent (PI) address space: Address space that an end Provider Independent (PI) address space: Address space that an end
site obtains directly from a Regional Internet Registry (RIR) for site obtains directly from a Regional Internet Registry (RIR) for
addressing its devices. The main advantage (for the end site) is addressing its devices. The main advantage (for the end site) is
that it does not have to return those addresses (and renumber its that it does not have to return those addresses (and renumber its
site) upon changing providers. However, PI address blocks are not site) upon changing providers. However, PI address blocks are not
aggregatable and thus each individual PI assignment results in an aggregatable and thus each individual PI assignment results in an
individual prefix being injected into the DFZ. individual prefix being injected into the DFZ.
Site: Any topologically and administratively distinct entity that Site: Any topologically and administratively distinct entity that
skipping to change at page 6, line 34 skipping to change at page 6, line 34
factors (discussed below). It should be noted that the overall factors (discussed below). It should be noted that the overall
routing update rate is dependent on two factors: the number of routing update rate is dependent on two factors: the number of
individual prefixes and the mean per-prefix update rate. While it individual prefixes and the mean per-prefix update rate. While it
is clear that the overall number of prefixes is increasing super- is clear that the overall number of prefixes is increasing super-
linearly, further study is needed to determine whether the mean linearly, further study is needed to determine whether the mean
per-prefix update rate is increasing as well [1]. per-prefix update rate is increasing as well [1].
This super linear growth presents a scalability challenge for current This super linear growth presents a scalability challenge for current
and/or future routers. There are two aspects to the challenge. The and/or future routers. There are two aspects to the challenge. The
first one is purely technical: can we build routers (i.e., hardware & first one is purely technical: can we build routers (i.e., hardware &
software) actually capable handling the control plane load, both software) actually capable of handling the control plane load, both
today and going forward? The second challenge is one of economics: today and going forward? The second challenge is one of economics:
is the cost of developing, building and deploying such routers is the cost of developing, building and deploying such routers
economically sustainable, given current and realistic business models economically sustainable, given current and realistic business models
that govern how ISPs operate as businesses? that govern how ISPs operate as businesses?
Finally, the scalability challenge is aggravated by the lack of any Finally, the scalability challenge is aggravated by the lack of any
limiting architectural upper-bound on the growth rate and a weakening limiting architectural upper-bound on the growth rate and a weakening
of traditional social constraints on the growth rate that have helped of traditional social constraints on the growth rate that have helped
restrain growth so far. Going forward, there is considerable restrain growth so far. Going forward, there is considerable
uncertainty whether future growth rates will continue to be uncertainty whether future growth rates will continue to be
skipping to change at page 7, line 49 skipping to change at page 7, line 49
expense) when a new business opportunity is enabled as a result of an expense) when a new business opportunity is enabled as a result of an
upgrade. For example, an upgrade might be justified by an ability to upgrade. For example, an upgrade might be justified by an ability to
support increased traffic or an increase in the number of customer support increased traffic or an increase in the number of customer
connections, etc., where the upgrade can translate into increased connections, etc., where the upgrade can translate into increased
revenue. In contrast, it is more difficult to justify unplanned revenue. In contrast, it is more difficult to justify unplanned
upgrades in the absence of corresponding customer benefit (and upgrades in the absence of corresponding customer benefit (and
revenue) to cover the upgrade cost. It is generally desired that revenue) to cover the upgrade cost. It is generally desired that
deployed equipment remain usable over its planned lifetime. An deployed equipment remain usable over its planned lifetime. An
increase in the resources required to support larger or more dynamic increase in the resources required to support larger or more dynamic
routing tables is viewed as a sort of "unfunded mandate", in that routing tables is viewed as a sort of "unfunded mandate", in that
customers do not expect to have to pay more just to continue customers do not expect to have to pay more just to retain the same
retaining the same level of service as before, i.e., having all level of service as before, i.e., having all destinations be
destinations be reachable as was the case in the past. This reachable as was the case in the past. This undermining of planning
undermining of planning is particularly problematic when the increase is particularly problematic when the increase in routing demand
in routing demand originates external to the ISP, and the ISP has no originates external to the ISP, and the ISP has no way to control or
way to control or limit it (e.g., the increased demand comes from limit it (e.g., the increased demand comes from being part of the
being part of the DFZ). DFZ).
From a business perspective, it is desirable to maintain or increase From a business perspective, it is desirable to maintain or increase
the useful lifespan of routing equipment, by improving the scaling the useful lifespan of routing equipment, by improving the scaling
properties of the routing and addressing system. properties of the routing and addressing system.
3.3. Alignment of Incentives 3.3. Alignment of Incentives
Today's growth pattern is influenced by the scaling properties of the Today's growth pattern is influenced by the scaling properties of the
current system. If the system had better scaling properties, we current system. If the system had better scaling properties, we
would be able support and enable more widespread usage of certain would be able support and enable more widespread usage of certain
skipping to change at page 10, line 30 skipping to change at page 10, line 30
Outbound TE is typically accomplished by using internal IGP metrics Outbound TE is typically accomplished by using internal IGP metrics
to choose the shortest exit for two equally good BGP paths. to choose the shortest exit for two equally good BGP paths.
Adjustment of IGP metrics controls how much traffic flows over Adjustment of IGP metrics controls how much traffic flows over
different internal paths to specific exit points for two equally good different internal paths to specific exit points for two equally good
BGP paths. Additional traffic can be moved by applying some policy BGP paths. Additional traffic can be moved by applying some policy
to depreference or filter certain routes from specific BGP peers. to depreference or filter certain routes from specific BGP peers.
Because outbound TE is achieved via a site's own IGP, outbound TE Because outbound TE is achieved via a site's own IGP, outbound TE
does not impact routing outside of a site. does not impact routing outside of a site.
Inbound TE is performed by announcing a more specific route along the Inbound TE is performed by announcing a more-specific route along the
preferred path that "catches" the desired traffic and channels it preferred path that "catches" the desired traffic and channels it
away from the path it would take otherwise (i.e., via a larger away from the path it would take otherwise (i.e., via a larger
aggregate). At the BGP level, if the address range requiring TE is a aggregate). At the BGP level, if the address range requiring TE is a
portion of a larger address aggregate, network operators implementing portion of a larger address aggregate, network operators implementing
TE are forced to de-aggregate otherwise aggregatable prefixes in TE are forced to de-aggregate otherwise aggregatable prefixes in
order to steer the traffic of the particular address range to order to steer the traffic of the particular address range to
specific paths. specific paths.
TE is performed by both ISPs and customer networks, for three primary TE is performed by both ISPs and customer networks, for three primary
reasons: reasons:
skipping to change at page 11, line 34 skipping to change at page 11, line 34
also be a requirement due to contract or law. also be a requirement due to contract or law.
Multihoming can be accomplished using either PI or PA address space. Multihoming can be accomplished using either PI or PA address space.
A multihomed site advertises its site prefix into the routing system A multihomed site advertises its site prefix into the routing system
of each of its providers. For PI space, the site's PI space is used, of each of its providers. For PI space, the site's PI space is used,
and the prefix is propagated throughout the DFZ. For PA space, the and the prefix is propagated throughout the DFZ. For PA space, the
PA site prefix may (or may not) be propagated throughout the DFZ, PA site prefix may (or may not) be propagated throughout the DFZ,
with the details depending on what type of multihoming is sought. with the details depending on what type of multihoming is sought.
If the site uses PA space, the PA site prefix allocated from one of If the site uses PA space, the PA site prefix allocated from one of
its provider's (whom we'll call the Primary Provider) is used. The its providers (whom we'll call the Primary Provider) is used. The PA
PA site prefix will be aggregatable by the Primary Provider but not site prefix will be aggregatable by the Primary Provider but not the
the others. To achieve the same level of multihoming as described in others. To achieve the same level of multihoming as described in the
the case with PI addresses above, the PA site prefix will need to be case with PI addresses above, the PA site prefix will need to be
injected into the routing system of all of its ISPs, and throughout injected into the routing system of all of its ISPs, and throughout
the DFZ. In addition, because of the longest-match forwarding rule, the DFZ. In addition, because of the longest-match forwarding rule,
the Primary Provider must also advertise and propagate the individual the Primary Provider must also advertise and propagate the individual
PA site prefix; otherwise, the path via the primary provider (as PA site prefix; otherwise, the path via the primary provider (as
advertised via the aggregate) will never be selected due to the advertised via the aggregate) will never be selected due to the
longest match rule. For the type of multihoming described here, longest match rule. For the type of multihoming described here,
where the PA site prefix is propagated throughout the DFZ, the use of where the PA site prefix is propagated throughout the DFZ, the use of
PI vs. PA space has no impact on the control plane load. The PI vs. PA space has no impact on the control plane load. The
increased load is due entirely to the need to propagate the site's increased load is due entirely to the need to propagate the site's
individual prefix into the DFZ. individual prefix into the DFZ.
skipping to change at page 13, line 11 skipping to change at page 13, line 11
from its RIR. In order to be able to obtain additional address space from its RIR. In order to be able to obtain additional address space
that can be aggregated with the previously-allocated address space, that can be aggregated with the previously-allocated address space,
the RIR must keep a reserve of space that the requester can grow into the RIR must keep a reserve of space that the requester can grow into
in the future. But any reserved address space cannot be used for any in the future. But any reserved address space cannot be used for any
other purpose. Hence, there is an inherent conflict between holding other purpose. Hence, there is an inherent conflict between holding
address space in reserve to allow for the future growth of an address space in reserve to allow for the future growth of an
existing allocation and using address space efficiently. In IPv4, existing allocation and using address space efficiently. In IPv4,
there has been a heavy emphasis on conserving address space and there has been a heavy emphasis on conserving address space and
obtaining efficient utilization. Consequently, insufficient space obtaining efficient utilization. Consequently, insufficient space
has been held in reserve to allow for the growth of all sites and has been held in reserve to allow for the growth of all sites and
some allocations have had to me made from discontiguous address some allocations have had to be made from discontiguous address
blocks. For IPv6, a greater emphasis has been placed on aggregation. blocks. For IPv6, a greater emphasis has been placed on aggregation.
4.6. Dual Stack Pressure on the Routing Table 4.6. Dual Stack Pressure on the Routing Table
The recommended IPv6 deployment model is dual-stack, where IPv4 and The recommended IPv6 deployment model is dual-stack, where IPv4 and
IPv6 are run in parallel across the same links. This has two IPv6 are run in parallel across the same links. This has two
implications for routing. First, although alternative scenarios are implications for routing. First, although alternative scenarios are
possible, it seems likely that many routers will be supporting both possible, it seems likely that many routers will be supporting both
IPv4 and IPv6 simultaneously and will thus be managing both IPv4 and IPv4 and IPv6 simultaneously and will thus be managing both IPv4 and
IPv6 routing tables within a single router. Second, for sites IPv6 routing tables within a single router. Second, for sites
connected via both IPv4 and IPv6, both IPv4 and IPv6 prefixes will connected via both IPv4 and IPv6, both IPv4 and IPv6 prefixes will
need to be propagated into the routing system. Consequently, dual- need to be propagated into the routing system. Consequently, dual-
stack routers will maintain both an IPv4 and IPv6 route to reach the stack routers will maintain both an IPv4 and IPv6 route to reach the
same destination. same destination.
It is possible to make some simple estimates on the approximate size It is possible to make some simple estimates on the approximate size
of the IPv6 tables that would be needed if all sites reachable via of the IPv6 tables that would be needed if all sites reachable via
IPv4 today were also reachable via IPv6. In theory, each autonomous IPv4 today were also reachable via IPv6. In theory, each autonomous
system (AS) needs only a single aggregate route. This provides a system (AS) needs only a single aggregate route. This provides a
lower bound on the size of the fully-realized IPv6 routing table. lower bound on the size of the fully-realized IPv6 routing table.
(As of July 2007, [CIDR4] states there are 25,836 active ASes in the (As of July 2007, [3] states there are 25,836 active ASes in the
routing system.) routing system.)
A single IPv6 aggregate will not allow for inbound traffic A single IPv6 aggregate will not allow for inbound traffic
engineering. End sites will need to advertise a number of smaller engineering. End sites will need to advertise a number of smaller
prefixes into the DFZ if they desire to gain finer grained control prefixes into the DFZ if they desire to gain finer grained control
over their IPv6 inbound traffic. This will increase the size of the over their IPv6 inbound traffic. This will increase the size of the
IPv6 routing table beyond the lower bound discussed above. There is IPv6 routing table beyond the lower bound discussed above. There is
reason to expect the IPv6 routing table will be smaller than the reason to expect the IPv6 routing table will be smaller than the
current IPv4 table, however, because the larger initial assignments current IPv4 table, however, because the larger initial assignments
to end sites will minimize the de-aggregation that occurs when a site to end sites will minimize the de-aggregation that occurs when a site
must go back to its upstream address provider or RIR and receive a must go back to its upstream address provider or RIR and receive a
second, non-contiguous assignment. second, non-contiguous assignment.
It is possible to extrapolate what the size of the IPv6 Internet It is possible to extrapolate what the size of the IPv6 Internet
routing table would be if widespread IPv6 adoption occurred, from the routing table would be if widespread IPv6 adoption occurred, from the
current IPv4 Internet routing table. Each active AS (25,836) would current IPv4 Internet routing table. Each active AS (25,836) would
require at least one aggregate. In addition, the IPv6 Internet table require at least one aggregate. In addition, the IPv6 Internet table
would also carry more specific prefixes for traffic engineering. would also carry more-specific prefixes for traffic engineering.
Assume that the IPv6 Internet table will carry the same number of Assume that the IPv6 Internet table will carry the same number of
more specifics as the IPv4 Internet table. In this case one can take more specifics as the IPv4 Internet table. In this case one can take
the number of IPv4 Internet routes and subtract the number of CIDR the number of IPv4 Internet routes and subtract the number of CIDR
aggregates that they could easily be aggregated down to. As of July aggregates that they could easily be aggregated down to. As of July
2007, the 229,789 routes can be easily aggregated down to 150,018 2007, the 229,789 routes can be easily aggregated down to 150,018
CIDR aggregates [CIDR4]. That difference yields 79,771 extra more CIDR aggregates [3]. That difference yields 79,771 extra more-
specific prefixes. Thus if each active AS (25,836) required one specific prefixes. Thus if each active AS (25,836) required one
aggregate, and an additional 79,771 more specifics were required, aggregate, and an additional 79,771 more specifics were required,
then the IPv6 Internet table would be 105,607 prefixes. then the IPv6 Internet table would be 105,607 prefixes.
4.7. Internal Customer Routes 4.7. Internal Customer Routes
In addition to the Internet routing table, networks must also carry In addition to the Internet routing table, networks must also carry
their internal routing table. Internal routes are defined as more their internal routing table. Internal routes are defined as more-
specific routes that are not advertised to the DFZ. This primarily specific routes that are not advertised to the DFZ. This primarily
consists of prefixes that are a more specific of a provider aggregate consists of prefixes that are a more-specific of a provider aggregate
(PA) and are assigned to a single homed customer. The DFZ need only (PA) and are assigned to a single homed customer. The DFZ need only
carry the PA aggregate in order to deliver traffic to the provider. carry the PA aggregate in order to deliver traffic to the provider.
However, the provider's routers require the more specific route to However, the provider's routers require the more-specific route to
deliver traffic to the end site. deliver traffic to the end site.
This could also consist of more specific prefixes advertised by This could also consist of more-specific prefixes advertised by
multi-homed customers with the no-export community. This is useful multihomed customers with the no-export community. This is useful
when the fine grained control of traffic to be influenced can be when the fine grained control of traffic to be influenced can be
contained to the neighboring network. contained to the neighboring network.
For a large ISP, the internal IPv4 table can be between 50,000 and For a large ISP, the internal IPv4 table can be between 50,000 and
150,000 routes. During the dot com boom some ISPs had more internal 150,000 routes. During the dot com boom some ISPs had more internal
prefixes than there were in the Internet table. Thus the size of the prefixes than there were in the Internet table. Thus the size of the
internal routing table can have significant impact on the scalability internal routing table can have significant impact on the scalability
and should not be discounted. and should not be discounted.
4.8. IPv4 Address Exhaustion 4.8. IPv4 Address Exhaustion
The IANA and RIR free pool of IPv4 addresses will be exhausted within The IANA and RIR free pool of IPv4 addresses will be exhausted within
a few years. As the free pool shrinks, the size of the remaining a few years. As the free pool shrinks, the size of the remaining
unused blocks will also shrink and unused blocks previously held in unused blocks will also shrink and unused blocks previously held in
reserve for expansion of existing allocations or otherwise not used reserve for expansion of existing allocations or otherwise not used
due to their smaller size will be allocated for use. Consequently, due to their smaller size will be allocated for use. Consequently,
as the community looks to use use every piece of available address as the community looks to use every piece of available address space
space (no matter how small) there will be an increasing pressure to (no matter how small) there will be an increasing pressure to
advertise additional prefixes in the DFZ. advertise additional prefixes in the DFZ.
5. Pressures on Control Plane Load 5. Pressures on Control Plane Load
This section describes a number of trends and pressures that are This section describes a number of trends and pressures that are
contributing to the overall load of computing Internet paths. The contributing to the overall load of computing Internet paths. The
previous section described pressures that are increasing the size of previous section described pressures that are increasing the size of
the routing table. Even if the size could be bounded, the amount of the routing table. Even if the size could be bounded, the amount of
work needed to maintain paths for a given set of prefixes appears to work needed to maintain paths for a given set of prefixes appears to
be increasing. be increasing.
5.1. Interconnection Richness 5.1. Interconnection Richness
The degree of interconnectedness between ASes has increased in recent The degree of interconnectedness between ASes has increased in recent
years. That is, the Internet as whole is becoming "flatter" with an years. That is, the Internet as whole is becoming "flatter" with an
increasing number of possible paths interconnecting sites [3]. As increasing number of possible paths interconnecting sites [4]. As
the number of possible paths increase, the amount of computation the number of possible paths increase, the amount of computation
needed to find a best path also increases. This computation comes needed to find a best path also increases. This computation comes
into effect whenever a change in path characteristics occurs, whether into effect whenever a change in path characteristics occurs, whether
from a new path becoming available, an existing path failing, or a from a new path becoming available, an existing path failing, or a
change in the attributes associated with a potential path. Thus, change in the attributes associated with a potential path. Thus,
even if the total number of prefixes were to stay constant, an even if the total number of prefixes were to stay constant, an
increase in the interconnection richness implies an increase in the increase in the interconnection richness implies an increase in the
resources needed to maintain routing tables. resources needed to maintain routing tables.
5.2. Multihoming 5.2. Multihoming
Multihoming places pressure on the routing system in two ways. Multihoming places pressure on the routing system in two ways.
First, an individual prefix for a multihomed site (whether PI or PA) First, an individual prefix for a multihomed site (whether PI or PA)
must be propagated into the routing system, so that other sites can must be propagated into the routing system, so that other sites can
find a good path to the site. Even if the site's prefix comes out of find a good path to the site. Even if the site's prefix comes out of
a PA block, an individual prefix for the site needs to be advertised a PA block, an individual prefix for the site needs to be advertised
so that the most desirable path to the site can be chosen when the so that the most desirable path to the site can be chosen when the
path through the aggregate is sub-optimal. Second, a multi-homed path through the aggregate is sub-optimal. Second, a multihomed site
site will be connected to the Internet in more than one place, will be connected to the Internet in more than one place, increasing
increasing the overall level of interconnection richness. If an the overall level of interconnection richness. If an outage occurs
outage occurs on any of the circuits connecting the site to the on any of the circuits connecting the site to the Internet, those
Internet, those changes will be propagated into the routing system. changes will be propagated into the routing system. In contrast, a
In contrast, a singly-homed site numbered out of a Provider Aggregate singly-homed site numbered out of a Provider Aggregate places no
places no additional control plane load in the DFZ as the details of additional control plane load in the DFZ as the details of the
the connectivity status to the site are kept internal to the provider connectivity status to the site are kept internal to the provider to
to which it connects. which it connects.
5.3. Traffic Engineering 5.3. Traffic Engineering
The mechanisms used to achieve multihoming and inbound Traffic The mechanisms used to achieve multihoming and inbound Traffic
Engineering are the same. In both cases, a specific prefix is Engineering are the same. In both cases, a specific prefix is
advertised into the routing system to "catch" traffic and route it advertised into the routing system to "catch" traffic and route it
over a different path than it would otherwise be carried. When over a different path than it would otherwise be carried. When
multihoming, the specific prefix is one that differs from that of its multihoming, the specific prefix is one that differs from that of its
ISP or is a more-specific of the ISP's PA. Traffic Engineering is ISP or is a more-specific of the ISP's PA. Traffic Engineering is
achieved by taking one prefix and dividing into a number of smaller achieved by taking one prefix and dividing it into a number of
and more-specific ones, and advertising them in order to gain finer- smaller and more-specific ones, and advertising them in order to gain
grained control over the paths used to carry traffic covered by those finer-grained control over the paths used to carry traffic covered by
prefixes. those prefixes.
Traffic Engineering increases the number of prefixes carried in the Traffic Engineering increases the number of prefixes carried in the
routing system. In addition, when a circuit fails (or the routing routing system. In addition, when a circuit fails (or the routing
attributes associated with the circuit change), additional load is attributes associated with the circuit change), additional load is
placed on the routing system by having multiple prefixes potentially placed on the routing system by having multiple prefixes potentially
impacted by the change, as opposed to just one. impacted by the change, as opposed to just one.
5.4. Questionable Operational Practices? 5.4. Questionable Operational Practices?
Some operators are believed to engage in operational practices that Some operators are believed to engage in operational practices that
skipping to change at page 16, line 41 skipping to change at page 16, line 41
increases the load on the routing system, as any change in what is increases the load on the routing system, as any change in what is
advertised must ripple through the entire routing system. advertised must ripple through the entire routing system.
5.4.2. Anti-Route Hijacking 5.4.2. Anti-Route Hijacking
In order to reduce the threat of accidental (or intentional) In order to reduce the threat of accidental (or intentional)
hijacking of its address space by an unauthorized third party, some hijacking of its address space by an unauthorized third party, some
sites advertise their space as a set of smaller prefixes rather than sites advertise their space as a set of smaller prefixes rather than
as one aggregate. That way, if someone else advertised a path for as one aggregate. That way, if someone else advertised a path for
the larger aggregate (or a small piece of the aggregate), it will be the larger aggregate (or a small piece of the aggregate), it will be
ignored in favor of the more specific announcements. This increases ignored in favor of the more-specific announcements. This increases
both the number of prefixes advertised, and the number of updates. both the number of prefixes advertised, and the number of updates.
5.4.3. Operational Ignorance 5.4.3. Operational Ignorance
It is believed that some undesirable practices result from operator It is believed that some undesirable practices result from operator
ignorance, where the operator is unaware of what they are doing and ignorance, where the operator is unaware of what they are doing and
the impact that has on the DFZ. the impact that has on the DFZ.
The default behavior of most BGP configurations is to automatically The default behavior of most BGP configurations is to automatically
propagate all learned routes. That is, one must take explicit propagate all learned routes. That is, one must take explicit
skipping to change at page 18, line 46 skipping to change at page 18, line 46
shift the load from one place in the system to another, without shift the load from one place in the system to another, without
reducing the overall load as a whole. reducing the overall load as a whole.
3. Allows any end site wishing to multihome to do so 3. Allows any end site wishing to multihome to do so
4. Supports ISP and enterprise TE needs 4. Supports ISP and enterprise TE needs
5. Allows end sites to switch providers while minimizing 5. Allows end sites to switch providers while minimizing
configuration changes to internal end site devices. configuration changes to internal end site devices.
6. Provides meaningful benefits to the parties who bear the costs of 6. Provides end-to-end convergence/restoration of service at least
deploying and maintaining the technology. comparable to that provided by the current architecture
The problem statement in this document has purposefully been scoped The problem statement in this document has purposefully been scoped
to focus on the growth of the routing update function of the DFZ. to focus on the growth of the routing update function of the DFZ.
Other problems that may seem related, but do not directly impact the Other problems that may seem related, but do not directly impact the
route scaling problem are not considered to be "in scope" at this route scaling problem are not considered to be "in scope" at this
time. For example, Mobile IP [[RFC2002], RFC3775] and NEMO time. For example, Mobile IP [RFC3344] [RFC3775] and NEMO [RFC3963]
[[RFC3963]] place no pressures on the routing system. They are place no pressures on the routing system. They are layered on top of
layered on top of existing IP, using tunneling to forward packets via existing IP, using tunneling to forward packets via a care-of
a care-of addresses. Hence, "improving" these technologies (e.g., by addresses. Hence, "improving" these technologies (e.g., by having
having them leverage a solution to the multihoming problem), while a them leverage a solution to the multihoming problem), while a
laudable goal, is not considered a part of this problem statement. laudable goal, is not considered a part of this problem statement.
7. Security Considerations 7. Security Considerations
TBD TBD
8. Acknowledgments 8. IANA Considerations
This document contains no IANA actions.
9. Acknowledgments
The initial version of this document was produced by the Routing and The initial version of this document was produced by the Routing and
Addressing Directorate (http://www.ietf.org/IESG/content/radir.html). Addressing Directorate (http://www.ietf.org/IESG/content/radir.html).
The membership of the directorate at that time included Marla The membership of the directorate at that time included Marla
Azinger, Vince Fuller, Vijay Gill, Thomas Narten, Erik Nordmark, Azinger, Vince Fuller, Vijay Gill, Thomas Narten, Erik Nordmark,
Jason Schiller, Peter Schoenmaker, and John Scudder. Jason Schiller, Peter Schoenmaker, and John Scudder.
Comments should be sent to ram@iab.org or to radir@ietf.org. Comments should be sent to ram@iab.org or to radir@ietf.org.
9. Informative References 10. Informative References
[RFC2002] Perkins, C., "IP Mobility Support", RFC 2002, [RFC3344] Perkins, C., "IP Mobility Support for IPv4", RFC 3344,
October 1996. August 2002.
[RFC3775] Johnson, D., Perkins, C., and J. Arkko, "Mobility Support
in IPv6", RFC 3775, June 2004.
[RFC3963] Devarapalli, V., Wakikawa, R., Petrescu, A., and P. [RFC3963] Devarapalli, V., Wakikawa, R., Petrescu, A., and P.
Thubert, "Network Mobility (NEMO) Basic Support Protocol", Thubert, "Network Mobility (NEMO) Basic Support Protocol",
RFC 3963, January 2005. RFC 3963, January 2005.
[RFC4116] Abley, J., Lindqvist, K., Davies, E., Black, B., and V.
Gill, "IPv4 Multihoming Practices and Limitations",
RFC 4116, July 2005.
[RFC4632] Fuller, V. and T. Li, "Classless Inter-domain Routing [RFC4632] Fuller, V. and T. Li, "Classless Inter-domain Routing
(CIDR): The Internet Address Assignment and Aggregation (CIDR): The Internet Address Assignment and Aggregation
Plan", BCP 122, RFC 4632, August 2006. Plan", BCP 122, RFC 4632, August 2006.
[RFC4984] Meyer, D., Zhang, L., and K. Fall, "Report from the IAB
Workshop on Routing and Addressing", RFC 4984,
September 2007.
[1] <http://www3.ietf.org/proceedings/06mar/slides/grow-3.pdf> [1] <http://www3.ietf.org/proceedings/06mar/slides/grow-3.pdf>
[2] <http://www.cidr-report.org/as2.0/, http://www.cidr-report.org/ [2] <http://www.cidr-report.org/as2.0/, http://www.cidr-report.org/
cgi-bin/ cgi-bin/
plota?file=%2fvar%2fdata%2fbgp%2fas2.0%2fbgp%2das%2dcount%2etxt& plota?file=%2fvar%2fdata%2fbgp%2fas2.0%2fbgp%2das%2dcount%2etxt&
descr=Unique%20ASes&ylabel=Unique%20ASes&with=step,http:// descr=Unique%20ASes&ylabel=Unique%20ASes&with=step,http://
www.potaroo.net/tools/asn32/> www.potaroo.net/tools/asn32/>
[3] <http://www.potaroo.net/bgprpts/bgp-average-aspath-length.png> [3] <http://www.cidr-report.org/as2.0/>
[4] <http://www.potaroo.net/bgprpts/bgp-average-aspath-length.png>
Author's Address Author's Address
Thomas Narten Thomas Narten
IBM IBM
Email: narten@us.ibm.com Email: narten@us.ibm.com
Full Copyright Statement Full Copyright Statement
 End of changes. 35 change blocks. 
74 lines changed or deleted 95 lines changed or added

This html diff was produced by rfcdiff 1.34. The latest version is available from http://tools.ietf.org/tools/rfcdiff/