Internet Experiment Note No. 200
INTERNET PROJECT RESEARCH PLANNING REPORT
David D. Clark
MIT Laboratory for Computer Science
Computer Systems and Communications Group
1. Introduction
This paper represents a first attempt to provide a structure to the current
and planned activities in the Internet project. It is particularly concerned
with the evolution of the protocol architecture, and the identification of new
functions which the protocol is to support. It attempts to identify the basic
decisions which must be made as part of the planning process.
-N.B.: This document is a draft. It is distributed at this time for
comment and discussion only. Anyone using this document as a basis
for project planning at this stage is extremely misguided.
-The changes and extensions described here are not intended to have an
impact on current host implementations of TCP and IP, except for
hosts specifically involved in future research and demonstrations.
Implementors only involved in realizing the currently defined level
of service need not be concerned with this report.
The Internet project has reached a point where it must provide a stable and
usable service to its users. This is a painful but crucial stage in any system
project which hopes to prove its point in the real world. Thus, our planning
must now take into account two requirements, first that we provide a functional
service, and second that we develop and integrate new functions into the
existing architecture. There are several reasons why new functions must be
added to the existing architecture. First, there are additional service
requirements, which are not currently supported but which will be required
before the service meet the currently forseen needs. Second, there are
explicit research goals which have been established for the project, with
announced demonstration dates. Third, there are additional research goals
which are of interest to the participants in the project, and which are
generally perceived as being appropriate and compatible with the internet
philosophy. This last collection of topics can easily grow without bound,
2 leading to a project so large as to be intractible. One of the first planning issues to be addressed is precisely what problems we choose to solve in this next research cycle, and what problems we choose to ignore. 2. Possible Research Topics The following list catalogues a number of areas in which future evolution of the architecture has been proposed. The comments provided with each section attempt to give some idea of the amount of thought which has already gone into this topic, and the range of options available for approaching each topic. 2.1. Types of Transport Services: The issue here is what sort of services will the transport layers of the architecture provide to the applications above. Currently, TCP provides the only wide-spread service, a bidirectional, reliable data stream. It is unclear whether TCP is the most suitable protocol for high bandwidth, high delay occasions, such as bulk transfer of data, and it is certainly clear that TCP is inappropriate for speech and other low delay, loss-tolerant applications. Other functions which might be useful include multicast rather than point-to- point data transmission. Speech is a clearly defined goal for internet support, but it is currently being supported only outside the internet architecture, using alternative protocols. To provide serious speech support, IP may have to be evolved to provide the necessary support. Other services such as bulk, high bandwidth transfer and multi-casting are not specifically required for demonstration or service. Thus, it may be possible to ignore these issues. On the other hand, it is reasonable to consider exploring these topics because it is probable that they can be examined somewhat in isolation, without a global upheaval to the entire architecture. 2.2. Addressing and Routing: As we proceed from an assumption of an internet with 50 nets to an internet with 1,000 nets, substantial upheavals will occur in the architecture and its implementations. The simple routing algorithm of sending to every gateway the location of every net, would require exchanging tables with 1,000 entries, and not only would this cause an overflow of limited gateway storage space, but it would also use up a substantial part of the network bandwidth if the system were at all responsive. Currently, the internet has a vaguely hierarchical structure, in which at the top there are nets and beneath this level is a substructure which is understood only by the net itself. Presumably, it will
3 be necessary to generalize this structure by grouping nets into areas, which physically encompass several nets. However, this idea of areas is counter to an original internet goal, which is that individual networks should be connectable together in any configuration, with all possible physical paths being used as realizable routes through the internet. If there is a structure to the net, than only routes that conform to the structure will be realizable. I feel that a restriction of this sort on the internet is quite reasonable. An obvious structure for the internet is to imagine that there is a central cluster of nets which provide the function of long-haul transport. These transport nets would have connected to them additional sets of nets which provide an access function to the internet. Typical transport nets would be the ARPANET and SATNET: typical access nets would be local area nets and packet radio. The implication of this structure would be that the transport nets would never rely on the access nets for transport function. An additional complication of routing is the TYPE OF SERVICE field of IP. The TOS field is presumably used to affect the service provided by the individual nets: however, it also should serve the function of selecting among routes at the internet level. Currently, this function is completely missing. Adding it can only make the routing problem much more complicated. 2.3. Flow and Congestion Control: Currently, the internet has a simple mechanism for congestion control, the Source Quench Packet, which can variously be viewed as a choke packet or as an advisory overload packet. There is no evidence that this mechanism works very well; indeed there are substantial indications that it probably does not. As load builds up in the internet, and especially as we hook together networks whose basic speeds differ by orders of magnitude, it will be necessary to identify and implement a workable congestion mechanism. One decision currently undecided about the internet architecture is whether the congestion mechanism should include the concept of "enforcement". Most congestion mechanisms push back on their data sources in a manner that is not advisory, but mandatory. For example, networks selectively drop packets, disable communication, or simply refuse certain inputs. The internet can do none of these. If an input host ignores the congestion information currently offered, the increased degradation caused by this is not focused on the offending host, but is randomly distributed on all the traffic in the affected area. It would seem that some mechanism for enforcement would be appropriate. However, enforcement is very difficult, given one of the basic assumptions of the architecture, which is that the gateways have no state. Without state, it
4 is impossible to keep track of whichthe offending host is so that it can be discriminated against. I feel that an important idea in this respect is that of "soft state", in which the gateways attempt to build up a model of the world by observing the traffic passing through them. If they should crash and lose this state information, no transport services is disrupted, but as long as the information exists, it permits the gateway to discard selectively those packets which appear to be violating the congestion control restrictions currently in effect. 2.4. Distributed Control: An important question about the internet is the extent to which its ownership and management is decentralized. It was originally a design goal that various gateways would be implemented by different organizations, and the individual networks out of which the internet is built were certainly expected to be managed by separate organizations. Realistically, decentralized management is an important goal in the long term. However, it clearly makes development and operation much more difficult. Currently, essentially all of the central gateways as well as the important transport nets are all operated by BBN. A specific decision is required as to whether this situation is to be exploited, in order to simplify the next round of implementations, or whether distributed operation is to be deliberately injected into the system as an additional research goal. A problem closely related to this is the interaction between the existing service internet and some evolved internet being used as a testbed for these advanced topics. It will, at a minimum, be necessary to construct some sort of firewall between the experimental environment and the service environment, so that they can interact without destroying each other. 2.5. Partitioned Networks This is a critical research goal, because it has been explicitly identified for demonstration in the two to three year timeframe. The specific demonstration involves attaching high power packet radios to the ARPANET at appropriate locations, providing airborne packet radios which are able to communicate with these ground based radios, and then severing the landlines in the ARPANET and using the packet radios to reconstitute ARPANET service. There are two ways to approach this problem: at the network level or at the internetwork level. The network level solution, in which the imps are taught about the existence of the packet radio links, is clearly the simpler of the
5 two. Among other things, it requires no modification of the host software. The internetwork solution, in which knowledge of the packet radio links is restricted to the hosts and the gateways, is a more challenging solution, but one which is more in line with the original goals of the current cycle of research. I propose that the problem be solved at the internetwork level, but this requires an upgrade in the sophistication of the host routing algorithm, so that failures to communicate with the host apparently located on the same network trigger the kind of internet rerouting which would normally be invoked only for a pathway known to pass through intermediate gateways. There are other problems that resemble net partition. The "expressway routing" concept is that a host or gateway may select an internet path to bypass a network, even when the net is fully functional. This sort of action would require that the mechanisms designed to support partition be used under normal operations, not just under crisis conditions. 2.6. Improved Effectiveness: This somewhat vague title covers a number of important topics. Currently, the TCP specification describes logically correct operations, but makes only limited reference to efficient operation. Issues such as window management and the timing of acknowledgements are left as an exercise for the implementor. Bad design decisions in this area lead to the symptoms sometimes referred to as Silly Window Syndrome. It is important that the specification be expanded to include sufficient information to prevent this sort of inappropriate operation. An area in which this will become immediately visible is in the use of public data networks to carry internet packets. Most tariffs for public nets are based on the number of packets, and the window algorithms and acknowledgment generation algorithms strongly influence the number of packets required to send a given amount of data. There will be a pressing cost requirement to eliminate inappropriate implementations. Even where cost is not measured in dollars, the perceived effectiveness of these protocols is an important component of their acceptability to potential users. 2.7. Access Control: Plans are already underway in the short term to provide some controls over who can use the internet, in particular by the addition of passwords to the TAC. However, in the long run, much more must be done. The current protection strategy is based on the assumption that individual hosts are sufficiently responsible to restrain their users. As long as we have large time-shared computers, this assumption is still somewhat workable. However, we are moving
6 as rapidly as possible away from this position to a position in which personal computers are allocated to individual users, which provides no management strategy to ensure that the individual users of these computers, who are presumably attached to local nets over which no access control is imposed, refrain from going through these local nets to reach long haul nets whose use is limited. The most extreme example of this currently in the internet are the personal computers belonging to students at universities such as M.I.T. As part of M.I.T.'s policy toward its local net, these students are being invited to attach to the local net as rapidly as possible. However, the local net is attached to the ARPANET, and there is no mechanism which can prevent these students form utilizing the ARPANET from their personal computers. Clearly, it is ridiculous to expect the students to voluntarily refrain. The only possible mechanism is an access control algorithm in the gateway. But by the basic architectural assumptions of internet, this is almost impossible to provide, because the gateway lacks the information to discriminate between legitimate and undesirable users. The only information available to the gateway are the original and destination addresses, but short of a complete list enumerating all of the used addresses at M.I.T., which might potentially be very long, the address is not a sufficient indicator of validity. A decision must be made whether or not this problem is to be solved. If it is, some substantial work will be required. Access control and routing interact in an important way in the gateway. The access decision is not a simple yes/no control, but a selection of route based on privilege. For some nets, it will also be necessary to collect billing information, which requires the same sort of unforgable identification as does access control. 2.8. Performance Evaluation: There are currently a number of activities under way to evaluate the performance of the internet. Some of these are being done as part of operational management of the internet. Others are being done by various users of the internet from their particular vantage point to evaluate the service made available to them. These projects are not especially well coordinated at the moment, and the results they gather are not being used in other than a very general sense to evaluate and identify problems with the internet. Currently, it is difficult to improve the quality of the measurements being made, because there is not space available in the gateways to improve the metering and instrumentation which is installed there. A new version of the gateway code is now being developed by BBN which will have more space for new code. At a more philosophical level, we lack even a metric against which to compare
7 our work. What constitutes a "good" internet? Superficially, one thinks of things like low delay, high bandwidth, and reliability. But actually, depending on the class of service being used, these individual goals may or may not be desirable. In fact, a good internet is probably one which has the flexibility to conform to a variety of offered loads, rather than doing a superb job of meeting exactly one application class. In the long run, it would be worthwhile trying to identify what goal we are attempting to meet, before rather than after we meet it. 3. Discussion As the preceding list suggests, it is possible to put together a wish list of internet features which is unworkably large. One of the first problems we must face is doing some delicate but effective pruning upon this list to bring it to a managable size. Having done this, it will be necessary to make some rather difficult decisions about the general structure into which our future solutions must fit. The above list raises some very important questions about the shape of the internet. For example, the partitioning question most clearly raises the issue of whether or not hosts connected to each other over a single net view themselves as being connected to a network or to an internet. It is silly to proceed until we have an answer to that question. Two other important questions are the extent to which constraints are imposed on the workable topologies of the internet, and the extent to which the internet is assumed to be owned and operated by one or many organizations. The short time schedule for the announced demonstrations require that progress on this list of functions be fairly rapid. The need for quick progress suggests two conclusions to me. We will not achieve our goals if we attempt to get there by a series of small incremental steps from where we are now. Many of the functional requirements listed above interact with each other in a very strong way, and it will be necessary to take all of these requirements into account as we do a design for the final service. An incremental approach will only trick us into thinking that we could ignore some of these issues and attack others first. I believe that will not work in this case. We must attempt to stabilize the service we have now and then live with that service for some time, perhaps a year or more, during which time we develop the followup, which will ultimately be used for the demonstrations scheduled in the two to three year time frame. Given this conclusion, it then follows we should attempt to minimize those functions which we advertise as part of the current service we are attempting to stabilize, because any effort investigated in expanding the current service is effort which is diverted from the long-term design problem.
8 4. Milestones It is my hope that a working version of this document will be produced not later than the meeting of the ICCB in January. Anyone with comments is requested to send them to DClark at MIT-Multics before that time. -------
mirror server hosted at Truenetwork, Russian Federation.