THE CRUCIBLE                                                  INTERNET EDITION
August, 1989                                                Volume 1 : Issue 1
                                                                 (reprint)
   In this issue:
	A Critical Analysis of the Internet Management Situation

   THE CRUCIBLE is a moderated forum for the discussion of Internet issues.
   Contributions received by the moderator are stripped of all identifying
   headers and signatures and forwarded to a panel of referees.  Materials
   approved for publication will appear in THE CRUCIBLE without attribution.
   This policy encourages consideration of ideas solely on their intrinsic
   merit, free from the influences of authorship, funding sources and
   organizational affiliations.

   THE INTERNET CRUCIBLE is an eleemosynary publication of Geoff Goodfellow.

Mail contributions to:	                         crucible@fernwood.mpk.ca.us

------------------------------------------------------------------------------

	 A Critical Analysis of the Internet Management Situation:
		       The Internet Lacks Governance


				  ABSTRACT

At its July 1989 meeting, the Internet Activities Board made some
modifications in the management structure for the Internet.  An outline of
the new IAB structure was distributed to the Internet engineering community
by Dr. Robert Braden, Executive Director.  In part, the open letter stated:

   "These changes resulted from an appreciation of our successes, especially
    as reflected in the growth and vigor of the IETF, and in rueful
    acknowledgment of our failures (which I will not enumerate).  Many on
    these lists are concerned with making the Internet architecture work in
    the real world."

In this first issue of THE INTERNET CRUCIBLE we will focus on the failures
and shortcomings in the Internet.  Failures contain the lessons one often
needs to achieve success.  Success rarely leads to a search for new
solutions.  Recommendations are made for short and long term improvements 
to the Internet.


			A Brief History of Networking

	The Internet grew out of the early pioneering work on the ARPANET.
This influence was more than technological, the Internet has also been
significantly influenced by the economic basis of the ARPANET.

	The network resources of the ARPANET (and now Internet) are "free".
There are no charges based on usage (unless your Internet connection is via
an X.25 Public Data Network (PDN) in which case you're well endowed, or
better be).  Whether a site's Internet connection transfers 1 packet/day or
a 1M packets/day, the "cost" is the same.  Obviously, someone pays for the
leased lines, router hardware, and the like, but this "someone" is, by and
large, not the same "someone" who is sending the packets.

	In the context of the Research ARPANET, the "free use" paradigm was
an appropriate strategy, and it has paid handsome dividends in the form of
developing leading edge packet switching technologies.  Unfortunately,
there is a significant side-effect with both the management and technical
ramifications of the current Internet paradigm: there is no accountability,
in the formal sense of the word.

	In terms of management, it is difficult to determine who exactly is
responsible for a particular component of the Internet.  From a technical
side, responsible engineering and efficiency has been replaced by the
purchase of T1 links.

	Without an economic basis, further development of short-term
Internet technology has been skewed.  The most interesting innovations in
Internet engineering over the last five years have occurred in resource
poor, not resource rich, environments.

	Some of the best known examples of innovative Internet efficiency
engineering are John Nagle's tiny-gram avoidance and ICMP source-quench
mechanisms documented in RFC896, Van Jacobsen's slow-start algorithms and
Phil Karn's retransmission timer method.

	In the Nagle, Jacobsen and Karn environments, it was not possible
or cost effective to solve the performance and resource problems by simply
adding more bandwidth -- some innovative engineering had to be done.
Interestingly enough, their engineering had a dramatic impact on our
understanding of core Internet technology.

	It should be noted that highly efficient networks are important
when dealing with technologies such as radio where there is a finite amount
of bandwidth/spectrum to be had.  As in the Nagle, Jacobsen and Karn cases,
there are many environments where adding another T1 link can not be used to
solve the problem.  Unless innovation continues in Internet technology, our
less than optimal protocols will perform poorly in bandwidth or resource
constrained environments.

	Developing at roughly the same time as Internet technology have
been the "cost-sensitive" technologies and services, such as the various
X.25-based PDNs, the UUCP and CSNET dial-up networks.  These technologies
are all based on the notion that bandwidth costs money and the subscriber
pays for the resources used.  This has the notable effect of focusing
innovation to control costs and maximize efficiency of available resources
and bandwidth.  Higher efficiency is achieved by concentrating on sending
the most amount of information through the pipe in the most efficient
manner thereby making the best use of available bandwidth/cost ratio.

	For example, bandwidth conservation in the UUCP dial-up network has
multiplied by leaps and bounds in the modem market with the innovation of
Paul Baran's (the grandfather of packet switching technology) company,
Telebit, which manufactures a 19.2KB dial-up modem especially optimized for
UUCP and other well known transfer protocols.  For another example,
although strictly line-at-a-time terminal sessions are less "user friendly"
than character-oriented sessions, they make for highly efficient use of
X.25 PDN network resources with echoing and editing performed locally on
the PAD.

	While few would argue the superiority of X.25 and dial-up CSNET and
UUCP, these technologies have proved themselves both to spur innovation and
to be accountable.  The subscribers to such services appreciate the cost of
the services they use, and often such costs form a well-known "line item"
in the subscriber's annual budget.

	Nevertheless, the Internet suite of protocols are eminently
successful, based solely on the sheer size and rate of growth of both the
Internet and the numerous private internets, both domestically and
internationally.  You can purchase internet technology with a major credit
card from a mail order catalog.  Internet technology has achieved the
promise of Open Systems, probably a decade before OSI will be able to do so.


			   Failures of the Internet

	The evolution and growth of Internet technology have provided the
basis for several failures.  We think it is important to examine failures
in detail, so as to learn from them.  History often tends to repeat itself.


Failure 1:- Network Nonmanagement

	The question of responsibility in todays proliferated Internet is
completely open.  For the last three years, the Internet has been suffering
from non-management.  While few would argue that a centralized czar is
necessary (or possible) for the Internet, the fact remains there is little
to be done today besides finger-pointing when a problem arises.
 
	In the NSFNET, MERIT is incharge of the backbone and each regional
network provider is responsible for its respective area.  However, trying
to debug a networking problem across lines of responsibility, such as
intermittent connectivity, is problematic at best.  Consider three all too
true refrains actually heard from NOC personal at the helm:

	"You can't ftp from x to y?  Try again tomorrow, it will
	 probably work then."

	"If you are not satisfied with the level of [network] 
         service you are receiving you may have it disconnected."

	"The routers for network x are out of table space for routes,
	 which is why hosts on that network can't reach your new
	 (three-month old) network.  We don't know when the routers will
	 be upgraded, but it probably won't be for another year." 

	One might argue that the recent restructuring of the IAB may work
towards bringing the Internet under control and Dr. Vinton G. Cerf's recent
involvement is a step in the right direction.  Unfortunately, from a
historical perspective, the new IAB structure is not likely to be
successful in achieving a solution.  Now the IAB has two task forces, the
Internet Research Task Force (IRTF) and the Internet Engineering Task Force
(IETF).  The IRTF, responsible for long-term Internet research, is largely
composed of the various task forces which used to sit at the IAB level.
The IETF, responsible for the solution of short-term Internet problems, has
retained its composition.

	The IETF is a voluntary organization and its members participate
out of self interest only.  The IETF has had past difficulties in solving
some of the Internet's problems (i.e., it has taken the IETF well over a
year to not yet produce RFCs for either a Point-To-Point Serial Line IP or
Network Management enhancements).  It is unlikely that the IETF has the
resources to mount a concerted attack against the problems of today's ever
expanding Internet.  As one IETF old-timer put it: "No one's paid to go do
these things, I don't see why they (the IETF management) think they can
tell us what to do" and "No one is paying me, why should I be thinking
about these things?"
 
	Even if the IETF had the technical resources, many of the
Internet's problems are also due to lack of "hands on" management.  
The IETF, 

	o  Bites off more than it can chew;
	o  Sometimes fails to understand a problem before making a solution;
	o  Attempts to solve political/marketing problems with technical
	     solutions;
	o  Has very little actual power.

	The IETF has repeatedly demonstrated the lack of focus necessary to
complete engineering tasks in a timely fashion.  Further, the IRTF is
chartered to look at problems on the five-year horizon, so they are out of
the line of responsibility.  Finally, the IAB, per se, is not situated to
resolve these problems as they are inherent to the current structure of
nonaccountability.

	During this crisis of non-management, the Internet has evolved into
a patch quilt of interconnected networks that depend on lots of
seat-of-the-pants flying to keep interoperating.  It is not an unusual
occurrence for an entire partition of the Internet to remain disconnected
for a week because the person responsible for a key connection went on
vacation and no one else knew how to fix it.  This situation is but one
example of an endemic problem of the global Internet.


Failure 2:- Network Management

	The current fury over network management protocols for TCP/IP is
but a microcosm of the greater Internet vs. OSI debate going on in the
marketplace.  While everyone in the market says they want OSI, anyone
planning on getting any work done today buys Internet technology.  So it is
with network management, the old IAB made the CMOT an Internet standard
despite the lack of a single implementation, while the only non-proprietary
network management protocol in use in the Internet is the SNMP.  The dual
network management standardization blessings will no doubt have the effect
of confusing end-users of Internet technology--making it appear there are
two choices for network management, although only one choice, the SNMP has
been implemented.  The CMOT choice isn't implemented, doesn't work, or
isn't interoperable.

	To compound matters, after spending a year trying to achieve
consensus on the successor to the current Internet standard SMI/MIB, the
MIB working group was disbanded without ever producing anything: the
political climate prevented them from resolving the matter.  (Many
congratulatory notes were sent to the chair of the group thanking him for
his time.  This is an interesting new trend for the Internet--congratulating
ourselves on our failures.)

	Since a common SMI/MIB could not be advanced, an attempt was made
to de-couple the SNMP and the CMOT (RFC1109).  The likely result of RFC1109
will be that the SNMP camp will continue to refine their experience towards
workable network management systems, whilst the CMOT camp will continue the
never-ending journey of tracking OSI while producing demo systems for trade
shows exhibitions.  Unfortunately the end-user will remain ever confused
because of the IAB's controversial (and technically questionable) decision
to elevate the CMOT prior to implementation.

	While the network management problem is probably too large for the
SNMP camp to solve by themselves they seem to be the only people who are
making any forward progress.


Failure 3:- Bandwidth Waste

	Both the national and regional backbone providers are fascinated
with T1 (and now T3) as the solution towards resource problems.  T1/T3
seems to have become the Internet panacea of the late 80's.  You never hear
anything from the backbone providers about work being done to get hosts to
implement the latest performance/congestion refinements to IP, TCP, or
above.  Instead, you hear about additional T1 links and plans for T3 links.
While T1 links certainly have more "sex and sizzle" than efficient
technology developments like slow-start, tiny gram avoidance and line mode
telnet, the majority of users on the Internet will probably get much more
benefit from properly behaving hosts running over a stable backbone than
the current situation of misbehaving and semi-behaved hosts over an
intermittent catenet.


Failure 4:- Routing

	The biggest problem with routing today is that we are still using
phase I (ARPANET) technology, namely EGP.  The EGP is playing the role of
routing glue in providing the coupling between the regional IGP and the
backbone routing information.  It was designed to only accommodate a single
point of attachment to the catenet (which was all DCA could afford with the
PSNs).  However with lower line costs, one can build a reasonably
inexpensive network using redundant links.  However the EGP does not
provide enough information nor does the model it is based upon support
multiple connections between autonomous systems.  Work is progressing in
the Interconnectivity WG of the IETF to replace EGP.  They are in the
process of redefining the model to solve some of the current needs.  BGP or
the Border Gateway Protocol (RFC1105) is an attempt to codify some of the
ideas the group is working on.

	Other problems with routing are caused by regionals wanting a
backdoor connection to another regional directly.  These connections
require some sort of interface between the two routing systems.  These
interfaces are built by hand to avoid routing loops.  Loops can be caused
when information sent into one regional network is sent back towards the
source.  If the source doesn't recognize the information as its own,
packets can flow until their time to live field expires.

	Routing problems are caused by the interior routing protocol or
IGP.  This is the routing protocol which is used by the regionals to pass
information to and from its users.  The users themselves can use a
different IGP than the regional.  Depending on the number of connections a
user has to the regional network, routing loops can be an issue.  Some
regionals pass around information about all known networks in the entire
catenet to their users.  This information deluge is a problem with some
IGPs.  Newer IGPs such as the new OSPF from the IETF and IGRP from cisco
attempt to provide some information hiding by adding hierarchy.  OSPF is
the internets first attempt at using a Dykstra type algorithm as an IGP.
BBN uses it to route between their packet switch nodes below the 1822 or
X.25 layer.

	Unstable routing is caused by hardware or hosts software.  Older
BSD software sets the TTL field in the IP header to a small number.  The
Internet today is growing and its diameter has exceed the software's
ability to reach the other side.  This problem is easily fixed by
knowledgeable systems people, but one must be aware of the problem before
they can fix it.

	Routing problems are also perceived when in fact a serial line
problem or hardware problem is the real cause.  If a serial line is
intermittent or quickly cycles from the up state into the down state and
back again, routing information will not be supplied in a uniform or smooth
manner.  Most current IGPs are Bellman-Ford based and employ some
stabilizing techniques to stem the flow of routing oscillations due to
"flapping" lines.  Often when a route to a network disappears, it may take
several seconds for it to reappear.  This can occur at the source router
who waits for the route to "decay" from the system.  This pause should be
short enough so that active connections persist but long enough that all
routers in the routing system "forget" about routes to that network.  Older
host software with over-active TCP retransmission timers will time out
connections instead of persevering in the face of this problem.  Also
routers, according to RFC1009, must be able to send ICMP unreachables when
a packet is sent to a route which is not present in its routing database.
Some host products on the market close down connections when a single ICMP
reachable is received.  This bug flies in the face of the Internet parable
"be generous in what you accept and rigorous in what you send".

	Many of the perceived routing problems are really complex multiple
interactions of differing products.


			    Causes of the Failures

The Internet failures and shortcomings can be traced to several sources:
			
	First and foremost, there is little or no incentive for efficiency
and/or economy in the current Internet.  As a direct result, the resources
of the Internet and its components are limited by factors other than
economics.  When resources wear thin, congestion and poor performance
result.  There is little to no incentive to make things better, if 1 packet
out of 10 gets through things "sort of work".  It would appear that
Internet technology has found a loophole in the "Tragedy of The Commons"
allegory--things get progressively worse and worse, but eventually
something does get through.

	The research community is interested in technology and not
economics, efficiency or free-markets.  While this tack has produced the
Internet suite of protocols, the de facto International Standard for Open
Systems, it has also created an atmosphere of intense in-breeding which is
overly sensitive to criticism and quite hardened against outside influence.
Meanwhile, the outside world goes on about developing economically viable
and efficient networking technology without the benefit of direct
participation on the part of the Internet.

	The research community also appears to be spending a lot of its
time trying to hang onto the diminishing number of research dollars
available to it (one problem of being a successful researcher is eventually
your sponsors want you to be successful in other things).  Despite this,
the research community actively shuns foreign technology (e.g., OSI), but,
inexplicably has not recently produced much innovation in new Internet
technology.  There is also a dearth of new and nifty innovative
applications on the Internet.  Business as usual on the Internet is mostly
FTP, SMTP and Telnet or Rlogin as it has been for many years.  The most
interesting example of a distributed application on the Internet today is
the Domain Name System, which is essentially an administrative facility,
not an end-user service.

	The engineering community must receive equal blame in these
matters.  While there have been some successes on the part of the
engineering community, such as those by Nagel, Jacobsen and Karn mentioned
above, the output of the IETF, namely RFCs and corresponding
implementations, has been surprisingly low over its lifetime.

	Finally, the Internet has become increasingly dependent on vendors
for providing implementations of Internet technology.  While this is no
doubt beneficial in the long-term, the vendor community, rather than
investing "real" resources when building these products, do little more
than shrink-wrap code written primarily by research assistants at
universities.  This has lead to cataclysmic consequences (e.g., the
Internet worm incident, where Sendmail with "debug" command and all was
packaged and delivered to customers without proper consideration).  Of
course, when problems are found and fixed (either by the vendor's customers
or software sources), the time to market with these fixes is commonly a
year or longer.  Thus, while vendors are vital to the long-term success of
Internet technology, they certainly don't receive high marks in the short-term.


			       Recommendations

Short-term solutions (should happen by year's end):

	In terms of hardware, the vendor community has advanced to the
point where the existing special-purpose technologies (Butterfly, NSSs) can
be replaced by off-the-shelf routers at far less cost and with superior
throughput and reliability.  Obvious candidates for upgrade are both the
NSFNET and ARPANET backbones.  Given the extended unreliability of the
mailbridges, the ARPA core is an immediate candidate (even though the days
of net 10 are numbered).

	In terms of software, ALL devices in the Internet must be network
manageable.  This is becoming ever more critical when problems must be
resolved.  Since SNMP is the only open network management protocol
functioning in the Internet, all devices must support SNMP and the Internet
standard SMI and MIB.

	Host implementations must be made to support the not-so-recent TCP
enhancements (e.g., those by Nagle, Jacobsen and Karn) and the more recent
linemode TELNET.

	The national and regional providers must coordinate to share
network management information and tools so that user problems can be dealt
with in a predictable and timely fashion.  Network management tools are a
big help, but without the proper personnel support above this, the benefits
can not be fully leveraged.

	The Internet needs leadership and hands-on guidance.  No one is
seemingly in charge today, and the people who actually care about the net
are pressed into continually fighting the small, immediate problems.

Long-term solutions:

	To promote network efficiency and a free-market system for the
delivery of Internet services, it is proposed to switch the method by which
the network itself is supported.  Rather than a top-down approach where the
money goes from funding agencies to the national backbone or regional
providers, it is suggested the money go directly to end-users (campuses)
who can then select from among the network service providers which among
them best satisfies their needs and costs.

	This is a strict economic model: by playing with the full set of
the laws of economics, a lot of the second-order problems of the Internet,
both present and on the horizon, can be brought to heel.  The Internet is
no longer a research vehicle, it is a vibrant production facility.  It is
time to acknowledge this by using a realistic economic model in the
delivery of Internet services to the community (member base).

	When Internet sites can vote with their pocketbooks, some new
regionals will be formed; some, those which are non-performant or
uncompetitive, will go away; and, the existing successful ones will grow.
The existing regionals will then be able to use their economic power, as
any consumer would, to ensure that the service providers (e.g., the
national backbone providers) offer responsive service at reasonable prices.
"The Market" is a powerful forcing function: it will be in the best
interests of the national and regional providers to innovate, so as to be
more competitive.  Further, such a scheme would also allow the traditional
telecommunications providers a means for becoming more involved in the
Internet, thus allowing cross-leverage of technologies and experience.

	The transition from top-down to economic model must be handled
carefully, but this is exactly the kind of statesmanship that the Internet
should expect from its leadership.
-------