|
Unix Programming - Origins and History of Unix, 1969-1995 - TCP/IP and the Unix Wars: 1980-1990
TCP/IP and the Unix Wars: 1980-1990
The Berkeley
campus of the University of California emerged early as the single
most important academic hot-spot in Unix development. Unix research
had begun there in 1974, and was given a substantial impetus when Ken
Thompson taught
at the University during a 1975-76 sabbatical. The first BSD
release had been in
1977 from a lab run by a then-unknown grad student named Bill
Joy. By 1980
Berkeley was the hub of a sub-network of universities actively
contributing to their variant of Unix. Ideas and code from Berkeley
Unix (including the
vi(1)
editor) were feeding back
from Berkeley to Bell Labs.
Then, in 1980, the Defense Advanced Research Projects Agency
needed a team to implement its brand-new
TCP/IP protocol stack
on the VAX under Unix. The PDP-10s that
powered the ARPANET at that time were aging, and indications that DEC
might be forced to cancel the 10 in order to support the VAX were
already in the air. DARPA considered contracting DEC to implement
TCP/IP, but rejected
that idea because they were concerned that DEC might not be responsive
to requests for changes in their proprietary
VAX/VMS
operating system [Libes-Ressler]. Instead, DARPA
chose Berkeley Unix as a platform — explicitly because its
source code was available and unencumbered [Leonard].
Berkeley's
Computer Science Research Group was in the right place at the right
time with the strongest development tools; the result became arguably
the most critical turning point in Unix's history since its
invention.
Until the TCP/IP
implementation was released with Berkeley 4.2 in 1983, Unix
had had only the weakest networking support. Early experiments with
Ethernet were unsatisfactory. An ugly but serviceable facility called
UUCP (Unix to Unix Copy Program) had been developed at Bell Labs for
distributing software over conventional telephone lines via modem.[16] UUCP could forward Unix mail
between widely separated machines, and (after Usenet was invented in
1981) supported
Usenet, a distributed
bulletin-board facility that allowed users to broadcast text messages
to anywhere that had phone lines and Unix systems.
Still, the few Unix users aware of the bright lights of the
ARPANET felt like they were stuck in a backwater. No FTP, no telnet,
only the most restricted remote job execution, and painfully slow
links. Before TCP/IP,
the Internet and Unix cultures did not mix. Dennis
Ritchie's
vision of computers as a way to “encourage close
communication” was one of collegial communities clustered
around individual timesharing machines or in the same computing
center; it didn't extend to the continent-wide distributed
‘network nation’ that ARPA users had started to form in
the mid-1970s. Early ARPANETters, for their part, considered Unix a
crude makeshift limping along on risibly weak hardware.
After TCP/IP,
everything changed. The ARPANET and Unix cultures began to merge at
the edges, a development that would eventually save both from
destruction. But there would be hell to pay first as the result of
two unrelated disasters; the rise of
Microsoft and the
AT&T divestiture.
In 1981, Microsoft made its historic deal with
IBM over the new IBM
PC. Bill Gates bought QDOS (Quick and Dirty Operating System), a clone
of CP/M that its programmer Tim Paterson had thrown together in six
weeks, from Paterson's employer Seattle Computer Products. Gates,
concealing the IBM deal from Paterson and SCP, bought the rights for
$50,000. He then talked IBM into allowing Microsoft to market MS-DOS
separately from the PC hardware. Over the next decade, leveraging
code he didn't write made Bill Gates a multibillionaire, and
business tactics even sharper than the original deal gained Microsoft
a monopoly lock on desktop computing. XENIX as a product was rapidly
deep-sixed, and eventually sold to SCO.
It was not apparent at the time how successful
(or how destructive)
Microsoft was going
to be. Since the IBM PC-1 didn't have the hardware capacity to run
Unix, Unix people barely noticed it at all (though, ironically enough,
DOS 2.0 eclipsed CP/M largely because Microsoft's co-founder Paul Allen
merged in Unix features including subdirectories and
pipes). There were
things that seemed much more interesting going on — like the
1982 launching of Sun Microsystems.
Sun Microsystems founders Bill
Joy, Andreas
Bechtolsheim, and Vinod Khosla set out to build a dream Unix machine
with built-in networking capability. They combined hardware designed
at Stanford with the Unix developed at Berkeley to produce a smashing
success, and founded the workstation industry. At the time, nobody
much minded watching source-code access to one branch of the Unix tree
gradually dry up as Sun began to behave less like a freewheeling
startup and more like a conventional
firm. Berkeley was
still distributing BSD with source code. Officially, System
III source
licenses cost $40,000 each; but Bell Labs was turning a blind eye to
the number of bootleg Bell Labs Unix tapes in circulation, the
universities were still swapping code with Bell Labs, and it looked
like Sun's
commercialization of Unix might just be the best thing to happen to it
yet.
1982 was also the year that C first showed signs of establishing
itself outside the Unix world as the systems-programming language of
choice. It would only take about five years for
C to drive machine
assemblers almost completely out of use. By the early 1990s C and
C++ would dominate not
only systems but application programming; by the late 1990s all other
conventional compiled languages would be effectivelyobsolete.
When DEC canceled development on the
PDP-10's successor
machine (Jupiter) in 1983, VAXes running Unix began to take over as
the dominant Internet machines, a position they would hold until being
displaced by Sun workstations. By 1985, about 25% of all VAXes
would be running Unix despite DEC's stiff opposition. But the
longest-term effect of the Jupiter cancellation was a less obvious
one; the death of the MIT AI Lab's PDP-10-centered hacker culture
motivated a programmer named Richard
Stallman
to begin writing GNU, a complete free clone of Unix.
By 1983 there were no fewer than six Unix-workalike
operating systems for the IBM-PC: uNETix, Venix, Coherent, QNX, Idris,
and the port hosted on the Sritek PC daughtercard. There was still no
port of Unix in either the
System V or
BSD versions;
both groups considered the 8086 microprocessor woefully
underpowered and wouldn't go near it. None of the Unix-workalikes
were significant as commercial successes, but they indicated a
significant demand for Unix on cheap hardware that the major vendors
were not supplying. No individual could afford to meet it, either,
not with the $40,000 price-tag on a source-code license.
Sun
was already a success (with imitators!) when, in 1983, the
U.S. Department of Justice won its second antitrust case against
AT&T and broke
up the Bell System. This relieved AT&T from the 1958 consent
decree that had prevented them from turning Unix into a product.
AT&T promptly rushed to commercialize Unix System V—a move
that nearly killed Unix.
|
So true. But their marketing did spread Unix internationally.
|
|
| --
Ken Thompson
|
|
Most Unix boosters thought that the
divestiture was great news. We thought we saw in
the post-divestiture AT&T, Sun Microsystems, and Sun's smaller imitators the
nucleus of a healthy Unix industry — one that, using inexpensive
68000-based workstations, would challenge and eventually break the
oppressive monopoly that then loomed over the computer industry
— IBM's.
What none of us realized at the time was that the productization
of Unix would destroy the free exchanges of source code that had nurtured
so much of the system's early vitality. Knowing no other model than
secrecy for collecting profits from software and no other model than
centralized control for developing a commercial product, AT&T
clamped down hard on source-code distribution. Bootleg Unix tapes became
far less interesting in the knowledge that the threat of lawsuit might
come with them. Contributions from universities began to dry up.
To make matters worse, the big new players in the Unix market
promptly committed major strategic blunders. One was to seek
advantage by product differentiation — a tactic which resulted
in the interfaces of different Unixes diverging. This threw away
cross-platform compatibility and fragmented the Unix market.
The other, subtler error was to behave as if personal computers
and Microsoft were
irrelevant to Unix's prospects. Sun
Microsystems
failed to see that commoditized PCs would inevitably become an attack
on its workstation market from below.
AT&T, fixated on
minicomputers and mainframes, tried several different strategies to
become a major player in computers, and badly botched all of them. A
dozen small companies formed to support Unix on PCs; all were
underfunded, focused on selling to developers and engineers, and never
aimed at the business and home market that Microsoft was
targeting.
In fact, for years after divestiture the Unix community was
preoccupied with the first phase of the Unix
wars — an
internal dispute, the rivalry between System
V Unix and BSD
Unix. The dispute
had several levels, some technical (sockets vs. streams,
BSD tty vs. System V
termio) and some cultural. The divide was roughly between
longhairs and shorthairs; programmers and technical people tended to
line up with Berkeley and BSD, more business-oriented types with AT&T
and System V.
The longhairs, repeating a theme from Unix's early days ten years
before, liked to see themselves as rebels against a corporate empire;
one of the small companies put out a poster showing an X-wing-like
space fighter marked “BSD” speeding away from a huge
AT&T ‘death star’ logo left broken and in flames. Thus
we fiddled while Rome burned.
But something else happened in the year of the
AT&T divestiture that would have more long-term importance for
Unix. A programmer/linguist named Larry Wall quietly invented the
patch(1)
utility. The patch program, a simple tool
that applies changebars generated by
diff(1)
to a base file, meant that Unix developers could cooperate by passing
around patch sets — incremental changes to code — rather
than entire code files. This was important not only because patches
are less bulky than full files, but because patches would often apply
cleanly even if much of the base file had changed since the
patch-sender fetched his copy. With this tool, streams of development
on a common source-code base could diverge, run in parallel, and
re-converge. The patch program did more
than any other single tool to enable collaborative development over
the Internet — a method that would revitalize Unix after
1990.
In 1985 Intel shipped the first 386 chip, capable of addressing 4
gigabytes of memory with a flat address space. The clumsy segment
addressing of the 8086 and 286 became immediately obsolete. This was
big news, because it meant that for the first time, a microprocessor in
the dominant Intel family had the capability to run Unix without
painful compromises. The handwriting was on the wall for
Sun and the
other workstation makers. They failed to see it.
1985 was also the year that Richard
Stallman
issued the GNU manifesto [Stallman] and launched the
Free Software Foundation. Very few people took him or his
GNU project
seriously, a judgment that turned out to be seriously mistaken. In
an unrelated development of the same year, the originators of the X
window system released it as source code without royalties,
restrictions, or license code. As a direct result of this decision,
it became a safe neutral area for collaboration between Unix vendors,
and defeated proprietary contenders to become Unix's graphics
engine.
Serious standardization efforts aimed at reconciling the System
V and
Berkeley APIs also
began in 1983 with the /usr/group standard. This was followed in 1985
by the POSIX standards, an
effort backed by the IEEE. These described the intersection set of
the BSD and SVR3
(System V Release 3) calls, with the superior Berkeley signal
handling and job control but with SVR3 terminal control. All later
Unix standards would incorporate POSIX at their core, and later Unixes
would adhere to it closely. The only major addition to the modern Unix
kernel API to come afterwards was BSD sockets.
In 1986 Larry Wall, previously the inventor of
patch(1),
began work on Perl, which would become the first and most
widely used of the open-source scripting languages. In early 1987 the
first version of the GNU C compiler
appeared, and by the end of 1987 the
core of the GNU toolset was falling into place: editor, compiler, debugger, and
other basic development tools. Meanwhile, the X windowing
system was
beginning to show up on relatively inexpensive workstations. Together,
these would provide the armature for the open-source Unix developments
of the 1990s.
1986 was also the year that PC technology broke free of IBM's
grip. IBM, still trying
to preserve a price-vs.-power curve across its product line that would
favor its high-margin mainframe business, rejected the 386 for most of
its new line of PS/2 computers in favor of the weaker 286. The PS/2
series, designed around a proprietary bus architecture to lock out
clonemakers, became a colossally expensive failure.[17] Compaq, the most aggressive of the
clonemakers, trumped IBM's move by releasing the first 386 machine.
Even with a clock speed of a mere 16 MHz, the 386 made a tolerable Unix
machine. It was the first PC of which that could be said.
It was beginning to be possible to imagine that
Stallman's GNU project might mate with 386 machines to produce
Unix workstations almost an order of magnitude less costly than anyone
was offering. Curiously, no one seems to have actually got this far
in their thinking. Most Unix programmers, coming from the
minicomputer and workstation worlds, continued to disdain cheap 80x86
machines in favor of more elegant 68000-based designs. And, though a
lot of programmers contributed to the GNU
project, among
Unix people it tended to be considered a quixotic gesture that was
unlikely to have near-term practical consequences.
The Unix community had never lost its rebel streak. But in
retrospect, we were nearly as blind to the future bearing down on us
as IBM or
AT&T. Not even
Richard Stallman, who had declared a moral crusade against
proprietary software a few years before, really understood how badly
the productization of Unix had damaged the community around it; his
concerns were with more abstract and long-term issues. The rest of us
kept hoping that some clever variation on the corporate formula would
solve the problems of fragmentation, wretched marketing, and strategic
drift, and redeem Unix's pre-divestiture
promise. But worse was still to
come.
1988 was the year Ken Olsen (CEO of DEC) famously described Unix
as “snake oil”. DEC had been shipping its own variant of
Unix on PDP-11s since
1982, but really wanted the business to go to its proprietary
VMS
operating system. DEC and the minicomputer industry were in deep
trouble, swamped by waves of powerful low-cost machines coming out of
Sun Microsystems and the rest of the workstation
vendors. Most of those workstations ran Unix.
But the Unix industry's own problems were growing more severe.
In 1988 AT&T took a 20% stake in Sun Microsystems. These two
companies, the leaders in the Unix market, were beginning to wake up
to the threat posed by PCs, IBM, and Microsoft, and to realize that
the preceding five years of bloodletting had gained them little. The
AT&T/Sun alliance and the development of technical standards around
POSIX eventually healed
the breach between the System V and BSD Unix lines. But the second phase of the Unix
wars began when the
second-tier vendors (IBM,
DEC, Hewlett-Packard, and others) formed the Open Software Foundation
and lined up against the AT&T/Sun axis (represented by Unix
International). More rounds of Unix fighting Unix ensued.
Meanwhile,
Microsoft was
making billions in the home and small-business markets that the
warring Unix factions had never found the will to address. The 1990
release of Windows 3.0 — the first successful graphical
operating system from Redmond — cemented Microsoft's dominance,
and created the conditions that would allow them to flatten and
monopolize the market for desktop applications in the 1990s.
The years from 1989 to 1993 were the darkest in Unix's history.
It appeared then that all the Unix community's dreams had failed.
Internecine warfare had reduced the proprietary Unix industry to a
squabbling shambles that never summoned either the determination or
the capability to challenge Microsoft. The elegant Motorola chips
favored by most Unix programmers had lost out to Intel's ugly but
inexpensive processors. The GNU project failed to produce the free Unix kernel
it had been promising since 1985, and after years of excuses
its credibility was beginning to wear thin. PC technology was being
relentlessly corporatized. The pioneering Unix
hackers of the 1970s
were hitting middle age and slowing down. Hardware was getting
cheaper, but Unix was still too expensive. We were belatedly becoming
aware that the old monopoly of
IBM had yielded to a
newer monopoly of
Microsoft, and
Microsoft's mal-engineered software was rising around us like a tide
of sewage.
[an error occurred while processing this directive]
|
