|
Unix Programming - Unix Interface Design Patterns - The ‘Separated Engine and Interface’ Pattern
The ‘Separated Engine and Interface’ Pattern
In Chapter7 we
argued against building monster single-process monoliths, and that it
is often possible to lower the global complexity of programs by
splitting them into communicating pieces. In the Unix world, this
tactic is frequently applied by separating the ‘engine’
part of the program (core algorithms and logic specific to its
application domain) from the ‘interface’ part (which
accepts user commands, displays results, and may provide services such
as interactive help or command history). In fact, this
separated-engine-and-interface pattern is probably the one most
characteristic interface design pattern of Unix.
(The other, more obvious candidate for that distinction would be
filters. But filters are more often found in non-Unix environments
than engine/interface pairs with bidirectional traffic between them.
Simulating pipelines is easy; the more sophisticated IPC mechanisms
required for engine/interface pairs are hard.)
Owen Taylor, maintainer of the GTK+ library widely used for
writing user interfaces under X, beautifully brings out the
engineering benefits of this kind of partitioning at the end of his
note Why GTK_MODULES is
not a security hole; he finishes by writing "[T]he
secure setuid program is a 500 line program that does only what it
needs to, rather than a 500,000 line library whose essential task is
user interfaces".
This is not a new idea. Xerox PARC's early research into graphical user
interfaces led them to propose the
“model-view-controller” pattern as an archetype for
GUIs.
-
The “model” is what in the Unix world is usually
called an “engine”. The model contains the
domain-specific data structures and logic for your application.
Database servers are archetypal examples of models.
-
The “view” part is what renders your domain objects
into a visible form. In a really well-separated model/view/controller
application, the view component is notified of updates to the model
and responds on its own, rather than being driven synchronously by
the controller or by explicit requests for a refresh.
-
The “controller” processes user requests and passes
them as commands to the model.
In practice, the view and controller parts tend to be more
closely bound together than either is to the model. Most GUIs, for
example, combine view and controller behavior. They tend to be
separated only when the application demands multiple views of the
model.
Under Unix, application of the model/view/controller pattern is
far more common than elsewhere precisely because there is a strong
“do one thing well” tradition, and IPC methods are both
easy and flexible.
An especially powerful form of this technique couples a policy
interface (often a GUI combining view and controller functions) with
an engine (model) that contains an interpreter for a domain-specific
minilanguage. We examined this pattern in Chapter8, focusing on minilanguage design; now
it's time to look at the different ways that such engines can form
components of larger systems of code.
There are several major variants of this pattern.
In a configurator/actor pair, the interface part controls the
startup environment of a filter or daemon-like program which then runs
without requiring user commands.
The programs
fetchmail(1)
and
fetchmailconf(1)
(which we've already used as case studies in
discoverability
and data-driven programming and will encounter again as language case
studies in Chapter14) are
a good example of a configurator/actor pair.
fetchmailconf is the interactive dotfile
configurator that ships with
fetchmail.
fetchmailconf can also serve as a GUI
wrapper that runs fetchmail in either foreground or background
mode.
This design pattern enables both
fetchmail and
fetchmailconf to specialize in what they do
well, and indeed to be written in different languages appropriate to
their task domains. Fetchmail, which usually runs in background as a
daemon, need not be bloated with GUI code. Conversely,
fetchmailconf can specialize in elaborate
GUIness without exacting size and complexity costs from
fetchmail. Finally, because the information channels between them are
narrow and well-defined, it remains possible to drive
fetchmail from the command line and from
scripts other than fetchmailconf.
The term “configurator/actor” is my invention.
A slight variant of the configurator/actor pair can be useful in
situations that require serialized access to a shared resource in a
batch mode; that is, when a well-defined job stream or sequence
of requests requires some shared resource, but no individual
job requires user interaction.
In this spooler/daemon pattern, the spooler or front end simply
drops job requests and data in a spool area. The job requests and data
are simply files; the spool area is typically just a directory. The
location of the directory and the format of the job requests are agreed
on by the spooler and daemon.
The daemon runs forever in background, polling the spool
directory, looking there for work to do. When it finds a job request,
it tries to process the associated data. If it succeeds, the job
request and data are deleted out of the spool area.
The classic example of this pattern is the Unix print spooler
system,
lpr(1)/lpd(1). The
front end is
lpr(1);
it simply drops files to be printed in a spool area periodically
scanned by
lpd. lpd's job
is simply to serialize access to the printer devices.
Another classic example is the pair
at(1)/atd(1),
which schedules commands for execution at specified times. A
third example, historically important though no longer in wide use,
was UUCP — the Unix-to-Unix Copy Program commonly used as a mail
transport over dial-up lines before the Internet explosion of the
early 1990s.
The spooler/daemon pattern remains important in mail-transport
programs (which are batchy by nature). The front ends of mail
transports such as
sendmail(1)
and
qmail(1)
usually make one try at delivering mail immediately, through SMTP over an
outbound Internet connection. If that attempt fails, the mail will
fall into a spool area; a daemon version or mode of the mail transport
will retry the delivery later.
Typically, a spooler/daemon system has four parts: a job
launcher, a queue lister, a job-cancellation utility, and a
spooling daemon, In fact, the presence of the first three parts is
a sure clue that there is a spooler daemon behind them
somewhere.
The terms “spooler” and “daemon” are
well-established Unix jargon. (‘Spooler’ actually
dates back to early mainframe days.)
In this pattern, unlike a configurator/actor or spooler/server
pair, the interface part supplies commands to and interprets output
from an engine after startup; the engine has a simpler interface
pattern. The IPC method used is an implementation detail; the engine
may be a slave process of the driver (in the sense we discussed in
Chapter7) or the
engine and driver may communicate through sockets, or shared memory,
or any other IPC method. The key points are (a) the interactivity of
the pair, and (b) the ability of the engine to run standalone with its
own interface.
Such pairs are trickier to write than configurator/actor pairs
because they are more tightly and intricately coupled; the driver
must have knowledge not merely about the engine's expected startup
environment but about its command set and response formats as well.
When the engine has been designed for scriptability, however, it
is not uncommon for the driver part to be written by someone other
than the engine author, or for more than one driver to front-end a
given engine. An excellent example of both is provided by the programs
gv(1)
and
ghostview(1),
which are drivers for
gs(1),
the Ghostscript interpreter. GhostScript renders PostScript to various
graphics formats and lower-level printer-control languages. The gv and
ghostview programs provide GUI wrappers for GhostScript's rather
idiosyncratic invocation switches and command syntax.
Another excellent example of this pattern is the xcdroast/cdrtools
combination. The cdrtools distribution provides a program
cdrecord(1)
with a command-line interface. The
cdrecord code specializes in knowing
everything about talking to CD-ROM hardware.
xcdroast is a GUI; it specializes in
providing a pleasant user experience. The
xcdroast(1)
program calls
cdrecord(1)
to do most of its work.
xcdroast also calls other CLI tools:
cdda2wav(1)
(a sound file converter) and
mkisofs(1)
(a tool for creating ISO-9660 CD-ROM file system images from a list of
files). The details of how these tools are invoked are hidden from
the user, who can think in terms centered on the task of making CDs
rather than having to know directly about the arcana of sound-file
conversion or file-system structure. Equally important, the
implementers of each of these tools can concentrate on their
domain-specific expertise without having to be user-interface
experts.
|
A key pitfall of driver/engine organization is that frequently
the driver must understand the state of the engine in order to reflect
it to the user. If the engine action is practically instantaneous,
it's not a problem, but if the engine can take a long time (e.g., when
accessing many URLs) the lack of feedback can be a significant issue.
A similar problem is responding to errors. For example, the
traditional (although not very Unix-like) confirmation question about
whether it's OK to overwrite a file that already exists is kind of
painful to write in the driver/engine world; the engine, which detects
the problem, has to ask the driver to do the confirmation prompting.
|
|
| --
Steve Johnson
|
|
It's important to design the engine so that it not only does the
right thing, but also notifies the driver about what it's doing so
the driver can present a graceful interface with appropriate feedback.
The terms “driver” and “engine” are
uncommon but established in the Unix community.
A client/server pair is like a driver/engine pair, except that
the engine part is a daemon running in background which is not
expected to be run interactively, and does not have its own user
interface. Usually, the daemon is designed to mediate access to
some sort of shared resource — a database, or a transaction
stream, or specialized shared hardware such as a sound device.
Another reason for such a daemon may be to avoid performing
expensive startup actions each time the program is invoked.
Yesterday's paradigmatic example was the
ftp(1)/ftpd(1)
pair that implements FTP, the File Transfer Protocol; or perhaps two
instances of
sendmail(1),
sender in foreground and listener in background, passing Internet
email. Today's would have to be any browser/web server pair.
However, this pattern is not limited to communication programs;
another important case is in databases, such as the
psql(1)/postmaster(1)
pair. In this one, psql serializes access
to a shared database managed by the postgres daemon, passing it SQL
requests and presenting data sent back as responses.
These examples illustrate an important property of such pairs,
which is that the cleanliness of the protocol that serializes
communication between them is all-important. If it is well-defined
and described by an open standard, it can become a tremendous
opportunity for leverage by insulating client programs from the
details of how the server's resource is managed, and allowing
clients and servers to evolve semi-independently. All
separated-engine-and-interface programs potentially get this kind
of benefit from clean separation of function, but in the
client/server case the payoffs for getting it right tend to be
particularly high exactly because managing shared resources is
intrinsically difficult.
Message queues and pairs of named
pipes can be and have
been used for front-end/back-end communication, but the benefits of
being able to run the server on a different machine from the client
are so great that nowadays almost all modern client-server pairs use
TCP/IP sockets.
[an error occurred while processing this directive]
|