Forum on Risks to the Public in Computers and Related Systems
ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator
Volume 1: Issue 33
Wednesday, 1 Jan 1986
Contents
Star Wars and Bank of NY- Brint Cooper
- Chris Hibbert
- Jim Horning
- Lipton and SDI
- Herb Lin
- The robot sentry
- Martin Minow
- Murphy is watching YOU
- Rob Austein
- Re: Failure probabilities in decision chains
- Stephen Wolff
Re: Can Bank of New York Bank on Star Wars? [PGN's retitling]
Brint Cooper <abc@BRL.ARPA>
Mon, 23 Dec 85 17:38:18 EST
The idea of independent, non-communicating "battle groups" for an SDI system sounds great. But what about the "fratricide" problem? Brint
Re: Can Bank of New York Bank on Star Wars? [PGN's retitling]
<Hibbert.pa@Xerox.ARPA>
Mon, 30 Dec 85 12:00:25 PST
To: RISKS FORUM (Peter G. Neumann, Coordinator) <RISKS@SRI-CSL.ARPA> cc: horning@decwrl.DEC.COM (Jim Horning) ------------------------------------- From: horning@decwrl.DEC.COM (Jim Horning) Date: 20 Dec 1985 1413-PST (Friday) To: RISKS@SRI-CSL.ARPA Subject: Can Bank of New York Bank on Star Wars? [PGN's retitling] Last night in the debate at Stanford on the technical feasibility of the SDI, Richard Lipton chose the financial network as an example of the advantages of a distributed system (such as he is proposing for SDI) over a centralized one. "There have been no catastrophes." [What about the ARPANET collapse? PGN] ------------------------------------- The ARPANET collapse is a good contrasting case to show what Lipton was talking about. His point about the financial "network" is that it isn't a monolithic system, but a set of many (dozens?, scores?, hundreds?) independant systems. Any one of the systems could fail (even catastrophically) and it wouldn't be much of a problem for the whole system. ARPANet is a single monolithic system, centrally designed and administered. Most bugs manifest at exactly the same provocations at widely separated parts of the net. There are at least a couple of separate national networks of automatic teller machines, and if any one of them dies, it shouldn't have any effect on the others, or on any of the banks with only local networks, or no networks at all. It would take a collapse of the phone system to put them all out of commission. ARPANet on the other hand is a monolithic system. There is one protocol that all parts of the system must share, a common medium is used, and in there are only a few implementations of the protocols. It doesn't take much to blow the whole system out of the water. (For the most part it's as reliable as it is is only because it gets constant use, and new parts aren't put in until they are shown to work most of the time.) What Lipton was proposing at the Stanford debate was that we make an anti-missile shield from many separately designed and implemented parts so that their failure modes are more independant. This is a good idea, and if it were done, I would have plenty of faith in the system. However, that's not the way government gets things done. Since the DOD is running the program there's no way there would be more than three "separate" designs, and they would all go through the same approval process, removing many of the differences they started with. Back to the ARPANet example, if you look at a larger system than just ARPA, including UUCP, DECNet, IBM's internal network, as well as the SOURCE, TYMNet, Compuserve, etc., you find the same robustness. ARPANet may die and be out of commission for a long time, and most people will still be able to get work done through some other medium, since only a fraction of the people using computer networks depend on any one of them. Chris
Re: Can Bank of New York Bank on Star Wars? [PGN's retitling]
Jim Horning <horning@decwrl.DEC.COM >
30 Dec 1985 1419-PST (Monday)
Chris,
I agree with many of your comments, and feel that the $38 billion
problem at Bank of New York is much more typical of how problems in
nominally "independent" systems can propagate because of the intrinsic
need to communicate. (As an example of a non-obvious interaction,
recall its effects on the platinum futures market.)
In addition to the problems you cite, Lipton's scheme suffers from a
few other flaws, including:
- The "simulation" that indicates that "only 5-10% extra bullets" would
be needed apparently makes two dubious assumptions:
1) Independent "battle groups" (with sufficient "teraflops") can
pinpoint targets as accurately as a cooperating distributed
system.
2) Each "battle group" is able to recognize all "kills" by
any battle group. I.e., the "extra bullets" counted are only
those that are fired simultaneously at a target. With many
of the proposed weapons, targets would be disabled, rather
than disintegrated; with kinetic weapons, a single target
could disperse to form a threat crowd.
(Note that observation of kills is a form of communication
intrinsic to the problem.)
- There were good systems reasons (completely outside of the computing
requirements) that led the Fletcher commision to propose
cradle-to-grave tracking (especially for RV vs. decoy discrimination)
and a layered defense. Lipton gave no evidence of understanding
those reasons, let alone making credible alternate proposals.
- The systems that you cite, and that he cited, are all ones where each
component is in routine use under the exact circumstances that they
must be reliable for. No matter how many independent subsystems the
Lipton SDI is divided into, NONE of them will get this kind of routine
use under conditions of saturation attack where reliability will be
most critical. Thus there is a high probability that each of them would
fail (perhaps in independent ways!).
Jim H.
Lipton and SDI
Herb Lin <LIN@MC.LCS.MIT.EDU>
Mon, 23 Dec 85 18:09:59 EST
To: horning@DECWRL.DEC.COM
cc: LIN@MC.LCS.MIT.EDU, RISKS@SRI-CSL.ARPA
From: horning at decwrl.DEC.COM (Jim Horning)
More generally, I am interested in reactions to Lipton's proposal that
SDI reliability would be improved by having hundreds or thousands of
"independent" orbiting "battle groups," with no communication between
separate groups (to prevent failures from propagating), and separately
designed and implemented hardware and software for each group (to
prevent common design flaws from affecting multiple groups).
That is absurd on the face of it. To prevent propagation of failures,
systems must be truly independent. To see the nonsense involved,
assume layer #1 can kill 90% of the incoming threat, and layer #2 is
sized to handle a maximum threat that is 10% of the originally
launched threat. If layer 1 fails catastrophically, you're screwed in
layer #2. Even if Layers 1 and 2 don't talk to each other, they're
not truly independent.
The robot sentry
<minow%rex.DEC@decwrl.DEC.COM >
Friday, 27 Dec 1985 13:11:28-PST
The following appeared on USENET net.general today (Dec 27). Martin.
--------
"A much more sinister arrival on the robot scene is named Prowler.
Created by Robot Defense Systems in Colorado, Prowler has been designed
for use as a sentry to guard military installations, warehouses and
other sites where security is important. When made available in the
near future, this squat, sturdy, mobile device will carry
microcomputers, software and sensors capable of locating intruders.
Chillingly, buyers will be able to arm Prowler with machine guns and
grenade launchers; they'll also be able to program the robot to fire at
will. The manufacturer claims that interest in Prowler has been high,
both among domestic companies who see it as a comparatively low-cost
replacement for 24-hour human security, and certain foreign countries
where government officials might prefer guards that will never revolt."
-- US Air magazine
-- JP Massar, Thinking Machines Corporation, Cambridge, MA
-- ihnp4!godot!massar, massar@think.com.arpa
-- 617-876-1111
Posted Fri 27-Dec-1985 16:08 Maynard Time. Martin Minow MLO3-3/U8, DTN 223-9922
Murphy is watching YOU
Rob Austein <SRA@XX.LCS.MIT.EDU>
Mon, 23 Dec 1985 16:45 EST
About six hours after sending that message about mailers [RISKS-1.32], I found myself with the pleasant task of doing bit level reconstruction of XX's MAILQ: directory with DDT, because the system had crashed while MMAILR was in the middle of a disk transfer. Talk about ironic postscripts.... Cheers, Rob
Re: Failure probabilities in decision chains
Stephen Wolff <steve@BRL.ARPA>
Mon, 23 Dec 85 17:33:15 EST
* IF the overall decision is correct if and only if all five sub-decisions are correct, and * IF the sub-decisions are statistically independent, and * IF the probability that each sub-decision is correct is 0.9, * THEN the probability that the overall decission is correct is 0.9^5 = .59049 (vide any textbook in probability) which is *suspiciously* close to "59%". But when Bill Walsh mentioned this problem to me in LA he was adamant that this was NOT the explanation he wanted. -s
Report problems with the web pages to the maintainer