The Risks Digest

The RISKS Digest

Forum on Risks to the Public in Computers and Related Systems

ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator

Volume 4 Issue 63

Thursday, 12 March 1987


o Re: Teflon flywheels and safe software
Al Mok
o Re: Electronic Steering
Bob Ayers
o Inputs For Quantitative Risk Assessment
Hal Guthery
o Re: Active car suspension
Geof Cooper
o Ozone hole a false alarm?
Mark Brader
o Phone problems (RISKs in auto-dialers)
David Barto
o Re: Mode C Transponders
Jan Wolitzky
o Automatic Landing Systems
Hugh LaMaster
o F-111 Losses
Rob Fowler
o Re: Computers in the Arts (Computer lighting)
Shannon Nelson
o Info on RISKS (comp.risks)

Re: Teflon flywheels and safe software (RISKS 4.56)

Wed, 11 Mar 87 19:32:34 CST
         [Response to comments of G. Jones and B. Randell:]

     It is a truism that we, as engineers can only hope to minimize the
probability of failure in the systems we build (assuming that you believe in
quantum mechanics). The relevant question is of course how to build systems
in such a way that we can have as much confidence as possible that they will
meet specifications with reasonable resources. To build a system which has
critical timing constraints, we can allocate resources carefully so as to
enable us to prove that the system will invariably meet the specified timing
constraints, ASSUMING that the hardware functions correctly and the external
environment does not stress the system beyond what it is designed to handle.
The fact that the hardware may not function correctly or that the operating
conditions imposed by the external world may exceed the design limits with a
certain probability DOES NOT absolve us of the responsibility to try to
allocate resources carefully so as to invariably meet the critical timing
constraints. Yes, performance figures are ultimately probabilistic. But if a
real-time software designer can at all help it, the source of uncertainty
should not be due to the adoption of a resource allocation strategy which is
outside the control of (worse still, not understood by) the software
designer.  I do not want my real-time program to cause a plane to crash or
an oil-drill platform to topple over in rough seas just because I cannot
predict that a tight loop takes twice as much time to run when the processor
decides to flush its data cache at an inopportune moment! I think this type
of predictability is the "time determinancy" that Scott Guthery was
referring to in his message. It has nothing to do with sequential
programming. It is a property that I certainly prefer to see in a
safety-critical system. I suspect you would too.

     There are many interesting and important research problems involved in
designing predictable real-time systems, i.e., guaranteed to meet certain
behavioral and timing specifications. Sometimes, the implementation language
can get in the way, e.g., see [Volz and Mudge 86], [Mok 84]. Having a
multi-processor system does not necessarily make it very much easier to meet
timing constraints. Even if you have one processor for each process, you
still have to make sure that the communication subsystem can deliver all the
time-critical messages. This communication problem is likely to be harder to
solve the more processors you employ. (Formulated properly as a
combinatorial optimization problem, a variety of this communication problem
can be shown to be NP-hard, but that does NOT mean that practical solutions
do not exist.) On the practical side, there are always the engineering
tradeoffs that need to be researched, e.g., should we use as few processors
as possible to meet the specifications so that we can have as many extra
ones around to replace ones that fail on-line? If we use one processor for
each process, will the power supply generate so much heat that the avionics
becomes less reliable because of higher operating temperature? If I am
writing a really tight timing loop (talking about microseconds, not
milliseconds), does it suffice to be able to measure execution time in terms
of the programmer's source code in a high level language? And how do I find
out how the on-processor instruction/data cache affects execution timing, if
I am allowed to measure execution time only at the high level language
level? There are also many other issues which are more theoretical in
nature, especially about the part the scheduler plays in satisfying
behavioral/timing specifications (more about this later?). All these need to
be studied carefully so that we can design real-time embedded systems that
the public can trust.
                                         — Al Mok

[Volz and Mudge 86] "Instruction Level Mechanisms for Accurate Real-Time
      Task Scheduling", Proceedings of the IEEE Real-Time Systems Symposium,
      pp. 209-217, New Orleans, Dec 2-4, 1986.

[Mok 84] "The Design of Real-Time Programming Systems Based on Process Models",
      Proceedings of the IEEE Real-Time Systems Symposium, pp. 5-17, Austin,
      Dec 4-6, 1984.

Re: Electronic Steering (RISKS 4.62)

Bob Ayers <ayers@src.DEC.COM>
Thu, 12 Mar 87 09:47:04 pst
In Risks 4.62, mlbrown@nswc-wo cites several "things that manufacturers
should have caught" and mentions three, namely "Pinto gas tanks, Audi
5000's sudden acceleration, Ford E-350 ambulances."

Now from previous postings on Risks and elsewhere, it is clear to the
non-hysterical that there is no "Audi 5000's sudden acceleration" except
for that produced by the driver stomping on the accelerator.  (Experiments
where both the brake and accelerator were floored disprove the 60-Minutes
docu-drama theories.)  (And, to forestall weak replies that the "thing that
should have been caught" was only the pedal layout, I'll remark that I've
seen it stated and not denied that the Audi's pedal layout, while skewed,
is by no means the most skewed layout on the market.)

The perceptions about Pinto gas tanks, too, are largely the result of
public alarm (I might say hysteria), fanned by those in charge of selling 
newspapers. In actual government crash-tests, the Pinto  
  a) passed the government-defined government-given tests and  
  b) was not even at the bottom end of the vehicles that passed.

I haven't heard of the "Ford E-350 ambulances and their propensity to
catch on fire."

Inputs For Quantitative Risk Assessment

Thu, 12 Mar 87 09:52 EDT
While I realize that a totally time-deterministic system is unachieveable
(on quantum theoretical grounds if none other) I am unwilling to simply
throw up my hands and hack code until everything seems to work correctly.  

My definition of a real-time system is a system in which time is a 
quantitatively managed resource.  The key word here is QUANTITATIVELY.
Scheduling is obviously the very heart of time management. Not only am 
I repulsed by the notion that people are to be told that there are 
questions that they may not ask, the correctness of a real-time program 
depends first and foremost on how it is scheduled.  Telling a real-time
programmer not to care about scheduling is like telling a scientific
programmer not to care about units.

In doing quantitative time engineering, I am prepared to work with 
tolerances and with probabilities.  If I'm working in microseconds,
I can do my calculations with some nanosecond slop.  I can also carry
out my calculations with probability distributions rather than values.
But I want to be able to make statements like the following at the end:

    "The time between when you start to depress the brake pedal
     with velocity v until the brake shoe makes contact with the
     drum is k milliseconds plus or minus j microseconds."
    "After you start depressing the brake pedal with velocity v
     the brake shoe makes contact with the brake drum in at least
     m milliseconds with probability n."

In other words, I want to provide my customers with the same sort of
quantitative risk assessment information that the people who build
medical systems (drugs, procedures, treatments, therapies, etc.) are
required to provide their customers.

In the work that I do, the execution time of one instruction counts.
For some of the machines I'd like to work with I am unable to obtain
either tolerance-bounded or probabilistic measures of instruction
execution times.  I had an opportunity to chat with the designer of the
Transputer recently.  Not only did he not know the execution time 
distribution the instructions, he didn't really care what they were.
This is not a directed criticism.  All of the advanced system designers
I've spoken with (processor, language, operating system, network, etc.)
abandon time determinism at the drop of a hat.  It's the in thing to do.

What I'm making a pitch for is quantitative risk assessment because
with it will come quantitative system engineering including quantatitive
time engineering.  We all praise safe systems and deride unsafe ones
but this is just Monday morning quarterbacking.  The question is how do
you build a safe one and avoid building an unsafe one.  The scientific method
seems to have worked well in other sciences, maybe it's time we gave it
a whirl.  Or maybe we just having too much fun playing in the silicon.

Re: Active car suspension

Geof Cooper <imagen!apolling!geof@decwrl.DEC.COM>
Thu, 12 Mar 87 11:05:17 pst
The French auto manufacturer, Citroen, has been selling cars with active
hydraulic suspension since the advent of the DS series in the late 50's.  It
probably used an analog computer, which is a little off the topic here, but
the benefits and risks might be of interest to people considering suspension
controlled by digital computer.

The suspension compensates to give better traction when going around
corners, down or up hills.  It does this by actively tilting the car closer
to upright.  The car rides high at city speeds for better clearance of bumps
and gets closer to the rode the faster you go, for better highway stability.
You can also flip a lever in the cab and have the car rise a foot off the
ground to go over snow, on dirt roads, etc..  It gets higher than a jeep.

The hydraulic system replaces the conventional car jack.  To change a tire,
you raise the car to its highest position, put something akin to a jack
stand underneath the middle of one side, and lower the car again.  The flat
tire rises into the air (RISKs: doesn't work right if you're on a hill, you
need someone to sit on the car if you are just trying to rotate tires).

The same facility allows the car to drive on straight roads with only three
wheels.  Thus, a flat doesn't cause you to swerve, and a blowout doesn't
cause you to go out of control (RISK: my father once drove for five miles on
a flat because he didn't know anything had happened).

As you might imagine, driving in a Citroen 15-20 years ago was a bit like
driving in a concept car.  And the suspension is only one of the advanced
features it had.

There are some interesting RISKs.  The car "settles down" after you turn off
the engine.  Since in city-mode it runs higher than most cars, you could end
up settling down on someone else's bumper.  The hydraulics will not lift the
car up in that case.  If you forget that you have the suspension in the
raised position and go on the highway, the car is not as stable as it would
otherwise be.

My father once had a break in the rubber tubes carrying hydraulic fluid
while on the highway.  All the warning lights in the car went on at once,
including a large red light that said "STOP".  The car remained stable, but
he lost power brakes, power steering, and power suspension all at once, and
had to get towed away.  A normal precaution was to carry an extra can of
hydraulic fluid around with you.
                                              - Geof
Phone: (408) 986-9400 (work)
Postal-Address: IMAGEN, 2650 San Thomas Expressway, Santa Clara, CA 95052

Ozone hole a false alarm? (Response to Henry Spencer, RISKS-4.61)

Mark Brader <mnetor!>
Thu, 12 Mar 87 14:07:13 EST
Scientists studying the problem from Antarctica announced some results that
were covered by TV news recently.  They said that products such as chlorine
dioxide were found at 50 times the expected levels, which indicates that the
ozone really is combining with chlorofluorocarbons and the need for action
is urgent.  I don't know how the measurements were made, but they seemed
                                     Mark Brader    

  [This is getting a bit peripheral to Risks, but I don't think in view
  of the above that it's right to close the topic with the note from
  Henry, who, incidentally, has no TV. - msb]

     [[It is still relevant: there is still a problem if the computer
     model is incomplete.  But, I tend to put messages of lesser relevance 
     or lesser general interest toward the end of the issue.  PGN]]

Phone problems (RISKs in auto-dialers)

David Barto <scubed!megatek!barto@seismo.CSS.GOV>
12 Mar 87 10:43:16 PST (Thu)
Recently here at Megatek, we have had a couple of problems with our
phone systems.  Both are the fault of the operator NOT checking that
the information typed in was correct.  The first was mine, the second
someone else.

My mistake was in reversing 2 digits in the phone number.  Instead
of calling a computer, I called a person.  Every hour, for 4 days.
The person complained to the phone company who traced it to us.
I did not notice since I had 2 phone numbers to try, the first failed
and the second worked.  I thus ignored the problem, went to USENIX, and
while I was gone the problem was reported.  (See what you get for making
changes on a friday before going on a trip? :-)

The second was more serious.  To call a long distance number the prefix
was 1-919-XXX-XXXX, the person entering the number entered 91, followed
by 2 backspaces to enter the 1 long distance code.  The back spaces
were ignored and the resulting number was 911-XXX-XXXX.  Dialing 911
with a modem indicates you are a deaf person requiring help.  The
number was traced back to us (our rotary) and the first person to be
called and asked about it was ME!  (I made the first mistake, I made
the second one.  Sound reasonable... :-)  It was found to be another
machine doing the dialing, and was corrected.

In both cases the number written on the piece of paper was correct and
the number entered in the computer was wrong.  I wonder how often this
happens.  We are becomming more computer oriented (look at the number
of modem ads you see, and the number of PC and PC clones that are sold.)
Could this become a major RISK in the future, dialing wrong numbers
for hours on end?

David Barto     sdcsvax!sdcc6--\        
barto@sdcsvax.ARPA  ihnp4--!bigbang-!megatek!barto

Re: Mode C Transponders

Thu, 12 Mar 87 07:39:28 PST
Phil Karn writes:

> The simple fact is that your actions put others (like me) under involuntary
> risk, ...

What is "involuntary" about the risk?  When you step aboard a plane, you
know there's a risk that something will go wrong.  No one forces you on.

Does your argument also apply to cars?  Suppose I can afford to equip my
Rolls Royce with one of those new radar-based automatic braking systems,
making it much less likely that I'll plow into someone from behind.  Now,
don't I have the right to expect that everyone else out there will want to
make *ME* safer as well, by installing these systems in their cars?  The
technology is there, we could certainly cut down the number of involuntary
(think of all those innocent passengers) traffic deaths, if only people
weren't so selfish and independent-minded.  And we're not even talking here
about systems that cost more than the cars themselves, unlike some of the
aircraft collision-avoidance systems you want every plane owner to rush out
and buy.

Anyway, this whole discussion has little to do with computer system
risks, so let's shut it down.    [AGREED.  P.]

Jan Wolitzky, AT&T Bell Labs, Murray Hill, NJ; 201 582-2998; mhuxd!wolit
(Affiliation given for identification purposes only)

Automatic Landing Systems

Hugh LaMaster <>
Thu, 12 Mar 87 13:19:01 pst
There has been a lot of discussion on RISKS recently about air safety.  I
have three questions that perhaps someone out there has more detailed
information about.

The first is the Automatic Landing System (ALS) that has been used in
Europe.  Could someone summarize what is known about ALS as far as RISKS is
concerned?  Is it (believed to be) a fail-safe system? Is it run by a
digital computer (with software :-) ) ?  Are there steps being taken now to
bring such a system to the U.S.?

The second question is about active controls on commercial jet transports.
Somewhere, I read that the new McDonnell-Douglas MD-11 (follow on to the
DC-10) will have relaxed aerodynamic stability, made possible by (naturally)
active controls.  What happens after a lightning strike wipes out all the
avionics (it has happened)?  It does not follow that if it is OK for the F-16,
it is OK for a commercial transport.  I assume that there won't be zero-zero
ejection seats for each passenger.

The third question is whether there any completely fly-by-wire transports
out there now? I have read that there is a version of the Airbus with
fly-by-wire, but it didn't say whether it also had conventional controls.
The same questions as above apply.

  Hugh LaMaster, m/s 233-9,  UUCP {seismo,topaz,lll-crg,ucbvax}!
  NASA Ames Research Center                ames!pioneer!lamaster
  Moffett Field, CA 94035    ARPA
  Phone:  (415)694-6117      ARPA

("Any opinions expressed herein are solely the responsibility of the
author and do not represent the opinions of NASA or the U.S. Government")

F-111 Losses

Thu, 12 Mar 87 12:47:03 EST
Back when I used do computerized cartography and terrain modelling I'd
heard a story (unconfirmed rumor, interesting scuttlebutt) that some
part of the F-111 lossage was due to active (and simple, low
technology) countermeasures.  In particular, one or more low altitude
airbursts with an appropriate mortar round (chaff? ) a couple hundred
meters ahead of the plane would cause a very strong terrain-like radar
return to suddenly appear.  The G force limiting in the program was
implicit, with the flight path obtained by filtering the observed
terrain into a smooth curve.  This worked great as long as the
observed terrain is really static and doesn't do anything strange.
When the terrain follower suddenly observed a "mountain range" appear
immediately ahead of the aircraft it panicked by trying to climb over
it.  The resulting acceleration could be well outside specs.

Since the planes tended to reuse routes such as valleys, the
implementation of this alleged countermeasure is simple.  An observer
with a phone alerts the mortar crews down route that a plane is coming
and sould arrive in X seconds.  The timing of the airbursts is
non-critical.  If the mortar crews get it right the wings fall off.
If they are too far in front of the plane it pulls up hard anyway
making it vulnerable to conventional AA.  Even if the AA doesn't get
the plane, the crew has just had a very disturbing experience.  Their
confidence in the terrain following system is shaken.  Their mission
plan might be screwed up possibly causing them to miss navigational
checkpoints.  They've got lots of excuses to abort and go home.

The immediate solution: program the system so that it ignores sudden
changes in terrain.  It wasn't obvious what would happen if there was
a real hill on the other side of the burst.  The smart money says
don't rely on terrain following in hilly terrain.

-- Rob Fowler

Re: Computers in the Arts (Computer lighting)

Shannon Nelson <decvax!tektronix!reed!psu-cs!nelsons@ucbvax.Berkeley.EDU>
Wed, 11 Mar 87 15:52:41 PST
I've worked in several different theatres as a lighting 'techie', both as a
stagehand and as the lighting designer.  In at least two of the auditoriums
I've worked in, the lights were controlled by a computer; one was computer
assisted, the other was fully computerized.

In the computer-assisted setup, the computer was used to control fast,
highly complex fades and effects in addition to hand controlling of other
fades qued on often 'inspirational' actors.  This was a nice balance, as the
operator could override the computer at anytime with a full set of manual
controls (100+ channel 2 scene preset board).

In the fully computer controlled system, everything was done through the
computer:  setting levels, fade timing, special effects, etc.  It had a
ss/sd floppy drive on one side, and looked kinda like a tvi 910 terminal
with several slider controls instead of keys.  As long as the computer was
operating correctly, it was useable.  Unfortunately, it was originally
installed wrong (by a regular (subcontracted) electrician, not a specialized
theatre electrician) and was often unusable, aside from the "backup system".
It was eventually fixed, but still, when it goes, it's gone.

There was a (half-baked) backup of all 96 channels tied to twelve controls
(single scene, no presets).  If the computer died from heat or power glitch,
we'd turn the key over to 'backup' and hope that we could restart the
computer and return to the correct sequence without too much confusion.

In such systems, the lights often cannot be done by hand because 1) there
aren't enough hands (sometimes the case with over 100 seperate controls),
and/or 2) the system wasn't designed to be overridden.  In some cases, all
that's left to human control are the follow-spots, which do little for
flooding the whole stage.

Do I like working with computer lighting systems?  Of course.  They make my
work as a designer much more exciting with the effects that are possible.
Do I want full overrides, even if I don't have enough hands? You'd better
believe it!! --

Shannon Nelson            ...tektronix!psu-cs!nelsons

Please report problems with the web pages to the maintainer