The Risks Digest

The RISKS Digest

Forum on Risks to the Public in Computers and Related Systems

ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator

Volume 9 Issue 42

Monday 13 November 1989

Contents

o Equinox TV programme on A320
Bev Littlewood
Chris Dalton
o European Safety is not always BETTER
Bruce C. Brown
o Artificial lightning
PGN
o Another intrusive database with associated privacy problems
Bill Gorman
o Re: "Computer Error" in Durham N.C. election results
Gregory G. Woodbury
o Re: Computer errors and computer risks
Willis H. Ware
D. King
o Info on RISKS (comp.risks)

Equinox TV programme on A320

B.Littlewood <sd396@CITY.AC.UK>
13 Nov 1989 17:42:48-GMT
Brian Randell, in RISKS-9.39, gave a brief description of this programme,
which he thought was quite well done.  I have a few more reservations than
Brian, but this may be because most of the film of me ended up on the
cutting-room floor!

There were some interesting moments, though, not least in some of the
assertions still being made by Airbus and its constituent companies.
Throughout there was a curious belief in the power of the `systems approach'.
Perhaps the strongest statements came from Gilles Pichon (Chief Engineer, A320,
Aerospatiale), who said:

"Safety is ensured by the system approach: the computers, the sensors, the
whole environment, the power sources and the computer software.  All this has
to be analysed very closely for safety.  And the method of analysis, which is
called safety analysis, has been around for over 20 years.  It was used for
Concorde and it's still valid today."

Pretty unexceptionable, you may think, but this statement is quickly followed
in the film by a more detailed examination of the fault-tolerance of the
fly-by-wire system:

  Jacques Troyes (Flight Control Manager, Aerospatiale): "We work with three
  pieces of information completely independent of each other.  And we can only
  guarantee to protect the aircraft with certainty when we are sure of having
  at least two pieces of information."

I assume this is a reference to the A320 software fault tolerance, but if so it
is a bit confusing.  The voice-over questions whether developing all versions
from the same specification might introduce common faults.  Then cut to

  John Knight (University of Virginia): "It appears to be possible to build
  diverse programs where the programs will allow you to recover from certain
  kinds of difficulties that the software may get into.  The real issue is we
  have no way of predicting ahead of time how successful that kind of technology
  is going to be."

Voice-over then says Airbus have stated the system is designed to fail only
once 10**-9 hours [lovely slip that -- they mean 10**9!].  Then cut to me

  Bev Littlewood (City University): That means this system should fail every
  billion flight hours.  A billion flight hours is about 100,000 years.  There
  is no-one in this business believes you can design systems to that
  reliability."

  Pichon: "Having done all our safety analysis we are confident that we
  achieved the 10**-9.  And for those who have some doubts we can also say that
  even with no fly-by-wire the aircraft can fly safely because we have the
  mechanical back-up at the end."

Voice-over explains that mechanical back-up is really only meant to keep the
aircraft flying until computer system is got working again.

  Littlewood: "Airbus could have had a fully functioning mechanical back-up on
  that aircraft, so that in the event of total loss of the computer system it
  was still flyable.  Now what they've got is a vestigial mechanical back-up.
  Really all that you can control if you totally lose the computer system is
  the rudder and tail trim."

  Gordon Corps (Engineering Test Pilot, Airbus Industrie): "I had one Northwest
  Airlines pilot land the airplane totally satisfactorily using just the back-up
  system."

Later in the film, there is an interview with Michel Asseline, who was pilot in
charge of the A320 which crashed on the Mulhouse flight.

  Asseline: "When I pull the stick to up position, the flight controls, the
  elevator controls, go to down position . . . why?  That would be the good
  question."

Whereupon a man from the DGAC (the French certification agency) is asked
whether he had seen any evidence to support this claim.  He said he had not.
Then cut to

  Bernard Ziegler (Vice-President, Engineering, Airbus Industrie): "By no
  means, never, the computer want to land the aircraft, never.  I would even
  say, believe it or not, that we have put in our computer law to resist to
  land.  The pilot land the aircraft, and nobody else."

The voice-over then comments that, 15 months later, the official French report
into the crash has not been published, but it will almost certainly clear the
computer.

Later there are reports of other problems pilots have met, including the
following exchange

  Gino Scattolini (A320 pilot): "As we were coming in to land with the engines
  at idling speed, the two engines accelerated up to climbing speed, and as the
  automatic systems were not working we might have left the plane's flight path
  if the crew had not intervened.  But safety was never at risk."

  Corps: "There have been fine tuning changes done in some aspects of the
  software and I guess they will go on for some time, as we say just to cure some
  of the teething problems that we have seen.  But they haven't affected anything
  of significance associated with flight safety at all."

The film ends by looking to the future, and in particular the possibility of
unstable commercial aircraft.

  Ziegler: "It's clear that we say active control, which is a natural
  derivative of the fly-by-wire, we will be able to reduce the weight of the
  structure, to reduce the surface of the control.  That is also the next step
  and we are working also in this direction."

  Littlewood: "Now making an airliner unstable would bring enormous economic
  benefits because it would cut down drag and the aircraft would be much more
  fuel efficient.  But an unstable aircraft has to be controlled by computer
  all the time; there is no possibility of a mechanical control by the pilot.
  So that next step is one I think we ought to be worrying about."

  Brian Perry (UK Civil Aviation Authority): "There's nothing we know which
  would say we shouldn't consider such an approach.  We believe that if you
  take the system approach which looks at the hazards following system failure
  or system non-availability, and can satisfy yourself that the safety criteria
  are met, then the aircraft is potentially certificatable."

Certainly I agree with Brian that the film is worth seeing (I think it is to be
shown in the US -- probably on Public Broadcasting).  It would have been good to
have more debate and less lovely pictures of the A320 doing fancy things.  But
a couple of things did come out.

First, it seems that senior engineers (Pichon, above) are still trying to
convince us that they have achieved the mythical 10**-9.  Are they fools or
knaves?

Second, there seems to be confusion about exactly what can be expected of the
back-up system.  Do Airbus want us to believe that airline pilots will be able
to land on this, or that they will never need to do so?  (or both?)

Third, there have been software problems.  (I'm intrigued by the notion of
`fine tuning': is this similar to `it's not a bug it's a feature'?)

Fourth, this was the first formal statement I had heard that Airbus were
working on active control.  It seems to me that the certification agencies have
to take a more active role here than is represented by Perry's statement.

BEV LITTLEWOOD, Centre for Software Reliability, City University,
London EC1V 0HB


Mistake in Equinox "Fly-by-wire" programme

Chris Dalton <crd@hplb.hpl.hp.com>
Mon, 13 Nov 89 14:53:55 gmt
The Equinox programme mentioned by Brian Randell and Lindsay Marshall in Risks
9.39 and 9.40 has a glaring mistake in the script... I hope.

The announcer quite clearly explains at one point that the system was designed
to fail every "ten to the minus nine hours".  Moments later, an engineer says
they achieved the "10^-9 error rate".  (I'd recorded the programme, so I was
able to check what was said.)

A case of losing something in the translation?

Chris Dalton  Hewlett_Packard Labs, Bristol BS12 6UF, UK  +44 272 799910
crd@hplb.hpl.hp.com  crd@hplb.lb.hp.co.uk  ..!mcvax!ukc!hplb!crd


European Safety is not always BETTER

Bruce C. Brown <bcbrown%fnal.dnet@fngate>
Sun, 12 Nov 89 23:17:10 CST
Recent discussions in RISKS have suggested that safety standards in Europe are
superior to those we enjoy here, and indeed, some recent statistics suggest
that that may be true in some important senses.  However, we should beware in
adopting the stance that they have the answers and we have nothing to give.

In particular, I was in Hamburg, Germany for a six month assignment in 1987 and
was AMAZED to discover that the American safety requirement that all door open
OUT and that buildings have doors which are locked such that noone can be
locked in were unknown there.  If I forgot my keys and worked late, I could be
locked in at three separate levels: my own office, the office corridor, and the
building external doors.  MOST rooms had telephones, but...

Like everything else, we need to be careful about adopting anything wholesale,
without review.

Bruce C. Brown, Magnet Test Facilty,
Fermi National Accel Lab, Batavia, IL 60510


Artificial lightning

"Peter G. Neumann" <neumann@csl.sri.com>
Mon, 13 Nov 1989 16:19:48 PST
Lightning may be natural, or may actually be stimulated artificially by
man-made conditions in situations in which lightning might otherwise not occur.
The latter occurred in the second and third of the following cases:

  ... three spectacular lightning accidents involving aircraft or spacecraft:
  (i) In 1963, a Boeing 707 flying at 5000 feet near Elkton, Maryland, was
  struck and destroyed by lightning, killing all occupants (3).  Lightning
  apparently burned through one of the metal wings, or in some other manner
  entered the fuel tank inside that wing, and caused the fuel vapor there to
  explode. (ii) In 1969, Apollo 12 artificially initiated (or "triggered") two
  lightning flashes, one to ground and one intracloud (IC) discharge, when it
  was launched through a weak cold front that was not producing natural
  lighting (4).  Although this rocket-initiated lightning caused major system
  upsets and minor permanent damage, the vehicle and its crew survived and were
  able to complete their mission successfully.  (iii) In 1987, an unmanned
  Atlas-Centaur vehicle (AC/67) was launched into weather conditions that were
  similar to those present at the launch of Apollo 12 and triggered a lightning
  discharge to ground (5).  This discharge upset the computer memory in the
  vehicle guidance system and produced an unplanned yaw rotation, and the
  associated stresses caused the vehicle to break apart.

This paragraph is excerpted from an article in the 27 October 1989 issue of
_Science_, Natural and Artificially Initiated Lightning, by Martin A. Uman
and E. Philip Krider, pp. 457-464.  References 3-5 are given in the article.

The Atlas-Centaur case was previously reported in RISKS-4.70 (1 April 1987, no
joke) and RISKS-4.96 (6 June 1987).  The Apollo 12 case has not -- to the best
of my knowledge been noted here previously.  More generally, the detailed
discussion of artificially triggered lightning in the Science article should be
particularly interesting to RISKS readers.


Another intrusive database with associated privacy problems

"W. K. (Bill) Gorman" <34AEJ7D@CMUVM.BITNET>
Mon, 13 Nov 89 14:00:50 EST
               MEXICO AND USA SIGN TREATY TO ATTACK TAX EVASIONS

   The mexican Secretary of Treasury, Pedro Aspe Armella, and Nicholas Brady,
northamerican Secretary of Treasury signed a treaty to detect and combat tax
evasions on both countries.  By means of this treaty both nations will have
access to information concerning the income of mexicans living in the States
and of northamericans living in Mexico. This deal also establishes the
possibility to exchange information on people who evades taxes.  This data will
be exchanged only if the laws and rights of the citizens are respected in each
country. With this information, the fiscal authorities expect to detect
possible tax evasions by incomes obtained in another country and that are
frequently not reported to the Government.

  [... another privacy "loophole" via a shared data base. The comment about
  protecting privacy by "observing all laws of both countries" is absurd. Once
  the data is in the hands of any third party, individual, corporate or
  national, controls imposed from without are nothing more than "gentlemen's
  agreements" observed out of courtesy and/or convenience.  Bill]
    [Bill did not indicate where this appeared.  I edited it lightly to fix a
  few typos (e.g., Treasure), but left "northamerican Secretary".  PGN]


Re: "Computer Error" in Durham N.C. election results

Gregory G. Woodbury <ggw@wolves.uucp>
Tue, 14 Nov 89 02:51:39 GMT
>           [With a "Duke" as Governor of both Massachusetts and California,
>           I wonder if any Duke Univ. folks were governing this election?  PGN]

    Well, I work at Duke University, and I was also working as an
assistant to the precint registrar for my home precinct -- that might
count a "governing" this election ;-)

    It was an enlightening experience doing an election.  The machines
used around here are purely mechanical.  The only electricity used is for a
fluroescent light over the front panel.  After the election, a hand crank is
used to force the counter wheels against an NCR paper (or rather vice versa)
and the numbers are transcribed by hand to the official ballot reports.
    In my precinct, we had two calculators (electronic) to assist in the
tally, and I still caught an error by doing a simple parity check on the
numbers as they where called out.
    As for the "Computer Error" down at the Board of Elections...
What goes on there is simply a convenience for the press and candidates.
The BoE staff has a few PC's and spreadsheets set up to do simple calculations
and the person who got to put it together this year simply messed up one
of the cross-tabulations.  There is not, as far as I know, a specific
program that the BoE uses, just a PC set up in the County Commission meeting
room used for simple arithmetic.  Prior to last year they used simple hand
calculators and never had a problem.
    The "Official Election" comes about one week after the voting when
the registrars from each pricinct sit down "en banc" and canvass the actual
numbers from the machines in their precincts and double check each other
via whatever method is most convenient.  Some of the registrars actually
do the arithmetic in their heads and have the result written on their
scratch pads before the various calculator people can announce what they
get.  All in all, its still dependent on mechanisims and mental skill.

Gregory G. Woodbury, Sysop/owner Wolves Den UNIX BBS, Durham NC


Re: Computer errors and computer risks (Saltzer, RISKS-9.41)

"Willis H. Ware" <willis@rand.org>
Mon, 13 Nov 89 16:05:42 PST
<>   .............................  In a traditional library, it was
<>   possible to invade your privacy by making a list of all the books you
<>   have every checked out.  All an investigator had to do was open every
<>   book in the library and look to see if you had signed the card
<>   inside.  The information was publicly available, but actually it was
<>   benignly protected by an enormous collection cost, so noone every
<>   worried about it.

In privacy discussions, one frequently hears the point about convenience
of collection, magnitude of what can be obtained for little effort, etc.,
but the concept of "benign protection by the status quo" is a very adroit
way of capturing the point and of relating it lay folks.

His point also brings to mind one made very forcefully by Richard Hamming
(currently on the faculty of the USN Postgraduate School at Monetery, CA)
many years ago.  In paraphrase, he said: "when something changes by an
order of magnitude, there are fundamental new effects."

Certainly from the benign library of the past to the computerized one of
now, the effort to assemble one's reading list has changed by been many
orders of magnitude.

Hamming's Law is really what's behind so many of the computer-induced
effects, and it's also the underlying issue in having such effects
understood among the laity.  It's certainly a big part of the problem with
getting legislators to pay attention; they think everything is fine just
because it has been fine in the past.
                        Willis Ware

   [Hamming is also well known for not standing on Isaac Newton's feet.  PGN]


Re: Computer errors and computer risks (Saltzer, RISKS-9.41)

<king@kestrel.edu>
Mon, 13 Nov 89 09:54:08 PST
<> In <a href="/Risks/9/40">RISKS DIGEST 9.40</a>, Randy Davis says,
<> > . . . I suggest the simple test above: Ask, can the identical
<> > problem can arise in the absence of computers?
<> I claim that it is not that simple...

I think i must respectfully disagree.

Consider the two examples given ...  Yes, i will concede that the cost of
collecting library patron information precludes its use to send "appropriate"
junkmail.  The cost of collecting DMV information precluded its use for
junkmail as well.  But these are trivial invasions of privacy, and not the ones
i'm most worried about.

Consider the possibility of a new McCarthy Era.  During the old McCarthy Era,
readers of certain books in the library WERE found and used for purposes which
i would assume many would just as soon forget.  The fact that this information
was available in dilute form protected nobody.  Recall that both the imaginary
but believable society of 1984, and the real tyrrany of Nazi Germany, were
quite plausible/possible with only "human computers".

Consider the case of the skiptracer.  The cost of a DMV trip would be a
negligible portion of his cost of doing business; no doubt he would have
several cases he could service with a single trip.

So, in part, i support the original thesis that for serious breaches of privacy
[as opposed to trivial annoyances] lack of a computer is no protection against
data collection.  In part, i offer a possibility for a NEW protection.

It is practical for the head librarian of even the largest city to personally
walk the half-dozen disk packs containing circulation information to the
library's degausser, together with the appropriate tapes, and defend the
privacy of the more-than-two-month-old circulation information reasonably
absolutely.  It is possible for the populace to order the DMV to implement
access poicies.  In short, the compactness of the information implies that the
privacy afforded patrons of a particular service will not be the accidental
result of the way things happen to be, but the result of an explicit decision.

-dk

Please report problems with the web pages to the maintainer

Top