The RISKS Digest
Volume 9 Issue 26

Wednesday, 20th September 1989

Forum on Risks to the Public in Computers and Related Systems

ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator

Please try the URL privacy information feature enabled by clicking the flashlight icon above. This will reveal two icons after each link the body of the digest. The shield takes you to a breakdown of Terms of Service for the site - however only a small number of sites are covered at the moment. The flashlight take you to an analysis of the various trackers etc. that the linked site delivers. Please let the website maintainer know if you find this useful or not. As a RISKS reader, you will probably not be surprised by what is revealed…

Contents

Hospital problems due to software bug
Joe Morris
Man-Machine Failure at 1989 World Rowing Championships
Geoffrey Knauth
Responsibility, Doctors, Military vs Software Developers
Leslie DeGroff
Organizational Accreditation: More Thoughts
Frank Houston
Jon Jacky
An interesting answer to the distributed time problem
Roy Smith
Re: Risks of distributed systems
D. Pardo
Info on RISKS (comp.risks)

Hospital problems due to software bug

Joe Morris (jcmorris@mitre.arpa) <jcmorris@mitre.mitre.org>
Wed, 20 Sep 89 09:54:11 EDT
>From the _Washington_Post_, Wednesday, 20 September, page F-1 (the separate
business section), without permission...

SICK SOFTWARE CHECKS IN AT 100 HOSPITALS
Programming Error Affects Admissions

About 100 hospitals around the country, including Washington Hospital
Center, were forced yesterday to switch from computers to pen and paper
for major bookkeeping functions because a software program could not
figure out what day it was.

Officials said there was no permanent loss of data or threat to treatment
of patients.  But the incident, apparently caused by a mistake in programming,
demonstrates how institutions are accepting the risk that major disuptions
might occur in the workplace as more and more functions are handed to
computers.

The incident affected hospitals that use software and services provided
by a Pennsylvania company called Shared Medical Systems Corp.  The
company stores and processes information for hospitals on its own mainframe
computers and provides software that can be used on IBM equipment.

Problems began to appear at numerous hospitals early yesterday morning.
As call after call for help arrived at SMS headquarters, technicians there
realized a pattern was emerging and advised clients to shut down parts of
their computer systems as they searched for the cause.

The problem was traced some hours later to a program that allows hospitals
to automate the ordering and reporting of laboratory tests.  Due to a
fault in the aging software, the machines were unable to accept as valid
the date September 19, 1989, and went "into a loop," refusing to work,
spokesman A. Scott Holmes said.

By day's end, computer services at about 100 of SMS's 600-700 client
hospitals had been disrupted.

One computer specialist described the problem as a "birth defect", an
accidental fault put in a program in its early days that later threatened
the system's health, in contrast to a "virus," a program that is written
with the deliberate purpose of replicating itself and causing disruption.

The incident also underlined how exacting the computer programmer's art
can be.  To build a system that handles routine tasks of hospital
administration, programmers must write thousands of lines of "code," or
computer instructions, that are intended to let the machine know what to
do in every conceivable circumstance.

At Washington Hospital Center yesterday, nurses used paper forms to admit
patients, saving the information for entry when the computer system was
restored.  Staff members also had to manually request lab tests because
the automated system would not work.

Delays caused inconvenience but "patient care was not affected," said
Claire Fiore, directory of public affairs and marketing for Washington
Hospital Center.  By about 5 p.m., the hospital's admissions computers
were up and running again.

Also hit was Capitol Hill Hospital, which reported minor disruptions.  "We
were able to function normally," said spokeswoman Lisa Poulter.  "It may
just take a little longer to order services."


Man-Machine Failure at 1989 World Rowing Championships

Geoffrey Knauth <geoff@sunfs3.UUCP>
Tue, 19 Sep 89 10:27:09 EDT
In the Petite Finals of the 1989 World Rowing Championships in Bled,
Yugoslavia, five strokes into the Coxed Fours race the microphone wire came
loose from my cox-box (an amplifier and tiny computer).  I lay in the bow and
coxed the entire race unaware that the speaker in the stern was out.  Some of
my crew could hear me, some could not, and in the confusion we rowed vacantly,
far below our potential.  The truth of a loose wire discovered just after the
finish, that I had not fastened the connector well enough, and that a year's
worth of hard work and good speed had just been wrecked, contained enough pain
and horror to burn in my soul forever.

Geoffrey S. Knauth, Camex, Inc., 75 Kneeland St., Boston, MA 02111
(617) 426-3577                                standard disclaimers

     [Risks are certainly pervasive.  This problem is not life-or-death or
     even row-versus-wade, but nevertheless must have had a devastating effect
     on all those involved — but particularly the cox.  On the other hand, I
     am surprised that in a sport so close to being natural (apart from
     computer designed shells) a computer would be permitted on-board.  Next we
     might have on-line sensors monitoring crew heart rates to probe their
     limits, automated instructions from the computerized coxswain replacement
     using digitally synthesized voice and ear-phones, and even remote computer
     guidance from the coach overhead in a helicopter.  Technology knows no
     bounds!  PGN]


Responsibility, Doctors, Military vs Software Developers

Leslie DeGroff <DEGROFF@INTELLICORP.COM>
Mon 18 Sep 89 09:53:56-PST
  A varety of Risks messages have discussed medical risks, military risks and
software accreditation.  A recent one from Douglas Jones triggers me to present
a different perspective on the issues of risks and accreditations.  A
fundamental feature of the current medical systems including accredited
institutions is that Doctors have a very high level of personal responsibility
for what happens.  It is not the institution on the line, it's Dr. XXX and in
most larger institutions there are reviews of any "failure" that the Dr. must
answer to peers that understand the issues quickly after such a failure.  There
are are drawbacks to this system, in many cases Dr's over compensate to this
Command responsibility by disregarding or belittling input from patients or
nurses with more contact with patients.  In the military also there is a
somewhat corrupted notion of command responsibility, officers involved in major
incidents/accidents are brought to trial with both career and freedom on the
line if there have been fatalities. The breakdown in the military system comes
in the fact that the line officiers are often in catch 22, if they don't use an
automated system they are risking their ship and if they do and it is used to
attack the wrong targets they are still at risk.
   There are two different messages that come out of consideration of the
medical and military systems, one that creator/providers be made more
personally responsible and two that users themselves bear major responsibility
for results.  This would have a multiplicative effect on lowering risks,
personal responsibility (ownership) of creators tends to drive craftmanship and
quality and usage responsibility tends to slow down the adoption of risk edge
tools.
                           Les DeGroff (intellicorp support)


Organizational Accreditation for Computer Assurance: More Thoughts

Frank Houston <houston@itd.nrl.navy.mil>
Tue, 19 Sep 89 11:30:45 -0400
In hope of stimulating more thought and discussion, I submit the following
responses to comments about my "food for thought" message.

 >Date: Fri, 15 Sep 89 08:51:05 CDT
 >From: Douglas W. Jones
 >Subject: Medical accreditation: good for big shops only?
 >...
 >One primary difference, from the accreditation point of view, is that most
 >hospitals are big, while many quite competent software shops are small.
 >Hospitals with fewer than 100 employees are rare, while there are many
 >corporations in both the software and hardware fields that are smaller.

I suggest that Dr. Jones look around his home state and the neighboring states.
Many hospitals are not large, and university hospitals are exceptionally large
compared with hospitals in general.  It may be true that hospitals with fewer
than 100 employees are rare, but they are held to standards like JCAHO's
to receive Medicare and insurance payments.  Moreover, JCAHO accredits
many other organizations and facilities, such as nursing homes, which are
small.

 >Another difference is that most hospitals have only a small number of staff
 >physicians.  Most physicians practicing in a hospital are not on the staff,
 >but instead, have an associate relationship.

Good point.  It is also true that many software and engineering shops
supplement their staff with consultants, which sets up a similar situation.
Nevertheless, I believe that JCAHO reviews credentials and records of both
staff and associates when considering renewal of accreditation.  The task of
reviewing staff credentials for an engineering firm should not be any more
difficult.

 >Finally, a hospital is a clearly defined organizational unit; even if it is
 >part of a hospital chain, the physical and staff boundaries are easy to.
 >define.  Many software organizations are far harder to circumscribe. Finding
 >accreditable units in a large corporation may be quite difficult, and I doubt
 >that entire corporations (say Ford or Xerox) are appropriate units of
 >accreditation.

I do not suggest that accreditation will be an easy concept to implement.
Consider, however, the impact of a customer requirement that an organization
be "accredited for the development of critical software/systems."  I am sure
that Ford or Xerox would find a way to identify the appropriate units if FAA
made accreditation a prerequisite for award of ATC contracts.

 >These comments are reasonably representative of a fairly extreme libertarian
 >view that government regulation of the free market is inherently inept and has
 >a corrupting influence on the quality of services offered by the market.

I must point out that my proposal does not specify accreditation by a
government agency.  In fact, having close up and personal experience with
government regulation, I share some facets of the "libertarian view."
JCAHO is not a government agency; it is more like an industry consortium;
another reason why I suggested it as a preliminary model.

 >Government did not invent the Computing Sciences Accreditation Board, and it
 >did not invent the Institute for the Certification of Computing Professionals.
 >Right now, nobody is forced to seek accreditation or certification, but it is
 >not hard to imagine a few court cases establishing the principle of strict
 >liability for losses caused by software failures, and if this happens, I see
 >little hope for avoiding forced adherence to some kind of accreditation or
 >certification standards in the software industry.  If government does not
 >force these on us, the insurance industry that emerges to provide malpractice
 >insurance for programmers will do it.

I believe that government would be quite willing to defer to a private
sector institution for regulating developers of critical software and systems,
as it has in the case of JCAHO for medical practice.  Product liability
underwriters and other insurers probably would also.

 >Date: Wed, 13 Sep 89 10:46:37 PDT
 >From: Bob Ayers
 >Subject: Medical accreditation: based on "customer" clout?
 >...
 >First, it's not that Medicare accepts accreditation as quality-proof, but
 >will accept real proof too — rather they accept (as I understand it) _only_
 >accreditation.

Legally Medicare and, therefore, Medicaid accept JCAHO accreditation in lieu
of their own reviews.  JCAHO accredited hospitals are "deemed" to be in
compliance with most of the Medicare Conditions of Participation for Hospitals.

I suggest that both individuals and organizations need to be accredited.  If you
look carefully at my outline, I suggest that an accreditation board might
examine professional certifications.

Nancy Leveson has argued very persuasively that, paraphrasing, "...someone
competent must bear responsibility for software in critical systems ... some
sort of professional licensing or certification is needed."  I have been
skeptical, but now I think that, with the further incentive of accreditation,
professional certification could be made to work.

I do not think accreditation is a "silver bullet."  It is just as corruptible
as any system of standards; however it uses the power of the marketplace to
make corruption counterproductive.  Both industry and customers would soon
mistrust a corrupt accreditation system.

The views expressed above are my own and do not reflect in any way the policies
of FDA.
                                         Frank Houston


Organizational Accreditation (What is BS5750/ISO9001?)

Jon Jacky <JON@GAFFER.RAD.WASHINGTON.EDU>
Fri, 15 Sep 1989 16:50:58 PDT
In RISKS 9(23), Frank Houston (houston@itd.nrl.navy.mil) proposed consideration
of some kind of organizational accreditation for software quality --- the
idea was, the whole organization would be certified, rather than particular
people or products.

Does something like this already exist in the UK?  Inside the front cover of
SOFTWARE ENGINEERING 88, PROCEEDINGS OF THE SECOND IEE/BCS CONFERENCE, there is
an ad from a systems house bearing an official-looking seal that says "BSI -
REGISTERED FIRM".  The ad copy says, "First Systems House to Win BS5750 (now
ISO 9001)", without further explanation.

Can any British RISKS readers enlighten us?

- Jon Jacky, University of Washington


An interesting answer to the distributed time problem

Roy Smith <roy%phri@uunet.UU.NET>
7 Sep 89 00:05:09 GMT
    Lots of things in lots of places depend on having a consistant time
reference distributed to many places.  Lots of people have worked out high-tech
solutions to the problem.  Most of them don't work very well.  When's the last
time you saw a master clock system that worked?  The one in our building
certainly doesn't.  Well, today, I saw an interesting solution in Mt. Sinai
Hospital in New York.  Take any of the thousands of closed circuit TVs in the
hospital and set it to channel 6 and you get a picture of a clock.  And not a
digitized, computer generated picture either.  Somewhere there is a TV camera
pointed at a good old sweep-secondhand analog clock, and that's what you see on
channel 6.  Sometimes low-tech solutions are the best.

Roy Smith, Public Health Research Inst., 455 First Avenue, New York, NY 10016


Re: Risks of distributed systems

<pardo@june.cs.washington.edu>
Sun, 17 Sep 89 12:05:06 PDT
In RISKS 9.25, Eugene Miya writes (about the special risks of parallel and
distributed systems) that ``[I]t isn't the researchers ... who nead exposure
[to parallel/distributed issues], it's the students who need this [...]''.

There are a growing number of schools that are using Ada to replace Pascal and
Modula-2 in the introductory programming courses.  Using Ada gives a
standardized mechanism for introducing issues such as exceptions and
(concurrent) tasks.

I think that everybody will agree that distributed/parallel is
under-represented, under-standardized, and over-risky.  There is hope, however.

    ;-D on  ( Is parallel uniprocessor an oxymoron? )  Pardo

Please report problems with the web pages to the maintainer

x
Top