Please try the URL privacy information feature enabled by clicking the flashlight icon above. This will reveal two icons after each link the body of the digest. The shield takes you to a breakdown of Terms of Service for the site - however only a small number of sites are covered at the moment. The flashlight take you to an analysis of the various trackers etc. that the linked site delivers. Please let the website maintainer know if you find this useful or not. As a RISKS reader, you will probably not be surprised by what is revealed…
From: Friedrich von Henke (vonHenke@SRI-CSL)
I am temporarily acting as guest moderator of the Risks Forum.
Peter Neumann had to go rather abruptly on an overseas trip, and
the transition happened a bit disorderly. My apologies to all of
you who were eagerly awaiting their twice-weekly cost of RISKS readings
but had to go without. I hope to have things under control now.
Apparently the hiatus has had the effect of slowing down the stream of
contributions to a merely trickle; please don't hesitate to get the flow
going again!
Friedrich von Henke
----------------------------------------------------------------------
Date: 11 Oct 85 19:39:27 PDT (Fri)
From: Nancy Leveson <nancy@UCI-ICSD.ARPA>
To: RISKS@sri-csl.ARPA
Subject: medical software incidents
I was just on a panel concerned with Software Safety at an IEEE
conference on Computers in Medicine and heard about some more
incidents involving software faults.
The first was cited in a recent RISKS forum message (about the
programmable implanted pacemaker which was inadvertently reprogrammed by
emitted magnetic fields from an anti-theft device in a retail store),
but the patient
was cited as having survived. Unfortunately, his weakened heart was
unable to stand the increased pace, and he died.
Other recalls by the FDA involve:
1) An infusion-pump (used for insulin) had a software problem which
caused the infusion of insulin or dextrose to be delivered at the
maximum rather than the lower intended rate. This occurred when certain
valid data was entered according to user instructions.
2) A programmable pacemaker "locked-up" when being reset by an
external programming device. Luckily this occurred in a doctor's office,
and the doctor was able to revive the patient.
3) A multiple-patient monitoring system was recalled because the software
got patients' names mixed up with the wrong data.
4) An algorithm was incorrectly programmed in a diagnostic lab instrument
which caused certain patient data to be reported erroneously as all zeros.
The reference for these incidents (for those who want to quote them) is:
H. Bassen, J. Silberberg, F. Houston, W. Knight, C. Christman, and
M. Greberman. "Computerized Medical Devices: Usage Trends, Problems,
and Safety Technology," in Proc. 7th Annual Conference of IEEE
Engineering in Medicine and Biology Society, Sept. 27-30, 1985,
Chicago, Illinois, pp. 180-185.
Nancy Leveson
University of Calif. Irvine
I would like to bring some activities to your attention which are
going on in Europe, especially within and triggered by EWICS TC 7.
The European Workshop on Industrial Computer Systems (EWICS), TC on
Systems Reliability, Safety and Security (TC 7) is working since about
10 years in this area, having some 100 members from industry, research
and university. Previous work resulted in Position Papers on
Development of safety related software
Hardware of safe computer systems
Guidelines for verification and validation of safety related software
Guidelines for documentation of safety related computer systems
Techniques for verification and validation of safety related software
System requirements specification for safety related systems
Current working areas include:
System integrity
Software quality assurance and metrics
Design for system safety
Reliability and safety assessment
Besides conducting about four working meetings a year the TC is organizing
the IFAC/IFIP Workshop on Achieving Safe Real-Time Computer Systems
(Safecomp'79, '82, '83, '85, '86).
The results of the work of TC 7 are introduced into the standardisation
process (IEC, ISO, and national bodies) as well as used by companies
and licensing authorities.
Those interested in more information can either contact me or the
current Chairman of TC 7: Mr. J.M.A. Rata, Electricite de France,
1 Avenue du General de Gaulle, F-92141 CLAMART FRANCE.
There exists an American counterpart to EWICS TC 7, but it was not
possible to attract enough interested persons to keep it alive.
The Japanese counterpart is also active, but due to the language
barrier communication is minimal.
Udo
To: risks@SRI-CSL.ARPA
When someone gets pinned to the wall by a robot, something different is going
on as compared to when someone gets gunned down by an FBI agent operating
under incorrect information retrieved from the NCIC. Both cases may lead to
specific tragedies, yet the example of risks from robots seems to me to be
qualitatively different from many other computer-use risks.
The difference is that robots are used primarily in environments where
mechanically-oriented people are accustomed to balancing the risks of new
machinery against the benefits. These people have, over the years, learned to
deal with risks from gear trains and drive belts, from swinging tailends on
steamshovels, from runaway elevators, from inadequately supported cranes.
They watch out, they learn from accidents, their insurers offer advice, they
make mistakes and take risks, and they learn. To a first approximation, an
industrial robot presents a risk similar in kind to other new machinery, and
there is a moderately well-working system in place that is accustomed to
watching for the risks. If anything, the average mechanic is suspicious of a
new piece of machinery in direct proportion to its complexity, newfangledness,
and gadgetry level, so is probably expecting the robot to screw up in
marvelous ways. One might wish to argue with the particular balance that an
industry has struck between risks and benefits, but it is unusual to find one
in which mechanical risks are not understood at least moderately well.
The mechanic's suspicion of the new gadget is the essence of what seems to be
missing in many other applications of computers, and why it is so important to
raise awareness of the need to assess risks. I'm not convinced we need to
harass our colleagues in the robot business with risk-consciousness-raising.
We should be instead talking to their installers to find out what we can
learn.
Jerry Saltzer
By chance I saw an article in an issue of the
"Journal of Environmental Engineers" (published in England, date of
issue about 10 months ago, I believe) about the sensitivity of
a microprocessor-controlled automobile control system to external
electromagnetic radiation. As I recall, a CB transmitter near the car
could, at the right frequency, make the engine slow down or speed up.
Perhaps this article would interest some of your contributors.
Bennett Smith
IRCAM
31, Rue Saint Merri
75004 Paris, France {seismo,philabs,decvax}!mcvax!ircam
From: Dave Dyer <DDYER@USC-ISIB.ARPA> To: risks@SRI-CSL.ARPA The human element really is where the action is, and it is a completely two-edged sword; Human actions which have the power to "fix" something almost inherently also give the power to "break" things equally severely. Conversely, weighty check and balance systems intended to prevent abuse end up preserving the status quo, however good or bad that may be. The "police computer horror story" I'm most familiar with is illustrative. This is a well documented case I've been reading about in ACLU publications. It seems some poor soul had his wallet stolen, and some criminal adopted his identity and later was involved in a robbery/murder. Through some circumstances peculiar to the case, the stolen identity, but not the culprit, were known to the LAPD. The detectives working on the case put the stolen identity into a national police computer. Our hero was stopped for a routine traffic citation, the computer coughed his name up, and he ended up on ice for a few days as a murder suspect. So far, this is pretty harmless and understandable. Eventually the guy's identity and and non-involvement were establishd and he was turned loose. Then it happened again. And Again. The guy began carrying a letter from the local chief of police, saying he wasn't the guy the computer said was wanted, but that didn't cut it when he traveled. The problem was that the LAPD detectives who put in the original "want" refused to remove it. Eventually the guy (and the ACLU) got the courts to mandate expunging the computer. I think the detectives involved and the LAPD are being sued. Quite rightly. The point is, it is <<hard<> to design a system that can do its intended job, permit discovery and correction of errors, and resist unautherized or inappropriate use. I can't imagine a system that can do all three.
From: the tty of Geoffrey S. Goodfellow <Geoff@SRI-CSL.ARPA>
[forwarded to RISKS by Bill Keefe <keefe%milrat.DEC@decwrl.DEC.COM>
from TELECOM Digest Volume 5, Issue 55, October 24, 1985]
Americans' Privacy Exposed by New Technology, Congress Told
By LEE BYRD - Associated Press Writer
WASHINGTON (AP) - The explosion in communications technology has so
outpaced privacy laws that Americans have little or no protection
against a plethora of new ways for government or private adversaries
to pry into their lives, a congressional agency reported today.
The non-partisan Office of Technology Assessment found that 35 out
of 142 domestic federal agencies use or plan to use various
electronic surveillance methods, including modern devices not
governed by a landmark 1968 law that circumscribed the use of
wiretaps and bugs - concealed microphones.
The agency said 36 agencies, not counting those in foreign
intelligence, already use a total of 85 computerized record systems
for investigative or intelligence purposes, and maintain 288 million
files on 114 million people. The report raised the ''technically
feasible'' specter of these being linked into a single data base
network that could track untold numbers of citizens without due
cause.
The report, requested by House and Senate committees, noted that
many new and uncontrolled methods of surveillance are made possible
by the very technologies of which more and more Americans are
availing themselves - electronic mail, computer conferencing,
cellular and cordless telephones, beepers and electronic pagers.
Intercepting such devices is easy, and ''the law has not kept pace,''
the agency said.
But other devices, such as miniature television cameras and pen
registers - which monitor the numbers called on a given telephone
line - have enabled new ways to spy on people even if their own
communications habits are more old-fashioned, the agency noted.
Rep. Robert W. Kastenmeier, D-Wis., chairman of the House Judiciary
subcommittee on courts and civil liberties, said the study ''shows
how the law in this area has broken down; it is up to Congress to fix
it. If we fail to act, the personal and business communications of
Americans will not have the privacy protection they deserve.''
Sen. Charles McC. Mathias, R-Md., said the report ''documents how
new and more intrusive forms of snooping have followed in the wake of
the exciting advances in communications technology,'' and agreed
Congress must ''bring federal privacy laws up to date.'
Rep. Don Edwards, D-Calif., chairman of the House Judiciary
subcommittee on civil and constitutional rights, said, ''While the
attorney general of the United States is claiming that the civil
liberties granted by the Constitution should be limited to the
'original intentions' of the framers, the technological possibilities
for government surveillance have exploded. The framers knew nothing
of closed-circuit television, wiretapping and computer data banks.''
The report noted that the Fourth Amendment, which protects ''the
right of the people to be secure in their persons, houses, papers and
effects, against unreasonable searches and seizures,'' was written
''at a time when people conducted their affairs in a simple direct,
and personalized fashion.''
Neither, said the report, has Title III of the Crime Control and
Safe Streets Act of 1968, which was designed to protect the privacy
of wire and oral communications, kept pace.
''At the time Congress passed this act,'' the report said,
''electronic surveillance was limited primarily to simple telephone
taps and concealed microphones. Since then, the basic communications
infrastructure in the United States has been in rapid technological
change.''
The congressional agency said it could not estimate the extent of
electronic surveillance in the private sector, saying only ''it is
probable that many forms ... go undetected, and if detected, go
unreported.''
But in its survey of the federal bureaucracy, OTA found 35 agencies,
mostly in the Justice, Treasury and Defense departments, used or
planned to use:
-Closed circuit television, 29 agencies.
-Night vision systems, 22.
-Miniature transmitters, 21.
-Electronic beepers and sensors, 15.
-Telephone taps, recorders, and pen registers, 14.
-Computer usage monitoring, 6.
-Electronic mail monitoring, 6.
-Cellular radio interception, 5.
-Satellite interception, 4.
As for the 85 computerized record systems that could be used for
surveillance purposes, none of the operators provided statistics
requested by the OTA on record completeness and accuracy.
Under the 1968 law, wiretaps and bugs are prohibited without a court
order based on the affirmation of a high-ranking prosecutor that a
crime has occurred, that the target of the surveillance is involved,
and that other means of investigation would be ineffective.
According to the Administrative Office of the U.S. Courts, federal
and state judges approved 801 out of 802 requests last year for
electronic surveillance, primarily wiretaps and hidden microphones,
at an average cost of $45,000.
The agency said that while there is some promise in emerging
techniques for low-cost data encryption or other means to protect
communication systems from eavesdropping, ''there is no immediate
technological answer ... against electronic surveillance.''
Foreign intelligence cases are governed by a separate law, so the
CIA, National Security Agency and Defense Intelligence Agency were
not included in the survey.
To: "risks" <risks@sri-csl>
In a recent issue of one of the digests on the net, there was a problem
mentioned that seems to have a bearing on risks on computer systems, particu-
larly as use of computer networking increases in the future. At their request,
the names have been changed to preserve anonymity.
Apparently for the past month, the people who reside on the
bitnet have been unable to receive [this digest]. There is a long
story behind this [...]. This story is also a *lot*
of guesswork as to what happened.
At the beginning of September, [our system] changed its host
name to conform to the new domain name standards. The site we
were using to get to bitnet, did not recognize the new name and
began rejecting all mail from [our system]. We did not become
aware of this because we were not receiving any rejections or errors
back from the gateway. We were however, receiving mail *from* the
people on Bitnet who were asking what happened to their [...] digest.
We attempted to contact the people at [the gateway] but of
course the mail failed and they never did anything to correct the
problem which they, of course, were not aware of because nobody was
complaining. (Sounds like a Catch-22 situation if I ever heard
one).
In any case, the problem has now been resolved.
Unfortunately, these people have missed close to 50 digests. There
is no way I can tie up the mailer at [either the host or the gateway]
in order to remail the messages. I also understand that there is no
way to use FTP from the bitnet.
It appears that several incidents conspired together to cause the loss of this
information, and although the loss was not critical, it will take much time and
effort for the people involved to catch up. If this had been a more critical
information transfer, it might have been corrected faster; however, there is a
good chance that it would have gone undetected anyway. It seems to be one more
reason for information about any potential changes to be passed on to any sites
that may be affected and to be thoroughly checked on both ends to prevent this
kind of thing from reoccurring.
Lynne C. Moore (Moorel@Eglin-Vax.Arpa)
Some doubt has been expressed in this forum recently about people changing grades and living happily ever after. I can't talk about college systems, but in high school all the grades, attendence, etc. for my high school and several other high schools were kept on a mainframe in a centralized location. There is a system in Michigan called MOIS for vocational data, and on the back of my MOIScript on computer science was the transcript of a terminal session between the attendance people and the computer. The login message gave the phone number of the mainframe. The password was overprinted, but that is useless — you can learn to read through almost any overprinting. Access to the grading, course scheduling, and attendance programs was by providing a social security number which was echoed and not overprinted. I thus found myself able to do really miraculous things. Being sixth in my high-school class, I had no real motivation to use my knowledge maliciously, and informed the administration. Some safeguards were added (old social security numbers retired, certain social security numbers only giving access to certain programs, restricting access to certain programs to certain accounts), but I'm sure those could have been circumvented with minimal effort — I was a fairly good systems hack on the operating system, and there were people who could hack rings around me. The operating system was a simple three-tier ring system, and a lot of the features put in for the sake of usability made it very insecure. I send this to give first-hand evidence to those who have been posting doubts that such things happen.
One of the systems programmers here at RPI made a point about administrators and students sharing the same computer: You really aren't that much more secure when you have administrators and students on separate computers if there is a network or dial-up connection to the administrative computer than you are when administrators and students are on the same computer. If you have network or dial-up connections to an administrative computer, it isn't too difficult for a student with an autodial modem to write a program on a PC that simply tries all possible phone numbers for certain telephone exchanges and record the numbers that produce a carrier tone. Then the student could have another program that tries passwords unless the system disconnects the line after a certain number of unsuccessful tries. The major advantage of separating administrators and students is that it might be more difficult for a student to access an administrative file from a student account assuming the administrative file has the proper protection set.
You can create an extremely effective audit trail if you think about it for a
while. That just makes the problem more "challenging".
Suppose you do your auditing one day and find that there were unauthorized
grade changes made for every student in the CS department and 1/2 of them
requested (incorrect) printed transcripts. It seems unlikely that everybody
independently broke in on the same day. So who do you penalize? How many
transcripts have been mailed out already?
Suppose no grades were changed but there is a trojan horse waiting to raise
the grade only under certain circumstances?
My point is that the data doesn't have to stay changed forever. And you can't
check the auditing records for every transaction, if you expect to gain by
using the computer. You need to do as much as you can, of course, but beware
of the false sense of security...
Mark.
Please report problems with the web pages to the maintainer