Today's Washington Post (12-21) reports on a General Accounting Office study, "Financial Integrity Act" about Information Technology applications within the government. The overall message is that information systems technology is costly and risky. Here are some quotes: "Federal agencies operate over 53,000 unclassified automated systems...life cycle costs in the billions of dollars..." The article reports costs of $17 billion for fiscal 89 versus $9 billion in fiscal 82 for these computer applications. "Invariably these systems do not work as planned, have cost overruns in the millions and even hundreds of millions of dollars, and are not developed on time. Congressional interest..has increased..." Some specific examples are cited. "...defense [business as well as command and control] far exceeded their original costs estimates...fell significantly short of expectations...design flaws, misjudgments in requirements, poor program management." The article describes a Navy financial system whose costs grew from $33 million to $479 over nine years of development. Also, an IRS system is estimated to cost $1 billion, and has not shown benefits from currently operational components. Except for specific details, all of this is old news to many of us who have been involved in large systems of various kinds for a while. What does seem to be new are trends toward larger fiascoes and for increased government concern a by people who control purse strings. Also, do stories of such failures indicate that we reach an intellectual brisk wall when we try to develop large systems? Or, are we simply repeating dumb mistakes? David Davis, MITRE Corp., McLean, VA Standard Disclaimer
Excerpted from the S.F.Chronicle, 19 Dec 1989: The California Supreme Court cleared the way yesterday for the use of standardized psychological tests in criminal trials to prove that a defendant does not fit the personality type likely to have committed the charged crime. In a 5-2 ruling, the court rejected a comparison that likened personality tests to lie detectors or voiceprints, which are excluded from tials because their reliability is not commonly accepted by the scientific community. The court majority said introduction of standardized psychological tests in trials is not a revolutionary development and the tests reliability can be challenged by prosecutors. "We see no reason to subject (these tests) to the special restrictions governing admission of new, novel or experimental scientific techniques not previously accepted by the courts," wrote Justice David Eagleson for the majority. Chief Justice Malcolm Lucas dissented, saying the decision opened the way for new "mini-trials" focusing not on a defendant's guilt or innocence but on his personality profile and whether it conforms to "the profile displayed by the average child molester, robber, arsonist, or whomever." He acknowledged that personality tests have been admitted by some courts to ahow a defendant's mental state at the time of the crime. But that is "far different than using them to exclude defendant from the relevant class of defenders in much the same manner as a blood test or voice print," Lucas wrote. With the vote, the court reversed the child molestation convictions of a Kern County couple found guilty of committing lewd acts with four young boys in 1983 and 1984. During the trial of Margie Grafton and Timothy Palomo, they attempted to call as an expert witness a psychologist who had given them two commonly used tests -- the Minnesota Multiphasic Personality Inventory and the Millon Clinical Mutiaxial Inventory. The psychologist, Roger Mitchell of Bakersfield, was prepared to testify on the basis of the test results and his interviews with Grafton and Palomo that they showed no indications of deviance and were unlikely to be involved with the charged crimes. Out of the presence of the jury, Mitchell told the trial court judge that the 566-question Minnesota test, copyrighted in 1943, had a reliability rating of over 70% [sic!!!] in diagnosing the illness of some patients and included hidden questions that detected lies by the person taking the exam. Many experts believed that the test makes it impossible [sic!!!] to conceal an abnormal personality profile, Mitchell told the judge. But the trial judge ruled that Mitchell could not testify because the defense had failed to prove the tests met the legal standard of general acceptance in the scientific community. The court yesterday overturned that ruling, saying the judge should have allowed Mitchell to testify. The majority also found that if his testimony had been allowed, it may have changed the outcome of the case. What has this to do with comp.risks? The tests at issue are all wholly computerized. Moreover, as if common sense were not enough, it is well established (the tests were statistically debunked in the 1960s) that the maximum accuracy of diagnosis, in most unrealistically favorable circumstances, is of the order of 20% -- hardly an improvement over a guess. Besides, the test is readily foolable, so much so that it is generally regarded as per se invalid the second time it's taken by the same person. Moreover, the "reliablity" pertains only to the crudest mental types of disability (schitzophrenia, paranoia, and five other yes/no nasties), whereas the computer tests are generally preprogrammed to spew out pages of rambling mumbo-jumbo analogous to daily horoscopes, execept that long psychiatric words are used. Such print-outs more often than not contradict themselves in details. I was once compelled to take such a test, by a California judge. The examiner, who actually gave classes in psychology to high-power groups of attorneys and judges, without blush permitted me to answer difficult questions by tossing a coin, because I said that was my "natural response" to the test. Still, the computer nevertheless reported that the test was "valid." On one page it reported that a compelling aversion to publicity, on another that I avidly desired publicity. One amusing diagnosis was the computer's finding that I lacked a sense of humor! I think this is worth the long posting, because these computerized tests are administered almost universally now, and decide everything from employability to the suitability of a mother to be a mother.
A few weeks ago there appeared an article in RISKS that reported on a computer software firm that had been successfully sued for "software malpractice". I didn't keep the article, so my details are sketchy. As I recall, the judge found that a programmer was culpable since he did not abide by ACM standards for software, and since he was an ACM member he should have adhered to those standards. This has been nagging at me for days. I've been an ACM member for some eight years and I've never even SEEN these standards. Furthermore, I do not necessarily endorse ACM standards just because they are from ACM. The industry certainly hasn't embraced all ACM standards (or any one else's for that matter). Even if one DOES endorse the standards, one wouldn't necessarily use them in all cases. For example, when writing experimental or demonstration software, formal development methods are often not used. The efficacy of the "quick and dirty" approach not withstanding, there are time when this is done UNDER THE DIRECTION of management AND the customer. It boggles the mind to consider the possibility that one could be sued for "software malpractice" without there being a formal definition of it or a legal standard. Breach of contract might have been a reasonable finding, but this???? I've heard it said in many arenas that the legal system in this country is out of control. This seems to be yet another example of a system out of kilter. Will this lead to a situation where no one dares to sell software to another party? Will programmers seek to defend themselves from their employers for fear of software quality violations? Stay tuned...
The Brooklyn Public Library has recently put in a computerized card catalog system. The branch nearest me (the main branch, as it turns out) has about 4 terminals in the main lobby, which also contains the card files (in GOK how many thousands of drawers). It hasn't taken people long to become totally dependent on the computerized system. Typically there are lines with 2-4 people waiting at each terminal (probably a 5-20 minute wait) and not a single soul using the card files. Unless there happens to be a terminal free (very rare) I just do it the old fashioned way. I can look up 3 or 4 books in much less time than it would take me to wait for a turn at the terminal. I wonder how long the average person will wait to use the "new fast computerized catalog" before resorting to the "old slow way", even if the old way is faster.
The Bellevue, Washington, Journal American ran an article on telephone glitches collected from its readers. o "... a dark stormy night, a desperate woman, a telephone from Kafka". Using a pay phone at a service station along the highway, she dialed 0 then the number and the phone went dead. She tried again and again. She finally reached an operator and found out that (a) the phone was owned by a private company (not AT&T), (b) collect calls could not be made, and (c) she could not be connected with an AT&T operator. o Another woman received hourly calls with the recorded message "The maximum dollar amount is exceeded by the number 4-4-4-4-4-4." The problem was traced to a pay phone at a local gas station with a full coin box. The phone was programmed to call someone when the coin box was full. Unfortunately, it was programmed with the wrong number. o For six months a woman had long distance calls to Mexico City on her bill. The phone company finally discovered that the woman's line was cross wired with a neighbor's line. The twist in the story was that the neighbor had recently moved into the house and did not realize it had TWO lines (the phone company had failed to disconnect the second line when the previous owner moved out). The neighbor's bill looked normal since most of his calls were on his primary line. Only when he used a secondary phone were the calls billed elsewhere. o One family had phones that rang three times then stopped. Friends said they called and let the phone ring 20 times and no one answered. "After extensive investigation [GTE] found an electronic glitch at a nearby central office." The article concluded: "the letters we received showed that people are dependent on the telephone and, when things go wrong, hardly in a mood to hear a pitch about the values of consumerism. True phones don't go wrong often, they said, But when they do ..."
In the recent Electronic Design News (a trade newspaper), the cover story is about Fujitsu's recent claims of 200,000 hrs MTBF on some of its hard drives. That is nearly 23 years of 24-hour continous use. Needless to say, this number is not obtained from units in the field, but extrapolated from their test data. Other manufacturers consider that number to be marketing strategy, although some have large (but not that large) numbers, too. If its any consolation, the article said that the drives had a 5 year warantee... I could not help but be reminded of the 10^-9 claims of some software producers...
I worked with WWMCCS in 1985/6 and many of their problems stemmed from a technology bet that they had made 3-4 years earlier. They had a software first philosophy that stressed using as much commercial-off-the-shelf (COTS) software as possible. They bet that by 1986, respondents to the RFP would be able to bid COTS 1) multi-level secure operating systems and 2) distributed heterogeneous DBMSs. It is 1989 and there are still precious few (if any) examples of either. When it became obvious that neither was going to appear by 1985/6 when they were scheduled publish the RFP, they were not prepared and the program began scrambling to find stop gaps. It was downhill from there.
The November House Judiciary Committee hearing on computer virus legislation will be shown on C-Span on December 23 (8:45 am) and December 24 (1:30 am). This was an interesting and timely event with representatives from NIST, ADAPSO, CBEMA, CPSR, and members of Congress discussing technical and legal responses to the issues raised by computer viruses. The prepared CPSR statement on computer virus legislation is available from the CPSR Washington office. Please send me a note if you would like a copy. Marc Rotenberg.
In the last few days I have learned the risks of posting to comp.risks! 1) No, Convex's "lint" does not edit files, I meant "indent"! Oops. (Re: Fun and Games with Indent, RISKS-9.54) 2) I also learned the dangers of using questionable terms like "yellow peril". I thought these days that "yellow peril" was so outdated that it carried about the same force as "Redcoats". Evidently I was wrong. If I upset anybody, I'm sorry. You can stop educating me as to the correct current popular definition now! Hopefully in a few decades or so my usage will become correct. (Re: Risks of Mail, RISKS-9.55)
Please report problems with the web pages to the maintainer