Risks associated with developing and using computer systems have been documented widely (e.g., by PGN) and have become part of popular awareness. Economic costs resulting from these risks are huge, though presently unquantified. They include the costs of system failures, abandoned system developments, and lost opportunities to build valuable systems whose complexity is deemed beyond present art. Despite the widespread awareness of this situation, nothing fundamental has been done to change it. New system technologies attempt to improve matters by giving system builders better tools. Large corporate and government initiatives to improve system trustworthiness have been announced. Despite many advances, system development risks have not abated. New systems keep getting developed whose defects are discovered too late to be repaired economically. Repairs become patches and basic defects remain embedded in the system. These problems are pervasive, both in safety and infrastructure-critical applications and in the mundane data-processing applications that support the national economy. With all the awareness of the hazards of system building, why does this bad situation continue? We suggest that the reason is the weakness of current risk assessment for new systems. Warnings about computer system risks that are given in an early stage do not have the force of warnings in other disciplines such as medicine and civil engineering and so they are ignored or discounted. What can be done to improve the believability of warnings about development hazards? We do not envision a super-powerful tool that can generate a high-confidence hazard assessment for all situations. Rather we see the need for a profession of hazard auditors who have earned acceptance based on their scientific skills and experience. The need for their skills should be assumed and demanded in all system development efforts. Their observations (and if necessary, testimonies) should be communicated to purchasers, builders and users. Tools should be developed to support their analyses. Building such a profession would be a substantial effort but the effort would surely be justified by the enormous cost of current development deficiencies. Government agencies, corporations, universities and professional associations all have clear roles to perform.
It's said that every new technology carries with it the possibility of a new crime, and in a similar way, every new safety feature has some possibility of actually causing more problems than it's supposed to solve. This was brought home to me the other night at Logan airport where I noticed a shuttle bus whose destination sign persisted in showing the special legend "Call 911", despite the best efforts of the driver and several dispatchers whose conversations I was able to listen in on from the driver's radio of the bus I was on. I don't know if they eventually got the sign turned off or had to take the bus out of service, nor do I know how many calls to 911 that night there were about it.
What made an Airbus rudder snap in mid-air? David Rose - *The Guardian* (UK), The Observer, 13 March 2005 http://observer.guardian.co.uk/international/story/0,6903,1436374,00.html When Flight 961 literally began to fall apart at 35,000 feet, it increased fears of a fatal design flaw in the world's most popular passenger jet. At 35,000 feet above the Caribbean, Air Transat flight 961 was heading home to Quebec with 270 passengers and crew. At 3.45 pm last Sunday, the pilot noticed something very unusual. His Airbus A310's rudder - a structure 28 feet high - had fallen off and tumbled into the sea. In the world of aviation, the shock waves have yet to subside. ... In November 2001, 265 people died when American Airlines flight 587, an Airbus A300 model which is almost identical to the A310, crashed shortly after take-off from JFK airport in New York. According to the official report into the crash, the immediate cause was the loss of the plane's rudder and tailfin, though this was blamed on an error by the pilots. ... There have been other non-fatal incidents. One came in 2002 when a FedEx A300 freight pilot complained about strange 'uncommanded inputs' - rudder movements which the plane was making without his moving his control pedals. In FedEx's own test on the rudder on the ground, engineers claimed its actuators - the hydraulic system which causes the rudder to move - tore a large hole around its hinges... The Observer has learnt that after (an earlier) disaster, more than 20 American Airlines A300 pilots asked to be transferred to Boeings, although this meant months of retraining and loss of earnings. Some of those who contributed to pilots' bulletin boards last week expressed anger at the European manufacturer in vehement terms. One wrote that having attended an Airbus briefing..., he had refused to let any of his family take an A300 or A310 and had paid extra to take a circuitous route on holiday purely to avoid them: 'That is how convinced I am that there are significant problems associated with these aircraft.'
"Oyster card fault causes problems on London Underground" "Automatic updates cause journey renewal problems" Daniel Thomas, *Computing*, 10 Mar 2005 Londoners were faced with travel problems this morning after an IT error meant hundreds of commuters could not renew journeys on their Oyster card. The error, which affected the whole of the London Underground (LU) and Docklands Light Railway (DLR), was caused when an overnight electronic updating process went wrong. Transport for London (TfL) and TranSys - the consortium that operates the Oyster card scheme - automatically updates the system each night to add new records and block stolen and canceled cards. But a glitch in the system early this morning means commuters are unable to use machines at Underground or DLR station this morning to add new journeys onto the smart cards. 'Every morning information goes out about stopped cards and it was an error in the data that caused the problem,' said a spokeswoman for TranSys. Passengers that have already paid for their journey or using prepay can still use the system as normal. TfL and TranSys identified the error at 4am this morning and starting issuing a fix to the problem by 8.30am. 'We hope everything to be up and running again by the end of the morning,' said the TranSys spokeswoman. 'We are now looking into what actually caused the error and ways of ensuring this doesn't happen again.'
The 9 Mar 2005 issue of the *Journal of the American Medical Association* contains two articles and an editorial that should be of interest to Risks readers. ROLE OF COMPUTERIZED ORDER ENTRY SYSTEMS IN FACILITATING MEDICATION ERRORS discusses a variety of issues including poor interface design requiring a physician to look at as many as 20 screens to see all the information about a patient, misleading and frequently misinterpreted dosage information, dosage change requires adding the new and deleting the old, poor integration of multiple systems, poor handling of discontinuation and resumption of medications, loss of orders and others. This article appears to be the result of a well done comprehensive study at one specific hospital. The Editorial, COMPUTER TECHNOLOGY AND CLINICAL WORK: STILL WAITING FOR GODOT makes a number of good points such as, "The misleading theory about technology is that technical problems require technical solutions; ie, a narrowly technical view that leads to a focus on optimizing the technology. In contrast, a more useful approach views the clinical workplace as a complex system in which technologies, people, and organizational routines dynamically interact." Anyone interested in systems design will find this interesting. The other Article, EFFECTS OF COMPUTERIZED CLINICAL DECISION SUPPORT SYSTEMS ON PRACTITIONER PERFORMANCE AND PATIENT OUTCOMES: A SYSTEMATIC REVIEW provides a comprehensive review of the topic. If you have access to this journal and the time to read these articles, you will find them interesting.
Recent coverage of a JAMA article on the patient errors (cited by R. Akerman in RISKS-23.78) caused by computers will likely be cited by those who resist the movement towards an electronic medical record. This despite the fact that all acknowledge that the current mixed state of computerized and non-computerized medical systems is abysmal. My perspective on this is that we often miss the core truth of most medical mistakes: they are caused by humans, not computers. In the 1990's I developed several programs designed to find medical mistakes. As such, I spent a lot of time analyzing mistakes, and dealing with defensive reactions by physicians and nurses to the mistakes found. The most common mistake, at its core, was raw human misunderstanding: conceptual misunderstanding leading to misinterpretation of medical data (surgeons who thought the higher the bacterial MIC number, the better the antibiotic, when the reverse is true, and therefore put the patient on an antibiotic guaranteed to be ineffective). A close second was communication failures, where a key report was pocketed, lost or otherwise not communicated to others who would understand its importance. However, in all these cases, the typical hospital political hierarchy sought to turn each of these medical errors into a computer error, lest a human (particularly a Doctor human) be found at fault. While I was grumpy about this at first, I soon realized that there was at least some truth in it, in that more easily understood medical reports, that highlighted and provided some interpretation to key information, and were more widely distributed were in fact improvements worth making to medical systems, and certainly would prevent far more errors than my mistake finding programs would ever find. The problem was however, that as the concept of the electronic medical record began taking shape, resistance to it often cited the end of incident analysis that blamed the computer, rather than the physician or nurse who was primarily at fault. The JAMA cases certainly sound like real problems with the human/computer interface, but they sound suspiciously like the final reports we used to end up on real mistakes made by real humans. The medical environment is extremely complex, understaffed and wrought with automated and semi automated systems that all can fail or conflict whether they are computerized or not. I routinely saw problems with continuation of standing order dosing long before those standing orders were computerized. Blaming the computer misses the point, even if it does point out how the computer system could be made better. The risk is one I often see in The Risks Digest: problems with computerized systems seem to get more attention than the usually much greater problems in the existing non-computerized systems. [Bob, Remember the full name of the Risks Forum! Actually, I get scolded now and then for running noncomputer-related items, but I try to keep noting how the noncomputer computer are also illustrative. PGN]
http://www.dailynews.com/Stories/0,1413,200~20954~2758409,00.html > Los Angeles City Clerk Frank Martinez ordered election workers Tuesday > night to use blue highlighter pens to re-ink thousands of voters' ballots > that had "bubbles" partially or faintly filled in, ... Los Angeles is using the Inkavote system. The voting machines and ballots look like the punchcard system (as in Florida 2000) but the ballot is poked by a special marker that puts a nice round ink blot in the round hole. The company that makes the counting equipment says that this highlighting step isn't needed since their machines are supposed to be able to count ballots that have even a small amount of ink in the right place. There were only a few thousand votes between the 2nd and 3rd place finishers in this election - the top two are headed for a runoff in May.
Bruce Schneier, on his blog recently, mentioned the paper "A Model Regime of Privacy Protection" by Daniel J. Solove & Chris Jay Hoofnagle. His link and discussion is at http://www.schneier.com/blog/archives/2005/03/ideas_for_priva.html The paper's abstract and a link to download it can be found at http://papers.ssrn.com/sol3/papers.cfm?abstract_id=681902 There are a lot of good ideas in this paper, but one in particular struck me as potentially unwise, and certainly underdeveloped: In conjunction with the universal notice, the FTC shall develop a centralized mechanism for people to exercise their rights with respect to their personal information. Such a mechanism would mimic the Do Not Call website, which allows individuals to opt-out of telemarketing and verify their enrollment by visiting a single website. Many interesting RISKS are raised by this. How do you identify the people in the opt-out registry? How do you authenticate requests to deny distribution of certain information? (A malicious person might try to cause difficulties for someone by forging a request to deny all credit data to potential lenders.) How do you determine who may or may not search the registry or read information in it? How do you keep this from acting as the "central key" to all the information on a person, effectively moving us closer to having One Central Database, with all of the problems that brings? There's a huge can of worms here waiting to be opened. Personally, my first instinct would be to avoid such a central registry and instead make it the responsibility of the data collectors to contact each individual with information about what they're collecting and how they're using it, and solicit permission to do so, as well as offer the ability to review the information. This avoids any centralized system, and also avoids certain types of error. For example, if I'm contact regarding a file that appears to have nothing to do with me, I can point that out, rather than have a company mistakenly believe that this file does correspond with my life. (Or I might just say it does, and use the information for identity theft. Who knows?) Curt Sampson <email@example.com> +81 90 7737 2974 http://www.NetBSD.org
Marketscore (www.marketscore.com) offer a free proxy service web users. They offer accelerated downloads and e-mail virus scanning. To use their service users download and install software onto their PCs. Marketscore are quite explicit that they collect a wide range of information about their subscribers, and make information available to web site owners on usage patterns - a sort of "Neilson" for the net. Unfortunately, they also impersonate SSL sites. If a subscriber attempts to set up an SSL connection to say, her bank, the Marketscore proxy sends back it's certificate, and then establishes an SSL connection to the destination. Clearly for this to work, the servers have to decrypt then re-encrypt all of the traffic. Equally clearly, large numbers of credit card numbers, account names, passwords etc are passing through the Marketscore systems in the clear. There is a very good explanation of the problem here: http://www.shellnofcu.com/site/scams.html
Nu.nl Internet <url:http://www.nu.nl/rubriek.jsp?n=224&c=50>, a current news website (in Dutch) had the following two headlines following each other this morning (my translation): * MSN as form of payment http://www.nu.nl/news/495376/57/art.html * Viruses in chat programs increasing in popularity http://www.nu.nl/news/494989/50/art.html The first article is about the SNS bank in the Netherlands introducing a service for small payments via Microsoft's MSN Messenger. The second article is about chat programs like MSN Messenger being used increasingly for spreading viruses. With all the phishers and some of the virus-writers targeting online banks and sites such as Pay-pal, guess where this will be going.
Some months back, Microsoft announced the purchase of an antivirus company. For those in malware research, this appeared to be an indicator that Microsoft would be getting back into the field. Apparently, very few of us are old enough to recall the first time Microsoft "produced" an antivirus product, but those who are remember that the kindest way to describe the attempt would be "not fully thought through." Therefore, we did not look forward to this event with any great enthusiasm. Subsequently, Microsoft announced it had acquired an anti-spyware company. Then it announced a beta test version of an anti-spyware product. Then there was a flurry of announcements about legalities, copyright infringements, products that would be free, settlements of copyright infringement suits, products that would be charged for, and so forth, so I hope I can be forgiven for not recalling exactly where in that timeline came the announcement of a beta version of an antivirus product. I viewed the antivirus beta with some trepidation. The announcement was not particularly clear about the capabilities of the product. It did indicate that the antivirus would be a) limited to specific malware programs, b) concentrate on "worms," and c) there seemed to be hints that the program would run in the background. With apprehension I downloaded the beta antivirus and installed it on one machine. Nothing happened. Nothing appeared in the Start menu programs list. Nothing appeared in the "Program Files" directory. Nothing appeared in the "Remove Programs" list. Nothing disappeared from my malware samples directory. Subsequently, I have been receiving announcements from "Auto Update" that the "Windows Malicious Software Removal Tool" was ready for installation. Previously I found this completely bewildering. In the latest instance, if you choose "Custom Install," it does inform you that the tool will run once, and then be deleted from your computer. This makes a bit more sense. According to Microsoft, more information for this update can be found at http://www.microsoft.com/malwareremove. This page states the same "run and then disappear" process, along with the assertion that the program will generate a report on the status of your computer. (So far, in my experience, this hasn't happened.) The page lists seventeen pieces of malware that the program "cleans." The mention of "background" operation now seems to be tied to the Auto Update process, although it isn't completely clear that the antivirus itself doesn't run in the background. (The "run and delete" description would seem to indicate that the antivirus doesn't run in the background.) I am interested in results from any others who have studied the program in more detail, including issues related to where the program looks for infections, what is cleaned, removal of malware from memory, cleanup of the Registry, scanning of mail files (many of the malware items listed are spread via e-mail attachments), and so forth. firstname.lastname@example.org email@example.com firstname.lastname@example.org http://victoria.tc.ca/techrev or http://sun.soci.niu.edu/~rslade
(Rothwell, RISKS-23.77) The Delivered-To field is added by qmail (and other MTAs) when the message is delivered, so it's only visible to the recipient named in the field. For example, if I send a message to email@example.com and BCC firstname.lastname@example.org, the copy of the message that Joe receives might get a Delivered-To field like: Delivered-To: email@example.com And I'll get one like: Delivered-To: firstname.lastname@example.org But since these are completely different copies of the message residing on different servers, Joe won't be able to see that I received a copy of the message. Of course if I were to forward my copy of the message to Joe with its Delivered-To fields intact, that would expose the blind copy. Dave Sill, Author, The qmail Handbook
Actually, it has always seemed obvious to me that the real purpose of the photo ID check, and the reason the airlines cooperate, is that it interferes with a secondary market in non-refundable tickets.
Having worked in the bar and liquor store industry for the last 6 years, I can tell you the scanners are not the end all solution either. At my work we have a large collection of fake ID's that scan just fine. All the info matches what is printed on the ID, but the ID was still fake. You can buy a mag strip writer for a few hundred dollars off the Internet and write whatever info you want on there. If you are smart enough, you can generate the correct 3-D barcode, and print that on the ID, too, and all the high priced scanners will tell you the ID is fine, even when it is a fake. The fake ID people even can get the UV light seals nearly 100% correct. The other risk here is not checking what the scanner says. A large number of the fake IDs I've confiscated scan (reports the person is 21 or older), but the info doesn't match. The person producing the fake used a real ID to generate the barcode info. Some of our scanners just check age, and don't display the actual data. And some states encrypt that data, so the scanner is useless on those. At my previous job, we had no scanners, so detecting fakes was all visual inspection (or by touch, a lot of fakes just don't feel right). On a number of fakes I confiscated, I had people insist I scan it! Told them I had no scanner, and I knew it was fake, and the only way they were going to gain entrance was for a police officer to verify the ID. Had a few call my bluff, and the cops came and wound up giving them a ticket. I took some of those ID's over to another location that had a scanner, and sure enough the scanner said they were 21! The place I'm at now has scanners, and some of the staff rely too much on them. I've grabbed ID's out of other clerks hands to inspect them when they ran them through the scanner and it said it was ok, and a lot of times it does turn out to be fake.
AOL has changed their Terms of Service for users of their services - see <http://www.aim.com/tos/tos.adp>. Users of their services, for example AOL Instant Messenger (AIM) in particular should note the details, including: "by posting Content on an AIM Product, you grant AOL, its parent, affiliates, subsidiaries, assigns, agents and licensees the irrevocable, perpetual, worldwide right to reproduce, display, perform, distribute, adapt and promote this Content in any medium". Nice one! A few points: 1: We've seen this before: a company makes a niche, gains userbase, then turns bad in some way ans shafts the user. 2: I've often used IM services to send beta software, specs, and so on to clients, using the "send file" option. By transferring them via AIM, I've allowed them to be reproduced in any form. There goes client confidentiality. 3: Big companies use AOL - one of my clients is one of the largest investment banks in the world. Their desktop IM client includes an AIM bridge, I guess that some traders use it to communicate with clients (by using the bridge, the bank can log everything and keep regulators happy). What does this mean to the bank, or another large company? 4: What if some voice over IP (VoIP) provider pulls this one. Or even an e-mail provider? Jabber and ICQ are free (as in beer) IM services. I've used ICQ, but have no experience of Jabber <http://www.jabber.org/> and <http://www.icq.com/>. Added note, 14 Mar 2005 13:51:55 -0000 (GMT) AOL has subsequently clarified its ToS. This has been reported in a blog:<http://www.chron.com/cs/CDA/ssistory.mpl/tech/blog/3082956>, which itself reports on e-mail and telephone conversations with AOL spokesman Andrew Weinstein. However, the ToS still read as they did yesterday (I doubt AOL are going to modify them without a lawyer casting an eye on them, and this story broke over the weekend). The terms are still ambiguous and everyone should be careful! Alistair McDonald, InRevo Ltd (http://www.inrevo.com) 07017 467 396 Author of the SpamAssassin book: http://www.spamassassinbook.com/
Please report problems with the web pages to the maintainer