The Risks Digest

The RISKS Digest

Forum on Risks to the Public in Computers and Related Systems

ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator

Volume 31 Issue 51

Wedesday 18 December 2019


Human error installing SCADA system leads to 7.5 million gallons of raw sewage dumped in Valdosta, GA
Killer Robots Aren't Regulated. Yet.
Jonah M. Kessel
Earth Enters Unknown as Magnetic North Pole Continues Push Toward Russia, Crosses Greenwich Meridian
Sputnik News
SpaceX to Make Starlink Satellites Dimmer to Lessen Impact on Astronomy
Scientific American
Smart lock has a security vulnerability that leaves homes open for attacks
Scores of sex offenders have state licenses to be electricians, manicurists, and more. The official who found out got fired.
Is Alexa Always Listening? How Amazon, Google, Apple Hear, Record
Apple Used the DMCA to Take Down a Tweet Containing an iPhone Encryption Key
Phone-breaking Android hole revealed
Deepfakes are getting better. Should we be worried?
Luggage tracking apps aren't 100% accurate. People are the weak link
Internet of crap encryption: IoT gear is generating easy-to-crack keys
The Register
Prime Leverage: How Amazon Wields Power in the Technology World
Cloud flaws expose millions of child tracking smartwatches
Thief Stole Payroll Data of 29,000 Facebook Employees
Companies Ignoring Third-Party Breach Alerts
Security Boulevard
Insurer Races to Fix Security Flaws After Whistleblower Alert
Bank Infosecurity
Audit knocks Mass. tax-collection agency
The Boston Globe
How hacking the human heart could replace pill popping
Bates v Post Office litigation - reliability of computers
Re: Election Security regulations in the U.S.
Dick Mills
Re: What happens if your mind lives for ever on the Internet?
Martin Ward
Info on RISKS (comp.risks)

Human error installing SCADA system leads to 7.5 million gallons of raw sewage dumped in Valdosta, GA

Shawn Merdinger <>
Sun, 15 Dec 2019 22:04:46 -0500

"On December 9, 2019, the staff at the Withlacoochee Wastewater Treatment
plant notified Environmental Services personnel to inform them that flow
into the plant had decreased by 50% over the previous few days. After a
brief investigation, utility personnel noticed that a contractor working on
the city's SCADA system disconnected a reference cable at the Remerton Lift
Station for testing and failed to reconnect it. As a result of the incident,
the lift station's level indicator and alarm agent were disconnected. The
lift station's alarm agent system did not operate as it normally would,
bypassing the alert notification that is typically sent to utility staff
when there is an issue at a lift station.

Based on the flow information collected, approximately 7,592,910 gallons
discharged from a manhole into Sugar Creek adjacent to the 1800 block of
Norman Drive."

  [Garbage In, Garbage Out: with a coochee-coup.  PGN]

Killer Robots Aren't Regulated. Yet. (Jonah M. Kessel)

Dewayne Hendricks <>
December 14, 2019 18:53:07 JST
Jonah M. Kessel, *The New York Times*, 13 Dec 2019
*Killing in the Age of Algorithms* is *The New York Times* documentary
examining the future of artificial intelligence and warfare.

Times reporters traveled to Russia, Switzerland, California and Washington,
D.C., talking to experts in the commercial tech, military and AI
communities.  Below are some key points and analysis, along with extras from
the documentary.

Do I need to worry about a Terminator knocking on my door?

Most experts say you can rest easy, for now. Weapons that can operate like
human soldiers are not something they see in our immediate future. Although
there are varying opinions, most agree we are far from achieving artificial
general intelligence, or A.G.I., that would allow for Terminators with the
kind of flexibility necessary to be effective on today's complex

However, Stuart J. Russell, a professor of computer science at the University of California, Berkeley, who wrote an influential textbook on artificial intelligence, says achieving A.G.I. that is as smart as humans is inevitable.

So where are we now?

There are many weapons systems that use artificial intelligence. But instead
of thinking about Terminators, it might be better to think about software
transforming the tech we already have.

There are weapons that use artificial intelligence in active use today,
including some that can search, select and engage targets on their own,
attributes often associated with defining what constitutes a lethal
autonomous weapon system (a.k.a. a killer robot).

In his book *Army of None: Autonomous Weapons and the Future of War*, the
Army Ranger turned policy analyst Paul Scharre explained, “More than 30
nations already have defensive supervised autonomous weapons for situations
in which the speed of engagement is too fast for humans to respond.''

Perhaps the best known of these weapons is the Israel Aerospace Industries
Harpy, an armed drone that can hang out high in the skies surveying large
areas of land until it detects an enemy radar signal, at which point it
crashes into the source of the radar, destroying both itself and the target.

The weapon needs no specific target to be launched, and a human is not
necessary to its lethal decision making. It has been sold to Chile, China,
India, South Korea and Turkey, Mr. Scharre said, and the Chinese are
reported to have reverse-engineered their own variant..

“We call them precursors,'' Mary Wareham, advocacy director of the arms
division at Human Rights Watch, said in an interview between meetings at the
United Nations in Geneva.  “We're not quite there yet, but we are coming
ever closer.''

So when will more advanced lethal autonomous weapons systems be upon us?

“I think we're talking more about years not decades,'' she said.

But for the moment, most weapons that use AI have a narrow field of use and
aren't flexible. They can't adapt to different situations.

“One of the things that's hard to understand unless you've been there is
just the messiness and confusion of modern warfare,'' Mr. Scharre said in an

“In all of those firefights,'' he explained, “there was never a point
where I could very clearly say that it was 100 percent that the person I was
looking at down the scope of my rifle was definitely a combatant.

Soldiers are constantly trying to gauge—is this person a threat? How
close can they get to me? If I tell them to stop, does that mean that they
didn't hear me or they didn't understand? Maybe they're too frightened to
react? Maybe they're not thinking? Or maybe they're a suicide bomber and
they're trying to kill me and my teammates.''

Mr. Scharre added, “Those can be very challenging environments for robots
that have algorithms they have to follow to be able to make clear and
correct decisions.''

Although current AI is relatively brittle, that isn't stopping militaries
from incorporating it into their robots. In his book, which was published in
2018, Mr. Scharre wrote that at least 16 countries had armed drones, adding
that more than a dozen others were working on them.

Earth Enters Unknown as Magnetic North Pole Continues Push Toward Russia, Crosses Greenwich Meridian (Sputnik News)

geoff goodfellow <>
Sat, 14 Dec 2019 09:02:05 -1000
*Earlier this year, US National Oceanic and Atmospheric Administration and
the British Geological Survey (BGS) were forced to update the World Magnetic
Model a year ahead of schedule due to the speed with which the magnetic
north pole is shifting out of the Canadian Arctic and toward Russia's


The BGS and the US National Centers for Environmental Information has
released a new update to the World Magnetic Model this week, confirming that
the magnetic north pole, whose coordinates are crucial for the navigation
systems used by governments, militaries and a slew of civilian applications,
is continuing its push toward Siberia.

“The WMM2020 forecasts that the northern magnetic pole will continue
drifting toward Russia, although at a slowly decreasing speed—down to
about 40 km per year compared to the average speed of 55 km over the past
twenty years,'' the US agency said in a press statement.

The data confirmed that this year, the magnetic north pole passed to within
390 km of the geographic North Pole, and crossed the Greenwich (prime)
meridian. Compilers also confirmed that the Earth's magnetic field is
continuing to weaken, at a rate of about 5 percent every 100 years. [...]

SpaceX to Make Starlink Satellites Dimmer to Lessen Impact on Astronomy (Scientific American)

Richard Stein <>
Sat, 14 Dec 2019 16:47:27 +0800

See SpaceX's Starlink Could Change The Night Sky Forever, And Astronomers
Are Not Happy, for a brief note outlining astronomer's umbrage.

"So now the company plans to treat one of the Starlink satellites with a
special coating, when the next group goes in late December, according to
SpaceX president and chief operating officer Gwynne Shotwell."

I wonder what's in SpaceX's 'secret anti-reflective' sauce? Hopefully, the
coating won't chip or flake off the Starlink payload while deployed in

Smart lock has a security vulnerability that leaves homes open for attacks (CNET)

Amos Shapir <>
Sat, 14 Dec 2019 11:19:06 +0200
Yet another IOT vulnerability story:

There are no details in the article, but it seems to be a case of
unencrypted communication between a "smart lock" and the phone app which
controls it.

Scores of sex offenders have state licenses to be electricians, manicurists, and more. The official who found out got fired. (BostonGlobe)

Monty Solomon <>
Sun, 15 Dec 2019 11:17:50 -0500

Is Alexa Always Listening? How Amazon, Google, Apple Hear, Record (Bloomberg)

Gabe Goldberg <>
Sun, 15 Dec 2019 21:02:52 -0500

Apple Used the DMCA to Take Down a Tweet Containing an iPhone Encryption Key (VICE)

Gabe Goldberg <>
Thu, 12 Dec 2019 23:10:55 -0500

Phone-breaking Android hole revealed (Gadget)

Monty Solomon <>
Fri, 13 Dec 2019 03:05:16 -0500

Deepfakes are getting better. Should we be worried? (TheBostonGlobe)

Monty Solomon <>
Sat, 14 Dec 2019 16:12:46 -0500

Luggage tracking apps aren't 100% accurate. People are the weak link (LATimes)

Monty Solomon <>
Sun, 15 Dec 2019 10:48:17 -0500

Internet of crap encryption: IoT gear is generating easy-to-crack keys (The Register)

Monty Solomon <>
Mon, 16 Dec 2019 11:24:20 -0500

Prime Leverage: How Amazon Wields Power in the Technology World (NYTimes)

Gabe Goldberg <>
Tue, 17 Dec 2019 11:42:37 -0500

Cloud flaws expose millions of child tracking smartwatches (TechCrunch)

Lauren Weinstein <>
Wed, 18 Dec 2019 09:03:30 -0800

  [Also noted by Gabe Goldberg.  PGN]

Thief Stole Payroll Data of 29,000 Facebook Employees (CISOmag)

Monty Solomon <>
Tue, 17 Dec 2019 11:28:45 -0500

Companies Ignoring Third-Party Breach Alerts (Security Boulevard)

Monty Solomon <>
Tue, 17 Dec 2019 11:30:09 -0500

Insurer Races to Fix Security Flaws After Whistleblower Alert (Bank Infosecurity)

Monty Solomon <>
Tue, 17 Dec 2019 11:33:01 -0500
Report: Blue Cross and Blue Shield Minnesota Had Thousands of Old 'Critical'

Audit knocks Mass. tax-collection agency (The Boston Globe)

Monty Solomon <>
Tue, 17 Dec 2019 11:50:00 -0500
`Incredibly sensitive' data is open to cyberattack at Mass. tax-collection
agency, audit report says

How hacking the human heart could replace pill popping (

Richard Stein <>
Wed, 18 Dec 2019 15:50:10 +0800
This BBC article suggests that an implanted medical device can improve your
quality of life.

Get an implanted device, fill it with your prescription(s), and set the
dispensation timer (every X hours) or delivery trigger condition (blood
glucose threshold). Convenient, no? With an implant, the recipient is
relieved from fetching a glass of water to assist medicine consumption,
"where is my medicine" moments, or "fingertip prick, blood glucose measure,
and insulin inject" duties. Refill the reservoir periodically, like
recharging a mobile electronic device.

Device implantation is a highly personal choice: to sustain longevity, a
candidate recipient may have no other options available to manage a chronic
or acute condition. Elective device implantation is a significant
life-changing and potentially life-threatening decision.

What questions do you ask a medical provider who recommends device
implantation? What information do you need to make an informed decision?
What are the implanted device choices? What about post-implant quality of
life? How will the implant either change, diminish, or improve life quality?
How often are explants (device removals) performed for the candidate device
choice? What are implant risks and their occurrence probabilities? Why does
your physician recommend manufacturer X's device, and not a competitor Y's?
Does your physician receive payment or other incentive from manufacturer X
to implant their device? What criteria drive device selection that's
relevant to your case?

These questions are difficult for a patient to ask their physician. A
patient often consciously relies on physician trust to guide a "go or no-go"
decision. You hold your physician in high regard. You rely on them to treat
you according to the Hippocratic Oath --- that's their career-long pledge to
serve your interests. While you can often trust your physician, can you
automatically extend this trust to the manufacturer that supplies the tools
and devices a physician uses to treat your condition?

I cannot give a binary 'yes' or 'no' answer. Risk, especially risks for
implanted cardiac devices, constitutes a measure that is too important to

In this note, I attempt to estimate an probability for adverse event
experience arising in cardiac-related implantable device recipients: My
analysis attempts to answer: "What is the probability of experiencing a
malfunction or injury or death (identified as adverse events) following
implantation of a pacemaker or defibrillator or electrical stimulus/sensing

I use freely available public, and professionally vetted/reviewed,
literature and government sources as noted below. Basic arithmetic is used
for computation.

  [See for a notable book on
  implanted medical devices and their risks—especially as experienced by
  one person implanted with a neuro-stimulator.]

tools collate submitted device report records. They are used to capture
adverse events (identified by the FDA as: DEATH, INJURY, MALFUNCTION, NOT
SPECIFIED, OTHER) arising from, or possibly attributed to, implanted cardiac
devices (defibrillators, pacemakers, electrical leads, etc.).

To perform the analysis, I estimate an aggregate adverse event count over a
given 42 month interval comprising 01JAN2016-31JUL2019. I use public sources
of device implant rates to calculate a non-zero probability that an adverse
event will impact a recipient. That the aggregation is applied across
multiple product codes (as shown below), implies that a recipient is
implanted with a defibrillator or pacemaker + electrical stimulus leads.

I do not attempt to segregate and identify probabilities attributed to
partial implant/explant, such as electrical stimulus lead explantation and
implantation with a new one. The term 'device' used here implies pacemaker,
defibrillator, and leads. It may also mean a big component of a pacemaker
(pulse generator, but not the pacemaker's enclosure) or defibrillator that
needs to be explanted or implanted.

The FDA website clearly states a caveat about using MAUDE data to calculate
event rates: "MAUDE data is not intended to be used either to evaluate rates
of adverse events or to compare adverse event occurrence rates across
devices." (See
Caveat emptor!

MAUDE content shows that on some calendar days, over 500 medical device
reports are submitted. MAUDE's web interface will only retrieve a maximum of
100 reports for any single day of interest (e.g. start date: 29JAN2017 and
end date: 29JAN2017). Hitting the MAUDE retrieval limit during search may
align with a manufacturer device recall campaign that requires a report
submission storm to comply with regulations.

The analysis was aided by pulling the raw zip files from
to process and cleanse them to enable evaluation. A few simple PYTHON
programs were used in this process.

MAUDE and TPLC afford a means to aggregate, to count, adverse event
density. This density can be combined with published, peer-reviewed sources
to estimate a post-implant adverse event occurrence probability.  MAUDE
substantially captures adverse event reports submitted by US-based
healthcare providers, manufacturer device manufacturers, and recipients.
Device manufacturers apparently submit the vast majority of MAUDE reports.

A small percentage (guestimate is ~1-2%) are submitted from manufacturer
device representatives or healthcare providers for recipient adverse events
in other countries (e.g., Singapore-based device representatives or
healthcare provider submitted ~1000 reports between 01JAN2016-31JUL2019, if
memory serves). Other countries rely on the same manufacturers (MEDTRONIC,
MEDICAL, OSCOR, etc.) as the US healthcare system for implantable cardiac

A patient's medical condition(s), and/or change in condition(s), often
serves as a significant justification to prepare and submit a medical device
report that characterizes an adverse event. Comprehensive cardiac and
electrophysiological knowledge is required to accurately assess and properly
characterize an adverse event.

The investigation used the following MAUDE product codes, comprising 16
distinct cardiac implantable device types, to estimate post-implant adverse
event probability noted below.

DTB|Permanent Pacemaker Electrode
DTD|Pacemaker Lead Adaptor
DXY|Implantable Pacemaker Pulse-Generator
LWP|Implantable Pulse Generator, Pacemaker (Non-Crt)
LWS|Implantable Cardioverter Defibrillator (Non-Crt)
MRM|Defibrillator, Implantable, Dual-Chamber
MXC|Recorder, Event, Implantable Cardiac, (Without Arrhythmia Detection)
MXD|Recorder, Event, Implantable Cardiac, (With Arrhythmia Detection)
NIK|Defibrillator, Automatic Implantable Cardioverter, With Cardiac
Resynchronization (Crt-D)
NKE|Pulse Generator, Pacemaker, Implantable, With Cardiac
Resynchronization (Crt-P)
NVN|Drug Eluting Permanent Right Ventricular (Rv) Or Right Atrial (Ra)
Pacemaker Electrodes
NVY|Permanent Defibrillator Electrodes
NVZ|Pulse Generator, Permanent, Implantable
OJX|Drug Eluting Permanent Left Ventricular (Lv) Pacemaker Electrode
OSR|Pacemaker/Icd/Crt Non-Implanted Components
PNJ|Leadless Pacemaker

Each MAUDE product code identifier consist of 3 alphabetic characters.  They
are assigned to medical devices as part of FDA device registration and
approval processes. Each product code consists of devices of similar type
and function from different manufacturers. Thus, the NVY product code
encompasses the class of Permanent Defibrillator Electrodes manufactured or
sold into the global marketplace that is subject to FDA regulation.

The TPLC tool aggregates adverse events for product codes, but assigns
unique terms to segregate event attribution into defect categories. As an
example, the DTB product code (Permanent Pacemaker Electrode) reveals this
TOP-10 tabular summary (TOTAL COUNT == 59835) reported and full traceable to
the MAUDE system since 2016:


High Capture Threshold   9132
Under-Sensing            7738
Over-Sensing             7525
Adverse Event Without
Identified Device or Use
Problem                  7523
Device Dislodged or
Dislocated               7055
High impedance           6255
Failure to Capture       5155
Capturing Problem        3303
Fracture                 3299
Signal Artifact          2850

Under-sensing occurs when the pacemaker signal amplifier is too insensitive
-- the gain is too low—to detect a portion of the recipient's native
electrical heart activity. In contrast, Over-sensing occurs if the pacemaker
signal amplifier gain is too high, leading the device to detect
inappropriate signals, like skeletal muscle movements.

The TPLC counts, and their assigned categories, are prepared and maintained
by an FDA panel who review the MAUDE adverse event reports.  The 'DEVICE
PROBLEM' labels comprise an arcane lexicon that non-subject matter
specialists struggle to interpret. A dictionary of TPLC category labels was
not found in the FDA website.

Based on the raw MAUDE records (downloaded in AUG2019), an analysis reveals
adverse event records were reported between 01JAN2016-31JUL2019 (42 calendar
months) for the 16 scoped cardiac-specific product code set. This adverse
event population might arise from accelerated battery discharge, lead
displacement, inappropriate shock, and over 100 unique classification terms
that characterize MAUDE medical device reports in TPLC.

The adverse incident density is notable. It likely implies, but does not
guarantee, that ~240,000 UNPLANNED physician and emergency care center
visits by device recipients. Some events may have been reported via Internet
monitoring, and deemed not sufficient to merit a provider visit on
inspection by the attendant. But we assume this event set constitutes an
insignificant fraction (<<1%) during the 42 month reporting interval.

This paper
( from the
European Heart Rhythm Association (EHRA) estimates that 1.25 Million
pacemakers were implanted in 2016 worldwide. It further estimates an
implantation rate of ~520 per million (~52 per 100,000) population. The EHRA
pacemaker recipient average age is ~78 +/- 9 years.

In the US, the Agency for Healthcare Research and Quality (
reported 2015 statistics for pacemaker AND defibrillator implantation rate
of ~55 per 100,000 population, a value which substantially aligns with the
EHRA 2106 study. U.S. recipient's average age is ~72 years. The reporting
tool @ yields this report after a little

The total recipients for device implantation, in the US, is given by the
rate of implantation per 100,000 times the total population:

In 2016, US census estimates 328,677,530 population. That's 3286.7 * 100,000
persons. 55 recipients/100,000 * (3286.7 * 100,000) ~= 180,768 recipients of
defibrillator, pacemaker and device leads in 2016. This aggregate also
includes device explants—removal of pacemaker, defibrillator and leads.

For the 42 month MAUDE reporting interval (01JAN2016-30JUL2019), we have
240,232 adverse event reports or 5,720 reports per month.

5,720 adverse events per month DIVIDED by 180,768 cardiac device recipients
= 3.16% probability to experience a monthly adverse event per 2016 census

If ~3% of implanted cardiac device recipients experience unplanned
healthcare provider visitations, it represents a significant tax on the
delivery system—an extra ~5720 unplanned visits.

Device recipients often have no alternative other than implantation to
sustain their longevity. The estimated adverse event rate from implanted
cardiac devices suggests that device manufacturers must pursue methods to
suppress adverse events that initiate unplanned visits.

If implanted device sensing issues constitute a significant cause of
unplanned visits, it suggests that signal processing algorithms may require
enhancement. Sustained research to improve implanted device reliability must
become an industrial priority.

Before electing to receive a prescription-dispensing implant, especially for
cardiac care, ask your healthcare provider to offer statistics about adverse
events that may initiate an unplanned visit. It is imperative for
well-informed consumers to understand and consider the risks arising from
implanted devices BEFORE the procedure.

Glossy manufacturer product literature may not detail sufficient historical
adverse event probabilities for a device implant that informed consumer
choice requires.

Exploring FDA's medical device "systems of record," as embodied by the FDA's
MAUDE and TPLC data repositories and reporting tools, can be enlightening
and frightening. Substantial technical information about manufacturer
implanted device issues are identifiable that may impact your decision to
integrate them into your physiology. Implanted medical device manufacturer
success depends on consistently beneficial patient outcomes. While
apparently small, a demonstrable risk weighs against achievement.

Bates v Post Office litigation - reliability of computers

Stephen Mason <>
Tue, 17 Dec 2019 13:05:18 +0000
You might have picked up that the judge issued his (313 page) judgment
yesterday with 3 appendices in the English case of Bates v Post Office
Limited.  They are all available here:

I am told by Tim McCormack [] that the
judge went into detail about the meaning of *robust*—although only
discussing what the two parties had to say on the topic, and none of the
discussions in chapter 6 of Electronic Evidence were discussed at all [the
solicitors and barrister for the claimants were made aware of the
practitioner text Electronic Evidence].

  [Stephen, Don't forget *resilience*; robustness is not enough.  PGN]

Electronic Evidence is open source and a download from here:

Here are 3 relevant posts in relation to the opening speech of the barrister
for the Post Office:

The use of statistics and software code

The use of the word *robust* to describe software code

Robustness and reliability in computer systems

I will begin to read through the judgment over the next few days/weeks, and
compare it to the transcript of the trial of Seem Misra, where the
prosecution kept on asserting the system was robust then. I published the
complete transcript of the Seema Misra case here:


Transcript at the bottom of this page:

I'd appreciate people's thoughts on this when you get around to looking at

The importance of this case is this: Seema Misra and others were prosecuted,
and reliance was made on the robustness of the Horizon system without any
evidence that the system was robust or what robust meant. It also appears
that evidence given at her trial was dubious. I aim to bring out these
issues, and wondered whether if anybody had the time and patience to
consider an article for next year's Digital Evidence and Electronic
Signature Law Review? (also
available via the HeinOnline subscription service).

We are going to do a 5th edition of Electronic Evidence next year, coming
out in 2021, and it would be very helpful to have a technical view on these
issues for me to cite.

Central to the issues are the failure of judges to order greater disclosure
of software, which I pointed out in my article “Artificial intelligence: Oh
really? And why judges and lawyers are central to the way we live now—but
they don't know it'', Computer and Telecommunications Law Review, 2017,
Volume 23, Issue 8, 213--225.

Disclosure was an issue in Seema Misra's case - it appears that people were
happy to prosecute a person on the flimsiest of evidence.

I have another URL for the judgment - this includes the appendices:

Re: Election Security regulations in the U.S. (RISKS-31,50)

Dick Mills <>
Sun, 15 Dec 2019 11:45:10 -0500
I agree about election security, but I think the need for regulation of
recounts is even more urgent.

In the USA, we are cursed by close elections where every vote counts.

Recounts after close elections too often lead to viscous fights over recount
procedures.  It seems like every county makes up the rules as they go along.
Paper ballots or paper receipts multiply the possibilities for recount
fraud. IMO, recount flaws weaken public confidence even more than election

We need a detailed national standard for how to handle recounts.

Re: What happens if your mind lives for ever on the Internet? (Shapir, RISKS-31.50)

Martin Ward <>
Wed, 18 Dec 2019 15:06:03 +0000
> That's because in this context, "human intelligence" is a moving target.
> Until the 1960's, looking up a name and number in a phone book was
> considered a task of human intelligence;

This is incorrect. The definition of "machines as intelligent as humans" was
established back in 1950 in the seminal paper by Alan Turing: "Computing
Machinery and Intelligence" which described the "Turing Test". It should
(still) be required reading for any software engineer.

Please report problems with the web pages to the maintainer