The Risks Digest

The RISKS Digest

Forum on Risks to the Public in Computers and Related Systems

ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator

Volume 31 Issue 41

Monday 9 September 2019

Contents

An Op-Ed from the Future on Election Security
Alex Stamos
French air traffic control 'outage' hits UK flights
BBC
Voice-mimicking software used in major theft
WashPost
Robot hires human being in world first as AI conducts job interview
Daily Star
Bright Idea—Can't stop... (from News of the Weird, The Guardian)
Gabe Goldberg
Voice-mimicking software used in heist—in AI first
The Straits Times
Evading Machine-Learning Malware Classifiers
William Fleshman
No, this AI hasn't mastered eighth-grade science
Tiernan Ray
Stina Ehrensvärd is creating "a seatbelt for the Internet."
Fortune
Apple Finally Breaks Its Silence on iOS Hacking Campaign
WiReD
Convicted hacker called to testify to grand jury in Virginia
WashPost
Re: How Apple's HomePod turned my friends into rude troglodytes
Amos Shapir
Info on RISKS (comp.risks)

An Op-Ed from the Future on Election Security (Alex Stamos)

"Peter G. Neumann" <neumann@csl.sri.com>
Fri, 5 Sep 2019 09:17:15 PDT
  [This is a poignant delicious wonderful RISKS-worthy satirical item
  (truncated here, because you really should read the original on Alex's
  website).  Alex apparently wrote it for a less-techie audience that does
  not understand many of the past election fiascoes covered in RISKS and
  elsewhere.  Many of them actually appear in the context of Alex's piece --
  which is more than timely (in that it is dated 1 Jan 2021!).  Some of the
  URLs have strangely disappeared from my conversion of pdf to ascii here,
  so I urge you to go to the complete text in this URL:
    https://www.lawfareblog.com/topic/election-security PGN]

Alex's indroduction (excerpted):

  Below is a potential *Lawfare* piece from New Year's Day 2021, following a
  not-quite-worst-case scenario of election interference using real
  vulnerabilities in U.S. electoral systems, as well as social media,
  traditional media and the political sphere. For a more thorough discussion
  of weaknesses and recommended mitigations, please see the *election
  security report* <https://cyber.fsi.stanford.edu/securing-our-cyber-future>
  from my colleagues and me at Stanford's *Cyber Policy Center*
  <https://cyber.fsi.stanford.edu>.  [Alex]

1 Jan 2021

New Year' Day is traditionally spent recovering from the previous night's
revelry. This year, the United States awakens to the greatest New Year's
hangover in the country's almost 245-year history: a crisis of
constitutional legitimacy as all three branches of government continue to
battle over who will take the presidential oath of office later this
month. This coming Wednesday, Jan. 6, a joint session of Congress will meet
for what is a *traditionally perfunctory counting*
<https://www.law.cornell.edu/uscode/text/3/15> of the Electoral College
votes.  With lawsuits still pending in seven states, both major-party
candidates claiming victory via massive advertising campaigns and the
president hinting that he might not accept the outcome of the vote, it's
time to reflect on how everything went so very wrong.

The first signs of external interference were seen in the spring of 2020.
As the Democratic primary field narrowed, a group of social media accounts
that had voiced strong support for particular candidates early on pivoted
from supporting their first-choice candidates to alleging that the
Democratic National Committee (DNC) had unfairly rigged the primary. The
uniform nature of these complaints raised eyebrows, and an investigation by
Twitter, Google and Facebook *traced the accounts back to American employees
of a subsidiary of the Sputnik News Agency*
<https://www.nytimes.com/2019/01/17/business/facebook-misinformation-russia.htm\l>
-- an English-language media entity owned by the Russian state. Yet as these
groups were careful not to run political ads and to use U.S. citizens to
post the content, there was no criminal predicate for deeper law enforcement
investigations.

The activity around the election intensified in the summer, when medical
records for the son of the presumptive Democratic nominee were stolen from
an addiction treatment center and seeded to the partisan online media. But
that wasn't all: Less than 24 hours later, *embarrassing photos*
<https://www.nbcnews.com/tech/tech-news/pennsylvania-man-arrested-will-plead-gu\ilty-celebrity-hacking-n539166>
from the phone of the incumbent president's single, Manhattanite daughter
were released on the dark web. While the FBI has remained silent on the
matter, citing an ongoing investigation, the New York Times recently quoted
anonymous NSA officials attributing the first leak to Russia's SVR
intelligence service and the latter to the Chinese Ministry of State
Security. As to why Russia and China appear to be backing opposing
candidates, America's adversaries do not necessarily share the same
geopolitical goals, and it is clear that the Chinese are no longer willing
to sit on the sidelines of U.S. politics while the Russians interfere.

This multi-sided foreign interference dominated the headlines throughout the
last half of the campaign, drawing the media's attention away from
substantive policy debates and priming the U.S. electorate for the coming
catastrophe.  Election Day 2020 started quietly, with the familiar
television spots showing images of early lines at polling places, interviews
with proud citizens wearing `I Voted' footage of volunteers canvassing
neighborhoods.  The first signs of trouble appeared in Miami,
Ft. Lauderdale, Akron and Cleveland, as poll workers were surprised by the
unusually large number of mismatches between the voting rolls they had been
provided and the ID shown by people intending to vote.  [...]

  [The rest of this keeps getting better, and ever more scary.  It is highly
  recommended.  The pithy final paragraph cuts to the chase:

    “We couldn't have known,'' voices on Capitol Hill have argued again and
    again in the months since the election—including the Senate majority
    leader.  If only there was a way to go back in time and help them
    understand the risks of their inaction.

  Remember, this is a visionary perspective from January 2021.
  It really seems like 20-20 foresight.  PGN]


French air traffic control 'outage' hits UK flights (BBC)

Monty Solomon <monty@roscom.com>
Fri, 6 Sep 2019 13:51:23 -0400
https://www.bbc.com/news/uk-49541972


Voice-mimicking software used in major theft (WashPost)

Peter Houppermans <not.for.spam@houppermans.net>
Mon, 9 Sep 2019 09:19:53 +0200
Source: https://www.washingtonpost.com/technology/2019/09/04/an-artificial-intelligence-first-voice-mimicking-software-reportedly-used-major-theft/

"Thieves used voice-mimicking software to imitate a company executive's
speech and dupe his subordinate into sending hundreds of thousands of
dollars to a secret account, the company's insurer said, in a remarkable
case that some researchers are calling one of the world's first publicly
reported artificial-intelligence heists.

The managing director of a British energy company, believing his boss was on
the phone, followed orders one Friday afternoon in March to wire more than
$240,000 to an account in Hungary, said representatives from the French
insurance giant Euler Hermes, which declined to name the company."

Hmmm.  And no other feedback channel was used to verify this - especially
since the request was deemed "rather strange"?


Robot hires human being in world first as AI conducts job interview (Daily Star)

the keyboard of geoff goodfellow <geoff@iconia.com>
Thu, 5 Sep 2019 12:39:21 -1000
*Tengai is said to be "bias free" and will only hire the best person for
the job regardless of ethnicity, age or gender*

A robot has hired a human being for the first time in history as an AI was
left to do job interviews.  Robotic head Tengai has been commissioned to
carry out recruitment in the Upplands Bro Municipality, Sweden.  Tengai
resembles a head on a stick, with a friendly looking face beamed onto a
screen which wraps around his plastic skull.

The robot was developed by recruitment company TNG together with the tech
firm Furhat Robotics.  He is reported to have hired a man called Anders
Ornhed, from Jarfalla.  Anders has the honour of becoming the first person
ever to hired by an AI.  Swedish radio reported Anders got through the
interview process with Tengai.  He was given the job as digital coordinator
at the municipality office.

Tengai is boasted to be `bias free'.

The robot is not affected by the jobseeker=E2=80=99s age, gender of
ethnicity—he just wants the best person for the job.  [...]

https://www.dailystar.co.uk/news/world-news/robot-hires-human-being-world-1=
9572551


Bright Idea—Can't stop... (from News of the Weird, The Guardian)

Gabe Goldberg <gabe@gabegold.com>
Sun, 8 Sep 2019 23:32:01 -0400
A Twitter user known only as "Dorothy," 15, was banned from her phone by her
mom in early August after becoming distracted while cooking and starting a
fire, but that didn't stop her, reported The Guardian. First she tweeted
from a Nintendo 3DS gaming device, but Mom caught on quickly and posted that
the account would be shut down. The next day, Dorothy tweeted from her Wii
U, assuring followers that while Mom was at work, she'd be looking for her
phone. Finally, on Aug. 8, with no other options left, Dorothy reached out
to Twitter from an unlikely source: her family's LG smart refrigerator. "I
am talking to my fridge what the heck my Mom confiscated all of my
electronics again," she posted. The post went viral, even prompting LG to
tweet about it with the hashtag #FreeDorothy. [The Guardian, 8/13/2019]


Voice-mimicking software used in heist—in AI first (The Straits Times)

Richard Stein <rmstein@ieee.org>
Sun, 8 Sep 2019 18:33:13 -0700
https://www.straitstimes.com/world/europe/voice-mimicking-software-used-in-heist-in-ai-first

The precise voice impersonation synthesis method is not identified.  The
incident affirms an emerging business risk, supplementing the ever-growing
list of CxO fraud techniques and exploits.

Voice impersonation might be thwarted by multi-factor authentication,
including face-to-face verification, before payment approval authorization
completes.

Each authentication factor introduced into the payment approval life cycle
adds transactional friction to business effectiveness.

Business fraud losses rise as technologically-enabled theft becomes more
sophisticated than carbon-based operators can detect and deter. Can a
silicon-based operator successfully replace humans at fraud detection with
an superior AUCROC (area-under-curve, receiver operating characteristic)
false-positive/negative result?

Insurance companies are noticing these incidents, and will raise premiums as
various fraud losses accrue.

https://catless.ncl.ac.uk/Risks/31/26#subj14.1 identifies one voice
simulator. https://catless.ncl.ac.uk/Risks/31/34#subj11.1 affirms the risk
magnitude to business and government operations.


Evading Machine-Learning Malware Classifiers (William Fleshman)

"Peter G. Neumann" <neumann@csl.sri.com>
Mon, 9 Sep 2019 13:18:28 PDT
  [Thanks to Ray Perrault.  PGN]

William Fleshman, 3 Sep 2019
Evading Machine Learning Malware Classifiers for fun and profit!
https://towardsdatascience.com/evading-machine-learning-malware-classifiers-ce52dabdb713

In this post, Im going to detail the techniques I used to win the Machine
Learning Static Evasion Competition announced at this years DEFCON AI
Village. The goal of the competition was to get 50 malicious Windows
Portable Executable (PE) files to evade detection by three machine learning
malware classifiers. Not only did the files need to evade detection, but
they also had to maintain their exact original functionality and behavior.
[...]

  [Nice Work.  Beautifully presented.  This is indeed a winner!  PGN]


No, this AI hasn't mastered eighth-grade science (Tiernan Ray)

Gene Wirchenko <gene@shaw.ca>
Fri, 06 Sep 2019 10:32:01 -0700
  [I thought these "learning" systems were rather more sophisticated than
  what appears to be the case presented here.  Is this actually a house of
  cards?]

Tiernan Ray, ZDNet, 5 Sep 2019

Researchers at the Allen Institute for AI have engineered a brilliant
mash-up of natural language processing techniques that gets high scores on
Regents exam questions for high school science, but the software is not
really learning science in the sense most people would think, it's just
counting words.
https://www.zdnet.com/article/no-this-ai-hasnt-mastered-eighth-grade-science/

One of the most mindless features of modern education are standardized
tests, which require pupils to regurgitate information usually committed to
memory in rote fashion. Fortunately, a machine has now been made that can
complete questions on a test about as well as the average student, perhaps
freeing humans for more worthwhile types of learning.

Just don't be confused that it has anything to do with learning as you
typically think of it.


Stina Ehrensvärd is creating "a seatbelt for the Internet."

Gabe Goldberg <gabe@gabegold.com>
Sat, 7 Sep 2019 22:02:24 -0400
The CEO and founder of Yubico, a startup that designs online
account-securing fobs, says as much as she enthusiastically slaps a package
on a table at Fortune's offices. Inside the plastic container: Her latest
product. It's the first Lightning-port compatible hardware security
key. Translation: the first security fob that works with Apple's latest
iPhones, generations 5 and later.

Hardware security keys come highly recommended by security experts. They
offer an additional layer of protection—a second-factor, in the parlance
-- over passwords alone. They're generally more secure than sending a
one-time code to your phone, or using a random number generating application
to produce the codes. Services such as Twitter, Facebook, and Dropbox
support the keys.

Before one dismisses the notion—why am I going to stick this dongle into
my phone every time I want to log into one of my accounts?—Stina
anticipates the objection. You only have to stick in the key every so
often. Google lets you have a 30-day grace period. Other services give you
more leniency. Besides: What's a minor inconvenience for so much peace of
mind?

https://fortune.com/2019/09/07/hardware-security-keys-a-seatbelt-for-the-internet-cyber-saturday/


Apple Finally Breaks Its Silence on iOS Hacking Campaign (WiReD)

Gabe Goldberg <gabe@gabegold.com>
Sat, 7 Sep 2019 16:40:19 -0400
https://www.wired.com/story/ios-hacks-apple-response/


Convicted hacker called to testify to grand jury in Virginia (WashPost)

Gabe Goldberg <gabe@gabegold.com>
Fri, 6 Sep 2019 15:15:32 -0400
FALLS CHURCH, Va.—A convicted hacker who's serving 10 years in prison for
breaking into computer systems of security firms and law-enforcement
agencies has been called to testify to a federal grand jury in Virginia.

Supporters of Jeremy Hammond, part of the Anonymous hacking group, say he's
been summoned to testify against his will to a grand jury in Alexandria on
Tuesday. Hammond, who admitted leaking hacked data to WikiLeaks, believes
the subpoena is related to the investigation of WikiLeaks and its founder
Julian Assange. Assange is under indictment in Alexandria and the U.S. is
seeking extradition.

Prosecutors declined comment.

Former Army intelligence analyst Chelsea Manning was also called to testify
to the WikiLeaks grand jury. She refused and is now serving a jail sentence
of up to 18 months for civil contempt.

Hammond's supports say he'll also refuse to testify.

https://www.washingtonpost.com/national/convicted-hacker-called-to-testify-to-grand-jury-in-virginia/2019/09/03/297a7596-ce5f-11e9-a620-0a91656d7db6_story.html


Re: How Apple's HomePod turned my friends into rude troglodytes (Wirchenko, RISKS-31.40)

Amos Shapir <amos083@gmail.com>
Mon, 9 Sep 2019 18:13:39 +0300
This seems to be a cultural thing.  In Israel (and I guess many other
countries) this is quite acceptable behavior, especially among good old
friends.

Technology just seems to bring the world together in many ways.

Please report problems with the web pages to the maintainer

Top