The RISKS Digest
Volume 33 Issue 67

Thursday, 1st January 1970

Forum on Risks to the Public in Computers and Related Systems

ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator

Please try the URL privacy information feature enabled by clicking the flashlight icon above. This will reveal two icons after each link the body of the digest. The shield takes you to a breakdown of Terms of Service for the site - however only a small number of sites are covered at the moment. The flashlight take you to an analysis of the various trackers etc. that the linked site delivers. Please let the website maintainer know if you find this useful or not. As a RISKS reader, you will probably not be surprised by what is revealed…

Contents

Speculative out-of-order execution on my part?
PGN
Airline baggage drops
JSX
How space storms miscue train signals
phys.org
Why Long Trains Keep Derailing
ProPublica
Trojanized Windows and Mac apps rain down on 3CX users in massive supply chain attack
Sentinel One
Chinese fraudsters: evading detection and monetizing stolen credit-card information
ATT
A Front Company and a Fake Identity: How the U.S. Came to Use Spyware It Was Trying to Kill.
NYTimes
It's like children turned loose on a jungle gym
CBC
AI application ChatGPT temporarily banned in Italy over data collection concerns
CBC
Even More on Trust & Safety and AI
Lauren Weinstein
Australian mayor prepares world's first defamation lawsuit over ChatGPT content
The Guardian
Pausing AI Developments Isn't Enough. We Need to Shut It All Down
Eliezer Yudkowsky
Forgive or Forget: What Happens When Robots Lie?
Catherine Barzler
I am not afraid of robots. I am afraid of people.
Gary Marcus
Are robot waiters the future? Some restaurants think so.
AP News
It's Their Content,You're Just Licensing it,
NYTimes
Stupid physical risk
Nextdoor via Phil Smith III
Re: DC Metro Will Retrofit Faregates To Cut Down On Fare Evasion
Stan Brown
Info on RISKS (comp.risks)

Speculative out-of-order execution on my part?

Peter G Neumann <neumann@csl.sri.com>
Thu, 06 Apr 2023 16:57:44 PDT
* In that I somehow managed to put out the 1 April issue as RISKS-33.68 one
day early, an off-by-one error in the issue number, so I now figure that I
should backdate this RISKS-33.67 issue five days to April Fools' Day, to
balance off my previous *post*-dated issue.  It seems only natural, but
was actually *not* an April-Fools prank.


Airline baggage drops (JSX)

Henry Baker <hbaker1@pipeline.com>
Sat, 01 Apr 2023 18:07:01 +0000
I just received this *April Fool's* email from JSX, a startup airline
serving California.

The amazing thing is that I suggested something eerily similar about
a decade ago.

My non-April-Fool's suggestion was to have Fedex/UPS simply dump
all their packages from ~10,000' altitude, and have them GPS-guided
to their destinations, JDAM-style:

https://en.wikipedia.org/wiki/Joint_Direct_Attack_Munition

"The JDAM is not a stand-alone weapon; rather it is a 'bolt-on' guidance
package that converts unguided gravity bombs into precision-guided munitions
(PGMs)."

I figured that UPS/Fedex could deliver packages with the same precision
as JDAM bombs.

Beating swords into plowshares...

  [In RISKS-26.78, I noted from my Bell Labs days that Vic Vyssotsky had a
  wonderful piece on a Cable-laying Satellite, programmed to drop a cable
  between two specified points, carefully engineered to avoid snap-back and
  collateral damage .  PGN]


How space storms miscue train signals (phys.org)

Richard Marlon Stein <rmstein@protonmail.com>
Sun, 02 Apr 2023 02:55:48 +0000
  [Re: Over 1,000 Trains Derail Each Year in America (NYTimes, RISKS-33.63.
  PGN]

https://phys.org/news/2023-03-space-storms-miscue.html

"Train track disruptions are particularly troublesome because space storms
can interfere with detection systems that prevent collisions. Railways
detect trains using electrical currents and send stop signals to others to
avoid crashes. But when Earth's magnetic field is disrupted, they might send
false signals to stop or go, affecting operations and potentially
endangering the freight and passengers on board."

Recent train derailings across the U.S. are being investigated.

Certain trains (in the U.S.) with HazMat cargoes are remotely piloted by
joystick—virtually crewed. They are currently exempt from certain safety
regulations.

https://www.nbcnews.com/politics/congress/remote-hazmat-trains-fall-congress-push-rail-regulation-rcna77667


Why Long Trains Keep Derailing (ProPublica)

Gabe Goldberg <gabe@gabegold.com>
Mon, 3 Apr 2023 14:59:12 -0400
Before that morning in Hyndman in August 2017, regulators had already
investigated seven long-train accidents in which the length was a culprit,
and the nation's largest rail-worker union had sounded alarms about a
pattern of problems.

None of this caused the Federal Railroad Administration, the agency in
charge of train safety, to intercede—even as more long trains crashed in
the years after the Hyndman derailment, sending cars spilling into other
communities.

Today, the rail administration says it lacks enough evidence that long
trains pose a particular risk. But ProPublica discovered it is a quandary of
the agency's own making: It doesn't require companies to provide certain
basic information after accidents—notably, the length of the train --
that would allow it to assess once and for all the extent agency of the
danger.

   ... [More on Hunter Harrison PGN-truncated]

https://www.propublica.org/article/train-derailment-long-trains


Trojanized Windows and Mac apps rain down on 3CX users in massive supply chain attack (Sentinel One)

geoff goodfellow <geoff@iconia.com>
Fri, 31 Mar 2023 20:19:13 -0700
Remember SolarWinds? A similar attack is playing out now against a new
software supplier.

Hackers working on behalf of the North Korean government have pulled off a
massive supply chain attack on Windows and macOS users of 3CX, a widely
used voice and video calling desktop client, researchers from multiple
security firms said.

Through means that aren't yet clear, the attack managed to distribute
Windows and macOS versions of the app, which provides both VoIP and PBX
services to 600,000+ customers <https://www.3cx.com/company/customers/>,
including American Express, Mercedes-Benz, and Price Waterhouse Cooper. The
attackers somehow gained the ability to hide malware inside 3CX apps that
were digitally signed using the company's official signing key. The macOS
version, according to <https://objective-see.org/blog/blog_0x73.html> macOS
security expert Patrick Wardle, was also notarized by Apple, indicating that
the company analyzed the app and detected no malicious functionality.

In the making since 2022

“This is a classic supply chain attack, designed to exploit trust
relationships between an organization and external parties,'' Lotem
Finkelstein, Director of Threat Intelligence & Research at Check Point
Software, said in an email.  “This includes partnerships with vendors or
the use of a third-party software which most businesses are reliant on in
some way. This incident is a reminder of just how critical it is that we do
our due diligence in terms of scrutinizing who we conduct business
with.''

Security firm CrowdStrike said the infrastructure and an encryption key
used in the attack match those seen in a March 7 campaign carried out by
Labyrinth Chollima, the tracking name for a threat actor aligned with the
North Korean government.

The attack came to light late on Wednesday, when products from various
security companies began detecting malicious activity coming from
legitimately signed binaries for 3CX desktop apps. Preparations for the
sophisticated operation began no later than February 2022, when the threat
actor registered a sprawling set of domains used to communicate with
infected devices. By 22 Mar 2023, security firm Sentinel One saw a spike in
behavioral detections
<https://www.sentinelone.com/blog/smoothoperator-ongoing-campaign-trojanizes-3cx-software-in-software-supply-chain-attack/>


Chinese fraudsters: evading detection and monetizing stolen credit-card information (ATT)

geoff goodfellow <geoff@iconia.com>
Wed, 5 Apr 2023 07:37:52 -0700
Cyber-attacks are common occurrences that often make headlines, but the
leakage of personal information, particularly credit-card data, can have
severe consequences for individuals. It is essential to understand the
techniques employed by cyber-criminals to steal this sensitive information.

Credit-card fraud in the United States has been on the rise, with total
losses reaching approximately $12.16 billion in 2021, according to Insider
Intelligence. Card-Not-Present (CNP) fraud constituted 72% of these losses,
with a substantial portion attributed to Chinese fraudsters.

This article discusses the tactics employed by Chinese cyber-actors in
committing CNP fraud and their value chain.

Chinese fraudsters primarily target the United States for two reasons: the
large population makes phishing attacks more effective, and credit-card
limits in the country are higher compared to other nations. These factors
make the U.S. an attractive market for card fraudsters.

Common methods for acquiring card information include phishing, JavaScript
injection through website tampering, and stealing data via Trojan horse
infections. Phishing is the most prevalent method, and this analysis will
focus on phishing tactics and the monetization value chain of stolen
credit-card information.  [...]

https://cybersecurity.att.com/blogs/security-essentials/chinese-fraudsters-evadi
ng-detection-and-monetizing-stolen-credit-card-information


A Front Company and a Fake Identity: How the U.S. Came to Use Spyware It Was Trying to Kill. (NYTimes)

Jan Wolitzky <jan.wolitzky@gmail.com>
Sun, 2 Apr 2023 20:00:24 -0400
The Biden administration has been trying to choke off use of hacking tools
made by the Israeli firm NSO. It turns out that not every part of the
government has gotten the message.

<https://www.nytimes.com/2023/04/02/us/politics/nso-contract-us-spy.html>


It's like children turned loose on a jungle gym (CBC)

Matthew Kruk <mkrukg@gmail.com>
Sat, 1 Apr 2023 14:39:49 -0600
https://www.cbc.ca/news/business/chatgpt-intelligence-ownership-column-don-pittis-1.6739025

In some ways the surprising thing about ChatGPT is how it caught not just
the general public, but even artificial intelligence experts by surprise.

People like Karina Vold, a philosopher of cognitive science and artificial
intelligence at the University of Toronto, knew this kind of thing was
around the corner, but the user-friendly accessibility that allowed almost
anyone with a few computer skills to try it out has been transformative.
She thinks even its creators were surprised.

“They are learning, I think, a lot from our own human feedback as we play
with the system, kind of like building a jungle gym and then releasing a
bunch of children onto it,'' said Vold.


AI application ChatGPT temporarily banned in Italy over data-collection concerns (CBC)

Matthew Kruk <mkrukg@gmail.com>
Fri, 31 Mar 2023 19:47:13 -0600
https://www.cbc.ca/news/world/italy-openai-chatgpt-ban-1.6797963

Italy's Data Protection Authority on Friday temporarily banned OpenAI's
ChatGPT chatbot and launched a probe over a suspected breach of the
artificial intelligence application's data-collection rules.

The agency, also known as Garante, accused Microsoft Corp-backed ChatGPT of
failing to check the age of its users who are supposed to be 13 and up.

  [This item even made it to the Palo Alto local Daily Post on 3 Apr.
  PGN]


Even More on Trust & Safety and AI

Lauren Weinstein <lauren@vortex.com>
Thu, 6 Apr 2023 10:38:28 -0700
In answer to some questions I've received, let me put it this way. The firms
pushing out these AI chat systems seem to lack an understanding of how
ordinary persons exposed to them would react and use them. This is not
altogether surprising, we've seen this pattern in tech repeatedly for many
years, especially (but not exclusively) on the Internet.

While the firms have generally had disclaimers present on these AI
chat systems, to expect them to be fully understood in context by
random users of these systems is both unreasonable and potentially
dangerous.

Attempting to pause or stop AI training or other related research is not
practical nor desirable. But better communication with the public is
absolutely necessary. These systems need to be explained in ways that
non-technical, busy persons will appreciate in the context of their own
lives and experiences. The technologists designing these systems need to
realize that if sufficient resources are not dedicated to these direct
public communication and education needs, the firms will be ever more
targeted by politically-motivated attacks, and risk their work being ever
more mis-characterized by entities with political motives of their own, to
the detriment of the firms, their users, and the community at large.

This must be understood and acted upon immediately, or the benefits of AI
will be consumed by false narratives and it will be too late for much more
than painful regrets.


Australian mayor prepares world's first defamation lawsuit over ChatGPT content

Lauren Weinstein <lauren@vortex.com>
Thu, 6 Apr 2023 09:21:46 -0700
https://www.theguardian.com/technology/2023/apr/06/australian-mayor-prepares-worlds-first-defamation-lawsuit-over-chatgpt-content


Pausing AI Developments Isn't Enough. We Need to Shut It All Down (Eliezer Yudkowsky)

geoff goodfellow <geoff@iconia.com>
Sun, 2 Apr 2023 11:07:55 -0700
https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/

AI Labs Urged to Pump the Brakes in Open Letter
<https://time.com/6266679/musk-ai-open-letter/>


Forgive or Forget: What Happens When Robots Lie? (Catherine Barzler)

ACM TechNews <technews-editor@acm.org>
Wed, 5 Apr 2023 11:44:07 -0400 (EDT)
Catherine Barzler, Georgia Institute of Technology, 30 Mar 2023,
via ACM Tech News

Georgia Institute of Technology (Georgia Tech) researchers aimed to
determine whether a robot could apologize after lying to rebuild trust. The
study involved 341 online and 20 in-person participants in a game-like
simulation in which they were tasked with driving a robot-assisted car to
rush their friend to the hospital. The robot assistant warned that there
were police ahead and to stay under the speed limit, but after arriving at
the hospital, participants were informed that there had been no police. The
robot assistant then randomly provided one of five responses, three of which
admitted to deception and two that did not. Forty-five percent of in-person
participants did not speed, mainly because they believed the robot knew more
about the situation. The researchers found that apologizing without
admitting deception outperformed the other apologies, but when told about
the deception, the apology most effective in repairing trust involved an
explanation.


I am not afraid of robots. I am afraid of people. (Gary Marcus)

Gabe Goldberg <gabe@gabegold.com>
Mon, 3 Apr 2023 00:04:05 -0400
Some thoughts on AI risks, near-term and long-term, some recent
controversies in AI, and why we are in trouble if we can't find a way to
work together

https://garymarcus.substack.com/p/i-am-not-afraid-of-robots-i-am-afraid

With this great illustration of not-problem-solving:
https://twitter.com/razorbelle/status/1642000591802204162


Are robot waiters the future? Some restaurants think so. (AP News)

geoff goodfellow <geoff@iconia.com>
Thu, 6 Apr 2023 09:08:47 -0700
You may have already seen them in restaurants: waist-high machines that can
greet guests, lead them to their tables, deliver food and drinks and ferry
dirty dishes to the kitchen. Some have cat-like faces and even purr when you
scratch their heads.

But are robot waiters the future? It's a question the restaurant industry is
increasingly trying to answer.

Many think robot waiters are the solution to the industry's labor
shortages. Sales of them have been growing rapidly in recent years, with
tens of thousands now gliding through dining rooms worldwide.

“There's no doubt in my mind that this is where the world is going,'' said
Dennis Reynolds, dean of the Hilton College of Global Hospitality Leadership
at the University of Houston. The school's restaurant began using a robot in
December, and Reynolds says it has eased the workload for human staff and
made service more efficient.  [...]
  [Long article truncated for RISKS.  PGN]

https://apnews.com/article/robots-waiters-restaurants-84336d32667219776d4d0942c28caa46


It's Their Content,You're Just Licensing it. (NYTimes)

Monty Solomon <monty@roscom.com>
Tue, 4 Apr 2023 23:08:06 -0400
Recent automatic updates to e-book editions of works by Roald Dahl,
R.L. Stine and Agatha Christie are a reminder of who really owns your
digital media.

https://www.nytimes.com/2023/04/04/arts/dahl-christie-stine-kindle-edited.html

  [Sticking pins in the Dahl with widespread implications?  PGN]


Stupid physical risk

"Phil Smith III" <phsiii@gmail.com>
Mon, 3 Apr 2023 10:45:14 -0400
*Nextdoor* reports that some apartment complex of multiple buildings nearby
has identical keys for unit n in each building. Someone found out when she
woke up to find a stranger *in her apartment*, holding a key: he was a
prospective renter, was given key to check out unit, went to wrong building.

After some arguing with management, they sent locksmith to change at least
*her* locks. She got a few neighbors to verify that this was true for their
keys, too (presumably they knocked on other door, explained, then
demonstrated).

  [I Wonder how common this is.  Sure would make it easier for management to
  keep track of keys! /s]


Re: DC Metro Will Retrofit Faregates To Cut Down On Fare Evasion (RISKS-33.68)

Stan Brown <the_stan_brown@fastmail.fm>
Sat, 1 Apr 2023 07:07:03 -0700
My calculations come up with a different answer:

40,000 evasions per weekday
365*5/7 = about 261 weekdays per year (ignoring holidays)
40,000 * 261 = 10,440,000 evasions per year
Using your $5/fare(*) estimate, that's $52.2 million per year

Payback period, 70/52.2 = 1.34 years, or 1 year 4 months.

I'm sure there are plenty of shortsighted actions for which the Metro board
can be criticized, but a payback period of 16 months doesn't sound like one
of them.

  [Also noted by Martin Ward.  Opps.  Sorry.  I misread that as 40,000 each week...  BAD.  PGN]

Please report problems with the web pages to the maintainer

x
Top