The RISKS Digest
Volume 34 Issue 14

Saturday, 6th April 2024

Forum on Risks to the Public in Computers and Related Systems

ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator

Please try the URL privacy information feature enabled by clicking the flashlight icon above. This will reveal two icons after each link the body of the digest. The shield takes you to a breakdown of Terms of Service for the site - however only a small number of sites are covered at the moment. The flashlight take you to an analysis of the various trackers etc. that the linked site delivers. Please let the website maintainer know if you find this useful or not. As a RISKS reader, you will probably not be surprised by what is revealed…

Contents

Eclipse tourists should plan for overloaded cell networks
PGN
AI Researcher Takes on Election Deepfakes
NYTimes
ETH Zurich student requirement for Windows 11/MacOS, “safe browser”
Thomas Koenig
Assisted living managers say an algorithm prevented hiring enough
WashPost
Many-shot jailbreaking
Anthropic
Google fixes two Pixel zero-day flaws exploited by forensics firms
BleepingComputer
GPS shut down in parts of Israel
Jim Geissman
House, Senate leaders nearing deal on landmark online privacy bill
WashPost
For Data-Guzzling AI Companies, the Internet Is Too Small
WSJ
Re: When AI Meets Toast (Steve Bacher
????
Re: AI that targets civilians …
Amos Shapir
Re: Your boss could forward a mail message to you that shows you text he won't see, but you will
Geoff Kuenning
Re: The FTC is trying to help victims of impersonation scams get their money back
Steve Bacher
Re: Browsing in Google Chrome's incognito mode doesn't protect you as much as you might think
Steve Bacher
Re: Elon Musk's Starlink Terminals Are Falling Into the Wrong Hands?
Amos Shapir
Info on RISKS (comp.risks)

Eclipse tourists should plan for overloaded cell networks (WashPost)

Monty Solomon <monty@roscom.com>
Sat, 6 Apr 2024 19:34:59 -0400

A surge of eclipse visitors could bog down local cell service. Here's how to deal, including by downloading maps and movies ahead of time.

https://www.washingtonpost.com/technology/2024/04/02/cell-service-poor-solar-eclipse/


AI Researcher Takes on Election Deepfakes (NYTimes)

ACM TechNews <technews-editor@acm.org>
Fri, 5 Apr 2024 11:39:53 -0400 (EDT)

Cade Metz and Tiffany Hsu, The New York Times 2 Apr 2024

TrueMedia.org, founded by Oren Etzioni (pictured), founding chief executive of the Allen Institute for AI, has rolled out free tools that journalists, fact-checkers, and others can use to detect AI-generated deepfakes. Etzioni said the tools will help detect “a tsunami of misinformation” that is expected during an election year. However, he added that the tools are not perfect, noting, “We are trying to give people the best technical assessment of what is in front of them. They still need to decide if it is real.”


ETH Zurich student requirement for Windows 11/MacOS, “safe browser”

Thomas Koenig <tkoenig@netcologne.de>
Thu, 4 Apr 2024 19:53:37 +0200

ETH Zurich requires all students starting this fall or later to have a laptop with Windows 11 or a recent version of MacOS so they can install what is euphemistically called “Safe Exam Browser” for examinations.

What do you call a software which locks out the user and prevents him from doing things on his own computer? The usual term is “malware”, I believe. Requiring students to install such malware on their own computers is not so great.

There is also claim that the Safe Exam Browser cannot be run in a virtual machine. As students are notoriously inventive, it will be interesting to see how long that claim will stand the test of reality…

https://ethz.ch/en/studies/bachelor/beginning-your-studies/BYOD.html


Assisted living managers say an algorithm prevented hiring enough staff (The Washington Post)

Richard Marlon Stein <rmstein@protonmail.com>
Thu, 04 Apr 2024 21:14:26 +0000

https://www.washingtonpost.com/business/2024/04/01/assisted-living-algorithm-staffing-lawsuits-brookdale/

An algorithm optimizes senior-care labor scheduling (aka opex). Profit extraction wins, seniors (and their families) get [shorted.


Many-shot jailbreaking

Monty Solomon <monty@roscom.com>
Thu, 4 Apr 2024 14:47:46 -0400

We investigated a jailbreaking technique—a method that can be used to evade the safety guardrails put in place by the developers of large language models (LLMs). The technique, which we call many-shot jailbreaking, is effective on Anthropic's own models, as well as those produced by other AI companies. We briefed other AI developers about this vulnerability in advance, and have implemented mitigations on our systems.

The technique takes advantage of a feature of LLMs that has grown dramatically in the last year: the context window. At the start of 2023, the context window=E2=80=94the amount of information that an LLM can process as its input=E2=80=94was around the size of a long essay (~4,000 tokens). Some models now have context windows that are hundreds of times larger =E2=80=94 the size of several long novels (1,000,000 tokens or more).

The ability to input increasingly-large amounts of information has obvious advantages for LLM users, but it also comes with risks: vulnerabilities to jailbreaks that exploit the longer context window.

One of these, which we describe in our new paper, is many-shot jailbreaking. By including large amounts of text in a specific configuration, this technique can force LLMs to produce potentially harmful responses, despite their being trained not to do so.

Below, we'll describe the results from our research on this jailbreaking technique—as well as our attempts to prevent it. The jailbreak is disarmingly simple, yet scales surprisingly well to longer context windows. […]

https://www.anthropic.com/research/many-shot-jailbreaking

Paper https://www-cdn.anthropic.com/af5633c94ed2beb282f6a53c595eb437e8e7b630/Many_Shot_Jailbreaking__2024_04_02_0936.pdf


Google fixes two Pixel zero-day flaws exploited by forensics firms (BleepingComputer)

Monty Solomon <monty@roscom.com>
Fri, 5 Apr 2024 10:32:52 -0400

https://www.bleepingcomputer.com/news/security/google-fixes-two-pixel-zero-day-flaws-exploited-by-forensics-firms/


GPS shut down in parts of Israel

“Jim” <jgeissman@socal.rr.com>
Thu, 4 Apr 2024 19:06:07 -0700

Looks like GPS in parts of Israel is out to interfere with a possible Iranian counterattack. One wonders what critical services are disrupted by this. One risk of relying on advanced systems while in a country at war.


House, Senate leaders nearing deal on landmark online privacy bill (WashPost)

Monty Solomon <monty@roscom.com>
Fri, 5 Apr 2024 21:38:56 -0400

The leaders of two key congressional committees are close to an agreement on a national framework to protect Americans' personal data online.

https://www.washingtonpost.com/technology/2024/04/05/federal-privacy-interne= t-congress/


For Data-Guzzling AI Companies, the Internet Is Too Small (WSJ)

ACM TechNews <technews-editor@acm.org>
Fri, 5 Apr 2024 11:39:53 -0400 (EDT)

Deepa Seetharaman, The Wall Street Journal, 1 Apr 2024

Companies working on powerful AI systems are encountering a lack of quality public data online, especially as some data owners block access to their data. One possible solution to the data shortage is the use of synthetic training data, though this has raised concerns about the potential for severe malfunctions. DatologyAI is experimenting with curriculum learning, which feeds data to language models in a certain order to improve the quality of connections between concepts.


Re: When AI Meets Toast

Steve Bacher <sebmb1@verizon.net>
Fri, 5 Apr 2024 16:22:42 -0700

Some of us remember this gem from the 1990s. It seemed absurd at the time, but not so much now, eh?

The object oriented toaster

Once upon a time, in a kingdom not far from here, a king summoned two of his advisors for a test. He showed them both a shiny metal box with two slots in the top, a control knob, and a lever. “What do you think this is?”

One advisor, an Electrical Engineer, answered first. “It is a toaster,” he said. The king asked, “How would you design an embedded computer for it?”

The advisor: “Using a four-bit microcontroller, I would write a simple program that reads the darkness knob and quantifies its position to one of 16 shades of darkness, from snow white to coal black. The program would use that darkness level as the index to a 16-element table of initial timer values. Then it would turn on the heating elements and start the timer with the initial value selected from the table. At the end of the time delay, it would turn off the heat and pop up the toast. Come back next week, and I'll show you a working prototype.”

The second advisor, a software developer, immediately recognized the danger of such short-sighted thinking. He said, “Toasters don't just turn bread into toast, they are also used to warm frozen waffles. What you see before you is really a breakfast food cooker. As the subjects of your kingdom become more sophisticated, they will demand more capabilities. They will need a breakfast food cooker that can also cook sausage, fry bacon, and make scrambled eggs. A toaster that only makes toast will soon be obsolete. If we don't look to the future, we will have to completely redesign the toaster in just a few years.”

“With this in mind, we can formulate a more intelligent solution to the problem. First, create a class of breakfast foods. Specialize this class into subclasses: grains, pork, and poultry. The specialization process should be repeated with grains divided into toast, muffins, pancakes, and waffles; pork divided into sausage, links, and bacon; and poultry divided into scrambled eggs, hard- boiled eggs, poached eggs, fried eggs, and various omelette classes.”

“The ham and cheese omelette class is worth special attention because it must inherit characteristics from the pork, dairy, and poultry classes. Thus, we see that the problem cannot be properly solved without multiple inheritance. At run time, the program must create the proper object and send a message to the object that says, ‘Cook yourself.’ The semantics of this message depend, of course, on the kind of object, so they have a different meaning to a piece of toast than to scrambled eggs.”

“Reviewing the process so far, we see that the analysis phase has revealed that the primary requirement is to cook any kind of breakfast food. In the design phase, we have discovered some derived requirements. Specifically, we need an object-oriented language with multiple inheritance. Of course, users don't want the eggs to get cold while the bacon is frying, so concurrent processing is required, too.”

“We must not forget the user interface. The lever that lowers the food lacks versatility, and the darkness knob is confusing. Users won't buy the product unless it has a user-friendly, graphical interface. When the breakfast cooker is plugged in, users should see a cowboy boot on the screen. Users click on it, and the message ‘Booting UNIX v.8.3’ appears on the screen. (UNIX 8.3 should be out by the time the product gets to the market.) Users can pull down a menu and click on the foods they want to cook.”

“Having made the wise decision of specifying the software first in the design phase, all that remains is to pick an adequate hardware platform for the implementation phase. An Intel Pentium with 48MB of memory, a 1.2GB hard disk, and a SVGA monitor should be sufficient. If you select a multitasking, object oriented language that supports multiple inheritance and has a built-in GUI, writing the program will be a snap.”

The king wisely had the software developer beheaded, and they all lived happily ever after.


Re: AI that targets civilians … (RISKS-34.13)

Amos Shapir <amos083@gmail.com>
Fri, 5 Apr 2024 13:13:07 +0300

Actually, using face-recognition methods may be the most humane way to tell apart terrorists who hide among the civilian population. Especially when the alternative older methods were more like “kill them all and let God sort them out”.


Re: Your boss could forward a mail message to you that shows you text he won't see, but you will (RISKS-34.13)

Geoff Kuenning <geoff@cs.hmc.edu>
Fri, 05 Apr 2024 07:16:58 -0700

I am famous among my colleagues for my insistence on reading emails in plain text—to the point that when I receive an HTML-only email I will sometimes eye-parse it rather than feeding it into a decoder (although that's getting harder and harder as mailers insist on cluttering everything with selectors). And I always send in plain text.

My primary reason for using plain text has always been an aversion to web bugs and to size bloat, but now I have a new justification. Complicated things can break in ways that are just impossible with simple ones.


Re: The FTC is trying to help victims of impersonation scams get their money back

Steve Bacher <sebmb1@verizon.net>
Fri, 5 Apr 2024 16:32:26 -0700

I'm not impressed. The FTC is combatting this by creating a rule? Aren't these actions (mostly) already illegal? Though I'm glad to see that they're trying to outlaw fraudulent email sender addresses. That's way overdue.


Re: Browsing in Google Chrome's incognito mode doesn't protect you as much as you might think (RISKS-34.13)

Steve Bacher <sebmb1@verizon.net>
Fri, 5 Apr 2024 16:04:50 -0700

The Globe article unfortunately mixes descriptions of Google (Chrome)'s behavior with explanations from Mozilla (Firefox) on how incognito mode works. Chrome and Firefox have separate implementations of this and other modes, and I'm sure that Google has no inclination to follow what Mozilla says, nor does Mozilla care how Google implements it.

It would have been useful if the article had enlightened us as to whether Firefox has the same protection issues, since they bothered to quote the Mozilla Foundation to begin with (info they probably scarfed from a Mozilla web page anyway).


Re: Elon Musk's Starlink Terminals Are Falling Into the Wrong Hands? (Risks 34.12)

Amos Shapir <amos083@gmail.com>
Fri, 5 Apr 2024 13:08:49 +0300

The positive side of this is that Starlink is a communication link which falls under some US jurisdiction, and enables US security services to eavesdrop on communications in remote areas of the world which were off the grid till now, and therefore where outlaws and terrorists abound.

SpaceX's statement that they can “geolocate and turn off individual terminals when it detects illegal use”—and yet they haven't turned off many suspicious links, may indicate that Musk may be collaborating with such moves.

Please report problems with the web pages to the maintainer

x
Top