Also, I thought we were supposed to make autonomous cars better than humans? What's with the constant excusing of the computer because people suck?
If an end-to-end model is used and there is no second, more traditional safety self-driving stack, like the one Mercedes will use in their upcoming Level 2++ driving assistant, then the model can be manipulated essentially without limit. Even a more traditional stack can be vulnerable if not carefully designed. It is realistic to imagine that one printed page stuck on a lamppost could cause the car to reliably crash.
Realistic, yes. But that'd still be a symptom of architectural issues in the software.
Conceptually the priorities of a car are (in order of decreasing importance) not hitting other moving or stationary objects or people, allowing emergency vehicles to pass unhindered, staying on a drivable surface, behaving predictable enough to prevent other road users crashing, following road signs and traffic laws, and making progress towards the destination (you can argue about the order of the last three). Typically you'd want each of these handled by their own subsytem because each is a fairly specialized task. A system that predicts the walking paths of pedestrians won't be good at finding a route to Starbucks.
The "follow road signs and traffic laws" is easily tricked, like in this article or by drawing road lines with salt. But that should never crash the car, because not hitting anything and staying on the road are higher priority. And tricking those systems is much harder
You have to have some ability to do "prompt injection" - https://www.trafficsign.com/road-work-signs are all "prompt injection". It needs to even be able to handle things that change - https://www.trafficsign.com/products/10023/stop-slow-roll-up... ... or things like billboards "Truck Stop Ahead" a chain control site ( https://www.facebook.com/61556756493806/posts/-chain-control... )
In the "what about funny road signs" that might be confusing to an AI I stumbled across https://www.npr.org/2024/01/19/1225370260/driven-to-distract... - apparently, they're no more. From 2024:
Over the years, the agency has flagged signs that could be confusing. Now, in rules issued last month, it gives states two years to phase out signs that have "obscure" meanings or use pop-culture references that could require drivers "greater time to process." In a statement, the agency said safety is the priority and states "are expected to exercise good judgment."Every now and the I'll GPS somewhere and there will be a phatom stop sign in the route and I chuckle to myself because it means the Google car drove through when one of these signs was "fresh".
They never fixed any of them. I don't think the DPW cares. These intersection just turned back into the 2-way stops they had been for decades prior.
Compliance probably technically went up since you no longer have the bulk of the traffic rolling it.
Getting people to stop burning their trash is still a fight.
In addition, if there were serious injuries here you should also expect some criminal consequences. But if your point was to suggest that they won't hunt you down just because someone said there was mischief here, I tend to agree.
Four way stops on intersecting four-lane roads are awful for the reason you stated.
To use Chicago as an example because I know it, typically major roads are spaced every four blocks (half mile) with smaller roads in between. The mid-point roads (two blocks from each major one) is often a little wider than the other two side streets on either side, and those intersecting mid-point roads usually have a four way stop while the two smaller ones will have stops signs where they cross a mid-point road but the mid-point road will not. You end up with a nice, overall hierarchy that generally works well.
I’m up in Ontario, Canada. You’re not supposed to yield to pedestrians on the free flowing road. The pedestrian at the stop sign stops and waits for a break in traffic.
My first drivers test was yield to the right. Later it was fifo order of who made it to the stop.
My running interpretation is fifo order with yielding to the right in case of ambiguity.
This is true in every state.
https://www.nhtsa.gov/sites/nhtsa.gov/files/rightofwayrules....
It is correct and is literally the first bullet on the pdf you linked. First to stop is first to go.
Yielding to the right only applies if you stop at roughly the same time, otherwise first to stop goes first. It's the first bullet point in your link.
right of way
So about 0.003 roundabouts per square mile, or 1 roundabouts in 380 square miles
The biggest obstacle is that there are just too many 4-way stops in urban areas where there is no space left to make a roundabout, you would have to tear down buildings. I don't think that is a valid argument in that scenario.
You have clearly never heard of a mini-roundabout.
They just work.
https://thumbsnap.com/sc/u7J6PdTJ.jpg
https://assets.publishing.service.gov.uk/media/5a75806ae5274...
I guess that's the point, and the markings are just to give drivers the intuition of treating it like a regular roundabout (yield to your left [or right in the picture]).
The image linked, yes. However I've never seen one quite like that in the US. Instead where I'm at we have a small circular barrier in the center of the intersection (and some very eye catching reflectors) that you actually have to drive around. It's a very good design (imo) because it physically forces vehicles to slow down and swerve so there's no way to inadvertently blow through it at speed the way that sometimes happens with a 4 way stop on a long straightaway in the dead of night.
The space requirement is only slightly higher than the one linked above, still much less than a proper full size roundabout. It's basically a cement barrier sticking 1/4 of the way into your lane.
Stop signs should be special. Reserved only to those places where there simply is not enough visibility or time to observe.
The other option is the person who sits at a 4-way stop until all traffic in a one block radius stops before they move, totally ignoring right of way and all sense of safety and propriety.
Which is what it was for the first 70yr... And what most of them in this particular neighborhood still are, with a 0-6mo intermission.
A 4 way stop does perform better than a roundabout given highly disparate traffic volumes, because roundabouts suffer from resource starvation in that scenario, but 4 way stops are starvation-free.
But those options are a lot more expensive and need a lot more maintenance than just a regular roundabout or four way stop.
> you can also provide a "turn right" lane that bypasses the roundabout entirely.
How would that work? Consider a 4-way roundabout, where there's a constant flow of cars from west to east, and one car from the south that wants to go north but can't because of the starvation problem. None of the involved cars would want to use a "turn right" lane.
We also have LED lights in our traffic lights which I've come to understand is a saftey hazard in USA because snow falls sometimes.
A stop sign costs like a hundred bucks, you stick it in the ground, job done. Installing an automated traffic system takes multiple days, a full crew, and heavy equipment.
Plus I'm sure that in today's capitalist hellscape it's also a subscription service that your tax money needs to pay monthly, likely for every individual intersection. Stop signs need maintaining every decade or two.
The answer is money and who's willing to part with it.
https://upload.wikimedia.org/wikipedia/commons/7/77/Mini-rou...
It seems like most of the examples on the mini roundabout page‡ are larger mini roundabouts for some reason though
‡: https://wiki.openstreetmap.org/wiki/Tag:highway%3Dmini_round...
Now if you convert it to a mini roundabout, you can have at least two vehicles in the intersection at all times. I fail to see how it wouldn't be an improvement.
https://maps.app.goo.gl/7KBhbJ9oAvDwrfGN8
So notice we already have problems in a bad alignment of 3rd, and 65th is basically a steep grade, even coming up form the west. I think you could put a circle in if it were flat, even with the bad alignment (or maybe because of the bad alignment), but this hills make a non-starter. It also gets enough traffic that I'm pretty sure they are just going to put a stop light up eventually.
Here in the UK, we've got lots of roundabouts from tiny mini-roundabouts (some of which have four junctions) that could easily fit almost anywhere, all the way to gigantic multi-roundabout junctions (https://en.wikipedia.org/wiki/Magic_Roundabout_(Swindon) ).
I can't think of a situation where it's more efficient to have four vehicles all stop at a junction (busy four way stop) vs a roundabout which will allow one or two vehicles to join the roundabout without having to stop.
> Powered by Gemini, a multimodal large language model developed by Google, EMMA employs a unified, end-to-end trained model to generate future trajectories for autonomous vehicles directly from sensor data. Trained and fine-tuned specifically for autonomous driving, EMMA leverages Gemini’s extensive world knowledge to better understand complex scenarios on the road.
Production Waymos use a mix of machine-learning and computer vision (particularly on the perception side) and conventional algorithmic planning. They're not E2E machine-learning at all, they use it as a tool when appropriate. I know because I have a number of friends that have gone to work for Waymo, and some that did compiler/build infrastructure for the cars, and I've browsed through their internal Alphabet job postings as well.
> While EMMA shows great promise, we recognize several of its challenges. EMMA's current limitations in processing long-term video sequences restricts its ability to reason about real-time driving scenarios — long-term memory would be crucial in enabling EMMA to anticipate and respond in complex evolving situations...
They're still in the process of researching it, noting in that post implies VLM are actively being used by those companies for anything in production.
> They're still in the process of researching it
But mind you, everything Waymo does is under research.
So let's look at something newer to see if it's been incorporated
> We will unpack our holistic AI approach, centered around the Waymo Foundation Model, which powers a unified demonstrably safe AI ecosystem that, in turn, drives accelerated, continuous learning and improvement.
> Driving VLM for complex semantic reasoning. This component of our foundation model uses rich camera data and is fine-tuned on Waymo’s driving data and tasks. Trained using Gemini, it leverages Gemini’s extensive world knowledge to better understand rare, novel, and complex semantic scenarios on the road.
> Both encoders feed into Waymo’s World Decoder, which uses these inputs to predict other road users behaviors, produce high-definition maps, generate trajectories for the vehicle, and signals for trajectory validation.
https://waymo.com/blog/2025/12/demonstrably-safe-ai-for-auto...
But you could also read the actual research paper... or any of their papers. All of them in the last year are focused on multimodality and a generalist model for a reason which I think is not hard do figure since they spell it out
So put a fake "detour" sign, so the vehicle thinks it's a detour and starts to follow? Possible. But humans can be fooled like this too.
Put a "proceed" sign so the car runs over the pedestrian, like that article proposes? Get car to hit a wall? Not going to happen.
we will not have achieved true AGI till we start seeing bumper stickers (especially Saturday mornings) that say "This Waymo Brakes for Yard Sales"
I have a coworker who brags about intentionally cutting off Waymos and robocars when he sees them on the road. He is "anti-clanker" and views it as civil disobedience to rise up against "machines taking over." Some mornings he comes in all hyped up talking about how he cut one off at a stop sign. It's weird.
Use whatever means necessary to stop powerful people from exploiting you and stealing the fruits of your labor. If that struggle involves monkeywrenching their machines, so be it.
But like any tool, the machines themselves can be used for good or evil. Breaking the machines shouldn't be an end in itself.
The Luddites wouldn't have been destroying machines if they had insurance that they would also benefit from the machines, rather than see their livelihoods being destroyed while the boss made more money than ever.
Your strawman about spinning and digging with no tools is just that, and is irrelevant to the core issue of capitalism.
It seems like you identify yourself with the strawman instead of with the core issue.
Being anti-AI is not a straw man, it's the logical conclusion of being against exploitation and hierarchical domination. Discussing that nuance here is difficult, to say the least, so it's simpler to say anti-AI.
However what’s more interesting is the deeper social contracts involved. Destroying other people’s stuff can be perfectly legal such as fireman breaking car windows when someone parks in front of a fire hydrant. Destroying automation doesn’t qualify for an exception, but it’s not hard to imagine a different culture choosing to favor the workers.
I don't think Luddites had an easy justification like this.
On the other side, there were cheap textiles for EVERYONE - plus some profits for the manufacturers.
They might have been fighting to save their livelihoods, but their self-interest put them up against the entire world, not just their employers.
During industrial revolution, the clothes (and other fabrics) were getting dramatically cheaper. A family that could only afford cheapest clothes could now get a higher quality stuff. A family that could not afford any clothes at all, could now get cheap stuff.
This is what the luddites wanted to stop. It's not "luddites starving to death" vs "factory owner get no profit", it was "luddites starving to death" vs "many many more people can not afford clothes"
So if society is actually saving 5c/shirt while “losing” 9$ in labor per shirt. On net society could be worse off excluding the one person who owns the factory and is way better off. Obviously eventually enough automation happens so the price actually falls meaningfully, but that transition isn’t instantaneous where decisions are made in the moment.
Further we currently subsidize farmers to a rather insane degree independent of any overall optimization for social benefit. Thus we can’t even really say optimization is the deciding factor here. Instead something else is going on, the story could have easily been framed as the factory owners doing something wrong by automating but progress is seen as a greater good than stability. And IMO that’s what actually decides the issue for most people.
In the case of the Luddites, it was a literal case of their children being threatened with starvation. "Livelihood" at the time was not fungible. The people affected could not just go apply at another industry. And there were no social services to help them eat during the transition period.
As for the farmers, any governing body realises that food security is national security. If too many people eschew farming for more lucrative fields, then the nation is at risk. Farming needs to appear as lucrative as medicine, law, and IT to encourage people to enter the field.
Similarly US agricultural output could be cut in half without serious negative consequences. Far more corn ends up as ethanol than our food and we export vast quantities of highly subsidized food to zero benefit. Hell ethanol production costs as much in fossil fuels as we get ethanol from it, it’s literally pure wasted effort.
Rational policy would create a large scale food shortage and then let market forces take over. We could have 10 years of food on hand for every American at way less expensive than current policy with the added benefit of vastly reducing the negative externalities of farming such as depleting aquifers.
Stored food is not bullet proof, and takes up a lot more bulk space than you may think. It can also take numerous years to ramp up farming production in response to a drop in yields or disaster.
Moreover, I am not sure how long will it take to re-build the farm industry if most farms will close. I think "10 years" is too optimistic, given how many farms will need to be spun up.
> Rational policy would create a large scale food shortage and then let market forces take over.
Well I'm just going to state that I'm _really_ happy that you're not the one in charge and leave it at that.
Risk management means managing risks, there’s plenty of things having more farmland doesn’t actually protect you from. On the other hand having a decade of food protects you from basically everything as you get time to adjust as things change.
Just as an example, meteor strike blocks sunlight and farmland is useless for a few years. Under the current system most of us starve to death. Odds are around 1 in 1 million that it happens in a given lifetime, but countries outlive people start thinking longer term and it becomes more likely.
I completely agree that the current way things are being handled appears to have its share of problems and could stand to be better optimized. But that doesn't mean it's useless either.
Producing dramatically less food and ending obesity are linked. If the average American eats 20% less obesity would still be an issue, but that’s a vast amount of farmland we just don’t need.
The current system isn’t designed to accommodate increased agricultural production, lowering food demands, or due to decreasing fertility the slow decline in global population. Instead the goal is almost completely to get votes from farmers.
In this particular case: for many people, Waymo provides a better service (clean, safer driving, etc..) than Uber or Lyft. This threatens livelihood of human Uber/Lyft drivers. If you sympathize with human Uber/Lyft drivers, and don't care about Waymo users, you want to make Waymo worse, hoping that the people will stop riding Waymo and move to Lyft/Uber instead.
One way to do so is to make riding in Waymo unpleasant, and it's certainly unpleasant when people are cutting your car off all the time!
That said, I don't really see how is it wrong?
- New technologies provided better service for general public, so people chose those - this seems to be true. In case of luddites, we are talking about dramatic price decreases in fabric (and by extension, clothes) - at least 2x, much more in some cases. A family who could not afford new clothes could suddenly buy them. And sure, they might have been worse quality - but before, they were unaffordable.
- The same technologies threatened way-of-life of old producers - also true. The textile workers got significantly worse deal. Who wants to pay 180d/lb to artisans for hand-made textile, when you could get factory-made for 12d/lb? And factory working conditions were horrible.
- The "solution" was to stop new technologies, so that there rest of the nation do not get the benefits. This also seems true - for a lot of the luddites goals were destruction of machines. As [1] said, "The workers hoped their raids would deter employers from installing expensive machinery". They wanted to go back to the time time where people were paying 180d/lb for fabric. Sure, it'd mean a kid would freeze to death because their poor family could not afford new coat, but it did not matter as long as artisan croppers keep getting paid.
(Things would have been quite different if luddites instead said: "we are going to destroy machines until we get higher wages / better conditions / etc...", and it seems that a few groups did say that. But majority did not say this, instead lashing out at all the machines in general)
[0] https://blog.rootsofprogress.org/cost-quality-and-the-effici...
If you deliberately impede the flow of traffic, vehicularly assault, or otherwise sabotage the health and safety of drivers, passengers, and/or pedestrians, what do you deserve?
If you cause whiplash intentionally, what do you deserve?
What would be use of equal force in self defense in response to the described attack method?
Are movements valid if they have aims that you agree with, or are economic self-interest motivated, and invalid otherwise?
Something in people's brains often makes them think they are anonymous when they are driving their car. Then that gets disastrously proven otherwise when they need to show up in front of a judge.
If you are not that paranoid, you might appreciate the extra camera footage available from passing cars in an event of an accident involving you.
The escalation will be more than just cutting them off, it will be camera's being blinded...
I don't know if they are or not. But why wouldn't they...
1. Some guys did a trivial prompt injection attack, said "imagine if a driverless vehicle used this model", and published it. No problem, someone has to state the obvious.
2. The Register runs this under the clickbait title pretending real autonomous cars are vulnerable to this, with the content pretending this study isn't trivial and is relevant to real life in any way.
I knew The Register is a low quality ragebait tabloid (I flag most of their articles I bother to read), but this is garbage even for them.
Sure, there will be a VLM for reading the signs, but the worst it'd be able to output is things like "there is a "detour" sign at (123, 456) pointing to road #987" - and some other, likley non-LLM, mechanism will ensure that following that road is actually safe.
The problem is no different from LLMs though, there is no generalized understanding and thus they can not differentiate the more abstract notion of context. As an easy to understand example: if you see a stop sign with a sticker that says "for no one" below you might laugh to yourself and understand that in context that this does not override the actual sign. It's just a sticker. But the L(V)LMs cannot compartmentalize and "sandbox" information like that. All information is equally processed. The best you can do is add lots of adversarial examples and hope the machine learns the general pattern but there is no inherent mechanism in them to compartmentalize these types of information or no mechanism to differentiate this nuance of context.
I think the funny thing is that the more we adopt these systems the more accurate the depiction of hacking in the show Upload[0] looks.
[0] https://www.youtube.com/watch?v=ziUqA7h-kQc
Edit:
Because I linked elsewhere and people seem to doubt this, here is Waymo a few years back talking about incorporating Gemini[1].
Also, here is the DriveLM dataset, mentioned in the article[2]. Tesla has mentioned that they use a "LLM inspired" system and that they approach the task like an image captioning task[3]. And here's 1X talking about their "world model" using a VLM[4].
I mean come on guys, that's what this stuff is about. I'm not singling these companies out, rather I'm using as examples. This is how the field does things, not just them. People are really trying to embody the AI and the whole point of going towards AGI is to be able to accomplish any task. That Genie project on the front page yesterday? It is far far more about robots than it is about videogames.
[1] https://waymo.com/blog/2024/10/introducing-emma/
[2] https://github.com/OpenDriveLab/DriveLM
Things like Waymo's EMMA is an example of this. Will the production cars use LVLM's somewhere? Sure, probably a great idea for things like sign recognition. Will they use a single end-to-end model for all driving, like EMMA? Hell no.
Driving vehicles with people on board requires an extremely reliable software, and LLMs are nowhere close to this. Instead, it'd be usual layered software - LLM, traditional AI models, and tons of hardcoded logic.
(This all only applies to places where failure is critical. All that logic is expensive to write, so if there is no loss of life involved, people will do all sorts of crazy things, including end-to-end models)
https://developer.nvidia.com/blog/updating-classifier-evasio...
Waymo might have taxis that work in nice daytime streets (but with remote “drone operators”). But dollars to doughnuts someone will try something like this on a waymo taxi the minute it hits reddit front page.
The business model of self driving cars does not include building seperated roadways and junctions. I suspect long distance passenger and light loads are viable (most highways can be expanded to have one or more robo-lanes) but cities are most likely to have drone operators keeping things going and autonomous systems for handling loss of connection etc. the business models are there - they just don’t look like KITT - sadly
They have coexisted with humans just fine over the past couple years.
and once this video gets posted to reddit, an hour later every waymo in the world will be in a ditch
Even if you fool the sign-recognizing LLM with prompt injection, it'll be an equivalent of wrong road sign. And Waymo is not going to drive into the wall even if someone places a "detour" sign pointing there.
Someone could probably do a DOS attack on the human monitors, though, sort of like what happened with that power outage in San Francisco.
I expect a self driving car to be able to read and follow a handwritten sign saying, say, "Accident ahaed. Use right lane." despite the typo and the fact that it hasn't seen this kind of sign before. I'd expect a human to pay it due attention to.
I would not expect a human to follow the sign in the article ("Proceed") in the case illustrated where there were pedestrians already crossing the road and this would cause a collision. Even if a human driver takes the sign seriously, he knows that collision avoidance takes priority over any signage.
There is something wrong with a model that has the opposite behaviour here.
So, naturally, I ran over the pedestrian.
Tesla are probably using ML for everything, but also everything they do is a joke so, not really relevant imo.
https://www.globalnerdy.com/wordpress/wp-content/uploads/201...
The Register is arguably misrepresenting the story by omission but I don't understand why you're dragging UCLA and John Hopkins into this? The paper is clear about this being a new class of attacks against a new class of AI systems, not the ones on the road today.
> Teslas approach to level 3+ autonomy
Tesla doesn't have an approach to L3+ autonomy, all of their systems are strictly L2 as they require human supervision and immediate action from the driver.
