Multiple vulnerabilities in Regexp and String case conversion

293 points by runesoerensen 13 hours ago on hackernews | 108 comments

lousken | 12 hours ago

I need a vacation

donkey_brains | 12 hours ago

We all do, but we don’t know if our companies will still be there when we get back.

darknavi | 7 hours ago

maybe that's ok. We can just burn it down and restart ("make no mistakes" right??)

bluerooibos | 4 hours ago

Look on the bright side: if every piece of software is riddled with vulnerabilities, the hackers will have too many options to choose from - safety in numbers! /s

_pdp_ | 12 hours ago

Book it for 18 months from now. Oh, also bring your laptop.

cwmoore | 11 hours ago

I hear Java is nice

layer8 | 11 hours ago

You can find nice shells on the beaches.

NooneAtAll5 | 11 hours ago

java had vulnerability in base64 decode asm

https://calif.io/java4shell/

saghm | 10 hours ago

A full day ago! Practically ancient

koolba | 10 hours ago

We’re all fools for thinking something as inherently dangerous as decoding a base64 string or converting it to upper case should be performed in a non-airgapped environment.

cheeze | 9 hours ago

> Java RCE/Memory Corruption

> OpenJDK13

Serves those cutting edge technology adopters right. I'm sitting on JDK8 baby!

dboreham | 9 hours ago

Right with you.

vbezhenar | 7 hours ago

Thankfully my servers are too old for this vulnerability to work.

kawsper | 5 hours ago

Just don’t use jackson for parsing.

TZubiri | 12 hours ago

It should be pretty straightforward to replace regexes with good ole ifs, array acceses, and comparision operators, whether manually or with an LLM. Pretty low hanging fruit to shave off some kilobytes of attack vector

preinheimer | 11 hours ago

I think you’re underestimating the complexity of a lot of regular expressions.

marcus_holmes | 11 hours ago

Also that a lot of the regexes will be inside Rails code that we can't touch, Gems that need maintaining, etc.

TZubiri | 7 hours ago

dependencies that themselves are targets for deletion

account42 | 38 minutes ago

A lot of that complexity is because regex often needs complex constructs to express simple problems. I.e. it's often actually not a very suited tool but gets used anyway because it's there.

Spivak | 8 hours ago

What you're describing is using a parser generator to take a regex and turn it into code you can embed into your project. This already exists and you can use it now. https://re2c.org/

tptacek | 11 hours ago

There's what I think is probably an awesome bug here.

Windows-1252 encoding has a representation for lowercase German sharp 's' (ß), but not one for the uppercase. If you upcase a string containing a sharp 's', it gets normalized to "SS", which increases the length of the string by one character. The 1252 case conversion code doesn't handle this; it looks like it checks for headroom in the buffer one character at a time. So if the last character of a string you upcase is ß, it looks like you get a one-byte overflow (of 'S').

If that's really the bug (there's a claimed RCE POC), the heap grooming for it must be really fussy! I couldn't find any other character pairs that expand like this.

You can feed strings in 1252 to a Rails program through multipart POSTs, which get to pick their encodings.

The great thing about this bug, apart from how it illustrates how doomed we all are (if I'm right), is that it should by rights have the best vulnerability name of the year, because you get to work 'ß' into it.

cowlevel | 11 hours ago

Unicode also has this feature because SS is the correct uppercase form of ß

p-e-w | 10 hours ago

It’s not. There are multiple uppercase forms, including “SS”, “SZ”, and the uppercase ß character, all of which are well-established, which makes all of them correct according to the principle of linguistic descriptivism.

Georgelemental | 9 hours ago

But according to the "principle" of blindly following the Unicode standard with no tailoring, the "correct" form is SS. (And Unicode's stability guarantees preclude ever changing it to ẞ)

edflsafoiewq | 9 hours ago

>And Unicode's stability guarantees preclude ever changing it to ẞ

This seems like a bad property for a system that encodes human writing systems.

pmontra | 9 hours ago

Human writing systems themselves have that very same property. The stability of written language prevents them to adapt swiftly to changes to spoken language.

Knight / night is an example for English. That k should be gone, right? Or all those silent -ment and e in French.

Enter the computer and I'm not surprised that German speakers end up mostly with SS because that's what computers do when automatically upcasing words.

A lot of spelling changes are dictated by what's available on keyboards. In Italian we accept E' as È because we don't have that upper case character on our keyboard. Automatic uppercase on start of sentences fixed it where it's available and mobile keywords show an È as an option of you long press a shifted e. But what if you are writing Italian in an English locale form field with a hardware keyboard (here with an OS with English locale) or you have to write È in the middle of a sentence or, go back 30 or 40 years, you have to use a typing machine?

I'm am sure that every language that had to go through a phase of adapting to ASCII and to the limits on the layout of keyboards has similar stories to tell.

zelphirkalt | 5 hours ago

The writing systems don't have that property, because you can write slang and spell words differently all fine. Orthography might have that property, but that's in the nature of orthography.

yorwba | 3 hours ago

The stability guarantee precludes changing the default conversion, but not adding a new 'de-DE-2024' locale with updated rules, similar to the existing '1901' and '1996' locales.

eviks | 9 hours ago

What stops you from blindly following the standard with tailoring?

account42 | 58 minutes ago

Compatibility concerns, which can quickly become security concerns.

ben-schaaf | 8 hours ago

And according to linguistic perscriptivism, the upper case of ß is ẞ since 2017. (SS is also allowed, but no longer preferred)

dingensundso | 6 hours ago

The uppercase ß was introduced as an option in 2017 but only became the preferred option in 2024.

Interestingly Unicode included the uppercase ß since 2008.

tbh I'm German and just learned about this now. I don't even know which keyboard incantation I would have to use to create this abomination. My android keyboard does not seem to have an uppercase ß.

cyphar | an hour ago

> tbh I'm German and just learned about this now.

It's a little amusing how many Germans I've heard this from, meanwhile as a non-German I think I first heard about ß and ẞ at least a decade ago.

Maybe this is one of those "outsiders have fresh eyes" things where googling "wait, wtf is up with ß" is not something most Germans would do?

A similar thing happens with Japanese natives and ヷ/わ゛ -- most are not aware this character existed historically and they usually also incorrectly guess how it's read.

frangonf | 10 hours ago

Or so the Germans would have us believe...

sscaryterry | 10 hours ago

Who cares, I use ASCII :)

SoftTalker | 7 hours ago

I've been saying that for years.

eviks | 9 hours ago

Not only that, when you uppercase 1 char with precomposed diacritics, but uppercase doesn't have precomposed diacritics, so they expand into separate chars (max is 1 to 3 chars)

Both fixable, but as with a lot of Unicode stuff, might take a long while

Should we call it #upcaße2Shell?

tptacek | 11 hours ago

One of the discoverers suggested, on Twitter, Ruby2ßhell. My wife, who speaks and writes German, refused to entertain this.

luipugs | 6 hours ago

Typical German ßenße of humor.

zelphirkalt | 5 hours ago

It would be funnier with the ß at the beginning of a word, so that the name containing ß can readily be used for the exploit itself.

stouset | 4 hours ago

It would need to be at the end.
I am a native German speaker, and I'm ok with all of them. I draw the line with reading ß as b or beta.

Anon1096 | an hour ago

I have a modest proposal: it should be named Rußy2Shell

Hendrikto | 2 hours ago

ß is not s though. It is closer to th. ”ßhell“ would be pronounced like Mike Tyson would probounce ”sell“.

account42 | an hour ago

In what language? For german, ß gets transliterated to "ss" in ASCII environments for a reason and words where updated spelling rules no longer use ß also replaced it with "ss" and not "th". That's also why it's pretty uncommon at the start of a word, which is why we got along without an uppercase variant for so long.

testplzignore | 11 hours ago

I wonder if there is a similar bug for U+0130 which (I think) expands to two characters when lowercased.

tptacek | 11 hours ago

Maybe somewhere! But that character isn't in Windows-1252 at all, so it can't trip this string counting bug.

throw-the-towel | 7 hours ago

Wait, why would the Turkish dotted uppercase I expand to two characters?

yorwba | 3 hours ago

Because, unless you use a Turkish locale for the case-conversion, it turns into a lowercase i plus a combining dot above.

david_allison | 4 hours ago

I've encountered bugs with U+FB03 (Latin Small Ligature FFI) which expands to three characters when uppercased.

bee_rider | 11 hours ago

Haha, that exact character caused problems in a programming class I tutored for. Got to throw the first year students some curveballs.

watersb | 9 hours ago

> it should by rights have the best vulnerability name of the year, because you get to work 'ß' into it.

Holy sh!t

juliusdavies | 8 hours ago

Holy Scheiße
Heiße Scheiße!
values of ß will give rise to dom

dwattttt | 8 hours ago

As is tradition, Raymond Chen has had something to say about this. The windows-1252 code page was introduced with Windows 1.0, and according to Wikipedia its final version shipped with Win98. String operations are just plain too hard.

https://devblogs.microsoft.com/oldnewthing/20200804-00/?p=10...

https://devblogs.microsoft.com/oldnewthing/20241007-00/?p=11...

tptacek | 7 hours ago

There is no good reason for really any Rails app to accept Windows-1252! This is reminiscent of the YAML vulnerability, which surprised most Rails users with the fact that Rails endpoints all accepted YAML to begin with.

timcobb | 6 hours ago

What if I am reading old ID3 tags and posting them verbatim to my rails app? Should I be detecting encodings and converting to utf8 before rails?

throw-the-towel | 6 hours ago

Why not?

account42 | an hour ago

Because not accepting it would break backwards compatibility and the general expectation that in HTTP the sender can use any non-obscure encoding and the receive has to deal with it. Of course there is no need for the receiver to implement special uppercasing for each encoding - more sensible would be to convert it to UTF-8 at the earliest opportunity.

thibaut_barrere | 6 hours ago

Agreed on YAML.

It must be noted though that Excel exports CSV as Windows-1252 by default in France (and other countries).Apps that ingest these files and run regexps over them are quite frequent.

throw-the-towel | 6 hours ago

I'm mortified by the second article, C++ really is Lovecraftian-level insanity.

usrnm | 5 hours ago

To be fair, there aren't many languages out there where working with strings and encodings is not a nightmare

VorpalWay | 2 hours ago

C and C++ are amongst the worst for working with strings, substrings, etc. It has gotten a bit better in newer versions of C++ (and I don't keep track of changes in C, so not sure what is going on there these days), but it is still far behind most more modern languages.

Python has good string support, as does Rust. I think (though I don't know it myself) Perl is famous for good string handling? Even bash has ok support for working with strings.

So I don't really agree on "aren't many languages", but then again there are a ton of languages I don't know, so maybe those I mentioned are the exception.

nikanj | 3 hours ago

Your favourite language has the exact same insanity, it just hides it behind a curtain. Recommended reading: https://www.usenix.org/system/files/1311_05-08_mickens.pdf

tialaramex | an hour ago

Yeah, no. C++ is deliberately wrong about lots of things. You don't have to be wrong, choosing not to be wrong isn't somehow hiding the wrong behind a curtain.

If you try to do the same mistake in Rust it does not compile, this function takes an int (obviously shouldn't but that's a different design bug in C not C++) and we have a wchat_t which is not an int, so that does not compile at all. Footgun removed.

When you start trying to figure out how to write the equivalent mistake in Rust properly, you discover that Rust says uppercasing or lowercasing arbitrary Unicode char values gets you an Iterator and you can't convince your compiler that this cannot make the string longer -- because in fact it might make the string longer. The mistake is averted by design.

Sure, but behind the scenes, the string is stored in memory as a list of bytes, and the list of bytes gets longer if you lower-case it. Rust pulls a nice curtain in front of all the nasty pointers and registers, but this problem can't be eliminated.

srgb_vs_cmyk | an hour ago

UTF-16 in general was a good idea[1] at some point of time but that didn't last long.

[1] or according to some a rotten compromise, between those who though UTF-32 wastes memory and UTF-8 discriminates non-latin languages.

account42 | an hour ago

No it wasn't. Using UTF-16 for that reason is all about posturing and not based on reason. Even for mostly CJK documents or ther languages where UTF-8 happens to be longer on average, any ASCII-based markup will quckly turn the tables. And if storage is that much of a concern just compress the damned thing - then the UTF-16 vs. UTF-8 doesn't matter at all for size.

And to be clear, UTF-16 was not chosen anywhere for that reason. It only exists because people thought UCS-2 would be enough for everybody (even though that was always far-fetched and futilely trying to make it reality has permanently damaged Unicode) and UTF-16 was the easies quck fix hack to "fix" UCS-2 based systems. It's yet another short term solution that will keep causing problems forever.

ludovicianul | 6 hours ago

In my API fuzzer I use a bunch of similar cases that can expand when lowered/uppered: https://github.com/Endava/cats/blob/master/src/main/java/com...

It's interesting how people start discovering these edge cases again, now that AI can be put to work to find vulnerabilities like this.

Mutable strings bite again, but I never thought they would actually cause a buffer overflow like this.

yturijea | 46 minutes ago

Can we just eliminate windows-1252 for good. that encoding have given me so many headaches over the years, and there is no good reason to not just use utf-8 especially nowadays.

slopinthebag | 9 hours ago

oh boy. if only we had a language that could prevent this from happening. oh well, i guess there is nothing we can do but patch these things once they're discovered.

Gigachad | 9 hours ago

"There's no way this could be prevented", says users of only programming language where it regularly happens

slopinthebag | 9 hours ago

good developers simply would not have written these bugs.

Spivak | 8 hours ago

Just completely rewrite CRuby which includes a JIT and has to maintain ABI compatibility for compiled extensions. I mean I'm certainly not volunteering for that project.
Careful. The Genie is listening.

ethin | 9 hours ago

Oh boy if you could develop a solution that would port C code over with completely safe rust semantics where feasible and without people having to clean up the disaster of a result, that would be nice, instead of you guys trying to demonstrate how superior you and your favorite language are. Because that is the only way you are going to port the world to Rust. Oh, and you have to support COBOL too. Probably a bunch of other old languages that have been around for longer than a lot of us have been alive.

Seriously, all these snarky posts about Rust being the god language really comes off as "look how utterly superior we are as Rust programmers and look how superior the Rust language is, this would never ever happen in Rust the god language". Yes, I'm annoyed by stuff like this.

slopinthebag | 8 hours ago

actually rust devs are humble enough to understand their limitations and let the compiler help them out.

as for porting, well, llm go burrrrrr. we can even use llms to port gpl to mit.

rixed | 6 hours ago

If only we were not so busy here on HN posturing as wiser devs!

jiggawatts | 5 hours ago

That's all fine and good, but then why are people starting new projects in C in the year of our lord 2026!?

baggy_trough | 9 hours ago

exactly, which is why I use ruby.

rixed | 6 hours ago

I'm guessing you are thinking of rust, but you know that actually most languages would prevent this bug, right?

silon42 | 4 hours ago

There was Java... then someone tried to optimize Base64.

reillyrider | an hour ago

It's because of shitty cowboy C.

Writing *p++ = c;

to append to a buffer is just insanity, I don't know why so many people like to write "clever" code like this.

I have a buf_putc(struct buf *b, int c); which does bounds checking (and potentially growing the memory), and I've used this kind of abstraction for decades.

Same with "use after free" bugs, WHY are people malloc()ing and free()ing all over the place, and storing away pointers to things all over the place (people just looove their "objects" and stuffing pointers to things into structs and classes so now you have these insane lifetime couplings).

Ban pointer arithmetic, ban null terminated strings, ban malloc() and free() (and similar general purpose allocators), ban passing arrays without a length (and preferable have a struct ptrlen {} and pass it by value) and you eliminate 99.9999% of these bugs.

Multi threading in C is genuinely difficult because you get zero help from the compiler, but these low hanging bounds and lifetime problems can easily be eliminated with simple abstractions.

These are all C-based memory corruption vulnerabilities.

The reality of these type of bugs being found by autonomous AI models on established software should tell you by now that memory-unsafe languages are completely exposed.

This is who maintainers you are up against:

"Thanks to Stephan Lipp (Ada Logics) and Anthropic for reporting this issue."

Imagine attackers using abliterated models at scale who don't disclose these issues.

[0] https://github.com/ruby/ruby/security/advisories/GHSA-34m4-m...

userbinator | 8 hours ago

Imagine developers using abliterated models and fixing these issues before releasing any more code.

Gigachad | 4 hours ago

Problem is as a developer you need to find every single memory management issue. As an attacker you only have to find one.

Gigachad | 8 hours ago

We have been screaming this for years before AI showed up. But instead of removing C wherever possible it was decided fine to just patch the bugs as they showed up.

rixed | 6 hours ago

So screaming is not effective??

miohtama | 4 hours ago

Especially here on HackersNews this has been discussed multiple times.

People come to defend C.

stouset | 4 hours ago

I will say it’s been awhile since I’ve heard any of the usual “C isn’t that hard if you just follow some basic rules” which inevitably boil down to never making a mistake ever again

kalleboo | 55 minutes ago

The excuse these days is always "but look at this class of bugs that aren't C bugs" whataboutism

skybrian | 7 hours ago

I imagine Ruby programmers thought they were working in a memory-safe language, but apparently that's not enough?

zelphirkalt | 5 hours ago

Now I might better wait for Rust based Ruby. Or does it exist already? Or how far is JRuby? Is that still alive?

bingo-bongo | 5 hours ago

Or start writing stuff in https://crystal-lang.org/ - it’s great :)

zelphirkalt | 4 hours ago

I love exploring languages, but at some point the question turns to: Is the ecosystem there yet? What would you say about the ecosystem of Crystal?

mike_hearn | 3 hours ago

Both JRuby and TruffleRuby are very much alive and are not written in C. Shopify supports the TruffleRuby project, and it not only comes with more memory safety but also a JIT compiler!

And yes, there can be vulnerabilities in the compiler itself. But it's still better to have less memory unsafe code than more.

IshKebab | 5 hours ago

Yeah where are all the "Rust is less secure than battle tested C" now?

pentestercrab | 9 hours ago

  str = ("\x82\xA0" + ("A" * (2**32 + 4))).force_encoding(Encoding::Shift_JIS)
  
  puts "str.bytesize: #{str.bytesize}"
  puts "str.encoding: #{str.encoding}"
  puts "str.valid_encoding?: #{str.valid_encoding?}"
  
  new_str = str.downcase
  
  puts "new_str.bytesize: #{new_str.bytesize}"
  puts "new_str.bytes:    #{new_str.bytes}"
  puts "new_str.encoding: #{new_str.encoding}"
outputs on an unfixed version:

  str.bytesize: 4294967302
  str.encoding: Shift_JIS
  str.valid_encoding?: true
  new_str.bytesize: 6
  new_str.bytes:    [130, 160, 97, 97, 97, 97]
  new_str.encoding: Shift_JIS
also I'd suggest you check out CVE-2026-128084 (GHSA-4fqr-96q8-xrfx) if you are interested in C bugs impacting Ruby on Rails

jeppester | 4 hours ago

I'm worried that this can be used against a standard rails app somehow, as I'm sure there's regex parsing and downcasing somewhere in the stack.

However, if you need to do an operation on a malicious multi gb string to trigger this, it makes me slightly less concerned that regular rails apps are vulnerable.

But I'm no a security expert, is my assumption just wishful thinking?

pentestercrab | an hour ago

Rails accepts multi gb strings fine, as long as you have enough memory/swap (even with rack trying to prevent this). Having nginx in front isn't guaranteed to protect you either, see CVE-2026-95101 in nginx for an example.

nickcw | 4 hours ago

> CVE identifiers have been requested for these vulnerabilities but have not been assigned yet, so this advisory refers to them by their GitHub Security Advisory (GHSA) identifiers.

It sounds like they are using GitHub to assign CVEs. We do the same at the rclone project. GitHub are running at a 4 week delay assigning CVEs! They used to do it in 2-3 days.

CER10TY | 3 hours ago

Isn't this delay pretty standard nowadays, due to the sheer number of submissions? It's also not like you can just hop onto another CNA (eg GitLab) if the repo is on GitHub, as it's out of scope for other CNAs.

vintermann | 3 hours ago

Fun fact, I ordered something online to an address in Ålesund once, it took a long time to arrive, package tracking showed the idiots (Deutsche Post as I recall) had shipped it to a place called Lesund instead. Ignoring the post code.

Just some days ago I ordered some sheet music, and guess what, it was shipped to an address where ÆØÅ were replaced with ? too. Hopefully it will arrive.

The amount of code out there that still simply breaks on non-ASCII characters (even Latin-1 characters!) is unbelievable. No wonder the AI vulnerability scanners are having a field day.

account42 | 50 minutes ago

> Deutsche Post as I recall

If that's the case you are lucky to have had package tracking at all.

mpetrovich | 3 hours ago

You have a problem.

“I know, I’ll use regular expressions to solve it,” you think.

Now you have two problems.