This analysis seems off. Memory safety isn't a feature users will explicitly ask for or opt-in to. What users want is software that isn't buggy, and they just sort of ambiently expect it. A "Rust backend" is pretty meaningless to users, a bit like expecting users to pass a --dont-be-buggy flag to every invocation of the CLI tool.
As a user, I'd have questions like: if this backend is better then why isn't it the default? Why do I have to make a decision at all? I just want an HTTP library/tool that doesn't let me get hacked, how it achieves that is none of my concern really. Whether that's done using Rust or not isn't something that should appear in the user interface.
I also interpreted the article a bit that “users” here refers to developers that make use of libcurl, that would overall like to ensure that the application that they are writing does not segfault or similar. It seems plausible to assume that some of the Rust goodwill would translate to developers that may also try to migrate their overall code to Rust to try out that backend, and potentially contribute improvements. The Linux kernel and Git for example seem to attract a lot of attention of people that want to work on migrations towards more Rust.
I would agree that the curl project itself should have put more effort in fixing those bugs and eventually remove the C HTTP/1 backend, not the Rust code. That's what happened in Firefox for example, and other projects that adopted Rust incrementally.
As for why not making it the default, the post already made it clear? The new backend did not pass all the tests and therefore could not possibly be the default backend until that was fixed and turned out fixing the last few issues were quite hard. cURL has also supported different backends for a while now. For HTTP/3 for example you can swap out the backend for something else. This is like driving a car. If you literally don't care at all, just use the stock tires/engines and it will drive fine. But sometimes you may care and swap out the stock components for something else that suits your needs, and cURL allows that. cURL / libcurl gives a stable ABI as a glue but it's intentional that you can swap out the backend.
I guess it depends on what exactly you mean by "successfully". For example, while C is used for Linux and Linux is quite successful by most measures, that doesn't necessarily meant that there aren't areas where C isn't successful - in this case, a significant reason there's so much interest in Rust is precisely because there has not been success in producing reliably memory-safe code in C.
I think "forcing" might be a bit of a strong word for what's going on as well, but that's neither here nor there.
I feel "just because Rust" leans a bit towards the overly-dismissive side. I'd imagine Linus isn't interested in potentially adding support for Rust "just because Rust" - presumably he thinks there are potential concrete upsides that Rust offers over C that are worth expending time and effort to investigate despite the potential complications from introducing a new language.
> In the long run it will cause fragmentation and increase complexity to such a point that it could destabilize the original project in its entirety.
This sounds scary and all, but I'm admittedly a bit skeptical about this argument. Why would introducing Rust support necessarily lead to the outcome you describe? Are there examples of such an outcome happening to other projects?
Personally I wish libraries would just never remove/change anything from their API, that we'd be able to have some sort of immutable API. Only additions could be made, or you can fork it into another project if you want a different API.
cURL already supports other Rust-written backends as the article pointed out. See https://everything.curl.dev/internals/backends.html.
While these ideas are good defaults, unfortunately they lead to unmaintainable, monolithic, insecure code. One needs to make decisions on tradeoffs based on current needs and balance on how to move forward.
Remember how Tor anonymity was broken due to Postal's law? Or how many encryption downgrade attacks have resulted in breaches?
IMHO the better option, but a hard sell, is to focus on externalization and methods like ports and adapters and DDD to help you write interfaces that are more likely to be stable, while still having polices in place to help you move forward when needed.
The chaos report from the 90's and many many more studies since have showed that trying to get software projects perfect from the beginning is impossible.
There are just to many unknowns and trying to plan for them all is impossible.
In some situations like REST it is easier so that breaking changes can be released in a new API version, with a deprecation schedule.
With other things like graphql, ffi etc... it becomes far harder.
If changing the language results in API breakage, you weren't following the inversion principle.
Even with language features, what you call a 'stack' is an abstract data type, not a concrete implementation. While there may be very strong disintegration drivers like performance or language limitations that drive you leaking implementation details or limit your ability to confirm to the ADT while maintaining the other properties you want, that is not specifically the concrete implementation that results in that breaking.
ADT's like lists, stacks, or queues are defined by their semantics from the user's perspective.
Obviously Rust's design decisions may make duplicating the semantics while gaining the advantage of the switch challenging or possibly impossible.
While not really the target audience for either language, I would prefer zig's crashing behavior to hyper using unsafe evaluations of macros as an example.
https://github.com/hyperium/hyper/blob/30f2961e89eb306780d85...
Both are better than c's problems like `x[4]` actually being `*x` though. (IMHO)
The point being is, do target a forever API as a default ideal, but don't stick to that ideal too strongly or it will result in an outcome that is far worse for anything more complicated than say UNIX file operations which are simple enough to do so.
We are in a industry where the 'best' option rarely if ever exists and we almost always have to choose the least worst options.
If some external party can figure out how to refactor without breaking the ADT/API semantics, you shouldn't care except be impressed that they avoided leaking their implementation details to you.
> https://github.com/hyperium/hyper/blob/30f2961e89eb306780d85...
This looks basically like a port of something like Kotlin or C#'s ?. operator - the macro checks whether the pointer is null, returns err if it is, and dereferences the pointer otherwise. The part highlighted in the link looks like it's part of the macro implementation - there don't seem to be any "direct" uses of that particular rule in calling code and it's only invoked via the other rules in that macro definition. I think that should make that bit null-safe at least.
Lifetime safety might be a bit trickier, especially since that bit is part of hyper's C API so idk if there's anything Rust (or Zig, for that matter) could do to ensure the pointer is still valid when dereferenced.
Any reason why? On paper C doesn't seem to offer any benefit that Rust doesn't, nor Rust any harm that C doesn't.
https://lib.rs/curl is used in 1000 packages, https://lib.rs/hyper is in 21,000 packages.
Curl is big and supports lots of protocols, but Rust doesn't need a one-stop-shop for all of them. HTTPS covers majority of uses, and Rust has separate packages for mail, ftp, websockets, etc.
Disclaimer: Just reading TFA, not an active Rust programmer.
Only some existing Rust code has been added to curl, but the Rust ecosystem already has a better, safer way of using that code.
Curl is not planning to ever require Rust, so the rewrites are limited only to optional components, and can't fully guarantee that the code is safe. The Rust components are required to expose an unsafe unchecked C interface for the rest of curl. C compilers are unable to enforce the safety invariants of the interface, like the Rust compiler would in a program fully written in Rust.
> A rewrite of curl to another language is not considered
> This is not converting curl to Rust.
> Don’t be fooled into believing that we are getting rid of C in curl by taking this step.
Curl plans to live and die with C: https://daniel.haxx.se/blog/2017/03/27/curl-is-c/
This has been reaffirmed recently in https://daniel.haxx.se/blog/2024/08/06/libcurl-is-24-years-o...
> There was never any consideration to use another language than C for the library […] Not then, not now.
Not only curl is rejecting possibility of being rewritten in Rust, it's also committed to supporting completely-Rust-free curl, because they pride themselves on supporting a lot of retro/niche platforms that Rust doesn't exist on.
If there was some way to prove that the abstraction is safe, then that would be fine. But the inadequacy of communal auditing is the reason why C has security issues.
fn do_something() {
unsafe { ... }
}
// Somewhere in the program
do_something();
let is_tty = unsafe { libc::isatty(libc::STDERR_FILENO) } != 0;
In fact, there are exactly two "unsafe" blocks in all of my Rust projects, and the second one is not even needed anymore because of the language and ecosystem improvements, but the project is basically abandoned, so I'm probably not gonna fix it. There's just no need for unsafe in the vast majority of code.
I don't know where Rust critics get their statistics; probably from picking their noses, judging by their arguments. Most don't seem to even have read the official docs, the bare minimum required to form any conclusions at all about the language. I guess they don't read much Rust code and think there is no way we can write even semi-decently performing high-level code without resorting to unsafe hacks or calling C, because that's the way it's done in other languages.
You get some intuition for which methods to use, many structs have a pretty breadthy pallet, and then you can mostly just ignore it; quickly write your algorithm and let the compiler walk you through any oversights with the way you may have coded processing tasks.
Also helps to know that the docs support search by type signature since functions operate on or return owned, borrowed, or mutably borrowed, and generally just think of borrowed as a pointer for heuristic reasons.
It makes more sense than I do, usually.
This plus a stable ABI for handlers would also solve the writing backends in other languages problem because another team could contribute a Rust / hyper handler separately.
(I'm sure Daniel has solid reasons for not doing this, not to mention that it's a bunch of work which no one is volunteering to do.)
With separate curl handlers you'd also be able to find out what protocols your system is configured to access with a command like this [simulated]:
$ rpm -qa | grep curl-handler- | sort
curl-handler-ftp
curl-handler-http
curl-handler-https
curl-handler-smb
I wonder if there's a place for an additional level of abstraction here, so that the user would just see something like "subcomponent X of the curl package is being installed"...
The wrapper crate hasn't got everything exposed for the Easy interface (I'll probably be submitting PRs to expose some missing stuff at some point), but seems good enough for me.
Only issue really I've found so far is that heap fragmentation seems orders of magnitude higher than the C++ version, and so memory useage grows quite alarmingly, but from heaptrack, that all seems to be from within openssl, with things like this (a rediculous number of heap allocations per request):
https://github.com/openssl/openssl/issues/14837
but I'm not sure why the older C++ version doesn't suffer from the same issues as it's doing the same thing with HTTPS requests and is also using openssl, so I still need to look into that...
Currently doing:
unsafe { libc::malloc_trim(0); }
every 5 minutes keeps the memory use under control, but again, I don't need to do that in the C++ version, so there's some difference somewhere in the libcurl stack between the two...
I personally wouldn't even consider hyper as an HTTP client for use in most projects, but reqwest adds a much more approachable API on top of hyper.
[0] https://github.com/seanmonstar/reqwest/blob/master/Cargo.tom...
Much of that is due to not needing Tokio & co.
Part of the problem is there are so many, which do you choose?
I found one (I forget the name to protect the guilty) but my simple use case broke it.
I reached out to the authors, very helpful, I think they fixed it.
But by then I had moved on (time is money)
I sucked it up and went with importing all the tokio stuff I did not need (I absolutely did not need, almost no one does)
Rust has painted itself into a corner. A really very good idea at its core, almost all of it is really terrific, but some corner cases like these make it very hard to take advantage of it
This just isn't a significant problem at all.
Nonsense.
If you are steeped in HTTPS clients for Rust, maybe, but I can tell you from personal experience that coming fresh to https://crates.io/search?q=https%20client and friends is overwhelming
That would be interesting to dig in deeper. Wish they did that in the article. Or linked an article where they do.
The systemic security problems of cargo package distribution opens the door for state-sponsored threat actors to performing supply chain attacks. And when asked about these issues the only people who will comment using their real identities are based in China.
The security issues in the rust development experience are well-documented since several years, so my summary at [1] was no news to them.
But if you offer things like rustfmt, rustdoc, and a nice-to-use package distribution platform like cargo.io you should be a bit more concered with security imo.
Their lax attitude to these issues and substantial corporate interests not only from US-based but also China-based companies might be good for funding but for me raises many questions.
Do not post blatant lies.