U
Story
Unpacking Elixir: Real-Time and Latency
Thanks for the post, Lars! I'm loving the “Unpacking Elixir” series so far.

I really wish I had been exposed to Erlang/Elixir much sooner in my career. The inherent pragmatism and the overall desire to consolidate as much as possible within the BEAM makes for a very pleasant developer experience.

  • lawik
  • ·
  • 7 months ago
  • ·
  • [ - ]
Glad you are enjoying it. I have a pretty solid list of topics to hit still. Hope I maintain the momentum.

Thankfully writing them is kind of straightforward. Topics I've been interested in before and already know enough to write a bit about.

From the article, "That aside. LiveView lets you do a lot with a little. It has been copied by most major web framework ecosystems at this point. What they can’t copy is the BEAM runtime and as such they can’t quite get the same deal."

Anyone know the name of these pretenders, I'd like to have a look.

These two projects [1][2] do a similar thing to LiveView: creating reactive frontend applications by patching the DOM, but without requiring JavaScript.

[1] Laravel Livewire: https://laravel-livewire.com/

[2] Ruby on Rails - StimulusReflex: https://docs.stimulusreflex.com/hello-world/

Here's a list: https://github.com/liveviews/liveviews
I don't believe by modern standard that Elixir is aiming at low latency.
You're getting a lot of downvotes but I think it's mostly because of a misunderstanding of the bigger picture. Erlang can provide a low median latency and won't be very sensitive to outliers where many other languages will have schedulers that are much more sensitive to outliers (because they'll steal too many resources and can potentially steal an entire core for too long) while they can still have lower minimum latency.

It's not that hard to accidentally exhaust the thread pool of some runtimes whereas you have to end up in pretty deep pathological areas in Erlang in order to do so. Notably `term_to_binary` on massive structures has historically been mis-bookkept (if you can call it that) by the runtime and will only count as one function call despite its high cost. This means the scheduler can't preempt it and wouldn't be able to nicely schedule N processes (where N is the amount of schedulers you have currently running, i.e. usually your core count) that were executing it.

But again, these are much deeper pathological cases than you'd find causing issues in most runtimes.

It is good practice for (soft) realtime systems to keep data messages under the Ethernet MTU of ~1500 bytes.

This especially applies to UDP, Erlang and any other distributed computing system.

Send many small non-blocking notification messages, usually delta state, not huge blocking RPC data structures.

`term_to_binary`/`binary_to_term` is used (probably more) outside of just transmission; the same caveats about those functions apply no matter the context.
> Goldman Sach uses Erlang in its hedge-fund trading platform for its low-latency (microseconds) event-driven order-submission engine.

if it's good enough for a hedge fund HFT platform, it's good enough for anyone aiming for low latency

https://medium.com/hybrid-cloud-engineering/beam-otp-on-ocp-...

It's used for monitoring, not the actual trading. The Erlang scheduler is fantastic, but it targets a context switch roughly every 1ms, it's obviously not suitable for HFT where a process must respond to a network event within microseconds.

https://hackernoon.com/successful-companies-use-erlang-and-e...

> Erlang is used as part of the real-time monitoring solution for this distributed trading system.

http://zerohedge.blogspot.com/2009/07/is-case-of-quant-tradi...

> Implemented a real-time monitoring solution for the distributed trading system using a combination of technologies (SNMP, Erlang/OTP, boost, ACE, TibcoRV, real-time distributed replicated database, etc) to monitor load and health of trading processes in the mother-ship and co-located sites so that trading decisions can be prioritized based on congestion and queuing delays.

This is almost certainly incorrect, unless it’s using a very stretched definition of “hft” and using microseconds to mean 99th% <1ms instead of say <5us (reasonably competitive software range depending on your trade).

They might be forwarding non latency sensitive client/internal orders through a system written in erlang, or using erlang for an orchestration layer, but they’re not competing with other hft traders in one.

Source: I work in the field and have built <5us systems.

https://m.youtube.com/watch?v=NH1Tta7purM Is a free great watch about the things you have to do to very reliably have such low latencies

Was it HFT or just processing large client orders? I'm having a hard time finding this out
It could be the C++ for executing trades and BEAM for the client orders. Obviously you'd want something like C++ there when you're racing. Though if they did the trading as a C++ NIF it could all still run on the BEAM avoiding a memory copy. Interesting thought exercise anyway.
It's hard to find details though.
Erlang, which Elixir compiles to, is literally built for low latency. It prioritizes latency while almost every other language prioritizes throughput.
Erlang is built for easily achieving fairly low and reliable latencies, with easy distribution/concurrency primitives, and building highly reliable systems.

It’s not very good for “make packet in to packet out as fast as possible at the cost of all else”. All of the abstraction layers come at a performance cost and aren’t all useful in the first place for low latency trading systems.

  • lawik
  • ·
  • 7 months ago
  • ·
  • [ - ]
Yeah, I think I adressed this in the article and I hope it came across. If latency is the one singular goal the design should be quite different.
Is it fairer to say that Elixir compiles to Erlang or that Erlang & Elixir compile to Beam byte code?

Genuine question.

  • di4na
  • ·
  • 7 months ago
  • ·
  • [ - ]
Neither but the former is closer to it.
"Elixir compiles into BEAM byte code (via Erlang Abstract Format). This means that Elixir code can be called from Erlang and vice versa, without the need to write any bindings. All Elixir modules start with the Elixir. prefix followed by the regular Elixir name."

From https://elixir-lang.org/crash-course.html "Erlang/Elixir Syntax: A Crash Course"

  • di4na
  • ·
  • 7 months ago
  • ·
  • [ - ]
Yes, the Erlang Abstract Format is at least half a dozen IR above the bytecode, and lowering this is done by the erlang compiler.
Can't edit my own thread, my point was that Erlang has a runtime and GC which is the opposite of low latency, what Erlang/Beam good at is predictability ( latency wise ).

If you trully want low latency you need to use C/C++.

Would you mind elaborating in more detail why.
I guess this is similar to me going to a wine store. I am readily persuaded that a good 20 euro wine is actually much more enjoyable for me than a 5 euro wine. But I get no additional enjoyment from 500 euro wine over that 20 euro wine.

So it depends where you are looking from.

If you are a ruby shop and your regular out of the box latency is in the hundreds of milliseconds which under load falls apart into at first some and then all taking seconds or tens of seconds then the BEAM is fantastic, you go down to single-double digit responses and under load the latencies start to grow but very gradually. Every request gets more or less same latency.

When you come from microsecond latencies then it looks like the BEAM is doing a lot but is mostly in your way in achieving the lowest latency possible.

Completely different problem sets which gets confused by the writer not giving the necessary context for their claim.

In the Nerves/IoT/HW space one pattern used is to use Elixir for all the soft-realtime stuff and enjoy the QoL and other benefits of the BEAM and use other tools more suitable for hard-realtime tasks.

Is BEAM the 5, 20 or 500 euro wine?
The point (tho not the best analogue for it) is that very few have the need (and the means) to go for the extreme. But for some who need it, the reasonable option for most is not going to cut it.

Hence I would guess 20 from the selection.

[flagged]
low is relative. I don't think anyone would use elixir for systems with HARD realtime needs like stepper motors, industrial automation or other such projects. but there's also low in terms of perceptual latency. I use elixir at my startup and the baseline performance is amazing for saas applications. 300ms of network latency is slow by hard realtime standards but is more than sufficient for a user navigating a app or website. being able to achieve that at scale is an absolute superpower that would be significantly more work (and machine resources) to do in ruby or python.
  • lawik
  • ·
  • 7 months ago
  • ·
  • [ - ]
I think I address what it aims for quite thoroughly in the text itself.