“Any sufficiently complicated concurrent program in another language contains an ad hoc informally-specified bug-ridden slow implementation of half of
Erlang.” – Virding’s first rule of programming
Looks like Flawless goal is to persist the intermediate state of some workflow and restart it from midpoint in case of a failure. That's quite different from Erlang, which was mostly aimed at developing software while having access only to prototype versions of hardware with multiple hardware issues. To me this approaches are opposite: Flawless will be stuck in a loop trying to finish a workflow that crashes in the middle, while Erlang will happily discard 50% of the traffic that somehow hits the hardware bug.
I like to see this on a spectrum on 'reliable computation' and 'high availability' (and interruptible computing) with continuations/CallCC, checkpoint/restore (CRIU, DMTCP), VM snapshots or (live-)migration, to double/triple hardware, and even dual implementation, but also Kafka.
I find it a bit disheartening that these features are not more pervasive on language, runtimes, libraries and OSes, so very often the actual solutions are either very kludgy or very high maintenance, whereas the primitives are really interesting for many other endeavours.
Functional programmers think this way though. Everything is data, or a manipulation/transform of data, and we can wrap functions with higher level things to track inputs and outputs and persist that.
Erlang won’t do this out of the box but it’s a menial addition to your system.
I guess I just don’t understand how this is such a novel idea.
Not even the web assembly part is novel - Temporal does that for several languages.
This seems to be a new implementation of an existing idea, and may end up being cleaner - that part is to be seen since there’s no actual product or code visible yet.
Yeah. Actors and the BEAM are just the runtime. On top of that you build your system/domain logic and part of that is persisting state at regular intervals and being able to restore that state from any kind of failure. You build a durable app, or you don’t.
“It’s commonly known under the name durable execution, and is so new that most developers have never heard of it”
Serious? I feel like this post is very naive and perhaps disingenuous in thinking that Erlang/Elixir developers are not accounting for this. I’ve been building apps this way for a long time, regardless of language.
Rewriting everything in rust doesn’t solve any of these issues. State is state.
This tool is literally dagster or prefect but it’s claiming to be a revolutionary new idea.
This appears to be a novel way to represent how the application interacts with state though, an API plus runtime.
I've used workflow engines before, which provide similar capabilities: execute a distributed process to completion in the presence of failures. However, you have to be really careful when building workflow applications not to introduce accidental side-effects (that are not modeled in the workflow directly), or else the result will be nondeterministic (unspecified at best).
I haven't used Dagster or Prefect but it looks like this tool uses a different approach. An application developer doesn't need to "model" their workflow using this tool - just implement it on top of the API that Flawless provides. It reminds me a bit of the AWS Flow framework [1] for AWS SimpleWorkflow, but when developing Flow applications (which are Java apps) you still have to be extremely careful not to accidentally introduce local side effects or nondeterminism.
Because this approach provides a deterministic runtime, I see it being plausible that developers could be significantly more confident that their code is, in fact, deterministic.
> but it’s claiming to be a revolutionary new idea.
I haven't touched erlang in ages, but Joe Armstrong popped up in my youtube recommendations recently. After watching the video, I just thought about microservices and IaC for a while and thought "Ha, we're really fucking this up, aren't we?"
It wasn't a video pitching the language and certainly not drawing any explicit parallels to modern practices. Just something that got me thinking about the distributed concurrency system of erlang for the first time in a decade.
This seems... not the same as Erlang at all? Erlang solves the problem of persistent state by essentially eliminating it: more or less all state exists in message queues, or in an external database or something. Flawless seems to solve that problem with a technique similar to, but not quite the same as, filesystem journaling: take note of your side effects when performing them. In the case of filesystem journaling it's so that you can redo them if you crash, but in this case it's so that you don't need to.
It's not entirely clear to me how well this would work in which domains, but the overlap with domains where Erlang works well seems less than total.
While it might have tons of bugs, this one seems better; programming stuff as you normally would and have it distributed and durable automagically. At least that's what I got from it. And that's nice.
This is how people often describe Erlang and Elixir. But I guess it's fair to say that functional programming is not "programming as you usually would" for most!
"This is how people often describe Erlang and Elixir."
It is. I recall it fooling me when I first got into Erlang. But it's wrong. Erlang has some tools that help lead you in a more robust direction, but you still have to work to use them. They are not automatic and it's trivial to write an Erlang service that has a single point of failure, even accidentally.
Now, I don't want to criticize the tools it has and the fact that it does rather strongly lead you in a robust direction more than many systems. This is somewhere between "an easy mistake to make for a newbie" and "some sloppy advocacy sometimes by some people", 100% a people thing, not a criticism of the code base or technicals at all. But it is important for people considering Erlang/Elixir or early in the process to understand that it does not simply automatically make all your code run robustly in a cluster. It is only a strong push in the right direction and you still must understand it enough to make sure you don't break it.
Well, with Erlang you really have to plan these things and implement those. Functionality (pure/immutable more-so) helps, but it doesn't pick up everything as in, it's not drop-in; it's very possible, but you have to change habits, even if you were a functional programmer before.
Rust isn't exactly the normal style either unless you were used to some subset of modern c++ that probably only gets used in gaming or embedded.
While I wouldn't normally argue for rigor or anything tedious, creating computationally expensive abstraction layers to let you develop capabilities faster without having to worry about defects or edge cases seems the opposite of what rust is designed for. I think the language will fight you the whole way. The only aid I can think of is that you can wildly unwrap stuff everywhere not worrying about panics. Which does save some time.
> Rust isn't exactly the normal style either unless you were used to some subset of modern c++ that probably only gets used in gaming or embedded.
But, as I understand it, it works with 'normal Rust', which, if you enjoy Rust, would be normal right? It seems to compile the rust to wasm, run it in wasm, log the results while keeping pace of how far it came, and then rerun it with previous results. That's not quite nice to have, but, like you say I think, it has foot guns, and, one of the worst, is getting too comfortable thinking it has your back while this is not enough; side effects behind immutable variables are there and while this helps you a bit, it doesn't fix any of that and maybe makes it worse.
I am just a little annoyed this is not the norm but rather something special (and closed source?), why is it not everywhere? Probably because high amounts of magic?
