The Mojo is 68,000 times faster than Python type blogs are awesome, but can awesome comparisons be made with other languages too?

[nevdelap]

Mojo being 35,000 times faster than Python, 68,000 times faster than Python… it’s impressive, amazing, and cool, but to non-Python people and anti-Python who haven’t yet paid attention to Mojo yet its might not be so impressive. I’m a Python guy, have been for twenty years, but I work with a lot of Go people, and some very anti-Python people, and if they saw these comparisons their response would, I don’t doubt, be a sneering “sure, because Python is slow”.

Of course most people who say Python is slow don’t really know what they are talking about - not that it isn’t slow, but you have to know what that slowness means in real life to be able to make real judgements, and they are the same people who don’t know that developing in Python is very, very fast, that Python is extremely feature rich, and that its ecosystem is so enormous, and that it’s slowness isn’t even an issue in most use cases. They don't know that fast compiling/running Go is feature poor and easy but more work to do the same thing, that slow compiling/fast running Rust is feature rich but hard, that those languages and fast compiling/running C, as good as they are where they are the right choice, are inaccessible to a vast proportion of developers. And so speed comparisons on their own are missing all of those criteria.

So where am I going with this?

I assume, and please someone correct me if I’m assuming wrong, that in similar comparisons to the 35,000x and 68,000x faster blogs that Mojo would be significantly faster than C, Go, Rust, etc, because a great deal of the speed up is because of Mojo supporting all the amazing SIMD, vectorization, and parallelization, etc magic, and on top of the speed what makes it so impressive is that it supports these things ergonomically in ways that if it were done in those other high performance compiled languages would be very hard, if it's even possible.

I want to suggest that blog posts saying something like Mojo is N (impressive number) times faster than Go (or C or Rust) would really show the power of Mojo in ways that would make people outside the Mojo/Python community sit up and pay attention, and suggest that people who know those languages well enough to write such blogs, who are Mojo fans, might consider it.

I guess most important is to ask: Are my assumptions in the "I assume" paragraph wrong? Is what I suggest a possible reality?

[Mogball]

I think you raise a really good point: that many programmers who already don't use Python because "it is slow" won't be convinced by the headline number because it's not obvious how much faster Mojo is than other languages. I think this deserves a couple more blog posts! WDYT @abduld @shashank135sharma

[rd4com]

yes you've got a point, also the performance gained by simd is not the only benefit, there is the productivity factor that could be measured by comparing how many lines of codes it takes to program performant code in mojo vs other languages.

the challenge here is to demonstrate those exciting features without alienating other programming languages. mojo seem to reconcile performance and productivity. I think it is important to pay tribute to what benefits other languages have brought to the world in doing so!

[abduld]

There is a plan to play the benchmark game (https://benchmarksgame-team.pages.debian.net/benchmarksgame/index.html) and show that mojo is faster than most other languges out there. That should allow folks to compare mojo against go, c++, etc. and see performance for themselves.

[gryznar]

+1

[rodrigozayit]

any updates? 🙏

[MikeCase]

I wrote a simple Fibonacci algorithm in both python and mojo and the time difference is amazing to me. It's on my GitHub, both the python implementation and the mojo implementation. I'm no expert in any language so I'm sure it can be optimized better but you can certainly tell the difference.

Just my two cents.

edit
Oops... I haven't updated the repo to my latest work. You will need to comment out the argv stuff and replace the num constant in the do_fib function to an integer to get this working currently. I will update the repo when I get home this evening.

edit 2
I updated the repo on github, it should run if not let me know and I'll take another look at it. I'm at work right now and can't access my stuff at home to update the repo.

[Brian-M-J]

I've recently been thinking about topics like this. Many people talk about dev speed vs execution speed when comparing languages. I think they don't give much importance to what I call "practical execution speed" which is the ratio of execution speed to dev speed.

C++ and Python have great execution speed and dev speed respectively, but both have terrible practical execution speed. You need to spend a very long time studying the intricacies of those languages to maximize your code's performance.

I'm sure C/C++, Rust, Zig etc can theoretically run at the same speed as fully optimized Mojo. The question is, how much do you need to know about the language to reach this speed limit? Are there footguns that can make you unintentionally reduce the performance of your code by an order of magnitude? How complicated is it to take advantage of SIMD, multiple cores and accelerators, all of which can speed up your code by quite a bit?

[harshalgunjal69]

So, what is better in the end, Mojo or C/C++ ?

[ksandvik]

The days of figuring out perfect optimization by hand-tuning assembly code are over by a decade. I put my hope on smarter compilers that both optimize code and warn about performance bottlenecks. Mojo already has features like parallelize and autotuning that show the way smart compilers are heading. But it's good to know the basics of memory models, what compile time versus runtime means, and so on.

[jackos]

Internally the engineers do hand-write assembly to determine the maximum theoretical performance of different algorithms, and use it as a baseline to make sure Mojo is performing where it should.

[Brian-M-J]

Optimizing compilers are great, but they can't take care of everything. As recently as February this year, people talk about programming practices that slow your code. An example is this video: "Clean" Code, Horrible Performance

[soraros]

@Brian-M-J That's why the mojo compiler doesn't try to be too "clever" and the language made things like unroll and always_inline available to users.

Regarding the clean code video, you might find these discussions useful: reddit, HN

[PallHaraldsson]

I want to suggest that blog posts saying something like Mojo is N,000 times faster than Go (or C or Rust)

That's just not possible. Complied languages (with good type support), like Mojo, C, C++, Zig, and Julia are within 2x of each other. For at least single-threaded.

Some of the claims for Mojo are for multi-threaded (vs. single-threaded Python code), and then you can get almost arbitrary-high speedup. SIMD may be off by default in some (C/C++) compilers, is on by default in Julia. You need to opt into threads (I think Mojo too). Mojo has TCO I believe by default (it's not needed for performance, for real-world code, some benchmarks though, and Julia has a TCO macro available so opt-in). Julia has been benchmarked as comparable to Mojo, and is as easy to write as Python, and like Mojo (and like Mojo both have excellent support for GPUs), can reuse all of Python libraies through PythonCall.jl. It has by now over 10,000 packages, plus those of Python. You can also call Julia from Python (see that package), and many Julia packages already have a wrapper for Python.

[MoSafi2]

I understand how performance-wise, there may not but drastic differences between Julia, C++, Mojo (no 5x or 10x at least). However, the language environoment matters. My expereience with Julia that it is a language that lacks clear philosphoy unlike more principled languages as Python or even R. That's why it feels in use like bits and pieces from everywhere with no clear principled and coherent Julia way. In addition, lacking backing from big corporation, alot of common packages seems amateurish, trying to implement the largest amount of features but have problematic implementation. I wish Julia would have been better, it is very useful in scientific computing.

[manatlan]

Good point !
Here is a repo https://github.com/manatlan/sudoku_resolver ... where i try to implement a "sudoku solver", in different languages, to compare "run times".
The original one is the python version (13 lines of code) : it's a naive recursive algorithm of backtracing solver.
Yesterday, I've added a mojo version ... which takes 16s, whereas the py3.11 version took 1 minute (and the nim version is 8s)
I want to keep the sames logics between the differents versions.
But I'm pretty sure that the mojo version is weird (because it's not as fast as the nim version), and could be a lot better, using StringRef in place of String ... But I don't know how to do
I'd like to make it better, while preserving the simple logic/algorithm, with minimal lines of code.
COuld you help me to make it better ?

edit : finally, i've reached to make a simple mojo version which is as fast as the nim version ... happy with that !

[ziofil]

But then why is it "only" 6-7x faster than Python?

[manatlan]

For sure, it's far away the 35000x, 68000x which we read anywhere ;-)

In this repo, I want to compare the same algo, with same kind of types ... (grids are in "strings")
So "sudoku.mojo" is a 1to1 translation from "sudoku.py".
And in that case, it's only 6-7x faster.

If I specialize the types in the mojo version (sudoku_specialized.mojo), it's 20x faster.
(but pretty sure it could be better, but I'm not a mojo specialist yet)

[ksandvik]

Couple of ideas, buffer up print statements, also look at compilation time variables (see documentation).

[u3Izx9ql7vW4]

similar comparisons to the 35,000x and 68,000x faster blogs that Mojo would be significantly faster than C, Go, Rust,
I guess most important is to ask: Are my assumptions in the "I assume" paragraph wrong? Is what I suggest a possible reality?

Late to the party but I believe the assumptions are wrong. Most of the benchmarks I've seen that compare Mojo to C, Go, Rust, etc. should characterize Mojo as N times slower as opposed to faster.

I suspect the reason we're not seeing Mojo in the https://benchmarksgame-team.pages.debian.net/benchmarksgame/index.html after a year of the other commenter's post is because Mojo stacks up rather poorly, as of 2024.

As others have mentioned, Mojo meant to be a faster Python with built-in SIMD goodness. But being faster than Python is a pretty low bar to clear. If you compare Mojo to Python + Numpy/Numba, then the difference is within an order of magnitude. A user on Julia's forum benchmarked Mojo with Python+Numba and Mojo is slower:

https://discourse.julialang.org/t/why-mojo-can-be-so-fast/98458

[MoSafi2]

@u3Izx9ql7vW4 I am not sure that's true. did you check this for example? it is clear benchmarking results against vendor tools
https://www.modular.com/blog/the-worlds-fastest-unified-matrix-multiplication

Here is also a bunch of claims about the inference performance for a bunch of models
https://www.modular.com/max/performance
As both MAX and the models are available for download you can certainly go on and disprove those claims.

And TBH a poorly executed benchmark is still not informative regarldess of who is doing it.

[soraros]

The Sudoku solver utilizes Python for text input, essentially benchmarking the FFI with Python at that point. The prime test employs unbuffered I/O, which is a known issue #975; this remains unaddressed because it seems no one deems it urgent enough to implement a fix. If you eliminate the I/O and simply test the core loop or redirect the output to a list, the performance matches that of C++. Certainly, one could argue that text printing is a fundamental feature and should not perform poorly. I agree. The standard library is open source, so if you find its shortcomings intolerable, you can simply contribute improvements using Mojo itself (unlike in Python or Go).

Sure I don't care to speculate more on this, but for a company that frequently posts questionable benchmarks like this and this with outlandish headlines I find this hard to believe.

Have you read the blog in total? The author made the first version of the rebuttal post without even running the Mojo code. The author made a interesting point:

If it's really due to Mojo's groundbreaking compiler advances, why is my Julia implementation relatively faster compared to Needletail?

People seems to have this impression that Modular is saying they used compiler magic to make Python suddenly run fast. It's not the case, and actually, the exact opposite (Chris said himself that Mojo is about removing compiler magic from the language). Mojo is a system programming language, and code structured like reasonable C++/Rust (type-stable, non-error-raising, not overly dynamic, etc.) can be optimized, regardless of the language. This is well-known and further evidenced by tools like Numba and Julia.

Yep, I suppose it's only fair that Mojo should to be fast on its own terms.

On my laptop, fib(43) takes 1.543173 seconds to execute in Julia, and 0.87876580201555043 seconds in Mojo running within a Docker container. Does this prove Mojo is faster or scales better than Julia? Hardly. It's simply not a robust benchmark.

Your expectation is as likely to be wrong as mine if you haven't done the benchmarking yourself.

There's little speculation since I use Mojo (alongside Julia and Python's entire scientific suite) daily and contribute to its standard library, so I would think that I have a good sense of what to expect. The notion that modular programming makes Python code run as fast as C++ might sound far-fetched, but it isn’t entirely without merit.

They asked for a bunch of personal information in order to use it

Not anymore! They recently introduced a package management tool called magic (based on pixi), which fortunately doesn’t require that.

[u3Izx9ql7vW4]

@MoSafi2 No I've not seen it, thanks for the link. It seems like up until July of this year, Mojo's matrix multiplication was 4-5x slower than Numpy's (#2660) until a community member fixed it. (un)Surprisingly the blog titled the world's fastest matrix multiplication from Modular from last year didn't include Numpy as a comparison.

I don't put much weight on Modular's benchmarks for reasons I'm not going to repeat. I'll believe it if I see Mojo's performance on https://benchmarksgame-team.pages.debian.net/benchmarksgame/index.html.

@soraros I'm aware the author didn't bother to run Mojo's code and I don't blame them, it's not relevant to their point -- the original Mojo implementation cuts corners and it's not an apples to apples comparison, as is a theme of many Modular benchmarks.

Look I don't hang my hat on Mojo being fast or slow. It's a red flag when a library claims what something is capable of and when it doesn't measure up, they point you to a bunch of open tickets, saying yeah we know about this, and that, and so on. Who cares? I'm not your manager, I'm not looking for explanations.

Not anymore! They recently introduced a package management tool called magic (based on pixi), which fortunately doesn’t require that.

Sure, I'll give it a try when I have a chance.

Anyway, this is my last post. You guys have the last word

[MoSafi2]

@soraros I'm aware the author didn't bother to run Mojo's code and I don't blame them, it's not relevant to their point -- the original Mojo implementation cuts corners and it's not an apples to apples comparison, as is a theme of many Modular benchmarks.

Hey author of the original post of MojoFastTrim here, it is kinda intersting when the person benchmarking the code does not run the code, uses a wrong version of it, and ignores clarification of the benchmarking methdology, that does not seem to be reliable to me.
the original implementation of MojoFastTrim was following the rust implementation and they would pass the same test suite. you can also check the extensive test suite on the main repo and an additional implementation on FASTQ parser with progressive compile-time validation guarantees.

Here is an indendent datapoint for the same claim
https://news.ycombinator.com/item?id=39325821
You can also check the original implmentation (Though you will need Mojo 0.6 to run that)
https://github.com/MoSafi2/BlazeSeq/tree/MojoFastTrim
and Rust's implementation
https://github.com/onecodex/needletail

[MoSafi2]

@u3Izx9ql7vW4 please read the comments more carefuly, that was my response

The provided exmaples in the mojo repo is a toy example with what kinds of optimization possible in the language itself with zero-dependancies on 3rd party libraries. the performance attained by BLAS is possible to achieve and surpass in mojo (currently implemented in the closed-source max engine) and maybe it can be implemented in open source with more people intrested in the language.

[manatlan]

@soraros ...

The Sudoku solver utilizes Python for text input, essentially

benchmarking the FFI with Python at that point Yes, at this time, it was not possible to do it in pure mojo ... So I did it with python. Nowadays, is it possible to do that in pure mojo ? I'd like to remove that. If someone has an example, I take ;-) Le jeu. 29 août 2024, 23:02, soraros ***@***.***> a écrit :

…

The Sudoku solver utilizes Python for text input, essentially benchmarking the FFI with Python at that point. The prime test employs unbuffered I/O, which is a known issue #975 <#975>; this remains unaddressed because it seems no one deems it urgent enough to implement a fix. If you eliminate the I/O and simply test the core loop or redirect the output to a list, the performance matches that of C++. Certainly, one could argue that text printing is a fundamental feature and should not perform poorly. I agree. The standard library is open source, so if you find its shortcomings intolerable, you can simply contribute improvements using Mojo itself (unlike in Python or Go). Sure I don't care to speculate more on this, but for a company that frequently posts questionable benchmarks like this <https://stackoverflow.com/questions/77070883/performance-comparison-mojo-vs-python> and this <https://www.reddit.com/r/rust/comments/1al8cuc/comment/kpdsgzz/> with outlandish headlines I find this hard to believe. Have you read the blog in total? The author made the first version of the rebuttal post without even running the Mojo code. The author made a interesting point: If it's really due to Mojo's groundbreaking compiler advances, why is my Julia implementation relatively faster compared to Needletail? People seems to have this impression that Modular is saying they used compiler magic to make Python suddenly run fast. It's not the case, and actually, the exact opposite (Chris said himself that Mojo is about removing compiler magic from the language). Mojo is a system programming language, and code structured like reasonable C++/Rust (type-stable, non-error-raising, not overly dynamic, etc.) can be optimized, regardless of the language. This is well-known and further evidenced by tools like Numba and Julia. Yep, I suppose it's only fair that Mojo should to be fast on its own terms. On my laptop, fib(43) takes 1.543173 seconds to execute in Julia, and 0.87876580201555043 seconds in Mojo running within a Docker container. Does this prove Mojo is faster or scales better than Julia? Hardly. It's simply not a robust benchmark. Your expectation is as likely to be wrong as mine if you haven't done the benchmarking yourself. There's little speculation since I use Mojo (alongside Julia and Python's entire scientific suite) daily and contribute to its standard library, so I would think that I have a good sense of what to expect. The notion that modular programming makes Python code run as fast as C++ might sound far-fetched, but it isn’t entirely without merit. They asked for a bunch of personal information in order to use it Not anymore! They recently introduced a package management tool called magic (based on pixi), which fortunately doesn’t require that. — Reply to this email directly, view it on GitHub <#843 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABEZQQJECPBQTJAARNJDJ3ZT6D5LAVCNFSM6AAAAAA45U2NMKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANBZGE4DCMY> . You are receiving this because you commented.Message ID: ***@***.***>

[manatlan]

Just tested now ... (Nowadays, it is still not possible to read from stdin in a pure mojo way.) BTW, I removed the use of ffi/python (by reading from the file in a mojo way), on my celeron N100 : it goes from 89s to 77s : it's still far from rust, codon, or mojo0.4 ... But no problems, no flames ! Le ven. 30 août 2024 à 08:36, ***@***.*** ***@***.***> a écrit :

…

@soraros ... > The Sudoku solver utilizes Python for text input, essentially benchmarking the FFI with Python at that point Yes, at this time, it was not possible to do it in pure mojo ... So I did it with python. Nowadays, is it possible to do that in pure mojo ? I'd like to remove that. If someone has an example, I take ;-) Le jeu. 29 août 2024, 23:02, soraros ***@***.***> a écrit : > The Sudoku solver utilizes Python for text input, essentially > benchmarking the FFI with Python at that point. The prime test employs > unbuffered I/O, which is a known issue #975 > <#975>; this remains unaddressed > because it seems no one deems it urgent enough to implement a fix. If you > eliminate the I/O and simply test the core loop or redirect the output to a > list, the performance matches that of C++. Certainly, one could argue that > text printing is a fundamental feature and should not perform poorly. I > agree. The standard library is open source, so if you find its shortcomings > intolerable, you can simply contribute improvements using Mojo itself > (unlike in Python or Go). > > Sure I don't care to speculate more on this, but for a company that > frequently posts questionable benchmarks like this > <https://stackoverflow.com/questions/77070883/performance-comparison-mojo-vs-python> > and this > <https://www.reddit.com/r/rust/comments/1al8cuc/comment/kpdsgzz/> with > outlandish headlines I find this hard to believe. > > Have you read the blog in total? The author made the first version of the > rebuttal post without even running the Mojo code. The author made a > interesting point: > > If it's really due to Mojo's groundbreaking compiler advances, why is my > Julia implementation relatively faster compared to Needletail? > > People seems to have this impression that Modular is saying they used > compiler magic to make Python suddenly run fast. It's not the case, and > actually, the exact opposite (Chris said himself that Mojo is about > removing compiler magic from the language). Mojo is a system programming > language, and code structured like reasonable C++/Rust (type-stable, > non-error-raising, not overly dynamic, etc.) can be optimized, regardless > of the language. This is well-known and further evidenced by tools like > Numba and Julia. > > Yep, I suppose it's only fair that Mojo should to be fast on its own > terms. > > On my laptop, fib(43) takes 1.543173 seconds to execute in Julia, and > 0.87876580201555043 seconds in Mojo running within a Docker container. Does > this prove Mojo is faster or scales better than Julia? Hardly. It's simply > not a robust benchmark. > > Your expectation is as likely to be wrong as mine if you haven't done the > benchmarking yourself. > > There's little speculation since I use Mojo (alongside Julia and Python's > entire scientific suite) daily and contribute to its standard library, so I > would think that I have a good sense of what to expect. The notion that > modular programming makes Python code run as fast as C++ might sound > far-fetched, but it isn’t entirely without merit. > > They asked for a bunch of personal information in order to use it > > Not anymore! They recently introduced a package management tool called > magic (based on pixi), which fortunately doesn’t require that. > > — > Reply to this email directly, view it on GitHub > <#843 (reply in thread)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AABEZQQJECPBQTJAARNJDJ3ZT6D5LAVCNFSM6AAAAAA45U2NMKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANBZGE4DCMY> > . > You are receiving this because you commented.Message ID: > ***@***.***> >

[soraros]

Yea, I also tried it myself. I was wrong, the performance was just horrible.