Just want to mention that these standalone python builds have been super important for simplifying a lot of dev tooling, even if users may not see it directly — I work for a cloud infrastructure company, and they’re what allow us to give users a one-line way of adding Python to any Docker image they desire. It’s helpful to have reproducible, standalone Python builds of this quality (and release transparency). Thanks y’all for taking this on.
> Normally, when you build CPython on Linux or macOS, several system paths are hardcoded into the binary. This is fine if you're building and installing Python on a single machine, but it's a problem if you want to pre-build Python and then distribute it to other machines.
> So, for example, when you download Python on Linux (e.g., from python.org), what you're actually downloading is the CPython source, which is then built on your machine.
But python.org does provide prebuilt macOS binaries. How is that accomplished and why doesn't whatever they're doing generalize to Linux?
A big holdup seems to be who would maintain this in the CPython project. Perhaps some Astral folks could become core devs as well and maintain it upstream
Hardly anyone who does real work would want to contribute to CPython these days. The reputational risk is too high, the work would be taken, modified and slowly ruined by the mediocre influencers.
I would advise against Astral for maintaining anything inside the Python organization. Too much talk, power plays and no real software engineering.
The parent comment is inflammatory. That said, moving anything in the Python project requires a lot of energy, there is high friction and it's probably not wise to try to do that until something has established itself outside.
The discussions around lockfiles, dynamic metadata or PyBI (the PEP that wanted to address what python-build-standalone does) are good examples of how hard it is to cause change in that space.
I could never justify the time investment to upstream a lot of my python-build-standalone work. I made some attempts. But it always felt like I was swimming against a heavy current and the talk to meaningful action ratio was too high. The payoff would be there. But it was the kind of work someone would have to pay me to do: not how I would choose to spend my free time on nights and weekends.
I’m optimistic the Astral folks will have better success than me and I support them in their efforts. They have viable, popular solutions in hand. Hopefully that helps convert others to their cause. “If you build it they will come.”
The macOS builds in Python.org are not easy to use for tools like uv and rye. They have hardcoded paths and can only be installed to the one location on the file system. They are also framework builds which is untypical for pythons.
A framework build is a specific build of Python to make it work like a .framework on macOS. The original motivation of this was that you can also ship frameworks within .app bundles but the Python framework build has hardcoded paths so you cannot really do it.
One of the consequences of framework builds is that they have a different layout than a regular Python installation on the file system. The Python installer will also litter a bunch of files into /Applications which makes installing competing versions surprisingly annoying.
In theory a framework build of Python would be preferrable but the framework build would have to become fully relocatable for that benefit to pay off. They are not today.
This is a huge relief. I love the uv bootstrapping, but was unhappy with the built-in-Nebraska feel for how they sourced Python. Still not as great as being an official Python.org project, but an excellent step.
That was my thought as well. "Hopefully someone decides to take on the stewardship of PyOxidizer, too." That might not dovetail well enough with someone's business plans though.
A big thank you to Gregory and the Astral team. The binaries saved me tons of bullshit build time. The saved CPU cycles also helped reducing carbon footprint!
> With those modifications, it then builds Python from source across a wide matrix of Python versions, platforms, and build variants (e.g., optimized vs. debug builds), and publishes the built distributions to GitHub Releases.
Just want to mention that these standalone python builds have been super important for simplifying a lot of dev tooling, even if users may not see it directly — I work for a cloud infrastructure company, and they’re what allow us to give users a one-line way of adding Python to any Docker image they desire. It’s helpful to have reproducible, standalone Python builds of this quality (and release transparency). Thanks y’all for taking this on.
Oh you maintain PyTorch right?
I think GP is https://www.ekzhang.com/ (hi Eric!), not http://ezyang.com/
> Normally, when you build CPython on Linux or macOS, several system paths are hardcoded into the binary. This is fine if you're building and installing Python on a single machine, but it's a problem if you want to pre-build Python and then distribute it to other machines.
> So, for example, when you download Python on Linux (e.g., from python.org), what you're actually downloading is the CPython source, which is then built on your machine.
But python.org does provide prebuilt macOS binaries. How is that accomplished and why doesn't whatever they're doing generalize to Linux?
This is actively being discussed https://discuss.python.org/t/publish-linux-installer-on-pyth...
A big holdup seems to be who would maintain this in the CPython project. Perhaps some Astral folks could become core devs as well and maintain it upstream
Hardly anyone who does real work would want to contribute to CPython these days. The reputational risk is too high, the work would be taken, modified and slowly ruined by the mediocre influencers.
I would advise against Astral for maintaining anything inside the Python organization. Too much talk, power plays and no real software engineering.
?
Source needed.
Pun unintended.
The parent comment is inflammatory. That said, moving anything in the Python project requires a lot of energy, there is high friction and it's probably not wise to try to do that until something has established itself outside.
The discussions around lockfiles, dynamic metadata or PyBI (the PEP that wanted to address what python-build-standalone does) are good examples of how hard it is to cause change in that space.
I could never justify the time investment to upstream a lot of my python-build-standalone work. I made some attempts. But it always felt like I was swimming against a heavy current and the talk to meaningful action ratio was too high. The payoff would be there. But it was the kind of work someone would have to pay me to do: not how I would choose to spend my free time on nights and weekends.
I’m optimistic the Astral folks will have better success than me and I support them in their efforts. They have viable, popular solutions in hand. Hopefully that helps convert others to their cause. “If you build it they will come.”
The macOS builds in Python.org are not easy to use for tools like uv and rye. They have hardcoded paths and can only be installed to the one location on the file system. They are also framework builds which is untypical for pythons.
What implies being framework builds here (-> what's the difference)?
A framework build is a specific build of Python to make it work like a .framework on macOS. The original motivation of this was that you can also ship frameworks within .app bundles but the Python framework build has hardcoded paths so you cannot really do it.
One of the consequences of framework builds is that they have a different layout than a regular Python installation on the file system. The Python installer will also litter a bunch of files into /Applications which makes installing competing versions surprisingly annoying.
In theory a framework build of Python would be preferrable but the framework build would have to become fully relocatable for that benefit to pay off. They are not today.
And this works in macOS but not Linux because the required filesystem paths are always the same on macOS but not on Linux?
This is a huge relief. I love the uv bootstrapping, but was unhappy with the built-in-Nebraska feel for how they sourced Python. Still not as great as being an official Python.org project, but an excellent step.
Love the way you made that Nebraska reference!
For people as puzzled as me by the Nebraska reference, I think it's referring to this well known XKCD post:
https://xkcd.com/2347/
I think the next project adopted by Astral will be PyOxidizer.
That was my thought as well. "Hopefully someone decides to take on the stewardship of PyOxidizer, too." That might not dovetail well enough with someone's business plans though.
at this point they should just start their own Python implementation :-)
I really hope so
I hope this is also relevant to extending uv towards creating standalone python executables
Amazing, I'm really hoping Astral will stay in the game for long!
A big thank you to Gregory and the Astral team. The binaries saved me tons of bullshit build time. The saved CPU cycles also helped reducing carbon footprint!
> With those modifications, it then builds Python from source across a wide matrix of Python versions, platforms, and build variants (e.g., optimized vs. debug builds), and publishes the built distributions to GitHub Releases.
This should be illegal.
Why?
Supply chain risk.
Please explain your reasoning.
Somebody else is building your binaries. You've added another link in your software supply chain. How do you know they haven't inserted malware?
> Somebody else is building your binaries.
That happens all the time. Who builds the docker images you are using?
> You've added another link in your software supply chain. How do you know they haven't inserted malware?
You're installing untrusted random packages from PyPI. There are many much weaker points than Astral giving you malware for fun.
Sure it happens, but that doesn't mean you shouldn't think about reducing it.
How to handle this situation is literally defined in the LICENSE for any modern software project