back to article Fed up with Python setup and packaging? Try a shot of Rye

The recent debut of a Python tool called Rye has raised hopes that the programming language's long standing setup and packaging shortcomings may be solvable. Rye, a personal project developed by Armin Ronacher, an open source developer and creator of the Python web framework Flask, addresses two issues: bootstrapping (setup) …

  1. A Non e-mouse Silver badge

    LWN has a series of articles on work on packing in Python.

  2. David Harper 1

    No mention of pip and venv?

    How can you write an entire article about Python packaging and fail to mention venv and pip? Both are simple and easy to use, and -- in my experience, at least -- they work well on Linux and macOS.

    1. Tom 7

      Re: No mention of pip and venv?

      The work but they're not really a solution or simple, just a sticking plaster. Cant believe how many copies of python I have on my system.

    2. Tom7

      Re: No mention of pip and venv?

      They work if:

      * You want to use the version of Python that is installed by default on your system

      * You don't want to package your work

      As soon as you want to use a different version of Python for a project, or you want to publish your work on PyPI or distribute it in wheel form, there is a whole world of pain that venv and pip don't help you with.

      That said, rye seems to still have some pretty rough edges. After an hour of faffing around, I still can't get it to install pylint into a venv...

      1. Short Fat Bald Hairy Man

        Re: No mention of pip and venv?

        For me, they have worked with different versions too (ubuntu with deadsnakes ppa).

        Right now, I have 3.9 (for legacy), 3.10 (default OS) and 3.11 (always test on one ahead) right now. Seem to work ok.

        1. Phil Lord

          Re: No mention of pip and venv?

          They work but only provide the basics in terms of convenience. For example, if you create a venv you still have to remember to activate it, either through an IDE, or on the command line (for which I use direnv. The functionality of pip depends on whether you have: `pip install numpy` will do different things depending. And, pip is not a project tool. If you copy my git repo, you need to create a venv and then run pip on requirements.txt. And, because there is no lock file, what you get might still be different from what I get. And if I have run `pip install numpy` previously, my repo might not work for you because I forgot to add it to requirements.txt.

          And so on. pip and venv are potentially part of the solution, providing installation, dependency management and sandboxing. But they fall wall short in providing all of the solution. Python packaging is a mess.

          1. Blank Reg

            Re: No mention of pip and venv?

            After the absolute stupidity of whitespace having semantics, installing packages in python is my biggest complaint.I wasted two hours yesterday with pip, and it's not the first time.

          2. Missing Semicolon Silver badge

            Re: No mention of pip and venv?

            "And, because there is no lock file, what you get might still be different from what I get". You don't pin your dependencies? Tut.

        2. Charlie Clark Silver badge

          Re: No mention of pip and venv?

          Ugh deadsnakes! I have Python3.6 to 3.12 and PyPI on my MacBook all installed and managed using MacPorts (I could still have 2.7 if I wanted to). Debian/Ubuntu have probably the worst approach to Python I've come across.

        3. Tom7

          Re: No mention of pip and venv?

          I can't figure out what was wrong with this. I blew it all away and started again and it all just worked. It had somehow got it into its head that it needed quite an old version of pylint in that one project and nothing I did seemed able to convince it otherwise.

      2. AdamWill

        Re: No mention of pip and venv?

        I mean, I've never really had *that* much trouble. This is what the packaging/release process for all my Python projects looks like:

        (there are some simpler ways to do the version stuff these days, but that's what I'm used to). Run that script, release is done (published to pypi). I could ditch and just use pyproject.toml , but that'd maroon far too many older setups, so I keep it around.

        Local testing via tox works fine: (the isolated_build setting for tox makes it not use host system packages, which reduces any chance of different results on different systems), just needs the interpreters for whichever Python versions you want to test installed, which Fedora makes easy. CI just runs tox in containers:

        However, there's so much noise about this, it must be a real problem for some cases, I've just always had trouble grokking exactly how. I suspect it's partly much more of a problem if you don't develop on Linux (note all the discussion of getting Python installed in the first place, which is never going to be a problem on Linux...), and partly much more of a problem if you don't work in pure Python (so compilation is involved at some point)...

        I find Hynek Schlawak a really good source to keep up to date on stuff - .

    3. jilocasin

      Re: No mention of pip and venv?

      What about pyenv?

      Nice system to manage multiple versions of python on your system all the way down to on a per project basis.

      1. Tom7

        Re: No mention of pip and venv?

        As with a number of other systems, this only solves half the problem. It helps you manage a collection of other people's packages, not to package your own.

        1. David Harper 1

          Re: No mention of pip and venv?

          Probably best use Java then.

    4. Mozzie

      Re: No mention of pip and venv?

      Have an upvote.... in fact the only upvote for some reason.

      I'm with you on this. Nothing wrong with a requirements.txt, a supporting .md and a small directory with your systemd service files and startup scripts in.

      That said, Armin is a huge source of talent and Rye has just gone into my priority learning list. I'm still waiting for him to convince me asyncio is better than Gevent though.... or anybody for that matter.

    5. FatGerman

      Re: No mention of pip and venv?

      Venv is a hack solution to a problem that shouldn't exist in the first place. That's what the article is about.

  3. Crypto Monad Silver badge

    And while you're mentioning xkcd, don't forget 927.

    1. david 12 Silver badge

      According to the NYT , 1939 was the 200 year anniversary of the publication of "Joe Millers Joke Book". (That would make it 284 years old now)

  4. Anonymous Coward
    Anonymous Coward

    > bootstrapping (setup) and packaging

    An interpreter doesn't need "bootstrapping" outside of compiling the thing in the first place. Anyone who thinks it does is making things difficult for themselves.

    And by "packaging" what they actually mean is encouraging a culture of dragging in a load of technical debt and inadequate "square" wheels as dependencies.

    NPM, CPAN,, PIP, etc. No thanks. Short term convenience, rarely outweighs the long term maintainability.

    (not to undermine the work involved however. Orchestration of all these systems is very impressive, and quite a feat to maintain).

    1. Tom7

      So I'm curious - how do you install Python packages without pip? Or do you insist that all the Python you write use nothing but the standard library?

      1. Gunboat Diplomat

        Avoiding pip

        I'm guessing they look up the package source code online and type it in locally using vi with pins wedged under their fingernails while fighting a bear.

        Or maybe they use conda.

        1. Mark 65

          Re: Avoiding pip

          I'm guessing they look up the package source code online and type it in locally using vi with pins wedged under their fingernails while fighting a bear.

          Or maybe they use conda.

          There's a difference?

      2. Anonymous Coward
        Anonymous Coward

        The same way I install packages for any software...

        Sorry, I can't quite work out why you are asking such a seemingly elementary question. How do you install software in general? How do you think Python developers achieved anything before pip? How do all the other hundreds of languages who don't provide a language based package manager get around the issue?

        So many questions, but honestly none of them are something that someone working with IT should be asking.

        1. mpi Silver badge

          > How do you think Python developers achieved anything before pip?

          Well, I can only speak for myself, but, depending on the level of inconvenience and self-torture I was looking for, either by installing modules via the distros pkg manager, or by downloading them in whatever format the maintainer thought was most useful, and shoving them somewhere into `sys.path`.

          And both methods sucked. Hard. That's why first easyinstall and then pip where invented, that's why it became the dominant tool for installing modules. That's why almost every single major programming language other than C/C++ have at least SOME concept of a built in pkg manager.

          Distro pkg manager sucks because it limits me to what the distro offers, and makes modules with incompatible dependencies about as fun to deal with as a nest of wasps. But granted, it wasn't all bad, because at the very least, distro pkg managers automated dependency management and update things with a minimum of fuss (except when an update breaks something the package manager didn't install, and thus doesn't care about) in contrast to our next candidate, Manual installation.

          Which also sucks, because depending on the source I was either dealing with a repo (which wasn't always git), a tar archive, and sometimes FTP servers. Some of these included C-libs. Some of those were precompiled. Some I had to build myself. Which ofc means I have yet another set of dependencies, but this time it's C. Integrating projects with with large dependency graphs? FUN (Short for "Frekked up nightmare") Sure, one could just vendor everything. That shifts the nightmare from build to update. No, wait, updating was ALWAYS a nightmare regardless, because without a package manager, there is no dependency management, so whenever anything was updated, it could potentially break something else. Manually checking the dependency graph for 42+ 3rd party modules, not all of which provided their dependencies in a standardized format? Not fun.

          So yeah, by now I consider a package management system for a programming language essential. C/C++ excepted, because there the distros actually do a really good job of providing libs, because everything depends on them anyway.

        2. mpi Silver badge

          > How do all the other hundreds of languages who don't provide a language based package manager get around the issue?

          Out of curiosity, would you care to name some mainstream programming languages that don't have a modern package manager? I don't need hundreds, let's start with, say, 20.

          1. Anonymous Coward
            Anonymous Coward


            If you only need 20; just grab the i.e 2nd language in the list of each group. I am pretty sure that will land on one without a language specific package manager. Saves me typing them.

            1. mpi Silver badge

              My question was about MAINSTREAM PROGRAMMING LANGUAGES.

              That is:

              - Languages that actually matter as of right now, as opposed to being long irrelevant, extremely niche, hobbyist, esolang or not established.

              - Languages that are not DSL

              - Languages that actually have a concept of libraries and dependencies to begin with

              So, with these clarifications, I repeat my question: May I have a list of 20 mainstream programming languages satisfying these requirements, that do not feature a package manager? Actually, tell you what, I'll lower the requirement to 15 languages, how's that?

      3. david 12 Silver badge

        To install without PIP: I download some python modules, put them in a directory/folder, and add that directory/folder name to the python sys path.

        I do this when, because of old/small equipment, I am unable to use the encryption required by PIP, or total lack of networking.

        When they work, PIP and Conda manage dependencies. Frequently they mis-manage dependencies: I can't believe how unbelievably bad they both are at managing dependencies: I can only believe that my use cases must be outliers. I guess that for people who aren't managing obscure modules on fleets of obscure systems, both must work ok.

        Installing either way, either by hand or by using PIP, used to work much better before the python ecosystem acquired so many dependencies. PIP is a particularly obvious example, but lots of stuff I use requires TLS 1.3 and a handful of other library packages just to load, even for local filesystem operations, because it includes networking or authentication in the feature list.

        1. FatGerman

          Oh gd, so much this. Module A requires version X of module Z, but module B requires version Y of module Z. I need A and B, but version Y is not backwards compatible with version X and I can't have both. You can't resolve that with some godawful venv hack. You need a properly managed ecosystem.

    2. CowHorseFrog Silver badge

      Exactly come to think of it why even download the languaage itself, write your own parser and runtime.

      1. Anonymous Coward
        Anonymous Coward

        Exactly. Why actually write a working solution at all? Just play with bootstrapping scripts and juggling dependencies all day instead.

  5. chololennon

    How about Poetry?

    My experience with Poetry ( is very similar to npm or cargo (I also use maven/gradle with Java, and vcpkg/cmake with C++, but those are another story, especially with C++)

  6. Morten Bjoernsvik

    what we need another package bloat

    I've done java maven and node npm and they are terrible and bloated.

    a maven pom under 300 lines are seldom. and you need a artifactory or nexus local repo to build fast. and all the plugins.

    npm a bit better but the directory stucture completely messy. typescrip ,

    1. CowHorseFrog Silver badge

      Re: what we need another package bloat

      Maven has many faults but verbosity is not your enemy its your friend. There is nothing worse than a config file that has nothing and you or others think they know the defaults but it turns out the defaults are actually something else.

      Defaults be it config or runtime are one of the biggest enemies of all software types, because we all know how fun it is when others make a decision on what they think is a good default and you end up with problems like the log4j exploits.

      Writing everything down so you can read is always helpful.

  7. Michael Hoffmann Silver badge

    Not sure how it helps?

    I just watched the demo video and could not immediately see how it distinguishes itself from something like "pyenv".

    Also, wasn't the premise the focus on packaging, which I would take to include distribution? I suppose that in the end I'm still building some wheel, pushing that out and good luck that you don't have dependency hell! Finally, at this early stage, there's no integration with anything from IDEs to various other frameworks that start with "something init" and dump metadata in your directory (*sigh* AWS CDK *sigh*)

  8. bazza Silver badge

    Software Packaging is Hard

    Python is in a right mess, which is partly why I don't recommend it for actual software engineering.

    Python is not helped by the fact that python gets used in Linux distros / desktops as a system component in a way that exposes that Python environment to users. Of course it's going to get mixed up with their dev python environments. It's amateurish design and it's asking for trouble.

    But it's not that great elsewhere. If one looks at anything and everything from autotools to snap, there's some severe issues. Just try running two different versions of gcc on a system and build using the none standard one...

    At least with OSes like MacOS, Windows and FreeBSD there's certain guaranteed minima to work to.

    In Linux you can see why things like snap and flatpak get invented - about the only way of getting the right environment for software is to bring it with you these days - but feels like that is an insane way to go. Certain projects don't help either (cough GTK cough).

    The entire Linux distro world should take a look at Visual Studio. If I install that, I don't end up with a build environment that's entangled with my OS and desktop environment in a tangled knot of conflicting dependencies. I don't end up with compiled code that is platform unique. I can very easily get a packaged installer that'll work on, well, any windows installation. And, if it comes to that, it also does a pretty good job of letting me compile, debug and run C/C++ software for Linux too (though it is then at the mercy of what is inside the Linux distro, but with WSL one can have quite a few different Linuxes at one's beck and call). It does this a lot better than most of the Linux dev tools (which is probably VS Code these days anyway).

    With Linux distros as they are today, I really think that Linus and his colleagues in the kernel project are wasting their time. The kernel has done an admirably good job at keeping the kernel system interface consistent - "don't break userland". That's not achieving much if the all the "users" are going to do with software is either build it from source at installation, or run it inside a virtual machine. The kernel is written to support binary compatibility. The rest of the Linux world is so dysfunctional that that is an increasingly ignored and snubbed blessing.

    Until Linux distros stop being structured to drive software building in a me, me, me and only me direction, software distribution will remain a messy nightmare. Build / dev environments need to be entirely separate to the OS environment.

    1. Charlie Clark Silver badge

      Re: Software Packaging is Hard

      Python is not helped by the fact that python gets used in Linux distros / desktops as a system component in a way that exposes that Python environment to users.

      This is one of the many sins of the Linux distros. *BSD users understand the difference between system and user software and ports takes away all the pain that comes from conflating the two and lets you have pretty much all the versions of a language that you would like. On my Mac I have Python 3.6 to 3.12 and no conflicts.

    2. mpi Silver badge

      Re: Software Packaging is Hard

      > Of course it's going to get mixed up with their dev python environments.

      That is a long-solved problem, even ignoring virtualization entirely. `pyenv` provides shims and multi version installs that don't interfer with the OS's python installation. `venv` provides environments ie. package-isolation within these installations. Both can be used in concert to provide as many arbitrary combinations of interpreter and modules as I want, all on the same machine, without any virtualization, without any interference with the systems Python install.

      And again, that is before we even mention containers, which achieves the exact same effect, only in a vastly more convenient manner, that applies to every language and build system, not just python.

      If people run python projects in their production environment without these, that's their own problem. If I insist on throwing 40 kg of camping gear in my trunk and then find my fishing lines tangled with my socks and never see half the tent-pegs again, I have only myself to blame for not packaging properly.

      > Just try running two different versions of gcc on a system and build using the none standard one...

      You were saying?

      > but with WSL one can have quite a few different Linuxes at one's beck and call

      docker run -it debian /bin/bash

      docker run -it archlinux /bin/bash

      docker run -it redhat/ubi8 /bin/bash

      docker run -it alpine /bin/ash

      You were saying?

      > The rest of the Linux world is so dysfunctional that that is an increasingly ignored and snubbed blessing.

      Curious then, isn't it, that this "snubbed blessing" is powering almost the entirety of this planets sever infrastructure, as well as the vast majority of mobile personal tech from smartphones to tablets.

      > Build / dev environments need to be entirely separate to the OS environment.

      No they don't, because it's my environment, and I want to be able to make it as tidy or chaotic as I like. And because I can do all that, I can, on my machine, using one-liner terminal commands, spin up not one, not two, but an arbitrary amount of different dev/build/test/production environments, in seconds, up to and including simulating entire networks of microservices.

  9. Ochib

    Will it work with C?

    I descend upon your earth from the skies

    I command your very souls you unbelievers

    Bring before me what is mine

    At the 7 C's of Rye

  10. yoganmahew

    No man is an island, except a python user

    20 some posts, and nary an upvote.

    Not only do the 20 people not agree with each other, everyone else reading the thread doesn't either. Not disagree enough to downvote, that would be rude...

  11. mpi Silver badge

    Sorry if this sounds negative, but...

    > "A class of users expect a packaging tool that provides a cohesive experience (like npm (NodeJS), cargo (Rust), gem (Ruby), pub (Dart), dotnet (C#/.NET), etc)

    Yes, and what do most of such tools have in common? They were there for almost the whole lifetime of the project. That's why almost every tutorial, book, video, etc. on the topic features them.

    Sorry if I sound like a negative nelly here, but that ship hasn't just sailed for Python, it has sailed, came back, shanghaied a new crew, was sunk with a load of sugarcane, tobacco and spice, was risen from the depths by an ancient carribean curse, and is currently headed for Tortuga.

    This is something that projects have to get right from the beginning. If they don't then it doesn't matter how many new solutions there will be, or how good they are, the genies won't go back into their bottles. Look at C. How many different systems for building, dependency management, etc. are there for C? How unified is the landscape? And C had a lot more time to come up with good solutions for this. And it did. But it didn't help, because they were not there from the beginning, and by the time good tools came into existence, they were just more programs competing in the same mindspace.

    In python the situation is very similar. Like C, Python became an integral part of the Unix Landscape. Every distro packages it, so there are not just competing build systems, there is also the systems own package manager with its strong opinion about how things should be.

    None of that means that this project isn't a good idea. I absolutely like Ronachers work, and will definitely give Rye a whirl. But I don't think that this will solve the headaches of python deployement. It will only make it somewhat less of a headache.

    1. Missing Semicolon Silver badge

      Re: Sorry if this sounds negative, but...

      Pycharm does a nice job of isolating the environment changes to a project - basically by defaulting to creating a venv for it. That and a couple of deadsnakes installs, and you're golden.

  12. CowHorseFrog Silver badge

    Java has a far bigger packaging system than basically all the others put together and yet somehow doesnt get a mention.

  13. davidjensenusa


    Another language, Rust? What is the rational?:Python is getting faster. Was Rust supposed to be faster?

    1. Anonymous Coward
      Anonymous Coward

      Re: Rust?

      Obviously, if you want to create a system to help bootstrap Python, it helps if you can demonstrate that bootstrapping is really, really hard: so add another language in the mix!

      Otherwise it'll be far too easy to get Python onto a new environment.

    2. mpi Silver badge

      Re: Rust?

      Yeah, that one I don't get either.

      I mean, sure, a lot of people will just load the compiled binaries. However, some people will want, or need to, build it from scratch, and pulling in the entire rust buildchain, may not be a problem when the system is already using rust...but if it doesn't, and that's not an unlikely scenario, then requiring its build chain and libraries to bootstrap something in python, when every other management tool for it is built either in python itself, or lives in a C-library called from python code ... I mean, why?

  14. mpi Silver badge

    Am I the only one who thinks that golang...

    ...may have solved all of these issues in the most elegant way out of all mainstream languages?

    Because, what's a golang module? A repository with some special plaintext files in it. aka. where the code will eventually end up anyway. Packaging a golang module for others to use, happens the moment I press ENTER after typing `git push`.

    What's the import-path? Why it's a simple string that contains all the information (aka. the URL and package name) required for the toolchain, which lives in a single executable that ships with the language core, to find and download modules using tools that already git and mercurial.

    Dependency management? Transient dependencies are directly defined in the source. When I want to import package X, and X wants to import Y, simply reading the source tells the toolchain all it needs to know about transient dependencies, simply because of how import-paths work.

    Version & integrity control? The aforementioned plaintext files contain versions (go.mod) and checksums (go.sum). The toolchain handles both of them for the developer. And how are versions defined? Why, again using a system that already exists: semantic versioning via git-tags.

    Sorry for sounding like a massive fanboy, but this. is. just. brilliant. The entire ecosystem works without requiring a central package authority. Builds work using just the info in the source files + go.mod/sum. Yes, goproxys exist, but they are not needed for the system to work. As long as peoples repos are up, the ecosystem is up. And the toolchain can make use of a lot of capabilities and semantics that already exist independent of golang, put them together in a clever way, and throw the end-result into the compiler.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like