Sorry Python but it is what it is.
You must log in or register to comment.
Controversial opinion, but
npmis as terrible aspip. How are we forgettinggems?npm is objectively worse. Base pip packages aren’t getting hijacked.
Maybe I’m misremembering, but didn’t pip have it’s own security concerns earlier this year?
I believe that was just name squatting.
It’s less the name squatting and more pip not supporting a certain PyPI resolution order: https://github.com/pypa/pip/issues/8606
For example, I have A, B and C in my requirements.txt but I want to install C from my own private PyPI. Everything works fine until someone uploads a package name C to the public PyPI then suddenly I’m not installing my private package anymore.
Yeah, I remember now. the name squatting was from people putting malicious packages under misspelled names of well known packages, like “requets” instead of requests.
That’s not a controversial opinion. I’d say it’s worse than pip. At least pip doesn’t put nag messages on the console or fill up your hard drive with half a gigabyte of small files. OP is confused.
npm is so good there are at least 3 alternatives and every package instructs on using a different one.
About the only good thing about npm is that I can use one of the superior alternatives. Using npm is almost always a headache as soon as you start working with a decent number of packages.
In my experience npm is not great but it does work most of the time. I just tried installing bunch of stuff using pip and NONE of them worked. Python is backwards compatibility hell. Python 2 vs 3, dependencies missing, important libraries being forked and not working anymore. If the official installation instructions are ‘pip install X’ and it doesn’t work then what’s the point?
npm has A LOT of issues but generally when I do ‘npm i’ i installs things and they work.
But the main point is that cargo is just amazing :)
P.S. Never used ruby.
Well there’s your problem lol.
Don’t use 2 for anything, it’s been “dead” for almost 4 years.
The problem is 2 and modules for 2 still tend to worm their way in somehow. I always use python3 -m pip because I never trust that “pip” alone is going to be python3 pip and I think that’s what the people who have lots of trouble with pip aren’t doing.
Valid point.
I force everything to 3 and don’t accept any 2.
And in fairness, there were some moderate breaking changes 3.6-3.8
It would be weird to have python2-pip installed if you don’t have python2 installed, pip should be python2-pip by default on most systems.
I… Dunno, are you suggesting that sometimes pip2 is the default and that that somehow mixes 2 and 3 modules? Pip 2 should install into python 2’s directory and pip 3 to python 3’s. The only times I have had messy python environments is when I mix pipenv, conda and/or pip, and when people install into the main python with specific versioning, use a virtual env for God’s sake, that’s what npm does.
Ahh the blissful ignorance of not having to manage tech debt
No, I just don’t ignore it for 4 years.
The bliss is in having management that actually DOES manage the debt instead of ignoring it until it shits the bed
I don’t think it’s fair to blame pip for some ancient abandoned packages you tried to use.
The issues I had:
- packages installing but not working due to missing dependencies
- packages installing but not working due to broken dependencies (wrong lib version installed)
- packages not building and failing with obscure errors
- one package was abandoned and using Python 2.7
If a ‘pip install X’ completes successfully but X doesn’t work it’s on pip. And when it fails it could tell you why. Cargo does.
packages installing but not working due to missing dependencies
This is the fault of the package author/maintainer
packages installing but not working due to broken dependencies
Sometimes the fault of the package author/maintainer. Sometimes this is the fault of a different package you’re also trying to use in tandem. Ultimately this is a problem with the shared library approach python takes and it can be ‘solved’ by vendoring within your own package.
packages not building and failing with obscure errors
Assuming the package is good, this is a problem with your build system. It’s like complaining a make file won’t run because your system doesn’t have gcc installed.
one package was abandoned and using Python 2.7
Unfortunately there’s a ton of this kind of stuff. I suppose you can blame pypi for this, they should have some kind of warning for essentially abandoned projects.
Hmm, I personally haven’t seen that kind of issue myself though. I also tend to not use random packages from random authors though, so that might help.
I would recommend that you give
pnpma try. Or if you want something that’s blazing-fast, there’sbunx, but that comes under a completely different JS environment (BunJS) and also brings back the same issue that’s innpm. From what I’ve heard, they’re probably going to make it similar topnpm.The main issue with JS is that every 6 months someone comes up with the next great tool that misses half of basic features and dies after 6 months when someone comes up with the next great tool. But at least the old tested solution still works unlike in Python where the main goal seems to be breaking the backwards compatibility as often as possible.
I’m still rocking the fuck out of PHP (8) 😘
pnpm is already very well established, it’s not completely different from npm either so they didn’t have to reinvent the wheel, they just made some things much better.
Python is is just a mess on the other hand, a thousand tools all with some overlap in what they’re trying to achieve because they didn’t have the balls to make pip an all-in-one solution, there are 2 great alternatives that do almost everything though: poetry and pdm. I read a spot on analysis on this article, maybe it can help you make a choiceThis is great, thanks. Will definitely read even though I don’t do much work in python. It’s good to know how NOT to do things.
But at least the old tested solution still works unlike in Python where the main goal seems to be breaking the backwards compatibility as often as possible.
lol what. Node does a new major release every six months. And you’re shit talking python? There’s probably never going to be another major version change, and minor versions have several years of support
In like 10 years of python development I don’t think I’ve ever been mad about breaking changes in python.
I’d personally take PECL over npm and I loathe PECL.
Composer, though, is excellent.
Sorry but nah. My last job we had a couple different python microservices. There was pipenv, venv, virtualenv, poetry, Pipfile.lock, requirements.txt (which is only the top level???), just pure madness
Apparently all this shit is needed because python wants to install shit globally by default? Are you kidding?
Well, we also had a couple node microservices. Here’s how it went: npm install. Done.
Afraid you fucked something and want a clean environment? Here’s how you do it with node: delete
node_modules/. Done.Want a clean python env? Uhhhhhhhh use docker I guess? Maybe try reinstalling Python using homebrew? (real actual answers from the python devs who set these up)
Well what’s currently installed?
ls node_modules, or usenpm lsif you want to be fancy.In python land? Uhhhhhh
Let’s update some dep–WHY AREN’T PYTHON PACKAGES USING SEMVER
So yeah, npm may do some stuff wrong, but it seems like it does way more shit right. Granted I didn’t really put in the effort to figure out all this python shit, but the people who did still didn’t have good answers. And npm is just straightforward and “works”.
“But JS projects pull in SOOOO many dependencies” Oh boohoo, you have a 1TB SSD anyway.
Apparently all this shit is needed because python wants to install shit globally by default?
None of that was needed. It was just used because nobody at your company enforced a single standard for developing your product.
Afraid you fucked something and want a clean environment? Here’s how you do it with node: delete node_modules/. Done.
rm -rf venv/. Done.
Want a clean python env? Uhhhhhhhh use docker I guess?
python -m venv venv
Well what’s currently installed? ls node_modules, or use npm ls if you want to be fancy. In python land? Uhhhhhh
pip freeze. pip list if you want it formatted.
Let’s update some dep–WHY AREN’T PYTHON PACKAGES USING SEMVER
Janky, legacy python packages will have random versioning schemes. If a dependency you’re using doesn’t follow semver I would question why you’re using it and seek out an actively maintained alternative.
Im honestly surprised someone using Python professionally appears to not know anything about how pip/venv work.
The points you think you are making here are just very clearly showing that you need to rtfm…
More like rtfms. I really didn’t feel like learning 20 different tools for repos my team didn’t touch very often.
I really don’t see the hassle… just pick one (e.g. pip/venv) and learn it in like half a day. It took college student me literally a couple hours to figure out how I could distribute a package to my peers that included compiled C++ code using pypi. The hardest part was figuring out how to cross compile the C++ lib. If you think it’s that hard to understand I really don’t know what to tell you…
Sure, for a new project. But when inheriting code I’m not in a position to pick.
The point is that the state of python package managers is a hot fucking mess compared to npm. Claiming that “npm is just as bad” (or worse) honestly seems ridiculous to me.
(And isn’t pip/venv the one the
requirements.txtone? Completely flat, no way to discern the difference between direct dependencies and sub-dependencies? No hashes? Sucks when it’s time for updating? Yeah no thanks, I’d like a proper lock file. Which is probably why there are a dozen other tools.)
So you are saying that npm is better than pip?? I’m not saying pip is good, but npm?
npm has a lockfile which makes it infinitely better.
pip also has lock files
pip freeze > requirements.txtWould that just create a list of the current packages/versions without actually locking anything?
Would that just create a list of the current packages/versions
Yes, and all downstream dependencies
without actually locking anything?
What do you mean? Nothing stops someone from manually installing an npm package that differs from package-lock.json - this behaves the same. If you
pip install -r requirements.txtit installs the exact versions specified by the package maintainer, just likenpm installthe only difference is python requires you to specify the “lock file” instead of implicitly reading one from the CWDAs I understand, when you update npm packages, if a package/version is specified in
package-lock.json, it will not get updated past that version. But running those pip commands you mentioned is only going to affect what version gets installed initially. From what I can tell, nothing about those commands is stopping pip from eventually updating a package past what you had specified in therequirements.txtthat you installed from.The behaviour you mention is from npm install, which will put the same exact version from the package-lock.json, if present. If not it will act as an npm update.
npm update will always update, and rewrite the package-lock.json file with the latest version available that complies with the restrictions defined on the package.json.
I may be wrong but, I think the difference may be that python only has the behaviour that package-lock.json offer, but not the package.json, which allows the developer to put constraints on which is the max/min version allowed to install.
If you want min-max behaviours you need to use wrappers like pipenv or jump into conda/mamba. Pip offers basic functionality because there are more advanced tools that the community uses for the more advanced use cases.
But running those pip commands you mentioned is only going to affect what version gets installed initially.
I don’t follow. If my package-lock.json specifies package X v1.1 nothing stops me from manually telling npm to install package X v1.2, it will just update my package.json and package-lock.json afterwards
If a requirements.txt specifies X==1.1, pip will install v1.1, not 1.2 or a newer version. If I THEN install package Y that depends on X>1.1, the pip install output will say 1.1 is not compatible and that it is being upgraded to 1.2 to satisfy package Y’s requirements. If package Y works fine on v1.1 and does not require the upgrade, it will leave package X at the version you had previously installed.
That’s not a lockfile. This would be the equivalent of package.json
How is it not a lock file?
package.json doesn’t contain the exact version number of all downstream dependencies, this does
Lockfile contains exact state of the npm-managed code, making it reproducible exactly the same every time.
For example without lockfile in your package.json you can have version 5.2.x. In your working directory, you use 5.2.1, however on repo, 5.2.2 has appeared, matching your criteria. Now let’s say a new bug appeared in 5.2.2.
Now you have mismatched vendor code, that can make your code behave differently on your machine, and your coworker’s machine, making you hunt for bug that wasn’t even on your side.
Lockfile prevents that by saving an actual state of vendor code.
Yes, which is EXACTLY like a
pip freeze’d requirements.txt, storing the exact version of every package and downstream dependency you have installed
Pip has Pipfile.lock.
That’s pipenv. Pip just has the capability to read and write from a requirements.txt, which is a step that must be taken manually
Oh right, sorry.
I would say npm is shitty like a lot of tools are. pip takes it to the next level.
Yeah? I don’t recall having to wait a long time when setting up my project using pip.
Cargo is a pretty good tool, but it’ll happily fill up your disk with cached copies of crates that you downloaded somewhere in your user folder. Using a modern fs like BTRFS with extent deduplication helps to save space, but it doesn’t solve the problems.
pip and npm are practically equally bad, at least the maintained versions. If you’re using old versions that haven’t been supported for years (Python 2 etc.) then you’re in for a world of pain with both.
There are much worse alternatives. Anaconda, for example, takes half an hour to resolve dependencies while also autoloading itself into your shell, adding up to half a second (I timed this! I thought zsh was bugged!) of latency between shell prompts.
The worst tool is probably what many C(++) projects seem to do: use Git as a dependency manager by including entire git repos as submodules. I’m pretty annoyed at having to keep multiple versions of tokio around when building a Rust project, but at least I’m not downloading the entire commit history for boost every time I clone a project!
cached copies of crates that you downloaded
Meh, what else is it supposed to do, delete sources all the time? Then people with slow connections will complain.
Also size-wise that’s actually not even much (though they could take the care to compress it), what actually takes up space with rust is compile artifacts, per workspace. Have you heard of kondo?
Idk, maybe you can share the common packages across projects. (That can never go wrong, right? /s)
Sources are shared, sharing compile-time artefacts is done within workspaces.
Oh… I did mean sharing comptime artifacts
You can globally share compile artifacts by setting a global target directory in the global Cargo config.
In $HOME/.cargo/config.toml:
[build] target-dir = "/path/to/dir"The only problems I had when I did it where some cargo plugins and some dependencies with build.rs files that expected the target folder in it’s usual location.
I actually vastly prefer this behavior. It allows me to jump to (readable) source in library code easily in my editor, as well as experiment with different package versions without having to redownload, and (sort of) work offline too. I guess, I don’t really know what it would do otherwise. I think Rust requires you to have the complete library source code for everything you’re using regardless.
I suppose it could act like NPM, and keep a separate copy of every library for every single project on my system, but that’s even less efficient. Yes, I think NPM only downloads the “built” files (if the package uses a build system & is properly configured), but it’s still just minified JS source code most of the time.
With python and virtualenv you can also keep the entire source of your libraries in your project.
Python virtual environments feel really archaic. It’s by far the worst user experience I’ve had with any kind of modern build system.
Even a decade ago in Haskell, you only had to type
cabal sandbox initonly once, rather thansource virtualenv/bin/activate.shevery time youcdto the project dir.I’m not really a python guy, but having to start touching a python project at work was a really unpleasant surprise.
If this is from the perspective of a hobbyist or brand new Python dev, that’s a fair opinion to have, I suppose.
That said, if you’re using Python in a professional capacity, you really need to learn how to use the toolchain properly.
Python packaging and virtual environments are not difficult to understand, and I’d wager based on your comments elsewhere in this thread that your frustrations are born from not taking the time to understand why following the instructions from a fourteen-year-old blog post aren’t working.
99.99% of the time, the fault isn’t with pip, it’s with the maintainer of the broken package you’re trying to use.
This is programmer humor, 95% of the people here still get defeated by semicolons, have never used a debugger, and struggle to exit
vim.Sometimes I wish there was a community for more advanced users, where the concept of deciding on the best build tool chain per project is not a major hurdle. Venvs? Nbd. Pipenv? Nbd. Conda/mamba/micromamba? Nbd. Pure pip? Oh boy, I hope it a simple one, but I’ll manage. Maven? Fml, but sure. Npm? Sure. “Complex” git workflows, no problem.
Idk, that’s just setting up the work environment, if your brains get squeezed by that I’m not sure if you will then be able to the actually code whatever its being asked of you. Some people…
But yeah, this is a newbie space so I guess that we have to ignore some noise.
Seriously, I usually use Poetry these days for most projects, shit just works, build well and lets me distribute my code from PiPy just fine. Everything in one pyproject.yaml.
I like the pyproject.yaml, but checking dependencies with poetry takes 5 to 10 minutes for my projects.
Yeah, that’s rough. Is it any faster w PipEnv?
It actually is almost as instant as you would expect
Weird, Poetry gotta up their game then 😎
This article someone linked is not 14 years old and it perfectly describes the mess python and pip are: https://chriswarrick.com/blog/2023/01/15/how-to-improve-python-packaging/
My favorite part is:
Most importantly: which tool should a beginner use? The PyPA has a few guides and tutorials, one is using pip + venv, another is using pipenv (why would you still do that?), and another tutorial that lets you pick between Hatchling (hatch’s build backend), setuptools, Flit, and PDM, without explaining the differences between them
But yes, following old blog post is the issue.
Why not read the official python docs?
Hahaha!..
Oh shit, you’re serious.
They pretty simply describe how to handle a venv, pip, reqs, etc.
Friend, while I appreciate the time and effort on the docs, it has a rather tiny section on one of the truly worst aspects of pip (and the only one that really guts usability): package conflicts.
Due to the nature of Python as an interpreted language, there is little that you can check in advance via automation around “can package A and package B coexist peacefully with the lowest common denominator of package X”? Will it work? Will it fail? Run your tool/code and hope for the best!
Pip is a nightmare with larger, spawling package solutions (i.e. a lot of the ML work out there). But even with the freshest of venv creations, things still go remarkably wrong rather quick in my experience. My favorite is when someone, somewhere in the dependency tree forgets to lock their version, which ends up blossoming into a ticking time bomb before it abruptly stops working.
Hopefully, your experiences have been far more pleasant than mine.
If you’re using a manually managed venv, you need to remember to activate it, or to use the appropriate Python.
That really doesn’t seem like a big ask.
I’ve been using python professionally for like 10 years and package management hasn’t really been a big problem.
If you’re doing professional work, you should probably be using docker or something anyway. Working on the host machine is just asking for “it works on my machine what do you mean it doesn’t work in production?” issues.
No, actually most devs don’t use docker like that. Not java devs, not JS devs, not rust devs. That is because maven, npm and cargo manage dependencies per project. You use it for python exactly because pip does it the wrong way and python has big compatibility issues.
They’re not difficult by any means.
But they are tedious when compared to other solutions.
If we talk about solutions: python has plenty. Which might be overwhelming to the user.
I use Direnv to manage my python projects. I just have to add
layout pyenv 3.12.0on top and it will create the virtual environment for me. And it will set my shell up to use that virtual environment as I enter that directory. And reset back to default when I leave the directory.But you could use pipenv, poetry, pdm, conda, mamba for your environment management. Pip and python do not care.
I have to agree, I maintain and develop packages in fortrat/C/C++ that use Python as a user interface, and in my experience pip just works.
You only need to throw together a ≈30 line
setup.pyand a 5 line bash script and then you never have to think about it again.The only time I ever interacted with python packaging was when packaging for nixos. And I can tell you that the whole ecosystem is nuts. You have like ten package managers each with thirty different ways to do things, none of which specify dependencies in a way that can be resolved without manual input because y’all have such glorious ideas as implementing the same interface in different packages and giving each the same name and such. Oh and don’t get me started on
setup.pymaking http requests.
NPM is ghastly though
npm is just plain up terrible. never worked for me first try without doing weird stuff
I don’t know what cargo is, but npm is the second worst package manager I’ve ever used after nuget.
cargo is the package manager for the Rust language
I’ve never had an issue with nuget, at least since dotnet core. My experience has it far ahead of npm and pip
I’ll second this. I would argue that .Net Core’s package/dependency management in general is way better than Python or JavaScript. Typically it just works and when it doesn’t it’s not too difficult to fix.
It’s also much faster to install packages than npm or pip since it uses a local package cache and each package generally only has a few DLL files inside.
cargo is rust
what’s wrong with nuget? I have to say I like the “I want latest” “no, all your dependencies are pinned you want to update latest you gotta decide to do it” workflow. I can think of some bad problems when you try to do fancy things with it but the basic case of “I just want to fetch my program’s dependencies” it’s fine.
I’m guessing they only used it 10 years ago when it was very rough around the edges. It didn’t integrate well with the old .NET Framework because it conflicted with how web.config managed dependencies and poor integration with VS. It was quite bad back then… but so was .NET Framework in general. Then they rebuilt from the ground up with dotnet core and it’s been rock solid since
Or they just hate Microsoft, which is a common motif to shit on anything Microsoft does regardless of the actual product.
Imho the VS integration has always been good, it’s the web config that’s always been a trash fire, and that’s not new.
The project I’m on right now originally had the nuget.exe saved in source because they had to manually run it through build scripts, it wasn’t built in to VS until VS2012
Memes like this make me ever more confused about my own software work flow. I’m in engineering so you can already guess my coding classes were pretty surface level at least at my uni and CC
Conda is what I like to use for data science but I still barely understand how to maintain a package manager. Im lowkey a bot when it comes to using non-GUI programs and tbh that paradigm shift has been hard after 18 years of no CLI usage.
The memes are pretty educational though
Try not to learn too much from memes, they’re mostly wrong. Conda is good, if you’re looking for something more modern (for Python) I’d suggest Poetry
Never have heard of Poetry, but I’ll check it out tonight! I pretty much exclusively coded in Python and Julia up until I got out of uni. I learned after a couple of months of insanity swapping kernels, init systems, distributions and learning everything about file systems only leads to further insanity and productivity hindrance.
Something something recommending someone who doesn’t know what a shell is to use emacs and make a Lua/Neovim config. Thanks for the tip!
Tbh, I’m always ending up having issues using poetry and conda. I prefer using penv and pip.
100% this. I remember really really trying to get the hang of them and eventually just giving up and doing it manually every time. I somehow always eventually mess something up or god forbid someone who isn’t me messes it up and I end up spending 4 hours dependency hunting. Venv and pip while still annoying are at least reliable and dead simple to use.
However, a container is now my preferred way of sharing software for at least the past 6 years.
Yup. A container i slow to rebuild, but at least the most robust. This is my preferred way to share python code when there are system dependencies involved.
This is why I use poetry for python nowadays. Pip just feels like something ancient next to Cargo, Stack, Julia, npm, etc.
Stack?
Used for Haskell development: https://docs.haskellstack.org/en/stable/
git pullbrewcurl | sudo bash
cat << EOF > main.c; gcc main.c -o main
What about CPAN?
You can’t even use it without the documentation of the program that you want to install because some dependencies have to be installed manually, and even then there’s a chance of the installation not working because a unit test would fail.
i will get hated for this but: cargo > composer > pip > npm
maven not on the list because they aren’t even playing the same sport.
What’s so bad about pip? Imho, the venv thing is really nice
vevn is not pip. The confusing set of different tools is part of the problem.
cough npm,yarn,grunt,esbuild,webpack,parcel,rollup,lasso,rollup,etc.,etc.cough
I’m not saying that Python’s packaging ecosystem isn’t complicated, but to paint JavaScript as anything other than nightmare fuel just isn’t right.
I don’t think that’s a fair comparison, the only two libraries that are related to the actual packaging system in that list is yarn and NPM. The rest of them have to do with the complexities of actually having your code runnable in the maximum number of browsers without issue. If python was the browser scripting language, it’d likely have the same issue.
Is there a python package that transpiles and polyfills python3 to work in python 2? 2.7? 2.5?
Also, unrelated to your comment, a lot of people are dunking on npm for the black hole that is node modules (which is valid), but also saying it’s not pip’s fault a lot of packages don’t work. It’s not npm’s fault the package maintainers are including all these dependencies, and there are some 0-dependency packages out there.
Is there a python 3 package that polyfills to Python 2?
Plenty of 'em. You ever heard of the
sixlibrary?
Well I guess you could say the tools that integrate with pip are really good, then
It’s not that confusing. There’s like 5 main different tools in total, what are you going to code if you can’t even set up the workspace? That’s much simpler than an installation that depends on cuda or spark, and those only require setting up environment variables after installation anyway.
As a programmer you’ll encounter several redundant libraries and tools in your life where each has an edge in some use cases and you’ll learn to use most to be able to adapt to the different projects you encounter, python’s package manager tools are simply one of those.
Of course there are more redundant tools. It doesn’t make pip suck any less though.
Just use poetry then
I genuinely don’t get the hate for pip, been using it for 5 years and never had an issue
the only time i’ve had issues with pip is when using it to install the xonsh shell, but that’s not really pip’s fault since that’s a very niche case and i wouldn’t expect any language’s package manager to handle installing something so fundamental anyways.
It’s all fun and games until the wheel variant you need for your hardware acceleration package conflicts with that esoteric math library you planned on using.
Isn’t this why you use venv?
This isn’t a pip issue though. Either these packages work together and the packaging is wrong, or they don’t work together.
The only thing I use pip for is to keep Proton-GE updated. ¯\_(ツ)_/¯
