Applications vs Libraries

Background on dependency hell¶

“Recently”, a tool called Poetry became popular with Python users. It made developing Python packages much easier, and made some strong decisions about user intent. One of these decisions was about package constraints — Poetry introduced upper bounds by default, e.g.

[project]
dependencies = [
   "numpy>=2.0,<3.0"
]

If the user is installing only your package, this isn’t a problem. But what about when users are installing lots of packages. As the number of packages $N\rightarrow \infty$, there will be many constraints. What happens when constraints overlap?

Certain tools like poetry and pdm may encounter incompatible constraints. When this happens, they will often try to use Back solving in order to find a version of the packages that doesn’t impose this constraint.

Back solving¶

When people write packages, they often start with “permissive” uncapped dependencies, e.g.

1
2
3
4
5
6
[project]
name = "arrow-to-knee"
version = "1.0.0"
dependencies = [
  "numpy >= 1"
]

Then, they discover that a new version of their dependency is incompatible with their package. So, they release a new version that introduces an upper cap:

1
2
3
4
5
6
[project]
name = "arrow-to-knee"
version = "1.0.1"
dependencies = [
  "numpy >= 1,<2"
]

If a user requires both arrow-to-knee and numpy>=2, what will happen? This problem is made worse by defensive caps that presume a new version of the dependency will be incompatible (i.e. before it’s known) to be the case).

Applications vs Libraries¶

This brings us back to the question of applications vs libraries.

The distinction between applications and libraries lies in how they’re installed. An Application should be installed by a tool manager like pipx or uvx, while an Library can be added as a dependency to other people’s pyproject.toml files.