Skip to content

Dependency resolution when pip install-ing a poetry managed package #4025

Description

@Lalufu
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • I have searched the documentation and believe that my question is not covered.

Issue

I have a question regarding dependency resolution in poetry when using pip install to install a package that is managed by it. I did not find a clear answer to this in the documentation.

My understanding of dependency resolution in poetry is the following:

  • In pyproject.toml I define my dependencies for development and runtime
  • These dependencies can be in caret form, which gives some leeway in the actual version that's picked. The caret form is the default when using poetry add.
  • poetry install will take all these dependencies, build the tree, find a solution that satisfies all the different direct and indirect dependency requirements, installs those packages in the versions determined, and writes the chosen version numbers into poetry.lock. poetry.lock should be version controlled, so that multiple people working on the same project get the same versions of dependencies.
  • Updates to dependencies require an explicit poetry update, which will reevaluate the tree and update poetry.lock, if required.

So far this is clear to me and makes sense.

However, when a poetry managed package is installed via pip install, what seems to happen is that the pip install process runs another dependency resolution process, looking only at pyproject.toml, and not at poetry.lock. This is even though build-backend = "poetry.core.masonry.api" is present in the pyproject.toml file. This might lead to pip install picking different dependency versions than poetry install did. I would expect pip install with the poetry build backend to look at poetry.lock to make sure end users get the exact same dependency versions that the package was developed and tested with.

  • Is my observation correct that pip install does not consult poetry.lock?
  • Is that intentional?
  • Is there a way to change this?

Activity

  1. finswimmer commented on Jun 4, 2021

    @finswimmer
    Member

    Hello @Lalufu,

    Is my observation correct that pip install does not consult poetry.lock?

    Yes.

    Is that intentional?

    Yes, pip calls the build backend as described in PEP 517. The result is the wheel file you will get if you run poetry build -f wheel on your project. The wheel file contains the general version ranges for the dependencies.

    Is there a way to change this?
    Not yet. If PEP 650 will ever be accepted it might be possible that pip or another tool interact as the universal installer, which then initiate a poetry install.

    Another possible solution that might come up soon, is a poetry plugin that can build a wheel with pinned version. Than you can pip install this wheel.

    fin swimmer

  2. locked and limited conversation to collaborators on Jun 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions