Hello.
I have an issue while running test in my project.
As one could notice I could successfully run mypy from command line from good working directory. But when I execute tests with python setup.py test it fails because of strange working direcotry.
What could be wrong?
Thank you.
My initial guess is that something in your setup is changing the working directory to /tmp/*, since without anything extra, os.getcwd() returns $TRAVIS_BUILD_DIR; e.g., https://travis-ci.org/BanzaiMan/travis_production_test/jobs/553256620#L176-L177
FWIW using setup.py test is highly discouraged, you should invoke the test runner directly.
As for the error, I’m pretty sure some of your tests creates a temporary directory, os.chdir()s there and doesn’t chdir back to the original directory after the test completion so the subsequent tests use with that temporary directory (because chdir syscall affects the whole process).
@BanzaiMan, @webknjaz thank you very much. I really do os.chdir() in code under test. Though it worked in my local environment. So I don’t know why this doesn’t work in Travis CI.
FWIW using
setup.py testis highly discouraged, you should invoke the test runner directly.
Why?
setup.py test is the default way to test a package, and should always work. It may not be using the best test runner, or may not even run all of the tests, but this is the best way to ensure that the runtime and test dependencies inside setup.py are correct and that packagers pass on those dependencies to end users.
The most common source of packaging bugs is when the dependencies change in the codebase, but a) setup.py install_require or test_requires are not updated, and/or b) the .spec ran the pytest and it didnt fail on the newly released tarball because the tests were skipped because they code maintainers intended to de-support old dependencies without a hard breakage, and the RPM maintainer didnt notice that the skips in the build log, or notice that minimum requirements had changed via setup.py or a softer mechanism like a note in README.
I recommend having one job which acts more like a user(and/or packager) than a developer, and uses setup.py install and setup.py test. Also build the wheel in CI, and then try to install and run tests against the wheel installed version (this is what tox does by default under the covers). Maybe make that job non-voting (and/or master only) so-as to not burden newbies with maintaining it, and the project maintainers can fix it before a release.
The grounds for that seem flimsy at best so it’s not a given if you ask me.
and uses
setup.py install
This is even more strongly discouraged. Users must never do this. It breaks installed resources tracking and pip is unable to uninstall it properly.
Why?
Historically it didn’t gain popularity and didn’t become a standard way for running tests. Current ecosystem maintainers (which is effectively PyPA) have come to conclusion that it’s bad to have some arbitrary non-standardized install-time chunk of code. There’s PEP517 which addresses this problem better, even though implementations aren’t perfect it’s worth talking a look. If you want to follow the latest discussions, go to https://discuss.python.org under Packaging section. There’s a lot of interesting stuff going on + some threads are follow-ups on the Packaging Mini-Summit we’ve had at PyCon in May.
AFAICS this PEP’s implementation is incomplete and/or experimental so it’s a bit premature to say that it “solves” anything and declare all the other things “strongly discouraged”.
I didn’t say that all other things are discouraged. I just explicitly listed those and explained why. setup.py test is discouraged since about 3–4 years ago. setup.py install is misleading because when pip installs dists it patches things to track all files which get installed, while setuptools just dump things to folders with no way to track what was installed. So for a long time pip uninstall used to report successful uninstallation but what was really happening is that it just removed some metadata files, not actual artifacts. Recent versions of Pip now explicitly inform user that it cannot actually uninstall those. That’s why it’s not recommended to mention setup.py install anywhere. It can still be used for packaging, but not for installation.
And I also explicitly said that PEP 517 is not completely implemented. There’s no in-tree build backends. And editable mode is intentionally omitted because of corner cases but this was discussed at the Packaging Mini-Summit where folks agreed on a solution and making a prototype.
You are confusing setuptools with distutils. setuptools does create metadata that pip can use. It’s another matter that it installs things as a compressed .egg by default unless zip_safe=False is passed to setuptools.setup()…
Yes, setuptools creates metadata, but there was something incomplete about it. I don’t remember what exactly.
This page has moved - pip documentation v23.3.1 only lists:
pip is able to uninstall most installed packages. Known exceptions are:
- Pure distutils packages installed with
python setup.py install, which leave behind no metadata to determine what files were installed.- Script wrappers installed by
python setup.py develop.
develop is an exception rather than a rule so I wouldn’t count it as a serious drawback.