Dbus-python Dependency (any version) does not work

Environments
, ,
1

Hi,

I recently added a small feature to my program (apprise) that requires the dbus library to be present for unit testing. I’ve see this issue which seems to define my problem, but the OP never returned to say whether or not he did to solve the problem. There is also reference to a StackOverflow post (from another related thread you guys handled) that doesn’t appear to exist any more.

There are numerous builds failing on me because dbus-python can’t be included/loaded (Python 2.7, Python 3.4, and Python 3.7. I’m using more versions of python, but those 3 seem to show all of the error variants.

Based on what I’ve found, here is what the .travis.yml file looks like. I added the sudo line, the before_install line and virtualenv just recently (without any luck).

language: python
sudo: required

matrix:
  include:
    - python: "2.7"
      env: TOXENV=py27
    - python: "3.4"
      env: TOXENV=py34
    - python: "3.5"
      env: TOXENV=py35
    - python: "3.6"
      env: TOXENV=py36
    - python: "3.7"
      dist: xenial
      env: TOXENV=py37
    - python: "pypy2.7-5.8.0"
      env: TOXENV=pypy
    - python: "pypy3.5-5.8.0"
      env: TOXENV=pypy3


before_install:
  - sudo apt-get update -qq
  - sudo apt-get install -qq python-dbus python3-dbus


install:
  - pip install .
  - pip install codecov tox
  - pip install -r dev-requirements.txt
  - pip install -r requirements.txt


# run tests
script:
  - tox


after_success:
  - tox -e coverage-report
  - codecov


virtualenv:
  system_site_packages: true


notifications:
  email: false

Any advice or help would be fantastic! :slight_smile:

2

Hi @caronc,

Thanks for opening this thread and sorry for the hassle.

Could you please also share a link to one of the builds that were failing before including the virtualenv approach you mention, so that we can check the original error as well?

In the meantime, I noticed at Issues with dbus that someone suggested installing dbus directly with pip. Is this something you tried already?

pip install dbus-python

Looking forward to hearing back from you!

3

I can just address your first part at this time (about sharing the link to the builds failing). I’ll update my requirements.txt and remove the dbus-python from there and add it directly into the travis.yml later this evening when I get home.

for the second part: Sure!

Failure #1:
My first change was to just build with what i had (no reference to dbus-python anywhere or ubuntu packages for that matter). This was destin to fail, but hey; got to start somewhere:
travis.yml / Travis CI Build Failures

Failure #2:

  • added dbus-python to requirements.txt

But I can see how this isn’t a good idea now (while writing this). So i’ll move the pip install reference into the travis.yml as a direct call (as per your suggestion) tonight! But here is the output none the less:
travis.yml / Travis CI Build Failures

Failure #3:

  • added the virtualenv directive to travis.yml

travis.yml / Travis CI Build Failures

Failure #4:

  • added the before_install directive to travis.yml

travis.yml / Travis CI Build Failures

Failure #5:

  • added the sudo: required directive to travis.yml

travis.yml / Travis CI Build Failures

Thoughts?

4

@iriberri, i took your advice and directly added the pip install dbus-entry and it unfortunately failed again. :slightly_frowning_face:

Failure #6:

Back to you!

5
6

Thanks for your reply.

You’ll have to forgive my ignorance, but i’m still quite new to Travis-CI; most of what I’ve learned was based on other examples throughout GitHub. But i’m very confused by your response as i don’t think it helps me in any way. But i’ll do my best to explain because i’m still interested in any further feedback you might have:

So you need to migrate to dist: xenial. Or use an older version of dbus-python.

I honestly don’t care what version of dbus I have installed. I just need ‘a’ Python dbus library present per version of Python. Shouldn’t Travis CI allow me to do this?

I also don’t define anywhere that dbus needs to be v1.8 or higher or even lower for that matter. I presume it’s getting this dependency value from somewhere inside the container? Maybe there is a bug/issue here?

I would think if changing over to use Xenial was a solution, then the one branch I have already set up to use it (Python v3.7) would have succeeded (and the rest would have still failed). But even the Xenial environment fails too.

Also the link I gave earlier says that you shouldn’t run with system site packages.

I’m completely fine not using site_packages… Again, just to be clear it was only part of my trial and error experience to get this dbus library to install. Just like virtualenv, system_site_packages didn’t work or solve my problem, so it’s been removed. You can see this in the latest Failure #6 i shared above which doesn’t reference either of these directives.

I need the system site packages since my application depends on a lot of libraries installed with apt-get .

From this point on, you’re quoting someone else’s problem and the response you gave them. I don’t need system_site_packages and have removed the reference to it.

My problem is: I just need the dbus Python library (regardless what version it is) installed in my Travis CI containers for each Python version i’m testing against. :slightly_smiling_face:

Thoughts?

7

In the latest failure you report, I see this:
https://travis-ci.org/caronc/apprise/jobs/490298221#L674-L679

  configure: error: Package requirements (dbus-1 >= 1.8) were not met:
  
  No package 'dbus-1' found
  
  Consider adjusting the PKG_CONFIG_PATH environment variable if you
  installed software in a non-standard prefix.

How does it work on your development machine?

It is not clear to me what package dbus-python is looking for, but I’d assume something like Ubuntu package; e.g.,

https://packages.ubuntu.com/trusty/dbus

I don’t think this is pre-installed on our images, so you’ll have to install that.

8

A minimal configuration that seems to work is:

dist: xenial
addons:
  packages:
    - libdbus-1-dev
install:
  - pip install dbus-python

https://travis-ci.org/BanzaiMan/travis_production_test/builds/490777540#L199

1 Like
9

@BanzaiMan you are amazing! We’re sooo close all thanks to you! I really appreciate your insight here! :clap: :trophy:

Failure Build #7 only fails to build a pypy and pypy3 build now. But the previous 5 build beautifully! :grinning:

At this point I could drop support for the pypi builds, but i’d be happier keeping them if I could. There errors now have flipped to failing (because of dbus errors) to just not having the ability to build at all (no package to fetch).

Can I poll all of you for more advice just close off this issue for good? :pray:

In the meantime, at least i’ve got a new error; so I’ll also Google around and make myself useful!

Edit: Just to answer your other question:

How does it work on your development machine?

I use CentOS 7.x; so python2-dbus (for python v2.7) and python3-dbus are indeed installable packages. But like any other distribution, CentOS builds their packages straight from PyPI so it wouldn’t be much different then their equivalent deb packages (for Ubuntu). Or even for those who just use pypi directly (assuming they have a compiler and dependencies installed) who install it on the fly.

Edit 2: I found this link which at least lets me fetch the updated packages, but i still fail to build on dbus related issues.
Failure Build #8 :slightly_frowning_face:

10

There are TypeErrors in your build logs. I suggest you track them down. Can you compile dbus-python with PyPy on your machine?

11

@BanzaiMan, The TypeErrors are being issued from something outside of my product. the Travis-CI modules just don’t want to seem to build dbus-python in these 2 specific environments.

BUT; it’s all good! I’m okay with this because all of the coverage for this one module is already met with the previous build environments that are running.

After about 10+ build failures later (more trial and error) which included:

  • a pytest.skip() call i added to one my dbus-python unit test files (for the pypi environment)
  • changes to the tox.ini and travis.yaml to just avoid dealing with the dbus packaging completely (again… for pypi environment only)

I got everything to build! :tada:

The final travis.yaml file looks like this:

language: python

dist: xenial

addons:
  apt:
    packages:
      - libdbus-1-dev

matrix:
  include:
    - python: "2.7"
      env: TOXENV=py27
    - python: "3.4"
      env: TOXENV=py34
    - python: "3.5"
      env: TOXENV=py35
    - python: "3.6"
      env: TOXENV=py36
    - python: "3.7"
      env: TOXENV=py37
    - python: "pypy2.7-6.0"
      env: TOXENV=pypy
    - python: "pypy3.5-6.0"
      env: TOXENV=pypy3

install:
  - pip install .
  - pip install codecov tox
  - pip install -r dev-requirements.txt
  - pip install -r requirements.txt
  - if [[ $TRAVIS_PYTHON_VERSION != 'pypy'* ]]; then travis_retry pip install dbus-python; fi

# run tests
script:
  - tox

after_success:
  - tox -e coverage-report
  - codecov

notifications:
  email: false

I learned that I could do conditional install inclusions based on the environment by just looking at how tornado structured their travis.yaml file. I took this idea and tweaked it to fit my pypi environment issue and it seemed to do the trick.

I couldn’t have done any of this without your help though! So thank you very much @BanzaiMan!! But thanks to everyone else who took the time to reply; the advice helped me look in other directions too! :slight_smile:

You guys rock!

12

Great to hear you were able to resolve the issue!

p.s. Please consider dropping ‘guys,’ since not all of us are. :slight_smile: