Python2.7 has been replaced with 3.7 by default

Languages Python
1

Hi there,

I’ve noticed that Travis CI build in KhronosGroup/SPIRV-LLVM-Translator has started to fail recently.

Looks like something has happened in the environment.
Here is the link to the last successful build: https://travis-ci.org/KhronosGroup/SPIRV-LLVM-Translator/jobs/556046035#L629
And here is the link to the first unsuccessful build: https://travis-ci.org/KhronosGroup/SPIRV-LLVM-Translator/jobs/557055315#L629

By some reason, python3.7 was found instead of 2.7 and it was identified by cmake as “1.4” and that caused an error.

.travis.yml explicitly sets “xenial” environment and according to the docs I would expect python 2.7 to be default.

Could you please help me to investigate what is going on?

2

– Found PythonInterp: /opt/pyenv/shims/python3.7 (found version “1.4”)

The default Python version is the one available as python for language: python if you don’t provide a python: clause. Your language is cpp so this doesn’t apply to you. (I’ve provided a PR with documentation improvement that better explains what is available where.)

/opt/pyenv/shims/python3.7 has nothing to do with that. That directory has pyenv shims for all the custom Python versions preinstalled (they are listed in the doc). Why CMake picks that instead of something more fitting is CMake’s problem.

You can try selecting preinstalled 2.7.15 as python with pyenv local 2.7.15 before calling CMake. Or debug CMake’s logic by running it with --trace or --trace-expand (lots of output!).

3

Hi @native-api, thanks for you suggestions!

I’ve used cmake --trace to check why python 3.7 was identified as 1.4. Here is the full log if you are interested, but the thing is that cmake tries to launch the following commands in order to extract python version:

  • ${PYTHON_EXECUTABLE} -c import sys; sys.stdout.write(';'.join([str(x) for x in sys.version_info[:3]]))
  • ${PYTHON_EXECUTABLE} -c import sys; sys.stdout.write(sys.version)

Looks like both of them fails and cmake decides that python is pretty old.

So, I’ve tried to launch the same commands by myself to see what’s happening. Here is the results:

Looks like /opt/pyenv/shims/python3.7 command is not available anymore if certain python version wasn’t specified explicitly via pyenv

I think this might be considered as solved (or at least work-arounded), but actually, it would be good if /opt/pyenv/shims/python3.7 wasn’t there at all since it doesn’t work by default.
I haven’t investigated why cmake has selected exact this path to see if it is possible to change its behavior. However, this might not be easy, since FindPythonInterp module hasn’t launched directly by cmake from KhronosGroup/SPIRV-LLVM-Translator, but it is launched from CMakeLists.txt from another project - we have to update it there to fix our problem.

4

The output you got – pyenv: python3.7: command not found – is the expected one for the default pyenv configuration – which is “system” selected, as there’s no system-provided python3.7 in Xenial. So no bug here.

This is a problem with CMake – or the way you are using it.

As I can see from the trace, it looks through existing executables in the form pythonX.Y, from the highest version down. But it doesn’t check for error when running import sys; <etc> but blindly sets the version to “1.4” and returns that instead of trying further. (That might be justified actually if Python didn’t have sys.version before 1.5. But the fact they don’t also check if the exit code is 1 is still a bug in my book – command not found produces code 127 (…okay, drawback :slight_smile: . They clearly didn’t anticipate pyenv's machinery when writing that.))

AFAIK it’s possible to instruct CMake to find a Python version in the specific range. Check out the opencv project how they do that. This way, any unusable executables will be bypassed (unless you provide a version range that includes “1.4” :wink: ).

Note that you still need to select a preinstalled version with pyenv if you wish to use one of those instead of the distro-provided ones.

5

How is your build looking for the Python executable? On the Xenial image with language: cpp, you’d find

$ which python && python --version
/opt/pyenv/shims/python
Python 2.7.12
$ which python3 && python3 --version || true
/opt/pyenv/shims/python3
Python 3.5.2
$ /usr/bin/python --version || true
Python 2.7.12
$ /usr/bin/python3 --version || true
Python 3.5.2
6

@native-api, @BanzaiMan,

As I’ve said, KhronosGroup/SPIRV-LLVM-Translator doesn’t use python directly, it uses it via llvm/llvm-project using this code.

I definitely will not be able to leave only one version there.

As I can see from the trace, it looks through existing executables in the form pythonX.Y , from the highest version down. But it doesn’t check for error when running import sys; <etc> but blindly sets the version to “1.4” and returns that instead of trying further.

This code comes from built-in CMake module, so it is also not so easy to change it :slight_smile:

Note that you still need to select a preinstalled version with pyenv if you wish to use one of those instead of the distro-provided ones.

Okay then, I will configure the environment to make python3.7 available and work-around this problem.

1 Like