Infinite loop of virtualenv: Windows

YakDriver · July 10, 2019, 2:00pm

Python 3.7.4 + virtualenv + Windows results in an infinite loop of sadness.

In the following config, the source line (source $HOME/venv/Scripts/activate) is never reached because of the death, destruction, and mayhem (i.e., infinite loop, memory error, and timeout).

Config:

    - stage: test
      os: windows
      language: shell
      env: PATH=/c/Python37:/c/Python37/Scripts:$PATH
      before_install:
        - choco install python
        - python -m pip install virtualenv
        - virtualenv $HOME/venv
        - source $HOME/venv/Scripts/activate

Logs:

    $ virtualenv $HOME/venv
    You are using pip version 19.0.3, however version 19.1.1 is available.
    You should consider upgrading via the 'python -m pip install --upgrade pip' command.
    Running virtualenv with interpreter c:\python37\python.exe
    Running virtualenv with interpreter c:\python37\python.exe
    Running virtualenv with interpreter c:\python37\python.exe
    Running virtualenv with interpreter c:\python37\python.exe
    Running virtualenv with interpreter c:\python37\python.exe
    Running virtualenv with interpreter c:\python37\python.exe
    Running virtualenv with interpreter c:\python37\python.exe
    Running virtualenv with interpreter c:\python37\python.exe
    Running virtualenv with interpreter c:\python37\python.exe
    ...

native-api · July 9, 2019, 6:46pm

You can debug virtualenv by running it via bash -x and using a useful PS4 and request debug more for better efficiency.

YakDriver · July 10, 2019, 1:19am

The Windows shell doesn’t seem to support bash -x. This issue is only affecting our Windows jobs. Using virtualenv on OSX and Linux is working fine.

I’ve tried pipenv on Windows to create the virtualenv and we get the same. Infinite loop until timeout:

39.82s$ python -m pip install pipenv
Collecting pipenv
  Downloading https://files.pythonhosted.org/packages/13/b4/3ffa55f77161cff9a5220f162670f7c5eb00df52e00939e203f601b0f579/pipenv-2018.11.26-py3-none-any.whl (5.2MB)
Collecting certifi (from pipenv)
  Downloading https://files.pythonhosted.org/packages/69/1b/b853c7a9d4f6a6d00749e94eb6f3a041e342a885b87340b79c1ef73e3a78/certifi-2019.6.16-py2.py3-none-any.whl (157kB)
Collecting virtualenv-clone>=0.2.5 (from pipenv)
  Downloading https://files.pythonhosted.org/packages/ba/f8/50c2b7dbc99e05fce5e5b9d9a31f37c988c99acd4e8dedd720b7b8d4011d/virtualenv_clone-0.5.3-py2.py3-none-any.whl
Requirement already satisfied: pip>=9.0.1 in c:\python37\lib\site-packages (from pipenv) (19.1.1)
Collecting virtualenv (from pipenv)
  Downloading https://files.pythonhosted.org/packages/c4/9a/a3f62ac5122a65dec34ad4b5ed8d802633dae4bc06a0fc62e55fe3e96fe1/virtualenv-16.6.1-py2.py3-none-any.whl (2.0MB)
Requirement already satisfied: setuptools>=36.2.1 in c:\python37\lib\site-packages (from pipenv) (40.8.0)
Installing collected packages: certifi, virtualenv-clone, virtualenv, pipenv
Successfully installed certifi-2019.6.16 pipenv-2018.11.26 virtualenv-16.6.1 virtualenv-clone-0.5.3
174.26s$ pipenv shell
Creating a virtualenv for this project�
Pipfile: C:\Users\travis\build\YakDriver\scratchrelaxtv\Pipfile
Using c:\python37\python.exe (3.7.4) to create virtualenv�
Running virtualenv with interpreter c:\python37\python.exe
Running virtualenv with interpreter c:\python37\python.exe
Running virtualenv with interpreter c:\python37\python.exe
Running virtualenv with interpreter c:\python37\python.exe
Running virtualenv with interpreter c:\python37\python.exe
Running virtualenv with interpreter c:\python37\python.exe
Running virtualenv with interpreter c:\python37\python.exe
Running virtualenv with interpreter c:\python37\python.exe
Running virtualenv with interpreter c:\python37\python.exe
Running virtualenv with interpreter c:\python37\python.exe
Running virtualenv with interpreter c:\python37\python.exe
Running virtualenv with interpreter c:\python37\python.exe
...
[800-900 lines later]
...
Running virtualenv with interpreter c:\python37\python.exe
Running virtualenv with interpreter c:\python37\python.exe
FAIL
['Traceback (most recent call last):\n', '  File "c:\\python37\\lib\\site-packages\\pipenv\\utils.py", line 501, in create_spinner\n    yield sp\n', '  File "c:\\python37\\lib\\site-packages\\pipenv\\core.py", line 935, in do_create_virtualenv\n    extra=[crayons.blue("{0}".format(c.err)),]\n', 'pipenv.exceptions.VirtualenvCreationException: Failed to create virtual environment.\n']
[pipenv.exceptions.VirtualenvCreationException]:   File "c:\python37\lib\site-packages\pipenv\cli\command.py", line 390, in shell
[pipenv.exceptions.VirtualenvCreationException]:       pypi_mirror=state.pypi_mirror,
[pipenv.exceptions.VirtualenvCreationException]:   File "c:\python37\lib\site-packages\pipenv\core.py", line 2156, in do_shell
[pipenv.exceptions.VirtualenvCreationException]:       three=three, python=python, validate=False, pypi_mirror=pypi_mirror,
[pipenv.exceptions.VirtualenvCreationException]:   File "c:\python37\lib\site-packages\pipenv\core.py", line 574, in ensure_project
[pipenv.exceptions.VirtualenvCreationException]:       pypi_mirror=pypi_mirror,
[pipenv.exceptions.VirtualenvCreationException]:   File "c:\python37\lib\site-packages\pipenv\core.py", line 506, in ensure_virtualenv
[pipenv.exceptions.VirtualenvCreationException]:       python=python, site_packages=site_packages, pypi_mirror=pypi_mirror
[pipenv.exceptions.VirtualenvCreationException]:   File "c:\python37\lib\site-packages\pipenv\core.py", line 935, in do_create_virtualenv
[pipenv.exceptions.VirtualenvCreationException]:       extra=[crayons.blue("{0}".format(c.err)),]
[pipenv.exceptions.VirtualenvCreationException]: MemoryError
Failed to create virtual environment.
The command "pipenv shell" failed and exited with 1 during .
Your build has been stopped.

native-api · July 10, 2019, 2:05am

Sorry, I forgot that virtualenv is a Python script :-\
(It creates a bash activate script even in Windows, and Travis runs things via Git Bash, so if the problem was in activate, this would do.)

In that case, I would run virtualenv via <python> -m pdb or add some debug printing into site-packages\virtualenv.py around the place where the repeating line is printed.

YakDriver · July 18, 2019, 8:11pm

Looks like this is an issue with virtualenv and Python 3.7.4 running on Windows. (“Windows, why u do me that way?!”)

You can fix your config with this:

choco install python --version 3.7.3

YakDriver · July 18, 2019, 8:11pm

In Windows, it’s no longer necessary to pin Python at 3.7.3. The maintainers fixed the issue in virtualenv version 16.6.2, currently the latest release.