Description
- On TravisCI macOS images, when running Python3 via the
language: Bashcommand, the Travis terminal misreports the size of the terminal window. - This can be replicated using macOS system Python3 and Conda installed versions
- The issue cannot be replicated on a locally run macOS with the same build parameters
- This incorrect terminal window size reporting behaviour is inconsistent across Travis’ Windows, Linux and macOS build environments
Lightweight example
When running TPUT (Unix standard for getting terminal size):
tput cols
The output is correct:
80
When running python shutil.get_terminal_size() on a Travis macOS environment, from a script called terminal_size.py:
import shutil
print(shutil.get_terminal_size())
and calling this like this:
python3 -m terminal_size
The output is incorrect:
os.terminal_size(columns=0, lines=0)
this doesn’t occur on Travis CI Windows builds or Linux builds, where the output is as expected:
os.terminal_size(columns=80, lines=40)
When running on a local macOS MacBook Pro, with the same config as the Travis Mac Pipeline, the output is the same as the Travis CI Windows and Linux Builds. This confirms my understanding that the error is to do with the macOS Image Configurations on Travis.
Why this is important
This error can result in inconsistent behaviours across OS Images and can break doctest.
Consider a Sphinx Doctest example where the below is inside an index.rst file in the documentation:
.. testcode:: quick_start
import pandas as pd
df = pd.DataFrame({
"column1": [1, 4, 0, 10, 9],
"column2": [-1.3, -1.4, -2.9, -10.1, -20.4],
"column3": ["value_1", "value_2", "value_3", "value_2", "value_1"],
})
print(df)
.. testoutput:: quick_start
column1 column2 column3
0 1 -1.3 value_1
1 4 -1.4 value_2
2 0 -2.9 value_3
3 10 -10.1 value_2
4 9 -20.4 value_1
this will cause a doctest failure only on macOS only because when a terminal window is too small, pandas will only show the first and last columns, while the central column is swapped for ellipsis inside pandas’ repr:
python -m sphinx -E -W -b=doctest "docs/source" "docs/_build"
Running Sphinx v2.3.0
...
Document: index
---------------
**********************************************************************
File "index.rst", line 83, in quick_start
...
print(df)
Expected:
column1 column2 column3
0 1 -1.3 -0.3
1 4 -1.4 2.6
2 0 -2.9 -2.9
3 10 -10.1 -0.1
4 9 -20.4 -11.4
Got:
column1 ... column3
0 1 ... -0.3
1 4 ... 2.6
2 0 ... -2.9
3 10 ... -0.1
4 9 ... -11.4
i.e. the incorrect reporting of terminal size causes pandas to print incorrect ellipsis, which fails doctest. This is a false positive failure.
Detailed Examples
To help provide a further example of the problem I have provided code samples and links to Travis Pipelines.
Please note: both examples cover macOS, Linux and Windows because the point is to show the inconsistency across systems.
Detailed Example 1 - Using macOS System Python3
language: python
# ===== Linux ======
dist: xenial
python:
- 3.5
- 3.6
- 3.7
matrix:
include:
# ======= OSX ========
- name: "Python 3.6.5 on macOS 10.13"
os: osx
osx_image: xcode9.4 # Python 3.6.5 running on macOS 10.13
language: shell # 'language: python' is an error on Travis CI macOS
before_install:
- python3 --version
- pip3 install -U pip
- pip3 install pandas sphinx
script: |
echo "terminal columns reported by TPUT:"
tput cols
echo "terminal columns reported by python shutil get_terminal_size:"
python3 -m terminal_size
python3 -m sphinx -E -W -b=doctest "docs/source" "docs/_build"
You can see in action here and the overall pipeline showing Linux and Windows successes is here.
Detailed Example 2 - Using macOS with conda installed
language: bash
cache:
directories:
- $HOME/miniconda3
before_cache:
- rm -rf $CONDA_DIR/pkgs/cache
- rm -rf $CONDA_DIR/envs/hosts
- rm -rf $CONDA_DIR/conda-meta/history
- touch $CONDA_DIR/conda-meta/history
os:
- linux
- osx
- windows
env:
- PYTHON_VERSION="3.5"
- PYTHON_VERSION="3.6"
- PYTHON_VERSION="3.7"
before_install:
- |
export CONDA_DIR="$HOME/miniconda3"
if [ "${TRAVIS_OS_NAME}" == "windows" ]; then
export CONDA_DIR=`cygpath -w $CONDA_DIR`
export CONDA_BIN_DIR="$CONDA_DIR\scripts"
else
export CONDA_BIN_DIR="$CONDA_DIR/bin"
fi
export PATH="$CONDA_BIN_DIR:$PATH"
install:
# Install Conda
- |
if [ "${TRAVIS_OS_NAME}" == "windows" ]; then
choco install openssl.light
fi
if [ -d "$CONDA_DIR" ] && [ -e "$CONDA_BIN_DIR/conda" ]; then
echo "Miniconda install already present from cache: $CONDA_DIR"
rm -rf $CONDA_DIR/envs/hosts # Just in case...
else
echo "Installing Miniconda..."
rm -rf $CONDA_DIR # Just in case...
if [ "${TRAVIS_OS_NAME}" == "windows" ]; then
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Windows-x86_64.exe -O miniconda.exe || exit 1
cmd //c "start /wait "" miniconda.exe /S /D=$CONDA_DIR"
else
if [ "${TRAVIS_OS_NAME}" == "osx" ]; then
wget https://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-x86_64.sh -O miniconda.sh || exit 1
elif [ "${TRAVIS_OS_NAME}" == "linux" ]; then
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh || exit 1
fi
bash miniconda.sh -b -p "$CONDA_DIR" || exit 1
fi
fi
echo "Configuring Miniconda..."
conda config --set ssl_verify false || exit 1
conda config --set always_yes true --set changeps1 false || exit 1
echo "Updating Miniconda..."
conda update conda
conda update --all
conda info -a || exit 1
# Setup Conda Env
- |
echo "Creating a Python $PYTHON_VERSION environment..."
conda create -n hosts python=$PYTHON_VERSION || exit 1
source activate hosts
conda install pandas sphinx
echo "terminal columns reported by TPUT:"
tput cols
echo "terminal columns reported by python shutil get_terminal_size:"
python -m terminal_size
python -m sphinx -E -W -b=doctest "docs/source" "docs/_build"
Conclusion
Please let me know if you need more information, or if you think there’s a mistake in the above. I’ve tested thoroughly in multiple different setups/environments and believe that the terminal width is being reported incorrectly on Travis macOS alone.