openBLAS download with PowerShell failing, with no output

[thomassligon]

I am running a PowerShell script to install openBLAS on Windows
https://travis-ci.com/ICB-DCM/AMICI/jobs/275568973
I can see that the PS script amici/scripts/installOpenBLAS.ps1 runs and produces messages. However, there two issues:
1 (small): The environment variables created by PS are not visible to PS in later steps. However, creating them with BASH works, so I am using that now.
2 (critical): I can see no evidence that the directory C:\BLAS has been created or that the download and expansion of the zip file has worked. I can see that neither within the script nor later.
The final result is that the compiler cannot find cblas.h, so amici/src/cblas.cpp fails to compile.
amici\src\cblas.cpp(16): fatal error C1083: Cannot open include file: 'cblas.h': No such file or directory
In summary, the script is now running, thanks to fixing a typo in the Git BASH, but I can’t see the web client (System.Net.WebClient) or the unpacking (Expand-Archive). My guess is that this is something that Travis doesn’t support or it is a permissions issue on Windows. It works fine on my Windows PC.
The PS script is called in line 385, followed by a number of statements to test the results. It looks like this:
https://github.com/ICB-DCM/AMICI/blob/feature_msvc/scripts/installOpenBLAS.ps1

[native-api]

@thomassligon

In summary, the script is now running, thanks to fixing a typo in the Git BASH, but I can’t see the web client (System.Net.WebClient) or the unpacking (Expand-Archive)

Please update or (better, since there are no answers) rewrite your post to point to that problem, complete with the new build link and problem statement. Without seeing the build and knowing what you expect to be different, can’t say anything.

[thomassligon]

Thanks for the reply.
Nothing has changed since the post was written, so the links in the post are still current.
What I expect: The script should download the openBLAS file from the Internet and unpack it. That is what I meant when I said
I can see no evidence that the directory C:\BLAS has been created or that the download and expansion of the zip file has worked.
If you look at line 385 of the build, you can see the output of the script, consisting of the first and last lines (Write-Host), i.e. hello world, and everything involved with creating and displaying the environment variables. But I don’t see anything related to the web client or the Expand-Archive.
Perhaps more important, the next line in the build,
powershell -Command Get-ChildItem ‘C:\BLAS\OpenBLAS-0.3.6-x64’;
should display the contents of the unpacked openBLAS files. I don’t see anything there, either.
Finally, line 1827 of the build shows that the C compiler is failing to find an include file (cblas.h), which should be in the directory supplied to the compiler here:
/IC:\BLAS\OpenBLAS-0.3.6-x64\include
When I run the same script on my PC, I get
script installOpenBLAS.ps1 started

Directory: C:\

Mode LastWriteTime Length Name

---

d----- 11/23/2019 7:03 PM BLAS

PSPath : Microsoft.PowerShell.Core\Environment::BLAS_LIBS
PSDrive : Env
PSProvider : Microsoft.PowerShell.Core\Environment
PSIsContainer : False
Key : BLAS_LIBS
Value : /LIBPATH:C:\BLAS\OpenBLAS-0.3.6-x64\lib libopenblas.lib
Name : BLAS_LIBS

PSPath : Microsoft.PowerShell.Core\Environment::BLAS_CFLAGS
PSDrive : Env
PSProvider : Microsoft.PowerShell.Core\Environment
PSIsContainer : False
Key : BLAS_CFLAGS
Value : /IC:\BLAS\OpenBLAS-0.3.6-x64\include
Name : BLAS_CFLAGS

Directory: C:\BLAS\OpenBLAS-0.3.6-x64

Mode LastWriteTime Length Name

---

d----- 1/16/2020 9:51 PM bin
d----- 1/16/2020 9:51 PM include
d----- 1/16/2020 9:51 PM lib

Directory: C:\BLAS\OpenBLAS-0.3.6-x64\bin

Mode LastWriteTime Length Name

---

-a---- 8/14/2019 1:11 PM 32109225 libopenblas.dll

Directory: C:\BLAS\OpenBLAS-0.3.6-x64\include

Mode LastWriteTime Length Name

---

-a---- 8/14/2019 1:12 PM 46912 cblas.h
-a---- 8/14/2019 1:12 PM 47703 f77blas.h
-a---- 8/14/2019 1:12 PM 1271831 lapacke.h
-a---- 8/14/2019 1:12 PM 4388 lapacke_config.h
-a---- 8/14/2019 1:12 PM 474 lapacke_mangling.h
-a---- 8/14/2019 1:12 PM 33161 lapacke_utils.h
-a---- 8/14/2019 1:12 PM 4429 openblas_config.h

Directory: C:\BLAS\OpenBLAS-0.3.6-x64\lib

Mode LastWriteTime Length Name

---

d----- 11/23/2019 7:03 PM cmake
d----- 1/16/2020 9:51 PM pkgconfig
-a---- 8/14/2019 1:12 PM 40371644 libopenblas.a
-a---- 8/14/2019 1:13 PM 272719 libopenblas.def
-a---- 8/14/2019 1:12 PM 5416254 libopenblas.dll.a
-a---- 8/14/2019 1:12 PM 929762 libopenblas.exp
-a---- 8/14/2019 1:12 PM 1632678 libopenblas.lib

Directory: C:\BLAS\OpenBLAS-0.3.6-x64\lib\cmake

Mode LastWriteTime Length Name

---

d----- 1/16/2020 9:51 PM openblas

Directory: C:\BLAS\OpenBLAS-0.3.6-x64\lib\cmake\openblas

Mode LastWriteTime Length Name

---

-a---- 8/14/2019 1:13 PM 191 OpenBLASConfig.cmake
-a---- 8/14/2019 1:13 PM 284 OpenBLASConfigVersion.cmake

Directory: C:\BLAS\OpenBLAS-0.3.6-x64\lib\pkgconfig

Mode LastWriteTime Length Name

---

-a---- 8/14/2019 1:13 PM 580 openblas.pc

PSPath : Microsoft.PowerShell.Core\Environment::BLAS_CFLAGS
PSDrive : Env
PSProvider : Microsoft.PowerShell.Core\Environment
PSIsContainer : False
Key : BLAS_CFLAGS
Value : /IC:\BLAS\OpenBLAS-0.3.6-x64\include
Name : BLAS_CFLAGS

PSPath : Microsoft.PowerShell.Core\Environment::BLAS_LIBS
PSDrive : Env
PSProvider : Microsoft.PowerShell.Core\Environment
PSIsContainer : False
Key : BLAS_LIBS
Value : /LIBPATH:C:\BLAS\OpenBLAS-0.3.6-x64\lib libopenblas.lib
Name : BLAS_LIBS

PSPath : Microsoft.PowerShell.Core\Environment::BLAS_TESTW
PSDrive : Env
PSProvider : Microsoft.PowerShell.Core\Environment
PSIsContainer : False
Key : BLAS_TESTW
Value : Test2
Name : BLAS_TESTW

script installOpenBLAS.ps1 completed

[native-api]

@thomassligon

Nothing has changed since the post was written, so the links in the post are still current.

Okay, this was not obvious at all, it sounded like you solved the original problem and was having a completely different one, with another build. Thank you for an extensive explanation!

[native-api]

Now, the lack of output is (yet another) manifestation of

Stderr output is shown after the build log rather than inline

Reported in Python: os.system("git clone") does not shows any output from command. Related: Stderr tty appears later in log / visual studio build tools + vswhere. Test shows that stdout output is correctly displayed inline, but any stderr output is displayed at the end of the log, after the “Your build exited with…” message: $ which python; python --version python -c 'if True: import subprocess subprocess.check_call("echo bla-bla-bla", shell=True) subprocess.check_call("echo bla-b…

---

The error message from DownloadFile (and then from Expand-Archive etc) can be seen further below in the log (I found it searching the raw log for “DownloadFile” after checking locally what an error message from WebClient.DownloadFile looks like):

Exception calling “DownloadFile” with “2” argument(s): “The request was aborted: Could not create SSL/TLS secure channel.”

---

Looking up this error message shows that it’s usually solved by explicitly specifying TLS version(s) for .NET to use.

Alternatively, you can use a more user-friendly web client to download the file like curl, wget (which are preinstalled) or Python’s functionality.

[thomassligon]

Thanks! That fixes my problem, and the installation in now working.
Also thanks for your information about curl, wget, and Python. However, I will stick with PowerShell for now, because my task in this project is to do the installations on Windows, where PS is very much at home and very powerful.
By the way, the fact that I didn’t get the error on my Windows 10 client, whereas it did occur on your Windows server, indicates that your machine does not have the most modern default cryptography settings on it. After all, Microsoft must have had a reason to stop supporting TLSv1.
https://docs.microsoft.com/en-us/security/solving-tls1-problem