No space left on device for System Z

Worker information:

hostname: 48be7aa6-189e-4dce-be1e-db8186466c1f@1241.lxd-s390x-travis-ci-production-1-worker5-com
version: ? ?
instance: travis-job-radareorg-radare2-256856420 a5448d3ad7ae521a8c2821af8b0f9c020dda03de8393b95fd6081ad948bac7a3 (via amqp)
startup: 2.519641297s

Build system information

Build language: generic
Build group: stable
Build dist: trusty
Build id: 136642533
Job id: 256856420
Runtime kernel version: 5.3.0-19-generic
travis-build version: 119feb2eb

See https://travis-ci.com/radareorg/radare2/jobs/256856420#L158 for the full log

ar: lib/libgdbr.a: No space left on device
make[2]: *** [lib/libgdbr.a] Error 1
cc1: fatal error: can’t open ‘/tmp/ccpypDQn.s’ for writing: No space left on device
compilation terminated.
Cannot create temporary file in /tmp/: No space left on device
make[2]: *** [src/packet.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [src/sigutil.o] Aborted (core dumped)
ar: creating libr_ar.a
kern/env.c:257:1: fatal error: closing dependency file kern/env.d: No space left on device
 }
 ^
compilation terminated.
cc1: fatal error: can’t open ‘/tmp/ccr6rkP5.s’ for writing: No space left on device
compilation terminated.
make[2]: *** [kern/env.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [kern/disk.o] Error 1
Cannot create temporary file in /tmp/: No space left on device
make[2]: *** [src/ptrace_wrap.o] Aborted (core dumped)
make[1]: *** [preall] Error 2
make: *** [all] Error 2
+exit 1

Hi @XVilka, glad to see that you’re using s390x.
I updated the storage pool.
Apologies for the inconvenience and please let us know if you do run into any issues.
Happy building!

2 Likes

Thank you for the prompt answer! I confirm, it was fixed.

I am seeing this issue again:

The disk space is low on s390x: https://travis-ci.org/MarcoFalke/bitcoin-core/jobs/616893865#L77

There is plenty on arm64 or amd64: https://travis-ci.org/MarcoFalke/bitcoin-core/jobs/616893862#L117 or https://travis-ci.org/MarcoFalke/bitcoin-core/jobs/616893866#L182

@Damian I am still seeing this error. See for example here:

https://travis-ci.org/bitcoin/bitcoin/jobs/625880747#L8181

which shows that only 9% disk was left (9GB):

https://travis-ci.org/bitcoin/bitcoin/jobs/625880747#L96

I presume this 9GB is shared among travis users running on that machine?

Hi @Marco
Do you still see “Error: Disk space is too low”?
in df please look at instance storage e.g “/var/snap/lxd/common/lxd/storage-pools/instances/containers/travis-job-bitcoin-bitcoin-625880747/rootfs”

1 Like

Hi @Damian

thanks for following up. I can confirm that the instance storage is large enough to run the full build. However, we are running the build inside of docker. Our ci scripts are designed to run (also) locally and for safety mark the build folder read-only. Modifications only happen in the docker container.

So I believe the issue here is that docker does not put its data files in the s390x travis instance storage. According to https://beta.docs.docker.com/engine/reference/commandline/dockerd/#miscellaneous-options this can be changed on startup of the docker daemon.

Sometimes it is not even possible to start docker, because

write /var/lib/docker/tmp/GetImageBlob036386674: no space left on device

https://travis-ci.org/bitcoin/bitcoin/jobs/641563767#L138

Imprint