Support for Go 1.11+ modules was recently introduced (by me, mostly), and a problem has been raised around the presence of a go.mod file when a given project also needs to assume that the working directory is in ${GOPATH%%:*}/src/[go-import-path]. I would like to request anyone currently affected by this problem to please offer suggestions for desired behavior so that we can be sure to accommodate the needs of most projects.
This has broken most of the github.com/gobuffalo projects that use GOPATH. I would expect the behavior to mimic that of GO111MODULE=auto.
Case “auto”:
- go.mod is ignored and GOPATH is used
Case “on”: - go.mod is used
Case “off”: - go.mod is ignored.
Some example broken tests are here:
- https://travis-ci.org/gobuffalo/buffalo/jobs/489246239
- https://travis-ci.org/gobuffalo/packr/jobs/489238614
Both of those look at the GOPATH env to make certain checks and derive certain information about where the user is and what they are doing.
3 Likes
That would be my expectation too.
The explicit export GO111MODULE=on has broken rclone’s tests
rclone uses go mod to manage dependencies so there is a go.mod file, however we use go mod vendor to vendor all the dependencies so we don’t want GO111MODULE=on.
The cross compile build fails with GO111MODULE=on for reasons I’m not 100% sure of…
Looking at the changes I see that the presence of a go.mod is what caused GO111MODULE=on.
I think since go modules are still experimental that this probably shouldn’t happen for the moment.
1 Like
I’d like a way to disable the setting of GO111MODULES=on
Unfortunately setting GO111MODULES=off in env: doesn’t work 
As a next step, I would like to propose removing any explicit GO111MODULE variable setting and allow the existing environment (possibly as specified in one’s .travis.yml) to take precedence.
1 Like
I think that would be sensible at this moment in time.
Go modules are still experimental - allowing the user to turn them on in the env seems the best way for the moment. That may change in the future as they get more stability and more adoption. There needs to be a way to turn them off too!
1 Like
I would like to request this be rolled back until it’s fixed. It’s breaking a lot of builds out there right and has ground my work to halt. Please.
@markbates Yes, I very much understand. What I’m preparing right now is not a strict rollback, but my intent is the same.
This is the change that I intend to promote, which ensures that existing ${GO111MODULE} values are respected and retains the syncing of the git clone into the go import path: https://github.com/travis-ci/travis-build/pull/1651
The branch from this PR is currently deployed to production. Please provide feedback at your earliest convenience on whether or not the behavior meets your expectations. https://github.com/travis-ci/travis-build/pull/1651
@meatballhat this fixes non-modules builds, but those that have modules on are still failing because of the GOPATH not being correct.
For example we have integration tests where we disable modules to do tests in the GOPATH. This is failing to set the GOPATH correctly, or it’s not placing code into the GOPATH as it has before.
Verdict: still really broken.
Error Trace: core_test.go:36
Error: Received unexpected error:
currently not in a $GOPATH
github.com/gobuffalo/buffalo/genny/newapp/core.init
/home/travis/build/gobuffalo/buffalo/genny/newapp/core/errors.go:9
Notice the code isn’t properly placed inside the GOPATH.
I’m not sure this can be automated in a way that satisfies everyone’s needs. The go import path sync is being performed for all go jobs, as well as the setting of ${GOPATH}. The only difference at this point is whether or not the working directory is set to the original git clone location or the copied source tree within ${GOPATH%%:*}/src.
Said another way, the go module docs describe code location in a way that implies it’s a decision that the reader makes. I’m not sure what the correct decision is for the code location, so I’m ensuring the same code is in both the “normal” clone location and the go import path, but there remains the question of where the working directory should be prior to handing control to whatever is in one’s .travis.yml.
It used to work until yesterday. The could should be placed into /home/travis/build/src/github.com/gobuffalo/buffalo/genny/newapp/core/errors.go. Placing code in the GOPATH correctly works with/without mods. The end user should be in charge of whether their builds use modules or not.
Do I understand correctly that you believe any project that intends to use go modules should be responsible for setting GO111MODULE=on?
Yes. They’re experimental and should be up to the end user to opt-in to use them.
The behavior you describe sounds fine to me. As you may have guessed, the original intent of the change was to try to guess at the project’s intended behavior, but clearly that hasn’t gone well.
A lot of projects need to support GOPATH and Modules right now, so we need to be able to control that. That’s why, for example, the buffalo projects have 4 go1.11 builds. Windows (mods on/off) and Linux (mods on/off). Everybody has to make sure their codes work in both environments for now.
2 Likes
I just deployed the latest from the previously mentioned pull request. Can you confirm current behavior matches your expectations?
1 Like
So far so good. I’ll post again if there’s an issue. It takes a while to run those builds.
1 Like
Thanks @meatballhat for fixing that so quickly.
I should give a little background about why that broke us so badly. As part of our build process, we perform a set of code mutations on our tree after all the tests have passed to ensure that code generation has been performed by contributors (this is to make up for the absence of easy server-side hooks in git/any way to ensure pre-commit hooks enforced on contributors - it’s nasty, but it works … most of the time). The upshot of this is that we need to know where our repo has landed on the build system, something that GOPATH has made easy in the past.
The breakage did give me some valuable information about brittle code generation that I have fixed in the PR to address this problem that I posted in my original report, so that is a good thing.
My suggestions about how to deal with this in the longer term is that there be a YAML key for GO111MODULE, making it explicit in the .travis.yml, GOPATH be set independent of Go version and $GO111MODULE, and code be placed in the normal $GOPATH/src/path/to/code independent of $GO111MODULE. I think this should be the behaviour for Go versions that do not have good support for $GOPATH-independent code introspection; the Go team have said that this is current goal, but the tools do not exist to handle a GOPATH-free world yet.
Under this model, we (Gonum) would actually have GO111MODULE=on, taking advantage and testing modules compliance in our code, but still enabling us to perform our code generation.
1 Like