Skip to content

Commit a385e26

Browse files
thaJeztahthaJeztah
committed
implement module compatibility check
This package imports all "importable" packages, i.e., packages that: - are not applications ("main") - are not internal - and that have non-test go-files We do this to verify that our code can be consumed as a dependency in "module mode". When using a dependency that does not have a go.mod (i.e.; is not a "module"), go implicitly generates a go.mod. Lacking information from the dependency itself, it assumes "go1.16" language (see [DefaultGoModVersion]). Starting with Go1.21, go downgrades the language version used for such dependencies, which means that any language feature used that is not supported by go1.16 results in a compile error; # github.com/docker/cli/cli/context/store /go/pkg/mod/github.com/docker/cli@v25.0.0-beta.2+incompatible/cli/context/store/storeconfig.go:6:24: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) /go/pkg/mod/github.com/docker/cli@v25.0.0-beta.2+incompatible/cli/context/store/store.go:74:12: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) These errors do NOT occur when using GOPATH mode, nor do they occur when using "pseudo module mode" (the "-mod=mod -modfile=vendor.mod" approach used in this repository). As a workaround for this situation, we must include "//go:build" comments in any file that uses newer go-language features (such as the "any" type or the "min()", "max()" builtins). From the go toolchain docs (https://go.dev/doc/toolchain): > The go line for each module sets the language version the compiler enforces > when compiling packages in that module. The language version can be changed > on a per-file basis by using a build constraint. > > For example, a module containing code that uses the Go 1.21 language version > should have a go.mod file with a go line such as go 1.21 or go 1.21.3. > If a specific source file should be compiled only when using a newer Go > toolchain, adding //go:build go1.22 to that source file both ensures that > only Go 1.22 and newer toolchains will compile the file and also changes > the language version in that file to Go 1.22. This file is a generated module that imports all packages provided in the repository, which replicates an external consumer using our code as a dependency in go-module mode, and verifies all files in those packages have the correct "//go:build <go language version>" set. To test this package: make shell make -C ./internal/gocompat/ make: Entering directory '/go/src/github.com/docker/cli/internal/gocompat' GO111MODULE=off go generate . GO111MODULE=on go mod tidy GO111MODULE=on go test -v # github.com/docker/cli/templates ../../templates/templates.go:13:17: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) # github.com/docker/cli/cli/compose/template ../../cli/compose/template/template.go:98:45: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../cli/compose/template/template.go:105:27: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../cli/compose/template/template.go:141:28: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) # github.com/docker/cli/cli/compose/types ../../cli/compose/types/types.go:53:22: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../cli/compose/types/types.go:86:34: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../cli/compose/types/types.go:105:22: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../cli/compose/types/types.go:137:34: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../cli/compose/types/types.go:211:20: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../cli/compose/types/types.go:343:35: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../cli/compose/types/types.go:442:40: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../cli/compose/types/types.go:469:24: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../cli/compose/types/types.go:490:24: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../cli/compose/types/types.go:587:28: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../cli/compose/types/types.go:442:40: too many errors # github.com/docker/cli/cli/context/store ../../cli/context/store/storeconfig.go:6:24: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../cli/context/store/store.go:74:12: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../cli/context/store/store.go:75:23: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../cli/context/store/metadatastore.go:43:58: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../cli/context/store/metadatastore.go:48:22: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../cli/context/store/metadatastore.go:80:30: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) # github.com/docker/cli/cli/command/idresolver ../../cli/command/idresolver/idresolver.go:6:2: "github.com/docker/docker/api/types" imported and not used ../../cli/command/idresolver/idresolver.go:7:2: "github.com/docker/docker/api/types/swarm" imported and not used ../../cli/command/idresolver/idresolver.go:9:2: "github.com/pkg/errors" imported and not used ../../cli/command/idresolver/idresolver.go:28:49: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../cli/command/idresolver/idresolver.go:58:53: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) # github.com/docker/cli/cli/compose/schema ../../cli/compose/schema/schema.go:20:46: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../cli/compose/schema/schema.go:27:53: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../cli/compose/schema/schema.go:45:32: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../cli/compose/schema/schema.go:66:33: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) FAIL gocompat [build failed] make: *** [Makefile:3: verify] Error 1 make: Leaving directory '/go/src/github.com/docker/cli/internal/gocompat' [DefaultGoModVersion]: https://github.com/golang/go/blob/58c28ba286dd0e98fe4cca80f5d64bbcb824a685/src/cmd/go/internal/gover/version.go#L15-L24 [2]: https://go.dev/doc/toolchain Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
1 parent f1f4779 commit a385e26

File tree

14 files changed

+4483
-2
lines changed

14 files changed

+4483
-2
lines changed

‎internal/gocompat/.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
go.mod
2+
go.sum
3+
main.go
4+
main_test.go

‎internal/gocompat/Makefile‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
.PHONY: verify
2+
verify: generate
3+
GO111MODULE=on go test -v
4+
5+
.PHONY: generate
6+
generate: clean
7+
GO111MODULE=off go generate .
8+
GO111MODULE=on go mod tidy
9+
10+
.PHONY: clean
11+
clean:
12+
@rm -f go.mod go.sum main.go main_test.go

‎internal/gocompat/generate.go‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
package main
2+
3+
//go:generate go run modulegenerator.go
4+
5+
// make sure the modfile package is vendored.
6+
import _ "golang.org/x/mod/modfile"

‎internal/gocompat/modulegenerator.go‎

Lines changed: 193 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,193 @@
1+
//go:build ignore
2+
3+
package main
4+
5+
import (
6+
"bytes"
7+
"fmt"
8+
"log"
9+
"os"
10+
"os/exec"
11+
"strings"
12+
"text/template"
13+
14+
"golang.org/x/mod/modfile"
15+
)
16+
17+
func main() {
18+
if err := generateApp(); err != nil {
19+
log.Fatal(err)
20+
}
21+
if err := generateModule(); err != nil {
22+
log.Fatal(err)
23+
}
24+
}
25+
26+
func generateApp() error {
27+
cmd := exec.Command("go", "list", "-find", "-f", `{{- if ne .Name "main"}}{{if .GoFiles}}{{.ImportPath}}{{end}}{{end -}}`, "../../...")
28+
out, err := cmd.CombinedOutput()
29+
if err != nil {
30+
return err
31+
}
32+
33+
var pkgs []string
34+
for _, p := range strings.Split(string(out), "\n") {
35+
if strings.TrimSpace(p) == "" || strings.Contains(p, "/internal") {
36+
continue
37+
}
38+
pkgs = append(pkgs, p)
39+
}
40+
tmpl, err := template.New("main").Parse(appTemplate)
41+
if err != nil {
42+
return err
43+
}
44+
45+
var buf bytes.Buffer
46+
err = tmpl.Execute(&buf, appContext{Generator: cmd.String(), Packages: pkgs})
47+
if err != nil {
48+
return err
49+
}
50+
51+
return os.WriteFile("main_test.go", buf.Bytes(), 0o644)
52+
}
53+
54+
func generateModule() error {
55+
content, err := os.ReadFile("../../go.mod")
56+
if err != nil {
57+
if !os.IsNotExist(err) {
58+
return err
59+
}
60+
content = []byte("module github.com/docker/cli\n")
61+
if err := os.WriteFile("../../go.mod", content, 0o644); err != nil {
62+
return err
63+
}
64+
// Let's be nice, and remove the go.mod if we created it.
65+
// FIXME(thaJeztah): we need to clean up the go.mod after running the test, but need to know if we created it (or if it was an existing go.mod)
66+
// defer os.Remove("../../go.mod")
67+
} else {
68+
log.Println("WARN: go.mod exists in the repository root!")
69+
log.Println("WARN: Using your go.mod instead of our generated version -- this may misbehave!")
70+
}
71+
mod, err := modfile.Parse("../../go.mod", content, nil)
72+
if err != nil {
73+
return err
74+
}
75+
if mod.Go != nil && mod.Go.Version != "" {
76+
return fmt.Errorf("main go.mod must not contain a go version")
77+
}
78+
content, err = os.ReadFile("../../vendor.mod")
79+
if err != nil {
80+
return err
81+
}
82+
mod, err = modfile.Parse("../../vendor.mod", content, nil)
83+
if err != nil {
84+
return err
85+
}
86+
if err := mod.AddModuleStmt("gocompat"); err != nil {
87+
return err
88+
}
89+
if err := mod.AddReplace("github.com/docker/cli", "", "../../", ""); err != nil {
90+
return err
91+
}
92+
if err := mod.AddGoStmt("1.21"); err != nil {
93+
return err
94+
}
95+
out, err := mod.Format()
96+
if err != nil {
97+
return err
98+
}
99+
tmpl, err := template.New("mod").Parse(modTemplate)
100+
if err != nil {
101+
return err
102+
}
103+
104+
gen, _ := os.Executable()
105+
106+
var buf bytes.Buffer
107+
err = tmpl.Execute(&buf, appContext{Generator: gen, Dependencies: string(out)})
108+
if err != nil {
109+
return err
110+
}
111+
112+
return os.WriteFile("go.mod", buf.Bytes(), 0o644)
113+
}
114+
115+
type appContext struct {
116+
Generator string
117+
Packages []string
118+
Dependencies string
119+
}
120+
121+
const appTemplate = `// Code generated by "{{ .Generator }}". DO NOT EDIT.
122+
123+
package main_test
124+
125+
import (
126+
"testing"
127+
128+
// Import all importable packages, i.e., packages that:
129+
//
130+
// - are not applications ("main")
131+
// - are not internal
132+
// - and that have non-test go-files
133+
{{- range .Packages }}
134+
_ "{{ . }}"
135+
{{- end}}
136+
)
137+
138+
// This file import all "importable" packages, i.e., packages that:
139+
//
140+
// - are not applications ("main")
141+
// - are not internal
142+
// - and that have non-test go-files
143+
//
144+
// We do this to verify that our code can be consumed as a dependency
145+
// in "module mode". When using a dependency that does not have a go.mod
146+
// (i.e.; is not a "module"), go implicitly generates a go.mod. Lacking
147+
// information from the dependency itself, it assumes "go1.16" language
148+
// (see [DefaultGoModVersion]). Starting with Go1.21, go downgrades the
149+
// language version used for such dependencies, which means that any
150+
// language feature used that is not supported by go1.16 results in a
151+
// compile error;
152+
//
153+
// # github.com/docker/cli/cli/context/store
154+
// /go/pkg/mod/github.com/docker/cli@v25.0.0-beta.2+incompatible/cli/context/store/storeconfig.go:6:24: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
155+
// /go/pkg/mod/github.com/docker/cli@v25.0.0-beta.2+incompatible/cli/context/store/store.go:74:12: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
156+
//
157+
// These errors do NOT occur when using GOPATH mode, nor do they occur
158+
// when using "pseudo module mode" (the "-mod=mod -modfile=vendor.mod"
159+
// approach used in this repository).
160+
//
161+
// As a workaround for this situation, we must include "//go:build" comments
162+
// in any file that uses newer go-language features (such as the "any" type
163+
// or the "min()", "max()" builtins).
164+
//
165+
// From the go toolchain docs (https://go.dev/doc/toolchain):
166+
//
167+
// > The go line for each module sets the language version the compiler enforces
168+
// > when compiling packages in that module. The language version can be changed
169+
// > on a per-file basis by using a build constraint.
170+
// >
171+
// > For example, a module containing code that uses the Go 1.21 language version
172+
// > should have a go.mod file with a go line such as go 1.21 or go 1.21.3.
173+
// > If a specific source file should be compiled only when using a newer Go
174+
// > toolchain, adding //go:build go1.22 to that source file both ensures that
175+
// > only Go 1.22 and newer toolchains will compile the file and also changes
176+
// > the language version in that file to Go 1.22.
177+
//
178+
// This file is a generated module that imports all packages provided in
179+
// the repository, which replicates an external consumer using our code
180+
// as a dependency in go-module mode, and verifies all files in those
181+
// packages have the correct "//go:build <go language version>" set.
182+
//
183+
// [DefaultGoModVersion]: https://github.com/golang/go/blob/58c28ba286dd0e98fe4cca80f5d64bbcb824a685/src/cmd/go/internal/gover/version.go#L15-L24
184+
// [2]: https://go.dev/doc/toolchain
185+
func TestModuleCompatibllity(t *testing.T) {
186+
t.Log("all packages have the correct go version specified through //go:build")
187+
}
188+
`
189+
190+
const modTemplate = `// Code generated by "{{ .Generator }}". DO NOT EDIT.
191+
192+
{{.Dependencies}}
193+
`

‎internal/tools.go‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
package internal
2+
3+
// make sure the modfile package is vendored.
4+
import _ "golang.org/x/mod/modfile"

‎vendor.mod‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,8 @@ require (
4444
gopkg.in/yaml.v2 v2.4.0
4545
gotest.tools/v3 v3.5.1
4646
tags.cncf.io/container-device-interface v0.6.2
47-
)
47+
golang.org/x/mod v0.11.0
48+
)
4849

4950
require (
5051
github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect
@@ -80,7 +81,6 @@ require (
8081
go.opentelemetry.io/otel/metric v1.19.0 // indirect
8182
go.opentelemetry.io/otel/trace v1.19.0 // indirect
8283
golang.org/x/crypto v0.14.0 // indirect
83-
golang.org/x/mod v0.11.0 // indirect
8484
golang.org/x/net v0.17.0 // indirect
8585
golang.org/x/time v0.3.0 // indirect
8686
golang.org/x/tools v0.10.0 // indirect

‎vendor/golang.org/x/mod/internal/lazyregexp/lazyre.go‎

Lines changed: 78 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)