go run命令执行流程

由于CC最近在做的一个项目需要用到go的编译相关的东西,因此看了一下这块的源码。这里记录一下go run执行后的执行流程。

测试用环境

测试用的目录和测试文件如下:

thuanqin@thuanqin-pc MINGW64 /tmp/gorun $ go version go version go1.11.4 windows/amd64 thuanqin@thuanqin-pc MINGW64 /tmp/gorun $ ls lib.go main.go thuanqin@thuanqin-pc MINGW64 /tmp/gorun $ cat lib.go package main import "fmt" func Lib() { fmt.Println("IN LIB") } thuanqin@thuanqin-pc MINGW64 /tmp/gorun $ cat main.go package main func main() { Lib() } thuanqin@thuanqin-pc MINGW64 /tmp/gorun $ go run . IN LIB

--- 测试文件和目录

整体流程

整体流程
整体流程
使用到的package之间的相互关系及作用
使用到的package之间的相互关系及作用

入口代码

下面来看具体的代码。

当执行了go run后,会的调用下面的runRun函数:

func runRun(cmd *base.Command, args []string) { work.BuildInit() var b work.Builder b.Init() b.Print = printStderr

--- cmd\go\internal\run\run.go

runRun首先调用BuildInit,BuildInit会根据参数执行一些初始化动作。接着代码在这里定义了一个work.Builder的对象,这个对象会负责后面的具体构建,尤其是存放构建中的全局状态。这里的b.Init用于初始化builder的一些属性,例如编译过程中的临时目录(可以通过GOTMPDIR环境变量设置)等。在CC这里就都是默认的配置,没有设置过特别的环境变量。

接着代码根据用户指定参数的不同会走不同的逻辑,在CC这里会走中间的逻辑:

func runRun(cmd *base.Command, args []string) { i := 0 for i < len(args) && strings.HasSuffix(args[i], ".go") { i++ } var p *load.Package if i > 0 { files := args[:i] for _, file := range files { if strings.HasSuffix(file, "_test.go") { // GoFilesPackage is going to assign this to TestGoFiles. // Reject since it won't be part of the build. base.Fatalf("go run: cannot run *_test.go files (%s)", file) } } p = load.GoFilesPackage(files) } else if len(args) > 0 && !strings.HasPrefix(args[0], "-") { pkgs := load.PackagesAndErrors(args[:1]) if len(pkgs) == 0 { base.Fatalf("go run: no packages loaded from %s", args[0]) } if len(pkgs) > 1 { var names []string for _, p := range pkgs { names = append(names, p.ImportPath) } base.Fatalf("go run: pattern %s matches multiple packages:\n\t%s", args[0], strings.Join(names, "\n\t")) } p = pkgs[0] i++ } else { base.Fatalf("go run: no go files listed") } cmdArgs := args[i:]

--- cmd\go\internal\run\run.go

这里代码的核心是

load.PackagesAndErrors

, 它的作用是加载一个go的package。代码如下:

// PackagesAndErrors is like 'packages' but returns a // *Package for every argument, even the ones that // cannot be loaded at all. // The packages that fail to load will have p.Error != nil. func PackagesAndErrors(patterns []string) []*Package { if len(patterns) > 0 && strings.HasSuffix(patterns[0], ".go") { return []*Package{GoFilesPackage(patterns)} } matches := ImportPaths(patterns) var ( pkgs []*Package stk ImportStack seenPkg = make(map[*Package]bool) ) for _, m := range matches { for _, pkg := range m.Pkgs { p := loadPackage(pkg, &stk) p.Match = append(p.Match, m.Pattern) p.Internal.CmdlinePkg = true if m.Literal { // Note: do not set = m.Literal unconditionally // because maybe we'll see p matching both // a literal and also a non-literal pattern. p.Internal.CmdlinePkgLiteral = true } if seenPkg[p] { continue } seenPkg[p] = true pkgs = append(pkgs, p) } } // Now that CmdlinePkg is set correctly, // compute the effective flags for all loaded packages // (not just the ones matching the patterns but also // their dependencies). setToolFlags(pkgs...) return pkgs }

--- cmd\go\internal\load\pkg.go

在CC这里patterns的内容为 . 。这里ImportPaths会解析出import的路径,在CC这里依旧是 .。接着的for循环就会对.这个路径进行分析,通过

loadPackage

生成对应的pacakge,然后将相关的package返回。loadPackage的代码如下:

// loadPackage is like loadImport but is used for command-line arguments, // not for paths found in import statements. In addition to ordinary import paths, // loadPackage accepts pseudo-paths beginning with cmd/ to denote commands // in the Go command directory, as well as paths to those directories. func loadPackage(arg string, stk *ImportStack) *Package { if build.IsLocalImport(arg) { dir := arg if !filepath.IsAbs(dir) { if abs, err := filepath.Abs(dir); err == nil { // interpret relative to current directory dir = abs } } if sub, ok := hasSubdir(cfg.GOROOTsrc, dir); ok && strings.HasPrefix(sub, "cmd/") && !strings.Contains(sub[4:], "/") { arg = sub } } if strings.HasPrefix(arg, "cmd/") && !strings.Contains(arg[4:], "/") { if p := cmdCache[arg]; p != nil { return p } stk.Push(arg) defer stk.Pop() bp, err := cfg.BuildContext.ImportDir(filepath.Join(cfg.GOROOTsrc, arg), 0) bp.ImportPath = arg bp.Goroot = true bp.BinDir = cfg.GOROOTbin if cfg.GOROOTbin != "" { bp.BinDir = cfg.GOROOTbin } bp.Root = cfg.GOROOT bp.SrcRoot = cfg.GOROOTsrc p := new(Package) cmdCache[arg] = p p.load(stk, bp, err) if p.Error == nil && p.Name != "main" { p.Error = &PackageError{ ImportStack: stk.Copy(), Err: fmt.Sprintf("expected package main but found package %s in %s", p.Name, p.Dir), } } return p } // Wasn't a command; must be a package. // If it is a local import path but names a standard package, // we treat it as if the user specified the standard package. // This lets you run go test ./ioutil in package io and be // referring to io/ioutil rather than a hypothetical import of // "./ioutil". if build.IsLocalImport(arg) || filepath.IsAbs(arg) { dir := arg if !filepath.IsAbs(arg) { dir = filepath.Join(base.Cwd, arg) } bp, _ := cfg.BuildContext.ImportDir(dir, build.FindOnly) if bp.ImportPath != "" && bp.ImportPath != "." { arg = bp.ImportPath } } return LoadImport(arg, base.Cwd, nil, stk, nil, 0) }

--- cmd\go\internal\load\pkg.go

这里的代码主要是在路径上做一些文章,不是很复杂,在将相关的路径分析出来后,通过

LoadImport

进行后续的Import动作:

// LoadImport scans the directory named by path, which must be an import path, // but possibly a local import path (an absolute file system path or one beginning // with ./ or ../). A local relative path is interpreted relative to srcDir. // It returns a *Package describing the package found in that directory. // LoadImport does not set tool flags and should only be used by // this package, as part of a bigger load operation, and by GOPATH-based "go get". // TODO(rsc): When GOPATH-based "go get" is removed, unexport this function. func LoadImport(path, srcDir string, parent *Package, stk *ImportStack, importPos []token.Position, mode int) *Package { stk.Push(path) defer stk.Pop()

--- cmd\go\internal\load\pkg.go

在第一次调用的时候,参数中的path为.,srcDir为这里测试代码文件所在的目录。通过stk先将当前的path推到栈中。接着的两个代码应该都和go mod相关,暂时不看:

if strings.HasPrefix(path, "mod/") { // Paths beginning with "mod/" might accidentally // look in the module cache directory tree in $GOPATH/pkg/mod/. // This prefix is owned by the Go core for possible use in the // standard library (since it does not begin with a domain name), // so it's OK to disallow entirely. return &Package{ PackagePublic: PackagePublic{ ImportPath: path, Error: &PackageError{ ImportStack: stk.Copy(), Err: fmt.Sprintf("disallowed import path %q", path), }, }, } } if strings.Contains(path, "@") { var text string if cfg.ModulesEnabled { text = "can only use path@version syntax with 'go get'" } else { text = "cannot use path@version syntax in GOPATH mode" } return &Package{ PackagePublic: PackagePublic{ ImportPath: path, Error: &PackageError{ ImportStack: stk.Copy(), Err: text, }, }, } }

--- cmd\go\internal\load\pkg.go

接着这里会判断一下package的父子信息,从上面的代码可以看到目前是没有parent的:

parentPath := "" if parent != nil { parentPath = parent.ImportPath }

--- cmd\go\internal\load\pkg.go

下面的代码用于获取在临时目录下的package的路径:

// Determine canonical identifier for this package. // For a local import the identifier is the pseudo-import path // we create from the full directory to the package. // Otherwise it is the usual import path. // For vendored imports, it is the expanded form. importPath := path origPath := path isLocal := build.IsLocalImport(path) var modDir string var modErr error if isLocal { importPath = dirToImportPath(filepath.Join(srcDir, path)) } else if cfg.ModulesEnabled { var p string modDir, p, modErr = ModLookup(path) if modErr == nil { importPath = p } } else if mode&ResolveImport != 0 { // We do our own path resolution, because we want to // find out the key to use in packageCache without the // overhead of repeated calls to buildContext.Import. // The code is also needed in a few other places anyway. path = ResolveImportPath(parent, path) importPath = path } else if mode&ResolveModule != 0 { path = ModuleImportPath(parent, path) importPath = path }

--- cmd\go\internal\load\pkg.go

CC的这个测试例子里,代码会进入第一个if语句中,importPath的值为

_/C_/Users/thuanqin/AppData/Local/Temp/gorun

,即临时目录下的gorun目录。在获取到importPath后,会看下packageCache中是否已经存在相关信息,如果不存在则new相关对象并进行分析:

p := packageCache[importPath] if p != nil { p = reusePackage(p, stk) } else { p = new(Package) p.Internal.Local = isLocal p.ImportPath = importPath packageCache[importPath] = p // Load package. // Import always returns bp != nil, even if an error occurs, // in order to return partial information. var bp *build.Package var err error if modDir != "" { bp, err = cfg.BuildContext.ImportDir(modDir, 0) } else if modErr != nil { bp = new(build.Package) err = fmt.Errorf("unknown import path %q: %v", importPath, modErr) } else if cfg.ModulesEnabled && path != "unsafe" { bp = new(build.Package) err = fmt.Errorf("unknown import path %q: internal error: module loader did not resolve import", importPath) } else { buildMode := build.ImportComment if mode&ResolveImport == 0 || path != origPath { // Not vendoring, or we already found the vendored path. buildMode |= build.IgnoreVendor } bp, err = cfg.BuildContext.Import(path, srcDir, buildMode) } bp.ImportPath = importPath if cfg.GOBIN != "" { bp.BinDir = cfg.GOBIN } else if cfg.ModulesEnabled { bp.BinDir = ModBinDir() } if modDir == "" && err == nil && !isLocal && bp.ImportComment != "" && bp.ImportComment != path && !strings.Contains(path, "/vendor/") && !strings.HasPrefix(path, "vendor/") { err = fmt.Errorf("code in directory %s expects import %q", bp.Dir, bp.ImportComment) } p.load(stk, bp, err) if p.Error != nil && p.Error.Pos == "" { p = setErrorPos(p, importPos) } if modDir == "" && origPath != cleanImport(origPath) { p.Error = &PackageError{ ImportStack: stk.Copy(), Err: fmt.Sprintf("non-canonical import path: %q should be %q", origPath, pathpkg.Clean(origPath)), } p.Incomplete = true } }

--- cmd\go\internal\load\pkg.go

CC这里没有用到mod,因此会走最后一个else的代码,通过cfg.BuildContext.Import会生成具体的bp。cfg.BuildContext.Import会生成bp这个Package中关于各类文件的信息,例如这个Package下面有哪些.go文件,这些文件有哪些import的依赖,bin/pkg目录的路径等。在做import依赖时会用到下面的语句,注意最后的flag:

pf, err := parser.ParseFile(fset, filename, data, parser.ImportsOnly|parser.ParseComments)

--- go\build\build.go

总之经过cfg.BuildContext.Import分析后,这个Package相关的源码文件的相关信息就更加清楚了,但是实际的编译工作还没有真正开始。在cfg.BuildContext.Import后,真正干活的是

p.load

这个调用。在p.load之后LoadImport的代码主要是根据是否有错误信息进行提示等收尾工作:

// Checked on every import because the rules depend on the code doing the importing. if perr := disallowInternal(srcDir, parent, parentPath, p, stk); perr != p { return setErrorPos(perr, importPos) } if mode&ResolveImport != 0 { if perr := disallowVendor(srcDir, parent, parentPath, origPath, p, stk); perr != p { return setErrorPos(perr, importPos) } } if p.Name == "main" && parent != nil && parent.Dir != p.Dir { perr := *p perr.Error = &PackageError{ ImportStack: stk.Copy(), Err: fmt.Sprintf("import %q is a program, not an importable package", path), } return setErrorPos(&perr, importPos) } if p.Internal.Local && parent != nil && !parent.Internal.Local { perr := *p perr.Error = &PackageError{ ImportStack: stk.Copy(), Err: fmt.Sprintf("local import %q in non-local package", path), } return setErrorPos(&perr, importPos) } return p }

--- cmd\go\internal\load\pkg.go

因此重点看下p.load的代码:

// load populates p using information from bp, err, which should // be the result of calling build.Context.Import. func (p *Package) load(stk *ImportStack, bp *build.Package, err error) { p.copyBuild(bp) // The localPrefix is the path we interpret ./ imports relative to. // Synthesized main packages sometimes override this. if p.Internal.Local { p.Internal.LocalPrefix = dirToImportPath(p.Dir) } if err != nil { if _, ok := err.(*build.NoGoError); ok { err = &NoGoError{Package: p} } p.Incomplete = true err = base.ExpandScanner(err) p.Error = &PackageError{ ImportStack: stk.Copy(), Err: err.Error(), } return }

--- cmd\go\internal\load\pkg.go

copyBuild相当于做了一次深拷贝,如果之前Import的代码存在err的话,这里会生成相关Error信息并直接返回。接着代码根据是否为main决定是否使用bin目录:

useBindir := p.Name == "main" if !p.Standard { switch cfg.BuildBuildmode { case "c-archive", "c-shared", "plugin": useBindir = false } }

--- cmd\go\internal\load\pkg.go

在CC这里的例子里是使用了的,因此会进入下面的这个逻辑:

if useBindir { // Report an error when the old code.google.com/p/go.tools paths are used. if InstallTargetDir(p) == StalePath { newPath := strings.Replace(p.ImportPath, "code.google.com/p/go.", "golang.org/x/", 1) e := fmt.Sprintf("the %v command has moved; use %v instead.", p.ImportPath, newPath) p.Error = &PackageError{Err: e} return } _, elem := filepath.Split(p.Dir) if cfg.ModulesEnabled { // NOTE(rsc): Using p.ImportPath instead of p.Dir // makes sure we install a package in the root of a // cached module directory as that package name // not name@v1.2.3. // Using p.ImportPath instead of p.Dir // is probably correct all the time, // even for non-module-enabled code, // but I'm not brave enough to change the // non-module behavior this late in the // release cycle. Maybe for Go 1.12. // See golang.org/issue/26869. _, elem = pathpkg.Split(p.ImportPath) // If this is example.com/mycmd/v2, it's more useful to install it as mycmd than as v2. // See golang.org/issue/24667. isVersion := func(v string) bool { if len(v) < 2 || v[0] != 'v' || v[1] < '1' || '9' < v[1] { return false } for i := 2; i < len(v); i++ { if c := v[i]; c < '0' || '9' < c { return false } } return true } if isVersion(elem) { _, elem = pathpkg.Split(pathpkg.Dir(p.ImportPath)) } } full := cfg.BuildContext.GOOS + "_" + cfg.BuildContext.GOARCH + "/" + elem if cfg.BuildContext.GOOS != base.ToolGOOS || cfg.BuildContext.GOARCH != base.ToolGOARCH { // Install cross-compiled binaries to subdirectories of bin. elem = full } if p.Internal.Build.BinDir == "" && cfg.ModulesEnabled { p.Internal.Build.BinDir = ModBinDir() } if p.Internal.Build.BinDir != "" { // Install to GOBIN or bin of GOPATH entry. p.Target = filepath.Join(p.Internal.Build.BinDir, elem) if !p.Goroot && strings.Contains(elem, "/") && cfg.GOBIN != "" { // Do not create $GOBIN/goos_goarch/elem. p.Target = "" p.Internal.GobinSubdir = true } } if InstallTargetDir(p) == ToTool { // This is for 'go tool'. // Override all the usual logic and force it into the tool directory. if cfg.BuildToolchainName == "gccgo" { p.Target = filepath.Join(base.ToolDir, elem) } else { p.Target = filepath.Join(cfg.GOROOTpkg, "tool", full) } } if p.Target != "" && cfg.BuildContext.GOOS == "windows" { p.Target += ".exe" }

--- cmd\go\internal\load\pkg.go

这段代码的逻辑主要是为了分析出生成的target可执行文件的文件名及相关的目录信息。如果忽略和mod相关的代码的话不是很复杂。在这之后会根据执行环境的需求,添加默认的依赖package:

// Build augmented import list to add implicit dependencies. // Be careful not to add imports twice, just to avoid confusion. importPaths := p.Imports addImport := func(path string, forCompiler bool) { for _, p := range importPaths { if path == p { return } } importPaths = append(importPaths, path) if forCompiler { p.Internal.CompiledImports = append(p.Internal.CompiledImports, path) } } // Cgo translation adds imports of "unsafe", "runtime/cgo" and "syscall", // except for certain packages, to avoid circular dependencies. if p.UsesCgo() { addImport("unsafe", true) } if p.UsesCgo() && (!p.Standard || !cgoExclude[p.ImportPath]) && cfg.BuildContext.Compiler != "gccgo" { addImport("runtime/cgo", true) } if p.UsesCgo() && (!p.Standard || !cgoSyscallExclude[p.ImportPath]) { addImport("syscall", true) } // SWIG adds imports of some standard packages. if p.UsesSwig() { if cfg.BuildContext.Compiler != "gccgo" { addImport("runtime/cgo", true) } addImport("syscall", true) addImport("sync", true) // TODO: The .swig and .swigcxx files can use // %go_import directives to import other packages. } // The linker loads implicit dependencies. if p.Name == "main" && !p.Internal.ForceLibrary { for _, dep := range LinkerDeps(p) { addImport(dep, false) } }

--- cmd\go\internal\load\pkg.go

接着对文件名等进行检查:

// Check for case-insensitive collision of input files. // To avoid problems on case-insensitive files, we reject any package // where two different input files have equal names under a case-insensitive // comparison. inputs := p.AllFiles() f1, f2 := str.FoldDup(inputs) if f1 != "" { p.Error = &PackageError{ ImportStack: stk.Copy(), Err: fmt.Sprintf("case-insensitive file name collision: %q and %q", f1, f2), } return } // If first letter of input file is ASCII, it must be alphanumeric. // This avoids files turning into flags when invoking commands, // and other problems we haven't thought of yet. // Also, _cgo_ files must be generated by us, not supplied. // They are allowed to have //go:cgo_ldflag directives. // The directory scan ignores files beginning with _, // so we shouldn't see any _cgo_ files anyway, but just be safe. for _, file := range inputs { if !SafeArg(file) || strings.HasPrefix(file, "_cgo_") { p.Error = &PackageError{ ImportStack: stk.Copy(), Err: fmt.Sprintf("invalid input file name %q", file), } return } } if name := pathpkg.Base(p.ImportPath); !SafeArg(name) { p.Error = &PackageError{ ImportStack: stk.Copy(), Err: fmt.Sprintf("invalid input directory name %q", name), } return } if !SafeArg(p.ImportPath) { p.Error = &PackageError{ ImportStack: stk.Copy(), Err: fmt.Sprintf("invalid import path %q", p.ImportPath), } return }

--- cmd\go\internal\load\pkg.go

下面的代码开始处理依赖的package,其首先对每个依赖的package调用LoadImport加载相关package,然后分析依赖情况:

// Build list of imported packages and full dependency list. imports := make([]*Package, 0, len(p.Imports)) for i, path := range importPaths { if path == "C" { continue } p1 := LoadImport(path, p.Dir, p, stk, p.Internal.Build.ImportPos[path], ResolveImport) if p.Standard && p.Error == nil && !p1.Standard && p1.Error == nil { p.Error = &PackageError{ ImportStack: stk.Copy(), Err: fmt.Sprintf("non-standard import %q in standard package %q", path, p.ImportPath), } pos := p.Internal.Build.ImportPos[path] if len(pos) > 0 { p.Error.Pos = pos[0].String() } } path = p1.ImportPath importPaths[i] = path if i < len(p.Imports) { p.Imports[i] = path } imports = append(imports, p1) if p1.Incomplete { p.Incomplete = true } } p.Internal.Imports = imports deps := make(map[string]*Package) var q []*Package q = append(q, imports...) for i := 0; i < len(q); i++ { p1 := q[i] path := p1.ImportPath // The same import path could produce an error or not, // depending on what tries to import it. // Prefer to record entries with errors, so we can report them. p0 := deps[path] if p0 == nil || p1.Error != nil && (p0.Error == nil || len(p0.Error.ImportStack) > len(p1.Error.ImportStack)) { deps[path] = p1 for _, p2 := range p1.Internal.Imports { if deps[p2.ImportPath] != p2 { q = append(q, p2) } } } } p.Deps = make([]string, 0, len(deps)) for dep := range deps { p.Deps = append(p.Deps, dep) } sort.Strings(p.Deps) for _, dep := range p.Deps { p1 := deps[dep] if p1 == nil { panic("impossible: missing entry in package cache for " + dep + " imported by " + p.ImportPath) } if p1.Error != nil { p.DepsErrors = append(p.DepsErrors, p1.Error) } } // unsafe is a fake package. if p.Standard && (p.ImportPath == "unsafe" || cfg.BuildContext.Compiler == "gccgo") { p.Target = "" }

--- cmd\go\internal\load\pkg.go

根据是否开启cgo决定是否从package中去除某些文件,这些文件应该在后续的代码中就不会被进行处理了:

// If cgo is not enabled, ignore cgo supporting sources // just as we ignore go files containing import "C". if !cfg.BuildContext.CgoEnabled { p.CFiles = nil p.CXXFiles = nil p.MFiles = nil p.SwigFiles = nil p.SwigCXXFiles = nil // Note that SFiles are okay (they go to the Go assembler) // and HFiles are okay (they might be used by the SFiles). // Also Sysofiles are okay (they might not contain object // code; see issue #16050). }

--- cmd\go\internal\load\pkg.go

根据参数进行一些简单的检查:

setError := func(msg string) { p.Error = &PackageError{ ImportStack: stk.Copy(), Err: msg, } } // The gc toolchain only permits C source files with cgo or SWIG. if len(p.CFiles) > 0 && !p.UsesCgo() && !p.UsesSwig() && cfg.BuildContext.Compiler == "gc" { setError(fmt.Sprintf("C source files not allowed when not using cgo or SWIG: %s", strings.Join(p.CFiles, " "))) return } // C++, Objective-C, and Fortran source files are permitted only with cgo or SWIG, // regardless of toolchain. if len(p.CXXFiles) > 0 && !p.UsesCgo() && !p.UsesSwig() { setError(fmt.Sprintf("C++ source files not allowed when not using cgo or SWIG: %s", strings.Join(p.CXXFiles, " "))) return } if len(p.MFiles) > 0 && !p.UsesCgo() && !p.UsesSwig() { setError(fmt.Sprintf("Objective-C source files not allowed when not using cgo or SWIG: %s", strings.Join(p.MFiles, " "))) return } if len(p.FFiles) > 0 && !p.UsesCgo() && !p.UsesSwig() { setError(fmt.Sprintf("Fortran source files not allowed when not using cgo or SWIG: %s", strings.Join(p.FFiles, " "))) return } // Check for case-insensitive collisions of import paths. fold := str.ToFold(p.ImportPath) if other := foldPath[fold]; other == "" { foldPath[fold] = p.ImportPath } else if other != p.ImportPath { setError(fmt.Sprintf("case-insensitive import collision: %q and %q", p.ImportPath, other)) return } if cfg.ModulesEnabled { p.Module = ModPackageModuleInfo(p.ImportPath) if p.Name == "main" { p.Internal.BuildInfo = ModPackageBuildInfo(p.ImportPath, p.Deps) } }

--- cmd\go\internal\load\pkg.go

至此load的代码就结束了。从代码中可以看到load主要做了下面几件事情:

*  生成bin及target的信息
*  加载一些默认的package支持运行时环境
*  进行各类校验检查,例如文件名的检查
*  对package依赖的各个import的package执行LoadImport,并检查依赖情况
*  根据cfg的参数进行文件类型检查

现在回到runRun。经过上面的一遍源码阅读后,此时package相关的信息都已经被加载了,包括依赖包的信息也都生成了,但是目前还未涉及到真正的编译相关的工作。因此我们继续看runRun后面的代码。首先是检查一下有没有Error,以及是否有main这个package:

if p.Error != nil { base.Fatalf("%s", p.Error) } p.Internal.OmitDebug = true if len(p.DepsErrors) > 0 { // Since these are errors in dependencies, // the same error might show up multiple times, // once in each package that depends on it. // Only print each once. printed := map[*load.PackageError]bool{} for _, err := range p.DepsErrors { if !printed[err] { printed[err] = true base.Errorf("%s", err) } } } base.ExitIfErrors() if p.Name != "main" { base.Fatalf("go run: cannot run non-main package") }

--- cmd\go\internal\run\run.go

接着是开始真正action的代码:

p.Target = "" // must build - not up to date var src string if len(p.GoFiles) > 0 { src = p.GoFiles[0] } else if len(p.CgoFiles) > 0 { src = p.CgoFiles[0] } else { // this case could only happen if the provided source uses cgo // while cgo is disabled. hint := "" if !cfg.BuildContext.CgoEnabled { hint = " (cgo is disabled)" } base.Fatalf("go run: no suitable source files%s", hint) } p.Internal.ExeName = src[:len(src)-len(".go")] // name temporary executable for first go file a1 := b.LinkAction(work.ModeBuild, work.ModeBuild, p) a := &work.Action{Mode: "go run", Func: buildRunProgram, Args: cmdArgs, Deps: []*work.Action{a1}} b.Do(a) }

--- cmd\go\internal\run\run.go

这里首先随机的取第一个文件的文件名作为可执行文件的名字。接着生成一个LinkAction和一个go run的Action,后者依赖前者。之后通过b.Do执行这个Action。

LinkAction的构造方法如下:

// LinkAction returns the action for linking p into an executable // and possibly installing the result (according to mode). // depMode is the action (build or install) to use when compiling dependencies. func (b *Builder) LinkAction(mode, depMode BuildMode, p *load.Package) *Action { // Construct link action. a := b.cacheAction("link", p, func() *Action { a := &Action{ Mode: "link", Package: p, } a1 := b.CompileAction(ModeBuild, depMode, p) a.Func = (*Builder).link a.Deps = []*Action{a1} a.Objdir = a1.Objdir // An executable file. (This is the name of a temporary file.) // Because we run the temporary file in 'go run' and 'go test', // the name will show up in ps listings. If the caller has specified // a name, use that instead of a.out. The binary is generated // in an otherwise empty subdirectory named exe to avoid // naming conflicts. The only possible conflict is if we were // to create a top-level package named exe. name := "a.out" if p.Internal.ExeName != "" { name = p.Internal.ExeName } else if (cfg.Goos == "darwin" || cfg.Goos == "windows") && cfg.BuildBuildmode == "c-shared" && p.Target != "" { // On OS X, the linker output name gets recorded in the // shared library's LC_ID_DYLIB load command. // The code invoking the linker knows to pass only the final // path element. Arrange that the path element matches what // we'll install it as; otherwise the library is only loadable as "a.out". // On Windows, DLL file name is recorded in PE file // export section, so do like on OS X. _, name = filepath.Split(p.Target) } a.Target = a.Objdir + filepath.Join("exe", name) + cfg.ExeSuffix a.built = a.Target b.addTransitiveLinkDeps(a, a1, "") // Sequence the build of the main package (a1) strictly after the build // of all other dependencies that go into the link. It is likely to be after // them anyway, but just make sure. This is required by the build ID-based // shortcut in (*Builder).useCache(a1), which will call b.linkActionID(a). // In order for that linkActionID call to compute the right action ID, all the // dependencies of a (except a1) must have completed building and have // recorded their build IDs. a1.Deps = append(a1.Deps, &Action{Mode: "nop", Deps: a.Deps[1:]}) return a }) if mode == ModeInstall || mode == ModeBuggyInstall { a = b.installAction(a, mode) } return a }

--- cmd\go\internal\work\action.go

这里做了下面几件事:

*  通过cacheAction生成一个link的action并做cache
*  linkAction在这里会依赖一个compileAction
*  设置linkAction的各个属性,例如Func属性、built属性等
*  通过addTransitiveLinkDeps生成传递依赖,简单的说就是compileAction的依赖都会变为linkAction的依赖
*  compileAction会被加上一个nopAction的依赖,这个nopAction会依赖linkAction的除了compileAction以外的依赖,通过这种方式保证这些Action执行的顺序(即link在compile和其它其依赖后执行,compile在nop后执行,而nop会在link的其它非compile后执行)
*  最后在go run中,并没有installAction被生成

现在看下CompileAction的实现:

// CompileAction returns the action for compiling and possibly installing // (according to mode) the given package. The resulting action is only // for building packages (archives), never for linking executables. // depMode is the action (build or install) to use when building dependencies. // To turn package main into an executable, call b.Link instead. func (b *Builder) CompileAction(mode, depMode BuildMode, p *load.Package) *Action { if mode != ModeBuild && (p.Internal.Local || p.Module != nil) && p.Target == "" { // Imported via local path or using modules. No permanent target. mode = ModeBuild } if mode != ModeBuild && p.Name == "main" { // We never install the .a file for a main package. mode = ModeBuild } // Construct package build action. a := b.cacheAction("build", p, func() *Action { a := &Action{ Mode: "build", Package: p, Func: (*Builder).build, Objdir: b.NewObjdir(), } if p.Error == nil || !p.Error.IsImportCycle { for _, p1 := range p.Internal.Imports { a.Deps = append(a.Deps, b.CompileAction(depMode, depMode, p1)) } } if p.Standard { switch p.ImportPath { case "builtin", "unsafe": // Fake packages - nothing to build. a.Mode = "built-in package" a.Func = nil return a } // gccgo standard library is "fake" too. if cfg.BuildToolchainName == "gccgo" { // the target name is needed for cgo. a.Mode = "gccgo stdlib" a.Target = p.Target a.Func = nil return a } } return a }) // Construct install action. if mode == ModeInstall || mode == ModeBuggyInstall { a = b.installAction(a, mode) } return a }

--- cmd\go\internal\work\action.go

这里的代码和LinkAction很类似,特别要注意的地方是对于package的每个import,都会生成一个compileAction作为其依赖。

几个action之间依赖如下:

*  goRunAction 依赖 linkAction
*  linkAction 依赖 compileAction以及compileAction的传递依赖
*  compileAction 依赖 nopAction,同时依赖其package所依赖的package的compileAction
*  nopAction 依赖 linkAction的依赖中的非第一个compileAction的其它依赖

相关的Func等信息如下:

&Action{Mode: "nop", Deps: a.Deps[1:]}

--- nopAction

a := &Action{ Mode: "build", Package: p, Func: (*Builder).build, Objdir: b.NewObjdir(), }

--- compileAction

a := &Action{ Mode: "link", Package: p, } a1 := b.CompileAction(ModeBuild, depMode, p) a.Func = (*Builder).link a.Deps = []*Action{a1} a.Objdir = a1.Objdir

--- linkAction

a := &work.Action{Mode: "go run", Func: buildRunProgram, Args: cmdArgs, Deps: []*work.Action{a1}}

--- goRunAction

在这些action之间关系建立完成后,通过b.Do就开始执行这些action了。经过上面的分析CC可以知道,真正干货的是各个Action的Func属性,而b.Do只是根据action之间的依赖进行协调。因此CC重点看下相关Action的实现。

Action的Func逻辑

Compile Action

对于CompileAction,其Func为build:

// build is the action for building a single package. // Note that any new influence on this logic must be reported in b.buildActionID above as well. func (b *Builder) build(a *Action) (err error) { p := a.Package bit := func(x uint32, b bool) uint32 { if b { return x } return 0 } cached := false need := bit(needBuild, !b.IsCmdList || b.NeedExport) | bit(needCgoHdr, b.needCgoHdr(a)) | bit(needVet, a.needVet) | bit(needCompiledGoFiles, b.NeedCompiledGoFiles)

--- cmd\go\internal\work\exec.go

开始的几行代码都用于计算need这个变量。在CC这里这个need都是1。接着:

if !p.BinaryOnly { if b.useCache(a, p, b.buildActionID(a), p.Target) { // We found the main output in the cache. // If we don't need any other outputs, we can stop. need &^= needBuild if b.NeedExport { p.Export = a.built } if need&needCompiledGoFiles != 0 && b.loadCachedGoFiles(a) { need &^= needCompiledGoFiles } // Otherwise, we need to write files to a.Objdir (needVet, needCgoHdr). // Remember that we might have them in cache // and check again after we create a.Objdir. cached = true a.output = []byte{} // start saving output in case we miss any cache results } if need == 0 { return nil } defer b.flushOutput(a) }

--- cmd\go\internal\work\exec.go

这里会判断是否要使用cache,cache的key通过buildActionID计算得到,其会包含很多的东西,例如各个文件的哈希等。为了学习这份代码我们就不看cache命中的逻辑,而是看cache不命中的逻辑:

defer func() { if err != nil && err != errPrintedOutput { err = fmt.Errorf("go build %s: %v", a.Package.ImportPath, err) } if err != nil && b.IsCmdList && b.NeedError && p.Error == nil { p.Error = &load.PackageError{Err: err.Error()} } }() if cfg.BuildN { // In -n mode, print a banner between packages. // The banner is five lines so that when changes to // different sections of the bootstrap script have to // be merged, the banners give patch something // to use to find its context. b.Print("\n#\n# " + a.Package.ImportPath + "\n#\n\n") } if cfg.BuildV { b.Print(a.Package.ImportPath + "\n") } if a.Package.BinaryOnly { _, err := os.Stat(a.Package.Target) if err == nil { a.built = a.Package.Target a.Target = a.Package.Target if b.NeedExport { a.Package.Export = a.Package.Target } a.buildID = b.fileHash(a.Package.Target) a.Package.Stale = false a.Package.StaleReason = "binary-only package" return nil } a.Package.Stale = true a.Package.StaleReason = "missing or invalid binary-only package" if b.IsCmdList { return nil } return fmt.Errorf("missing or invalid binary-only package; expected file %q", a.Package.Target) }

--- cmd\go\internal\work\exec.go

这里做了些根据cfg产生的动作,不是非常重要,继续往下看代码:

if err := b.Mkdir(a.Objdir); err != nil { return err } objdir := a.Objdir if cached { if need&needCgoHdr != 0 && b.loadCachedCgoHdr(a) { need &^= needCgoHdr } // Load cached vet config, but only if that's all we have left // (need == needVet, not testing just the one bit). // If we are going to do a full build anyway, // we're going to regenerate the files below anyway. if need == needVet && b.loadCachedVet(a) { need &^= needVet } if need == 0 { return nil } } // make target directory dir, _ := filepath.Split(a.Target) if dir != "" { if err := b.Mkdir(dir); err != nil { return err } } gofiles := str.StringList(a.Package.GoFiles) cgofiles := str.StringList(a.Package.CgoFiles) cfiles := str.StringList(a.Package.CFiles) sfiles := str.StringList(a.Package.SFiles) cxxfiles := str.StringList(a.Package.CXXFiles) var objects, cgoObjects, pcCFLAGS, pcLDFLAGS []string if a.Package.UsesCgo() || a.Package.UsesSwig() { if pcCFLAGS, pcLDFLAGS, err = b.getPkgConfigFlags(a.Package); err != nil { return } }

--- cmd\go\internal\work\exec.go

这里会创建相关的obj目录并将相关源码文件赋值给相关变量,接着:

// Run SWIG on each .swig and .swigcxx file. // Each run will generate two files, a .go file and a .c or .cxx file. // The .go file will use import "C" and is to be processed by cgo. if a.Package.UsesSwig() { outGo, outC, outCXX, err := b.swig(a, a.Package, objdir, pcCFLAGS) if err != nil { return err } cgofiles = append(cgofiles, outGo...) cfiles = append(cfiles, outC...) cxxfiles = append(cxxfiles, outCXX...) }

--- cmd\go\internal\work\exec.go

这块是关于SWIG的(上次CC接触SWIG好像还是几年前给Python写CPP模块=-=),这里我们也不会用到,继续往下看:

// If we're doing coverage, preprocess the .go files and put them in the work directory if a.Package.Internal.CoverMode != "" { for i, file := range str.StringList(gofiles, cgofiles) { var sourceFile string var coverFile string var key string if strings.HasSuffix(file, ".cgo1.go") { // cgo files have absolute paths base := filepath.Base(file) sourceFile = file coverFile = objdir + base key = strings.TrimSuffix(base, ".cgo1.go") + ".go" } else { sourceFile = filepath.Join(a.Package.Dir, file) coverFile = objdir + file key = file } coverFile = strings.TrimSuffix(coverFile, ".go") + ".cover.go" cover := a.Package.Internal.CoverVars[key] if cover == nil || base.IsTestFile(file) { // Not covering this file. continue } if err := b.cover(a, coverFile, sourceFile, cover.Var); err != nil { return err } if i < len(gofiles) { gofiles[i] = coverFile } else { cgofiles[i-len(gofiles)] = coverFile } } }

--- cmd\go\internal\work\exec.go

这段代码是关于代码覆盖的,CC这里也没有用到,继续往下看:

// Run cgo. if a.Package.UsesCgo() || a.Package.UsesSwig() { ...... } b.cacheGofiles(a, gofiles) // Running cgo generated the cgo header. need &^= needCgoHdr // Sanity check only, since Package.load already checked as well. if len(gofiles) == 0 { return &load.NoGoError{Package: a.Package} } // Prepare Go vet config if needed. if need&needVet != 0 { buildVetConfig(a, gofiles) need &^= needVet } if need&needCompiledGoFiles != 0 { if !b.loadCachedGoFiles(a) { return fmt.Errorf("failed to cache compiled Go files") } need &^= needCompiledGoFiles } if need == 0 { // Nothing left to do. return nil }

--- cmd\go\internal\work\exec.go

这段代码是关于cgo、swig以及vet的,这里没有用到,因此继续往下看:

// Prepare Go import config. // We start it off with a comment so it can't be empty, so icfg.Bytes() below is never nil. // It should never be empty anyway, but there have been bugs in the past that resulted // in empty configs, which then unfortunately turn into "no config passed to compiler", // and the compiler falls back to looking in pkg itself, which mostly works, // except when it doesn't. var icfg bytes.Buffer fmt.Fprintf(&icfg, "# import config\n") for i, raw := range a.Package.Internal.RawImports { final := a.Package.Imports[i] if final != raw { fmt.Fprintf(&icfg, "importmap %s=%s\n", raw, final) } } for _, a1 := range a.Deps { p1 := a1.Package if p1 == nil || p1.ImportPath == "" || a1.built == "" { continue } fmt.Fprintf(&icfg, "packagefile %s=%s\n", p1.ImportPath, a1.built) } if p.Internal.BuildInfo != "" && cfg.ModulesEnabled { if err := b.writeFile(objdir+"_gomod_.go", load.ModInfoProg(p.Internal.BuildInfo)); err != nil { return err } gofiles = append(gofiles, objdir+"_gomod_.go") }

--- cmd\go\internal\work\exec.go

这段代码是关于import config的,CC查了一下没有查到其作用,感觉是个类似宏指令的东西。继续往下看:

// Compile Go. objpkg := objdir + "_pkg_.a" ofile, out, err := BuildToolchain.gc(b, a, objpkg, icfg.Bytes(), len(sfiles) > 0, gofiles) if len(out) > 0 { output := b.processOutput(out) if p.Module != nil && !allowedVersion(p.Module.GoVersion) { output += "note: module requires Go " + p.Module.GoVersion } b.showOutput(a, a.Package.Dir, a.Package.Desc(), output) if err != nil { return errPrintedOutput } } if err != nil { if p.Module != nil && !allowedVersion(p.Module.GoVersion) { b.showOutput(a, a.Package.Dir, a.Package.Desc(), "note: module requires Go "+p.Module.GoVersion) } return err } if ofile != objpkg { objects = append(objects, ofile) }

--- cmd\go\internal\work\exec.go

这里从注释看应该是调用工具链进行编译工作了,干活的为

BuildToolchain.gc

。这个方法CC后面再看,先把这个Func后面的逻辑看完:

// Copy .h files named for goos or goarch or goos_goarch // to names using GOOS and GOARCH. // For example, defs_linux_amd64.h becomes defs_GOOS_GOARCH.h. _goos_goarch := "_" + cfg.Goos + "_" + cfg.Goarch _goos := "_" + cfg.Goos _goarch := "_" + cfg.Goarch for _, file := range a.Package.HFiles { name, ext := fileExtSplit(file) switch { case strings.HasSuffix(name, _goos_goarch): targ := file[:len(name)-len(_goos_goarch)] + "_GOOS_GOARCH." + ext if err := b.copyFile(objdir+targ, filepath.Join(a.Package.Dir, file), 0666, true); err != nil { return err } case strings.HasSuffix(name, _goarch): targ := file[:len(name)-len(_goarch)] + "_GOARCH." + ext if err := b.copyFile(objdir+targ, filepath.Join(a.Package.Dir, file), 0666, true); err != nil { return err } case strings.HasSuffix(name, _goos): targ := file[:len(name)-len(_goos)] + "_GOOS." + ext if err := b.copyFile(objdir+targ, filepath.Join(a.Package.Dir, file), 0666, true); err != nil { return err } } } for _, file := range cfiles { out := file[:len(file)-len(".c")] + ".o" if err := BuildToolchain.cc(b, a, objdir+out, file); err != nil { return err } objects = append(objects, out) } // Assemble .s files. if len(sfiles) > 0 { ofiles, err := BuildToolchain.asm(b, a, sfiles) if err != nil { return err } objects = append(objects, ofiles...) }

--- cmd\go\internal\work\exec.go

这边是处理.h的头文件以及.c和.s的汇编文件,这些应该都没有用到。继续往下看:

// For gccgo on ELF systems, we write the build ID as an assembler file. // This lets us set the SHF_EXCLUDE flag. // This is read by readGccgoArchive in cmd/internal/buildid/buildid.go. if a.buildID != "" && cfg.BuildToolchainName == "gccgo" { switch cfg.Goos { case "android", "dragonfly", "freebsd", "linux", "netbsd", "openbsd", "solaris": asmfile, err := b.gccgoBuildIDELFFile(a) if err != nil { return err } ofiles, err := BuildToolchain.asm(b, a, []string{asmfile}) if err != nil { return err } objects = append(objects, ofiles...) } } // NOTE(rsc): On Windows, it is critically important that the // gcc-compiled objects (cgoObjects) be listed after the ordinary // objects in the archive. I do not know why this is. // https://golang.org/issue/2601 objects = append(objects, cgoObjects...) // Add system object files. for _, syso := range a.Package.SysoFiles { objects = append(objects, filepath.Join(a.Package.Dir, syso)) }

--- cmd\go\internal\work\exec.go

这里是针对linux下ELF格式的,也没有用到,加上后面的代码主要就是生产objects列表,继续往下看:

// Pack into archive in objdir directory. // If the Go compiler wrote an archive, we only need to add the // object files for non-Go sources to the archive. // If the Go compiler wrote an archive and the package is entirely // Go sources, there is no pack to execute at all. if len(objects) > 0 { if err := BuildToolchain.pack(b, a, objpkg, objects); err != nil { return err } } if err := b.updateBuildID(a, objpkg, true); err != nil { return err } a.built = objpkg return nil }

--- cmd\go\internal\work\exec.go

这里调用了工具链中的pack工具把object文件打包,这里其实和gcc对c的处理是一样的(.o和.a)。

现在来看下

BuildToolchain.gc

的实现。

func (gcToolchain) gc(b *Builder, a *Action, archive string, importcfg []byte, asmhdr bool, gofiles []string) (ofile string, output []byte, err error) {

--- cmd\go\internal\work\gc.go

这里参数中的gofiiles在CC的例子里就是lib.go和main.go。在gc的主要部分,都是用于根据当前的信息生成相关参数:

p := a.Package objdir := a.Objdir if archive != "" { ofile = archive } else { out := "_go_.o" ofile = objdir + out } pkgpath := p.ImportPath if cfg.BuildBuildmode == "plugin" { pkgpath = pluginPath(a) } else if p.Name == "main" && !p.Internal.ForceLibrary { pkgpath = "main" } gcargs := []string{"-p", pkgpath} if p.Standard { gcargs = append(gcargs, "-std") } compilingRuntime := p.Standard && (p.ImportPath == "runtime" || strings.HasPrefix(p.ImportPath, "runtime/internal")) // The runtime package imports a couple of general internal packages. if p.Standard && (p.ImportPath == "internal/cpu" || p.ImportPath == "internal/bytealg") { compilingRuntime = true } if compilingRuntime { // runtime compiles with a special gc flag to check for // memory allocations that are invalid in the runtime package, // and to implement some special compiler pragmas. gcargs = append(gcargs, "-+") } // If we're giving the compiler the entire package (no C etc files), tell it that, // so that it can give good error messages about forward declarations. // Exceptions: a few standard packages have forward declarations for // pieces supplied behind-the-scenes by package runtime. extFiles := len(p.CgoFiles) + len(p.CFiles) + len(p.CXXFiles) + len(p.MFiles) + len(p.FFiles) + len(p.SFiles) + len(p.SysoFiles) + len(p.SwigFiles) + len(p.SwigCXXFiles) if p.Standard { switch p.ImportPath { case "bytes", "internal/poll", "net", "os", "runtime/pprof", "runtime/trace", "sync", "syscall", "time": extFiles++ } } if extFiles == 0 { gcargs = append(gcargs, "-complete") } if cfg.BuildContext.InstallSuffix != "" { gcargs = append(gcargs, "-installsuffix", cfg.BuildContext.InstallSuffix) } if a.buildID != "" { gcargs = append(gcargs, "-buildid", a.buildID) } platform := cfg.Goos + "/" + cfg.Goarch if p.Internal.OmitDebug || platform == "nacl/amd64p32" || cfg.Goos == "plan9" || cfg.Goarch == "wasm" { gcargs = append(gcargs, "-dwarf=false") } if strings.HasPrefix(runtimeVersion, "go1") && !strings.Contains(os.Args[0], "go_bootstrap") { gcargs = append(gcargs, "-goversion", runtimeVersion) } gcflags := str.StringList(forcedGcflags, p.Internal.Gcflags) if compilingRuntime { // Remove -N, if present. // It is not possible to build the runtime with no optimizations, // because the compiler cannot eliminate enough write barriers. for i := 0; i < len(gcflags); i++ { if gcflags[i] == "-N" { copy(gcflags[i:], gcflags[i+1:]) gcflags = gcflags[:len(gcflags)-1] i-- } } } args := []interface{}{cfg.BuildToolexec, base.Tool("compile"), "-o", ofile, "-trimpath", trimDir(a.Objdir), gcflags, gcargs, "-D", p.Internal.LocalPrefix} if importcfg != nil { if err := b.writeFile(objdir+"importcfg", importcfg); err != nil { return "", nil, err } args = append(args, "-importcfg", objdir+"importcfg") } if ofile == archive { args = append(args, "-pack") } if asmhdr { args = append(args, "-asmhdr", objdir+"go_asm.h") } // Add -c=N to use concurrent backend compilation, if possible. if c := gcBackendConcurrency(gcflags); c > 1 { args = append(args, fmt.Sprintf("-c=%d", c)) } for _, f := range gofiles { args = append(args, mkAbs(p.Dir, f)) }

--- cmd\go\internal\work\gc.go

最后的两行才真正开始执行命令:

output, err = b.runOut(p.Dir, nil, args...) return ofile, output, err }

--- cmd\go\internal\work\gc.go

可以看到runOut应该是个公用的方法,类似linkAction之类的肯定也会用到。其源码为:

// runOut runs the command given by cmdline in the directory dir. // It returns the command output and any errors that occurred. func (b *Builder) runOut(dir string, env []string, cmdargs ...interface{}) ([]byte, error) { fmt.Printf("dir: %s, env: %v, cmdargs: %v\n", dir, env, cmdargs) cmdline := str.StringList(cmdargs...) for _, arg := range cmdline { // GNU binutils commands, including gcc and gccgo, interpret an argument // @foo anywhere in the command line (even following --) as meaning // "read and insert arguments from the file named foo." // Don't say anything that might be misinterpreted that way. if strings.HasPrefix(arg, "@") { return nil, fmt.Errorf("invalid command-line argument %s in command: %s", arg, joinUnambiguously(cmdline)) } } if cfg.BuildN || cfg.BuildX { var envcmdline string for _, e := range env { if j := strings.IndexByte(e, '='); j != -1 { if strings.ContainsRune(e[j+1:], '\'') { envcmdline += fmt.Sprintf("%s=%q", e[:j], e[j+1:]) } else { envcmdline += fmt.Sprintf("%s='%s'", e[:j], e[j+1:]) } envcmdline += " " } } envcmdline += joinUnambiguously(cmdline) b.Showcmd(dir, "%s", envcmdline) if cfg.BuildN { return nil, nil } } var buf bytes.Buffer cmd := exec.Command(cmdline[0], cmdline[1:]...) cmd.Stdout = &buf cmd.Stderr = &buf cleanup := passLongArgsInResponseFiles(cmd) defer cleanup() cmd.Dir = dir cmd.Env = base.MergeEnvLists(env, base.EnvForDir(cmd.Dir, os.Environ())) err := cmd.Run() // err can be something like 'exit status 1'. // Add information about what program was running. // Note that if buf.Bytes() is non-empty, the caller usually // shows buf.Bytes() and does not print err at all, so the // prefix here does not make most output any more verbose. if err != nil { err = errors.New(cmdline[0] + ": " + err.Error()) } return buf.Bytes(), err }

--- cmd\go\internal\work\exec.go

当前在CC的例子里,其调用的几个参数的值为:

*  dir: C:\Users\thuanqin\AppData\Local\Temp\gorun
*  env: []
*  cmdargs: [[] C:\Go\pkg\tool\windows_amd64\compile.exe -o C:\Users\thuanqin\AppData\Local\Temp\go-build155327718\b001\_pkg_.a -trimpath C:\Users\thuanqin\AppData\Local\Temp\go-build155327718\b001 [] [-p main -complete -buildid 1sTsABbygTANGeo-wT-Q/1sTsABbygTANGeo-wT-Q -dwarf=false -goversion go1.11.4] -D _/C_/Users/thuanqin/AppData/Local/Temp/gorun -importcfg C:\Users\thuanqin\AppData\Local\Temp\go-build155327718\b001\importcfg -pack -c=4 C:\Users\thuanqin\AppData\Local\Temp\gorun\lib.go C:\Users\thuanqin\AppData\Local\Temp\gorun\main.go]

结合runOut的源码中的exec.Command可以知道,这里主要是基于compile命令执行的编译动作。因此来看下compile的源码,其main的内容如下:

func main() { // disable timestamps for reproducible output log.SetFlags(0) log.SetPrefix("compile: ") archInit, ok := archInits[objabi.GOARCH] if !ok { fmt.Fprintf(os.Stderr, "compile: unknown architecture %q\n", objabi.GOARCH) os.Exit(2) } gc.Main(archInit) gc.Exit(0) }

--- cmd\compile\main.go

另外在compile的源码下有个README的介绍文件,介绍了这个go compile的一些概念,尤其是里边指出了go/*中的包并没有被这个compile使用,这些包是用于后期gofmt等工具操作源码用的,因此大家如果在看源码的时候不要被go/parser给绕进去(汗...本来CC打算看看代码熟悉一下的,然后看着看着居然说这里不用这些工具=-=)。

从README中可以看到,gc主要分为四个阶段:

*  cmd/compile/internal/syntax阶段,用于做词法分析以及生成CST,类似传统编译结构中的前端
*  cmd/compile/internal/gc阶段,用于生产AST,进行类型检查,以及对AST进行一定优化而对其做的一些转换,类似于传统编译结构中的中端
*  cmd/compile/internal/ssa阶段,以及之前的cmd/compile/internal/gc阶段,用于生成SSA格式的指令。在这个阶段也会进行一定程度的优化。
*  cmd/internal/obj阶段,以及之前的cmd/compile/internal/ssa阶段,用于生成机器字节码,类似于传统编译结构中的后端

CC的项目重点是要得到AST,因此下面的代码主要专注于前两个阶段。来看gc.Main的实现。gc.Main的前半部分都是在进行参数的解析,毕竟其是一个命令行工具,因此这里就不贴了。其重点的框架代码如下:

// Main parses flags and Go source files specified in the command-line // arguments, type-checks the parsed Go package, compiles functions to machine // code, and finally writes the compiled package definition to disk. func Main(archInit func(*Arch)) { timings.Start("fe", "init") ...... archInit(&thearch) ...... startProfile() ...... initUniverse() ...... timings.Start("fe", "loadsys") loadsys() timings.Start("fe", "parse") lines := parseFiles(flag.Args()) timings.Stop() timings.AddEvent(int64(lines), "lines") finishUniverse() ...... defercheckwidth() timings.Start("fe", "typecheck", "top1") for i := 0; i < len(xtop); i++ { n := xtop[i] if op := n.Op; op != ODCL && op != OAS && op != OAS2 && (op != ODCLTYPE || !n.Left.Name.Param.Alias) { xtop[i] = typecheck(n, Etop) } } ...... timings.Start("fe", "typecheck", "top2") for i := 0; i < len(xtop); i++ { n := xtop[i] if op := n.Op; op == ODCL || op == OAS || op == OAS2 || op == ODCLTYPE && n.Left.Name.Param.Alias { xtop[i] = typecheck(n, Etop) } } resumecheckwidth() ...... timings.Start("fe", "typecheck", "func") var fcount int64 for i := 0; i < len(xtop); i++ { n := xtop[i] if op := n.Op; op == ODCLFUNC || op == OCLOSURE { Curfn = n decldepth = 1 saveerrors() typecheckslice(Curfn.Nbody.Slice(), Etop) checkreturn(Curfn) if nerrors != 0 { Curfn.Nbody.Set(nil) // type errors; do not compile } // Now that we've checked whether n terminates, // we can eliminate some obviously dead code. deadcode(Curfn) fcount++ } } // With all types ckecked, it's now safe to verify map keys. checkMapKeys() timings.AddEvent(fcount, "funcs") ...... timings.Start("fe", "capturevars") for _, n := range xtop { if n.Op == ODCLFUNC && n.Func.Closure != nil { Curfn = n capturevars(n) } } ...... timings.Start("fe", "inlining") if Debug_typecheckinl != 0 { // Typecheck imported function bodies if debug['l'] > 1, // otherwise lazily when used or re-exported. for _, n := range importlist { if n.Func.Inl != nil { saveerrors() typecheckinl(n) } } if nsavederrors+nerrors != 0 { errorexit() } } ...... timings.Start("fe", "escapes") escapes(xtop) ...... if dolinkobj { timings.Start("fe", "xclosures") for _, n := range xtop { if n.Op == ODCLFUNC && n.Func.Closure != nil { Curfn = n transformclosure(n) } } ...... initssaconfig() ...... peekitabs() ...... timings.Start("be", "compilefuncs") fcount = 0 for i := 0; i < len(xtop); i++ { n := xtop[i] if n.Op == ODCLFUNC { funccompile(n) fcount++ } } ...... compileFunctions() ...... } timings.Start("be", "externaldcls") for i, n := range externdcl { if n.Op == ONAME { externdcl[i] = typecheck(externdcl[i], Erv) } } ...... timings.Start("be", "dumpobj") dumpobj() if asmhdr != "" { dumpasmhdr() } ...... timings.Stop() ...... }

--- cmd\compile\internal\gc\main.go

这里CC把这个方法的重点代码都列出来了,其实由于这里要做timing,因此每个timings.Start其实就开始了一个阶段。看下和parse相关的代码,首先是initUniverse:

// initUniverse initializes the universe block. func initUniverse() { lexinit() typeinit() lexinit1() }

--- cmd\compile\internal\gc\universe.go

initUniverse主要是初始化一些类型和已知符号、整型的极值等信息,用于在做词法分析的时候可以正确的识别到。在完成初始化后parseFiles会进行相关的编译前端工作,生成具体的CST:

// parseFiles concurrently parses files into *syntax.File structures. // Each declaration in every *syntax.File is converted to a syntax tree // and its root represented by *Node is appended to xtop. // Returns the total count of parsed lines. func parseFiles(filenames []string) uint { var noders []*noder // Limit the number of simultaneously open files. sem := make(chan struct{}, runtime.GOMAXPROCS(0)+10) for _, filename := range filenames { p := &noder{ basemap: make(map[*syntax.PosBase]*src.PosBase), err: make(chan syntax.Error), } noders = append(noders, p) go func(filename string) { sem <- struct{}{} defer func() { <-sem }() defer close(p.err) base := syntax.NewFileBase(filename) f, err := os.Open(filename) if err != nil { p.error(syntax.Error{Pos: syntax.MakePos(base, 0, 0), Msg: err.Error()}) return } defer f.Close() p.file, _ = syntax.Parse(base, f, p.error, p.pragma, syntax.CheckBranches) // errors are tracked via p.error }(filename) } var lines uint for _, p := range noders { for e := range p.err { p.yyerrorpos(e.Pos, "%s", e.Msg) } p.node() lines += p.file.Lines p.file = nil // release memory if nsyntaxerrors != 0 { errorexit() } // Always run testdclstack here, even when debug_dclstack is not set, as a sanity measure. testdclstack() } localpkg.Height = myheight return lines }

--- cmd\compile\internal\gc\noder.go

这里会并行的处理各个文件,处理的工作由

syntax.Parse

负责,其源码如下:

// Parse parses a single Go source file from src and returns the corresponding // syntax tree. If there are errors, Parse will return the first error found, // and a possibly partially constructed syntax tree, or nil. // // If errh != nil, it is called with each error encountered, and Parse will // process as much source as possible. In this case, the returned syntax tree // is only nil if no correct package clause was found. // If errh is nil, Parse will terminate immediately upon encountering the first // error, and the returned syntax tree is nil. // // If pragh != nil, it is called with each pragma encountered. // func Parse(base *PosBase, src io.Reader, errh ErrorHandler, pragh PragmaHandler, mode Mode) (_ *File, first error) { defer func() { if p := recover(); p != nil { if err, ok := p.(Error); ok { first = err return } panic(p) } }() var p parser p.init(base, src, errh, pragh, mode) p.next() return p.fileOrNil(), p.first }

--- cmd\compile\internal\syntax\syntax.go

这里就是一个典型的由词法分析到生成CST的流程,首先init进行初始化,然后next获取下第一个token,接着fileOrNil负责分析。因此重点看下fileOrNil:

// ---------------------------------------------------------------------------- // Package files // // Parse methods are annotated with matching Go productions as appropriate. // The annotations are intended as guidelines only since a single Go grammar // rule may be covered by multiple parse methods and vice versa. // // Excluding methods returning slices, parse methods named xOrNil may return // nil; all others are expected to return a valid non-nil node. // SourceFile = PackageClause ";" { ImportDecl ";" } { TopLevelDecl ";" } . func (p *parser) fileOrNil() *File { if trace { defer p.trace("file")() } f := new(File) f.pos = p.pos() // PackageClause if !p.got(_Package) { p.syntaxError("package statement must be first") return nil } f.PkgName = p.name() p.want(_Semi) // don't bother continuing if package clause has errors if p.first != nil { return nil } // { ImportDecl ";" } for p.got(_Import) { f.DeclList = p.appendGroup(f.DeclList, p.importDecl) p.want(_Semi) } // { TopLevelDecl ";" } for p.tok != _EOF { switch p.tok { case _Const: p.next() f.DeclList = p.appendGroup(f.DeclList, p.constDecl) case _Type: p.next() f.DeclList = p.appendGroup(f.DeclList, p.typeDecl) case _Var: p.next() f.DeclList = p.appendGroup(f.DeclList, p.varDecl) case _Func: p.next() if d := p.funcDeclOrNil(); d != nil { f.DeclList = append(f.DeclList, d) } default: if p.tok == _Lbrace && len(f.DeclList) > 0 && isEmptyFuncDecl(f.DeclList[len(f.DeclList)-1]) { // opening { of function declaration on next line p.syntaxError("unexpected semicolon or newline before {") } else { p.syntaxError("non-declaration statement outside function body") } p.advance(_Const, _Type, _Var, _Func) continue } // Reset p.pragma BEFORE advancing to the next token (consuming ';') // since comments before may set pragmas for the next function decl. p.pragma = 0 if p.tok != _EOF && !p.got(_Semi) { p.syntaxError("after top level declaration") p.advance(_Const, _Type, _Var, _Func) } } // p.tok == _EOF f.Lines = p.source.line return f }

--- cmd\compile\internal\syntax\parser.go

从注释中可以看到,每个文件一定是下面的格式:

SourceFile = PackageClause ";" { ImportDecl ";" } { TopLevelDecl ";" } .

--- 

因此这里的代码就是在将源码生成这个格式的CST,生成的逻辑分别由着三行注释引出:

*  // PackageClause
*  // { ImportDecl ";" }
*  // { TopLevelDecl ";" }

这里生成CST的代码都是手写的递归代码,并没有基于某些工具进行生成。例如在TopLevelDecl中,可以看到在_Func类型的时候,会去尝试获取Func类型的CST节点,对应的代码为:

// FunctionDecl = "func" FunctionName ( Function | Signature ) . // FunctionName = identifier . // Function = Signature FunctionBody . // MethodDecl = "func" Receiver MethodName ( Function | Signature ) . // Receiver = Parameters . func (p *parser) funcDeclOrNil() *FuncDecl { if trace { defer p.trace("funcDecl")() } f := new(FuncDecl) f.pos = p.pos() if p.tok == _Lparen { rcvr := p.paramList() switch len(rcvr) { case 0: p.error("method has no receiver") default: p.error("method has multiple receivers") fallthrough case 1: f.Recv = rcvr[0] } } if p.tok != _Name { p.syntaxError("expecting name or (") p.advance(_Lbrace, _Semi) return nil } f.Name = p.name() f.Type = p.funcType() if p.tok == _Lbrace { f.Body = p.funcBody() } f.Pragma = p.pragma return f }

--- cmd\compile\internal\syntax\parser.go

parseFiles完成后,CST就生成完毕了。在这之后就是做类型检查,类型检查主要基于typecheck进行:

// typecheck type checks node n. // The result of typecheck MUST be assigned back to n, e.g. // n.Left = typecheck(n.Left, top) func typecheck(n *Node, top int) *Node { // cannot type check until all the source has been parsed if !typecheckok { Fatalf("early typecheck") }

--- cmd\compile\internal\gc\typecheck.go

其逻辑主要就是判断各个node是否类型正确,同时进行类型信息的生成等,CC这里就不列出来了。

Link Action

下面来看下linkAction的Func逻辑。和上面compileAction一样,其Func会调用

BuildToolchain.ld

进行真正的链接工作,后者则会调用link执行相关动作。几个run的参数如下:

*  dir: .
*  env: []
*  cmdargs: [[] C:\Go\pkg\tool\windows_amd64\link.exe -o C:\Users\thuanqin\AppData\Local\Temp\go-build155327718\b001\exe\lib.exe -importcfg C:\Users\thuanqin\AppData\Local\Temp\go-build155327718\b001\importcfg.link [-s -w -buildmode=exe -buildid=in4DY2QOaiviAn8Vujza/1sTsABbygTANGeo-wT-Q/3_zHTJijC6DHSy2dES3S/in4DY2QOaiviAn8Vujza -extld=gcc] C:\Users\thuanqin\AppData\Local\Temp\go-build155327718\b001\_pkg_.a]

link的main如下:

// The bulk of the linker implementation lives in cmd/link/internal/ld. // Architecture-specific code lives in cmd/link/internal/GOARCH. // // Program initialization: // // Before any argument parsing is done, the Init function of relevant // architecture package is called. The only job done in Init is // configuration of the architecture-specific variables. // // Then control flow passes to ld.Main, which parses flags, makes // some configuration decisions, and then gives the architecture // packages a second chance to modify the linker's configuration // via the ld.Arch.Archinit function. func main() { var arch *sys.Arch var theArch ld.Arch switch objabi.GOARCH { default: fmt.Fprintf(os.Stderr, "link: unknown architecture %q\n", objabi.GOARCH) os.Exit(2) case "386": arch, theArch = x86.Init() case "amd64", "amd64p32": arch, theArch = amd64.Init() case "arm": arch, theArch = arm.Init() case "arm64": arch, theArch = arm64.Init() case "mips", "mipsle": arch, theArch = mips.Init() case "mips64", "mips64le": arch, theArch = mips64.Init() case "ppc64", "ppc64le": arch, theArch = ppc64.Init() case "s390x": arch, theArch = s390x.Init() case "wasm": arch, theArch = wasm.Init() } ld.Main(arch, theArch) }

--- cmd\link\main.go

这里依据体系结构的不同会有不同的代码执行链接操作。由于link对CC要做的项目并没有什么作用,因此就不细看了。值得注意的是,在做LSP Server的时候如果要做GoToSymbleDefinition的动作时,需要在link这里找源码,因为经过上面的分析可以看到在compile阶段是不会做这个事情的。

Go Run Action

最后来看下go run的action的逻辑。其Func的代码为:

// buildRunProgram is the action for running a binary that has already // been compiled. We ignore exit status. func buildRunProgram(b *work.Builder, a *work.Action) error { cmdline := str.StringList(work.FindExecCmd(), a.Deps[0].Target, a.Args) if cfg.BuildN || cfg.BuildX { b.Showcmd("", "%s", strings.Join(cmdline, " ")) if cfg.BuildN { return nil } } base.RunStdin(cmdline) return nil }

--- cmd\go\internal\run\run.go

可以看到其逻辑非常简单,就是执行一下cmdline,在CC这里cmdline的内容为 C:\Users\thuanqin\AppData\Local\Temp\go-build672636426\b001\exe\lib.exe,这里名字叫lib.exe是由于上面提到的,在temp里会随机的取第一个go文件的文件名作为最后的target的名字。

至此整个go run的执行流就基本初略的看完了。

本文微信分享二维码


本文由星空水景整理编写。
如需转载请注明出处并保留文章所有引用的资料来源。
欢迎关注 星空水景 微博[微博搜索 星空水景 或扫描下方二维码]。