Known limitations for Go support

Before you start using Go application monitoring, make sure that you are aware of the known limitations.

Support limited to official, stable Go releases

Go support is limited to official, stable Go releases compiled with the Golang toolchain.

OneAgent doesn't support binaries compiled using the gccgo toolchain.

Application binaries must be dynamically linked

OneAgent fully automatic injection requires dynamically linked application binaries. Dynamic linking is automatically applied when the application uses certain standard runtime library packages, for example, net/http.

In all other cases, you can enforce dynamic linking through the -ldflags '-linkmode=external' command line option. Note that disabling cgo, for example, using CGO_ENABLED=0, is not supported, and OneAgent will reject the resulting application binary.

gocopydownload
package main
import "fmt"

func main() {
    fmt.Print("Enter text: ")
    var input string
    fmt.Scanln(&input)
    fmt.Print(input)
}

shellcopydownload
go build GoMinimal.go
file GoMinimal
GoMinimal: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped

shellcopydownload
go build -ldflags '-linkmode=external' GoMinimal.go
file GoMinimal
GoMinimal: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32

Applications can't be built with -linkshared option

Go supports dynamic linking of the Go standard library. This build mode is rarely used, and OneAgent won't inject into applications built this way.

shellcopydownload
go install -buildmode=shared -linkshared std
go build -linkshared GoMinimal.go

Applications that load Go plugins aren't supported

A Go plugin is a package compiled using the -buildmode=plugin build flag to produce a shared object file. This build mode is rarely used, and OneAgent will disable deep monitoring when an application actually loads a Go plugin.

Vendored third-party packages aren't supported

Go vendoring is used to include local copies of external dependencies in the project repository. This approach was used to pin versions of third-party packages before Go module support was added.

OneAgent will not monitor vendored packages. For example, gRPC services are supported only if you use Go modules or if you import go-grpc directly without using a dependency management system.

Applications must contain a symbol table

OneAgent relies on the information stored in the symbol table of the application binary. By default, Go generates a symbol table into the application binary. However, if you suppress this with certain command line parameters or tools, OneAgent won't be able to monitor an application that has no application binary symbol table.

• Don't use the external tool strip (strip <Go binary>).
• Don't compile with go build -ldflags "-s". This flag strips away the symbol table.
• Don't run go run <application>. This rarely-used command builds and runs applications on the fly. Because the output application file is temporary (the file is deleted automatically after the app termination), the application binary contains no symbol table.

Applications built with race detector enabled aren't supported

An application built with -race flag contains a built-in data race detector. This build mode is mostly used in a development environment and OneAgent won't inject into applications built this way.

Creation stack profiling of OS threads is disabled

OneAgent does not support the predefined threadcreate profile. Thread creation profiling results of Go applications monitored by OneAgent will contain empty stack traces only.

Support for static monitoring

OneAgent version 1.203+

Automatic injection for statically linked Go binaries is supported by OneAgent if

• The parent process is dynamically linked. This also applies to applications running as container payload.

  Example

  In this example, the parent process is a /bin/sh shell that starts a statically linked Go binary. The following code launches the /bin/sh shell and executes the provided command.

  shellcopydownload
  /bin/sh -c '/StaticGoMinimal <optional app arguments>'

  You can use the file command to verify if an application is dynamically or statically linked.

  shellcopydownload
  $ file -L /bin/sh
  /bin/sh: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked
  $ file -L /StaticGoMinimal
  /StaticGoMinimal: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked

• Kubernetes Classic full-stack injection The statically linked Go binary is running as Docker container entrypoint.

  Example

  dockerfilecopydownload
  FROM alpine:3.11
  COPY StaticGoMinimal /
  ENTRYPOINT ["/StaticGoMinimal"]

To enable Go static application monitoring, see Enable Go static monitoring.

Limitations

• Static Go applications running as container entrypoints are not supported in cloud-native full-stack injection.

  Automatic injection of statically linked Go applications running as container entrypoints is not supported when using the cloud-native full-stack injection deployment option in Kubernetes/OpenShift.

  Workaround and example

  To overcome this limitation, you can change the container entrypoint from a statically linked Go application to a dynamically linked application such as a shell.

  For example, in a cloud-native full-stack injection deployment, the following code starts an unsupported statically linked Go application.

  dockerfilecopydownload
  FROM alpine:3.11
  COPY StaticGoMinimal /
  ENTRYPOINT ["/StaticGoMinimal"]

  By changing the container entryproint from a statically linked Go application to a dynamically linked application, we obtain the following code that launches /bin/sh and executes the /StaticGoMinimal command.

  dockerfilecopydownload
  FROM alpine:3.11
  COPY StaticGoMinimal /
  ENTRYPOINT ["/bin/sh", "-c", "'/StaticGoMinimal'"]

• Static Go applications that use cgo are not supported.

  OneAgent rejects monitoring of static Go binaries that use cgo and, therefore, have a static dependency on the C system library libc. This is because the statically linked version of libc might conflict with the one used by OneAgent.

  Workaround

  To overcome this limitation, you can build the Go application as a dynamically linked executable that dynamically links to libc. This will ensure that both the Go application and OneAgent use the same version of libc, which is the one available on the host.

• Static Go monitoring requires the SYS_PTRACE capability in Docker containers.

  The SYS_PTRACE capability is enabled by default for Docker 19.03.0+ and Linux Kernel 4.8+. It allows system calls between processes running in a container, which is a requirement for Go static monitoring.

  Workaround and example

  You can overcome this limitation for Docker versions earlier than 19.03.0 or Linux Kernel versions earlier than 4.8 by running the container with the SYS_PTRACE capability as shown below.

  shellcopydownload
  docker run --cap-add=SYS_PTRACE <container> ...

• Docker images that don't provide a C system library are not supported.

  OneAgent requires a C system library to be available on the monitored host.

  Workaround and example

  To overcome this limitation, you can change the base image of a container to one that provides a C system library.

  An example of a Docker image that doesn't provide a C system library is the scratch image.

  dockerfilecopydownload
  FROM scratch
  COPY StaticGoMinimal /
  CMD ["/StaticGoMinimal"]

  An example of an image that provides a C system library is the alpine image.

  dockerfilecopydownload
  FROM alpine:3.11
  COPY StaticGoMinimal /
  CMD ["/StaticGoMinimal"]

Side effects

The file proc/<pId>/exe refers to an executable named oneagentdynamizer instead of the Go application binary, it is contained in the proc pseudo-filesystem that provides an interface to kernel data structures of running processes. This may cause system tools like ps or top to display oneagentdynamizer instead of the Go binary name in their output.

Support for musl libc

The musl libc library is a drop-in replacement for the glibc library. Dynatrace supports musl-based Go applications, such as those built on Alpine Linux.

There is one additional requirement for building a dynamically linked application binary. You should use the Go toolchain for alpine (golang:<version>-alpine) and add -ldflags '-linkmode external' to the build command line to enforce usage of the system linker. This is not required for statically linked Go applications watched by Go static monitoring.

gocopydownload
package main
import "fmt"

func main() {
    fmt.Print("Enter text: ")
    var input string
    fmt.Scanln(&input)
    fmt.Print(input)
}

dockerfilecopydownload
# --- Stage 1:
# Use Golang toolchain for alpine to build the application.
FROM golang:1.13.5-alpine as builder
RUN apk update && apk add gcc libc-dev

# Copy local code, for example, GoMinimal.go, to the container image.
COPY ./GoMinimal.go ./GoMinimal.go

# Build dynamically linked Go binary.
RUN go build -ldflags '-linkmode=external' GoMinimal.go

# --- Stage 2:
# Use a Docker multi-stage build to create a lean production image.
FROM alpine:3.11
# Install ca-certificates and libc6-compat for Go programs to work properly.
RUN apk add --no-cache ca-certificates libc6-compat

# Copy the binary to the production image from the builder stage.
COPY --from=builder /go/GoMinimal /GoMinimal

# Run the application on container startup.
CMD ["/GoMinimal"]

shellcopydownload
docker build -t gominimal-alpine .
docker run --interactive gominimal-alpine