Production Minimus container images do not include a shell. While we recomment most customers use the -dev tagged Minimus images for troubleshooting, sometimes it may be necessary to debug your running container built from a Minimus Production image.

1

Prepare the debugger container

We can use the Minimus Busybox image to pre-configure our debugger container. We will create the container (without running it), extract its filesystem, and place it in a local directory.

docker create --name debugger /
reg.lab.mini.dev/busybox:latest-dev
# Create a container based on the Minimus busybox image

mkdir debugger
# Create a local directory to store the filesystem

docker export debugger | tar -xC debugger
# `docker export` saves the root filesystem of the container to a tar archive
# The pipe `|` passes the output directly into the next command, to avoid creating an intermediate tar file
# `tar -xC debugger` extracts the archive files and saves them to the local directory
2

Start the target container and mount the debugger image

docker run -d --rm \
  -v $(pwd)/debugger:/.debugger \
  --name my-image {Your distroless Minimus image}
3

Start the debugging session

Start the debug session, reusing the namespaces and cgroups from the target container (my-image). This is possible because the two containers share namespaces and cgroups.

$ docker exec -it my-image /.debugger/bin/sh
3

Update the system's executable search path

Before we can use the debugging tools from the mount, we’ll first need to modify the PATH environment variable by appending (or prepending) /.debugger/bin to it. This works because the busybox toolkit in our mounted volume is statically linked.

/ $# export PATH=${PATH}:/.debugger/bin   
#   The target container takes precedence

You can either prepend or append the debugger container. It just depends how you want the system to behave in the case of a name collision:

  • Appending the debugger means the binaries from the target container will take precedence.
  • Prepending the debugger means the binaries from the debugger container will take precedence.
5

Ready to debug!

That’s it. You are now ready to use the busybox toolkit as if it were natively in the target container.

For example, you can explore the filesystem of the target container and list the running processes and network interfaces:

# List files and directories in long format
ls -l
# List all running processes on the system in full detail
ps -ef
# List IP addresses for all network interfaces
ip addr

The output should show you the information for your target Minimus container, as if you were running the command directly inside it.

This section is inspired by insights from Ivan Velichko’s blog.