Private Go Modules on Azure DevOps

There is a long-running issue with go get imports of Azure DevOps repositories due to the fact that the HTTPS URL contains a “_git” segment:

cmd/go: add support for Azure DevOps Git repos to go get · Issue #28236 · golang/go

The case for using PAT instead of SSH

When a module that is imported in your Go code is missing, Go will attempt to fetch the package using go get path/to/package. If the package is missing in $GOPATH/src and $GOPATH/pkg/mod , an attempt is made with HTTPS.

In order for Git to access the private repository, you may choose to either use a Personal Access Token (PAT) or to register an SSH key and redirect traffic from the HTTPS URL to the SSH equivalent. No matter how you got about it however, there will always be a trailing .git at the end of the repo name.

The big downside to using SSH is that the module name will need to be aliased, which means that anyone consuming your module would also need to use SSH or create another custom Git URL redirect.

With a PAT on the other hand, the URL remains intact and usable.

Using a Personal Access Token

Start off by creating a new Personal Access Token that has read access to code. Here’s a guide that will step you through the process.

My process for creating a PAT is normally:

1. Create a new PAT in the UI
2. Schedule a meeting with the Ops team the date the PAT runs out, set a notification for a week or two before the date
3. Copy the PAT to an Azure Key Vault (or any other Vault available to me)
4. Copy the PAT into an “common” Azure Library that can be used by the build pipelines

Then put the token in the global Git config like so:

git config --global url."https://anythinggoeshere:$devops_token@dev.azure.com".insteadOf "https://dev.azure.com"

The anythinggoeshere is the username, and is disregarded by Azure DevOps in this case.

It is now possible to fetch the Go package with its regular URL + .git :

go get -v dev.azure.com/${ORG}/${PROJECT}/_git/${REPO}.git

Using with Docker

Simply run the .insteadOf Git command in your Dockerfile. IMPORTANT: remove the $HOME/.gitconfig or credentials will be baked into the image / running container.

Note that there are ways to extract build arguments from a Docker image, so it’s not entirely safe, but it’s a best effort.

FROM golang:1.12-alpine3.10 AS builder
RUN apk add --no-cache git bash
WORKDIR /code
ARG devops_token
RUN git config --global url."https://anythinggoeshere:$devops_token@dev.azure.com".insteadOf "https://dev.azure.com"
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN go build -o main .
RUN rm $HOME/.gitconfig

Then pass the token as an argument when running Docker build:

docker build \
  -t $IMAGE:$TAG \
  --build-arg devops_token=$DEVOPS_TOKEN \
  .

Bonus: bootstrapping the DevOps token using Makefile and Azure Key Vault

Putting this in your Makefile in projects will give developers an easy way to keep the URL up to date. Rotating the key would just force everyone to run make set-git-redirect once.

VAULT_NAME ?= "NAME-OF-YOUR-VAULT"
DEVOPS_TOKEN_SECRET_NAME ?= "devops-repo-pat"
set-git-redirect:
 @token=`az keyvault secret show --name $(DEVOPS_TOKEN_SECRET_NAME) --vault-name $(VAULT_NAME) --query value -o tsv`; \
  [ -z "$$token" ] && echo "failed to fetch token..." && exit 1; \
  git config --global --unset $(git config --global --get-regexp "dev.azure.com") &>/dev/null || "true" ; \
  git config --global url."https://anything:$$token@dev.azure.com".insteadOf "https://dev.azure.com"

Something similar can of course be done for the docker build process. YMMV.

Bonus: using an SSH key

I don’t recommend doing this but I’ll keep this for future reference.

Each Azure DevOps HTTPS endpoint has a SSH equivalent:

https://myorg@dev.azure.com/myorg/myproject/_git/myrepo
# has the SSH equivalent
git@ssh.dev.azure.com:v3/myorg/myproject/myrepo.git

Git can be configured to use SSH instead of HTTPS:

git config --global url."git@ssh.dev.azure.com:v3".insteadOf https://dev.azure.com

You can view the results in ~/.gitconfig.

To set up an SSH key on your machine, follow this guide.

Make sure to initialize your modules with the SSH path, including the .git postfix:

go mod init dev.azure.com/myorganization/myproject/mypackage.git

Once the package has been pushed to Azure DevOps, you can now access it with go get:

go get -v -u dev.azure.com/myorganization/myproject/mypackage.git

Also, you can use the module in your other go modules:

package main
import (
    hello "dev.azure.com/myorganization/myproject/mypackage.git/pkg/hello"
)
func main() {
    hello.SayHello()
}

For information on how to use this with Docker, check the SSH section of this guide.