There are sev­eral ways to deploy resources in Azure, but to do it in the most secure and optimal way, we do not have many options. This art­icle presents a solu­tion that stream­lines and sim­pli­fies the access man­age­ment for organ­iz­a­tions using Git­Hub repos­it­or­ies and Ter­ra­form for auto­mated deploy­ment to Azure.

Access to man­aged Azure Resources

When we are talk­ing about the con­tinu­ous integration/continuous deploy­ment for Azure resources, using the ser­vice prin­cipals (SP) is the only way to achieve it. In our case, a ser­vice prin­cipal is the iden­tity used by the auto­ma­tion tools and determ­ines which resources can be used at which level. In com­par­ison, a user account can only be used to run inter­act­ive deploy­ment, which requires the user to log in to Azure first. As the ser­vice prin­cipal can be used to run deploy­ment pipelines from Azure DevOps, Git­Hub or any other auto­ma­tion solu­tion, the main ques­tion will be how to organ­ize access to Azure resources in a secure way. There are two ways the SP can be used to grant access to man­aged Azure resources: dir­ectly through the sub­scrip­tion or the Key Vault.

The simplest way is shown in the pic­ture above. Deploy­ment server con­nects Azure sub­scrip­tion using SP and deploys resources based on the assigned per­mis­sions. In this case, all deploy­ment pipelines that have dir­ect access to sub­scrip­tions can cre­ate, modify, or remove Azure resources. Another approach is to con­nect the deploy­ment server to the sub­scrip­tion with read-only per­mis­sions and keep ser­vice prin­cipal cre­den­tials that have the required per­mis­sions for deploy­ment in the Key Vault, as shown in the fol­low­ing diagram:

This approach gives us more con­trol over alter­a­tions, because the deploy­ment server does not have dir­ect modi­fy­ing per­mis­sions to resources in Azure. Using dif­fer­ent ser­vice prin­cipals for vari­ous pro­jects or repos­it­or­ies increases secur­ity, but also the com­plex­ity of man­aging the ser­vice prin­cipals. To avoid unne­ces­sary com­plic­a­tions, the sub­scrip­tions with devel­op­ment or proof-of-concept envir­on­ments should use the first authen­tic­a­tion method with a ser­vice prin­cipal assigned the role of the owner to facil­it­ate deploy­ment from each repos­it­ory. It makes sense to cre­ate a ser­vice prin­cipal for each deploy­ment ser­vice, like Git­Hub or Azure DevOps. The second method is suit­able for a pro­duc­tion envir­on­ment where con­trol over deploy­ments is more import­ant. The first approach can also be used, but we need to care­fully define the role and scope on both sides, Azure and the deploy­ment ser­vice. The ser­vice prin­cipal name must con­tain inform­a­tion about the pur­pose to be eas­ily iden­ti­fi­able for audit­ing and mon­it­or­ing. A recom­men­ded pat­tern for nam­ing is:

sp-<subscription|application|project>[-<deployment server>]-<role>[-<environment>]

Although the use of spaces is accept­able, you should avoid them for bet­ter read­ab­il­ity and to pre­vent prob­lems with val­id­a­tion rules. To fol­low along you will need an Azure account with a ready to use Azure CLI, an Azure Key Vault and a Git­Hub repository.

Con­nect Actions to Azure in GitHub

Accord­ing to the Microsoft doc­u­ment­a­tion Con­nect Git­Hub and Azure | Microsoft Docs, there are two dif­fer­ent ways for Git­Hub Action authen­tic­a­tion with Azure:

  • OpenID con­nect with an Azure ser­vice prin­cipal using a Fed­er­ated Iden­tity Credential
  • Using ser­vice prin­cipal with secrets

The first method defines Git­Hub scopes (organization/repo/branch) in the ser­vice prin­cipal con­fig­ur­a­tion while the second one requires cre­at­ing secrets in each repos­it­ory that demands access to Azure sub­scrip­tion. Regard­less of which method you choose, you will first need a ser­vice prin­cipal. If we have an Azure sub­scrip­tion datainsights-external-dev, and the ser­vice prin­cipal has the con­trib­utor role on this sub­scrip­tion for Git­Hub ser­vice, the name will be the following:

sp-datainsights-external-dev-github-contributor

1. Cre­ate Git­Hub Ser­vice Principal

To cre­ate a ser­vice prin­cipal, we can use the fol­low­ing AZ CLI command:

sp="sp-datainsights-internal-dev-github-contributor"
subscription="datainsights-internal-dev"
az ad sp create-for-rbac \
    --name $sp \
    --role contributor \
    --scopes /subscriptions/$(az account show \
--subscription $subscription \
--query id \
-o tsv)

The com­mand not only cre­ates the SP, but also attaches the con­trib­utor role to the sub­scrip­tion and provides secur­ity details in the out­put, which will look like the following:

{
    "appId": "00000000-0000-0000-0000-000000000000",
    "displayName": "sp-datainsights-internal-dev-github-contributor",
    "name": "",
    "password": "0000-0000-0000-0000-000000000000",
    "tenant": "00000000-0000-0000-0000-000000000000"
}

This inform­a­tion will be needed for fur­ther steps, so keep it handy.

2. Cre­ate the con­nec­tion between Git­Hub and Azure

As men­tioned above, there are two ways to con­nect the tools:


a) Use the Azure login Action with OpenID connect

To use OpenID Con­nect, we first need to run the next com­mand, which enables fed­er­a­tion with Git­Hub to trust the repository:

APPLICATION_ID="00000000-0000-0000-0000-000000000000"
OBJECT_ID=$(az ad app show --id $APPLICATION_ID --query objectId -o tsv)
CREDENTIAL_NAME="githubCredential"
REPO="repo:DataInsightsGmbH/azure-terraform:ref:refs/heads/main"
az rest --method POST --uri "https://graph.microsoft.com/beta/applications/$OBJECT_ID/federatedIdentityCredentials"
    --body "{
                'name':'$CREDENTIAL_NAME',
                'issuer':'https://token.actions.githubusercontent.com',
                'subject':'$REPO',
                'description':'Testing Github Azure Connection',
                'audiences': ['api://AzureADTokenExchange']
            }"

Be care­ful: the issuer must not con­tain any slashes at the end. It is use­ful to set a descript­ive name for the CREDENTIAL_NAME and use the description field to provide more details. The subject field can be defined, depend­ing on the repos­it­ory work­flow. For repos­it­or­ies that use Git­Hub environments:

repo:<Organization/Repository>:environment:<Name>

For repos­it­or­ies with ref­er­ence to branch or tag:

repo:<Organization/Repository>:ref:<ref path>

An example of the last point would be:

repo:DataInsightsGmbH/azure-terraform:ref:refs/heads/main

Next, we use Git­Hub Actions, the CI/CD tool provided by Git­Hub. Before we cre­ate the Action, we need to cre­ate the secrets that will allow us to con­nect to Azure:

  • AZURE_CLIENT_ID, which is the appID
  • AZURE_SUBSCRIPTION_ID
  • AZURE_TENANT_ID

In your repos­it­ory set­tings go to secrets and then click Actions.

Then the list of secrets and the but­ton for adding new secrets will be dis­played. If you click on it, the fol­low­ing screen will appear:

Now that we have set the secrets, we can con­fig­ure the Action by going to Actions and then set­ting up a simple workflow.

Name the file and copy the fol­low­ing text that uses the Azure login Action:

name: Run Azure Login with OpenID Connect
on: [push]
permissions:
    id-token: write
jobs:
    build-and-deploy:
    runs-on: ubuntu-latest
    steps:
    - name: 'Az CLI login'
      uses: azure/login@v1
      with:
          client-id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
    - name: 'Run Azure CLI commands'
      run: |
          az account show
          az group list
          pwd


b) Use the Azure login Action with a ser­vice prin­cipal secret

The second method is much sim­pler but requires sav­ing cre­den­tials in Git­Hub for each repos­it­ory. This time all that is needed is a Git­Hub Action secret that stores all the inform­a­tion you have received and is named AZURE_CREDENTIALS.

To use ser­vice prin­cipal secrecy for authen­tic­a­tion in Azure, use a work­flow Action like the following:

on: [push]
jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Log in with Azure
        uses: azure/login@v1
        with:
          creds: '${{ secrets.AZURE_CREDENTIALS }}

Unfor­tu­nately, at this moment Ter­ra­form does not sup­port authen­tic­a­tion using az cli with ser­vice prin­cipal and there­fore it is required to use secrets for ser­vice prin­cipal to run Ter­ra­form deployment.

You should now be able to com­mit the changes, which in turn trig­gers a work­flow run. This should run without errors and with the out­put of the com­mands from the last step.

3. Add example Ter­ra­form file

To fully demon­strate the cre­ation of resources, we will add a small Ter­ra­form script which just adds a resource group to the desired loc­a­tion in Azure. Cre­ate a file on the root of the repos­it­ory named main.tf with the next contents:

# Configure the Azure provider
terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 2.65"
    }
  }
  required_version = ">= 1.1.0"
}
provider "azurerm" {
  features {}
}
resource "azurerm_resource_group" "rg" {
  name     = "myTFResourceGroup"
  location = "westeurope"
}

As soon as you com­mit the changes, a work­flow run is executed, but it only provides the same out­put as before.

4. Cre­ate Ter­ra­form Ser­vice Principal

For Ter­ra­form, we need to cre­ate a ser­vice prin­cipal, just like we did to con­nect to GitHub.

sp="sp-datainsights-external-dev-terraform"
subscription="datainsights-external-dev"
az ad sp create-for-rbac \
    --name $sp --role contributor \
    --scopes /subscriptions/$(az account show --subscription $subscription --query id -o tsv)
    {
       "appId": "00000000-0000-0000-0000-000000000000",
       "displayName": " sp-datainsights-external-dev-terraform",
       "name": "",
       "password": "0000-0000-0000-0000-000000000000",
       "tenant": "00000000-0000-0000-0000-000000000000"
    }


5. Store cre­den­tials in Key Vault and set policy

We are going to store the appId as client-id and password as client-password in your already cre­ated Key Vault for later retrieval, and since we have con­nec­ted to the Azure sub­scrip­tion, all that is left is to set a policy for access­ing the secrets.

az keyvault set-policy -n  --secret-permissions get list –spn  


6. Adjust work­flow to deploy Azure Resources using Terraform

Now we can modify the work­flow file as shown below. This time we use the Key Vault Action in the second step to retrieve the secrets, which are then ref­er­enced by the Ter­ra­form Deploy­ment step and set as envir­on­ment vari­ables. Make sure you change the name of the Key Vault to your own name:

name: Run Azure Login with OpenID Connect, get secrets from key Vault, and do terraform deployment
on: [push]
permissions:
      id-token: write
jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    steps:
    - name: 'Az CLI login'
      uses: azure/login@v1
      with:
          client-id: ${{ secrets.AZURE_CLIENT_ID }} # Secrets defined
          tenant-id: ${{ secrets.AZURE_TENANT_ID }} # in GitHub Settings
          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
    - name: 'Get secrets from key Vault'
      uses: Azure/get-keyvault-secrets@v1
      with:
        keyvault: "keyvaultname"
        secrets: 'client-id, client-password'
      id: GetSecretAction
    - name: 'Checkout'
      uses: actions/checkout@v2
    - name: 'Terraform deployment'
      run: |
        terraform init
        terraform validate
        terraform plan
        terraform apply
      env:
        # Secrets for connecting Terraform provider to Azure
        ARM_CLIENT_ID:       ${{ steps.GetSecretAction.outputs.client-id }}       # Secrets from key Vault
        ARM_CLIENT_SECRET:   ${{ steps.GetSecretAction.outputs.client-password }} # Secrets from key Vault
        ARM_SUBSCRIPTION_ID: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
        ARM_TENANT_ID:       ${{ secrets.AZURE_TENANT_ID }}

When the Ter­ra­form step is executed, the envir­on­ment vari­ables are picked up and used to cre­ate the resource group. When the work­flow is com­plete, you should be able to see the newly cre­ated resource group in the resource groups sec­tion in the Azure Portal.

As you can see, prop­erly con­figured access from Git­Hub repos­it­or­ies allows us to cent­ral­ize access man­age­ment in Azure and, along with KeyVault, provides us with an addi­tional layer of secur­ity for the pro­duc­tion environment.

With the fil­ter in the fed­er­a­tion iden­tity based on repos­it­ory details such as envir­on­ment, branch, tags, pull requests, we can cre­ate more gran­u­lar per­mis­sions. But that is a topic for another article.