Introduction

In the modern era of containerized applications and microservices, efficient management of container images becomes paramount for enterprises. Azure Container Registry (ACR) offers a robust solution, allowing organizations to securely store, manage, and deploy container images. In this comprehensive guide, we’ll walk you through setting up an end-to-end ACR infrastructure tailored for enterprise needs.

In this post I want to cover end to end secure Public ACR architecure with scope base access. I have use Standard ACR which has a public endpoint. In future article I am planning to cover a Private Secure ACR setup too.

High Level Architecure

In my design I have followings:

• I have 2teams/squads (devops squad and online squad) on my hypothetical organization.
• There is a 1 ACR which will share by entire organizaton and its deployed int shared resource group (rg-shared)
• Each team has their own Azure Key Vault where they store their build/pull tokens for ACR
• Build/Pull token will be created by terraform and have role base access to each repos.
• You can find source to this blog post in here : https://github.com/prageeshag/tech-blog-public/tree/main/acr-with-tokens/azure-terraform

Folder structure for terraform code

Terraform

Azure Container Registry

Lets create the Azure Container registry. First I am creating a resource group named rg-shared services to store the ACR. I have created a module to create Resource Groups, so we can resuse that module to create other Resource Groups. I assume that you have already setup provider.tf with the credentials which require terraform to access your Azure account and provision resources.

Lets create a module for the Azure Resource Group

Creating a Terraform module for managing Azure Resource Groups is a great way to standardize your infrastructure as code (IaC) and make it more reusable. Terraform is a popular infrastructure provisioning and management tool, and creating modules allows you to encapsulate common configurations and resources. Below, I’ll guide you through creating a Terraform module for Azure Resource Groups:

Step 1: Set Up Your Project Structure

First, create a project directory structure to organize your Terraform code. You can structure your project like this:

Step 2: Define Input Variables

In the variables.tf file, define the input variables that your module will accept. For a Resource Group module, you might want to include variables for the name of the resource group, the location, and any additional tags:

################################################################################
# This is the set of variable for the module 
# file name : modules/rg/variable.tf
################################################################################

variable "name" {
  default     = ""
}


variable "location" {
  default     = ""
}

Step 3: Create the Resource Group

In the main.tf file, define the resource group using the input variables you’ve specified:

################################################################################
# This is the main file of the module which has the resource
# file name : modules/rg/main.tf
################################################################################

resource "azurerm_resource_group" "rg" {
    name     = var.name
    location = var.location   
}

Step 4: Define Output Values

In the outputs.tf file, define any output values you want to expose to users of your module. For example, you might want to output the Resource Group’s ID:

################################################################################
# This is the output values 
# file name : modules/rg/output.tf
################################################################################

output "object" {
  value = azurerm_resource_group.rg
  description = "returns the full Object"
}

output "name" {
  value = azurerm_resource_group.rg.name
  description = "returns the name of the resurce group"
}

output "id" {
  value = azurerm_resource_group.rg.id
  description = "returns the ID of Azure Resource Group"
}

Lets create a module for the Azure Container Registry

Similarly lets create a module for ACR also.

1. Define your variables in variables.tf:

################################################################################
# This is the set of variable for the module 
# file name : modules/acr/variable.tf
################################################################################

variable "name" {
  default     = "prageeshaTechBlog"
}

variable "resource_group" {
  default     = ""
}

variable "location" {
    default = ""
  
}

2. Implement the azure container registry in main.tf:

################################################################################
# This is the main file of the module which has the resource
# file name : modules/acr/main.tf
################################################################################

resource "azurerm_container_registry" "acr" {
  name                = var.name
  resource_group_name = var.resource_group
  location            = var.location
  sku                 = "Standard"
  admin_enabled       = false
}

3. Define your outputs in output.tf:

################################################################################
# This is the output values 
# file name : modules/rg/output.tf
################################################################################

output "object" {
  value = azurerm_container_registry.acr
  description = "returns the full Azure Key Vault Object"
}

output "name" {
  value = azurerm_container_registry.acr.name
}

output "id" {
  value = azurerm_container_registry.acr.id
}

Lets create shared resources

I am planning to place my ACR and other shared resources in shared service resource group. Because this ACR will be use by multiple teams therfore I am going to place it in the shared service.

1. Create resource group and ACR by using the modules we already created in above steps

################################################################################
# This is the output values 
# file name : shared-services/main.tf

################################################################################
locals {
    prefix-shared       = "shared-services"
    shared-location       = "eastus"
    shared-resource-group = "rg-shared-services"
    acr_name              = "prageeshaTechACR"
}


module "rg_shared_services" {
  source  = "../modules/rg"
  name = local.shared-resource-group
  location = local.shared-location
}


module "acr_shared" {
  source  = "../modules/acr"
  name =  local.acr_name
  location = local.shared-location  
  resource_group = module.rg_shared_services.name
}

2. Define your outputs in output.tf:

################################################################################
# This is the output values 
# file name : shared-services/output.tf
################################################################################


output "acr_shared_name" {
  value = module.acr_shared.name
}

output "acr_shared_id" {
  value = module.acr_shared.id
}

output "rg_shared_name" {
  value = module.rg_shared_services.name
}

3. Initialize Terraform

Navigate to the directory where your Terraform configuration files are located. Open your terminal and run: This command initializes Terraform, downloads the necessary providers, and sets up your working directory.

terraform init 

Initializing the backend...
Initializing modules...

Initializing provider plugins...
- Reusing previous version of hashicorp/azurerm from the dependency lock file
- Using previously-installed hashicorp/azurerm v3.70.0

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

4. Plan the Configuration

After initializing Terraform, you can generate an execution plan to see what actions Terraform will take. Run:

terraform plan -out my-tf-plan.out
Acquiring state lock. This may take a few moments...

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # module.acr_shared.azurerm_container_registry.acr will be created
  + resource "azurerm_container_registry" "acr" {
      + admin_enabled                 = false
      + admin_password                = (sensitive value)
      + admin_username                = (known after apply)
      + encryption                    = (known after apply)
      + export_policy_enabled         = true
      + id                            = (known after apply)
      + location                      = "eastus"
      + login_server                  = (known after apply)
      + name                          = "prageeshaTechBlog"
      + network_rule_bypass_option    = "AzureServices"
      + network_rule_set              = (known after apply)
      + public_network_access_enabled = true
      + resource_group_name           = "rg-shared-services"
      + retention_policy              = (known after apply)
      + sku                           = "Standard"
      + trust_policy                  = (known after apply)
      + zone_redundancy_enabled       = false
    }

  # module.rg_shared_services.azurerm_resource_group.rg will be created
  + resource "azurerm_resource_group" "rg" {
      + id       = (known after apply)
      + location = "eastus"
      + name     = "rg-shared-services"
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + acr_shared_id   = (known after apply)
  + acr_shared_name = "prageeshaTechBlog"
  + rg_shared_name  = "rg-shared-services"

───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Saved the plan to: my-tf-plan.out

To perform exactly these actions, run the following command to apply:
    terraform apply "my-tf-plan.out"
Releasing state lock. This may take a few moments...

5. Apply the Configuration (Optional) If the plan looks correct and you’re ready to create or update resources, you can apply the configuration by running:

terraform apply my-tf-plan.out

No you can see that the ACR is created.

Lets give ACR access to our devops-squad

Now we have created the ACR to store container images. Lets see how we provide access to the ACR. Devops Squad is one of the squad that require access to this ACR. We are using container resgistry scopes to grant access to the repositories.

1. Create AKV for devops squad

I am creating Azure Key Vault for devops-squad so that they can store their secrets in this vault. When I am generating auth tokens for ACR access I am storing those token secret in this AKV.

Lets create a module for the Azure Key Vault first.

1. Define your variables in variables.tf:

################################################################################
# This is the set of variable for the module 
# file name : modules/akv/variable.tf
################################################################################

variable "name" {
  default     = ""
}

variable "location" {
  default     = ""
}

variable "resource_group" {
  default     = ""
}

variable "tenant_id" {
  default     = ""
}

variable "access_policies" {
  description = "List of access policies"
  type        = list(object({
    tenant_id          = string
    object_id          = string
    secret_permissions = list(string)
    storage_permissions = list(string)
    key_permissions = list(string)
  }))
}

2. Implement the azure key vault in main.tf:

################################################################################
# This is the main file of the module which has the resource
# file name : modules/akv/main.tf
################################################################################


data "azurerm_client_config" "current" {}

resource "azurerm_key_vault" "akv" {

  name                            = var.name
  location                        = var.location
  resource_group_name             = var.resource_group
  enabled_for_disk_encryption     = false
  enabled_for_deployment          = false
  enabled_for_template_deployment = false
  enable_rbac_authorization       = false
  tenant_id                       = var.tenant_id
  soft_delete_retention_days      = 7
  purge_protection_enabled        = false

  sku_name = "standard"

  # Create access policies
  dynamic "access_policy" {
    for_each = var.access_policies
    content {
      object_id          = access_policy.value.object_id
      tenant_id          = access_policy.value.tenant_id
      secret_permissions = access_policy.value.secret_permissions
      key_permissions = access_policy.value.key_permissions
      storage_permissions = access_policy.value.storage_permissions
    }
  } 
}

3. Define your outputs in output.tf:

################################################################################
# This is the output values 
# file name : modules/akv/output.tf
################################################################################

output "object" {
  value = azurerm_key_vault.akv
  description = "returns the full Azure Key Vault Object"
}

output "name" {
  value = azurerm_key_vault.akv.name
  description = "returns the name of Azure Key Vault"
}

output "id" {
  value = azurerm_key_vault.akv.id
  description = "returns the ID of Azure Key Vault"
}

output "vault_uri" {
  value = azurerm_key_vault.akv.vault_uri
  description = "returns the vault URI of Azure Key Vault"
}

Lets create Resurce Group and Azure Key Vault for devops-squad

locals {
  prefix-devops         = "devops-squad"
  devops-location       = "eastus"
  devops-resource-group = "rg-devops-squad"
}

data "azurerm_client_config" "current" {}

module "rg_devops_squad" {
  source   = "../modules/rg"
  name     = local.devops-resource-group
  location = local.devops-location
}

module "akv_devops_squad" {
  source         = "../modules/akv"
  name           = "${local.prefix-devops}-vault"
  location       = local.devops-location
  resource_group = module.rg_devops_squad.name
  tenant_id      = data.azurerm_client_config.current.tenant_id


  access_policies = [
    {
      # This is to grant AKV permissions to my terraform service principal so
      # that it can update secret values 
      tenant_id           = data.azurerm_client_config.current.tenant_id
      object_id           = data.azurerm_client_config.current.object_id
      key_permissions     = ["Get", "List", "Encrypt", "Decrypt", "Create"]
      secret_permissions  = ["Get", "List", "Delete", "Set"]
      storage_permissions = ["Get"]
    },
    {
      # This is to grant AKV permission to my azure web console user 
      # that it can read the secret values from web for this demo purpose
      tenant_id           = data.azurerm_client_config.current.tenant_id
      object_id           = var.my_ui_account_object_id
      secret_permissions  = ["Get", "List", "Set"]
      key_permissions     = []
      storage_permissions = []
    }
    # Add more access policies as needed
  ]

}

Once you apply above resource you should have a resource group and Azure Key Vault created for the devops-squad

The Challenge of Access Control

Azure Container Registry provides various mechanisms for access control, including Azure Active Directory (Azure AD) authentication, service principals, and managed identities. While these mechanisms offer robust security, there are situations where you need to provide temporary or scoped access to your registry without sharing credentials. This is where Azure Container Registry Token Management shines.

To streamline the process of creating and managing Azure Container Registry tokens, we can leverage Terraform. Terraform allows us to define and manage our Azure resources in a declarative way. Let’s break down the Terraform module step by step.

1. Container Registry Scope Map

resource "azurerm_container_registry_scope_map" "scope_map" {
  name                    = var.scope_map_name
  container_registry_name = var.container_registry_name
  resource_group_name     = var.resource_group_name
  actions                 = var.actions
}

The first step is to create a scope map. A scope map defines the level of access for a specific token. You can specify the actions permitted, such as read, write, or delete, and associate it with your Azure Container Registry. This allows for fine-grained control over what a token can and cannot do.

2. Container Registry Token

resource "azurerm_container_registry_token" "token" {
  name                    = var.scope_map_name
  container_registry_name = var.container_registry_name
  resource_group_name     = var.resource_group_name
  scope_map_id            = azurerm_container_registry_scope_map.scope_map.id
}

Once we have a scope map in place, we create a token based on that scope map. This token can be considered as a bearer token with temporary access to the resources defined in the scope map. It is a dynamic way to grant access without revealing any credentials.

3. Container Registry Token Password

resource "azurerm_container_registry_token_password" "token_pwd" {
  container_registry_token_id = azurerm_container_registry_token.token.id
  password1 {
    expiry = var.token_expiry
  }
}

Tokens are issued with an associated password, which serves as a shared secret between the token issuer and the consumer. The password can have an expiration date to ensure that the access is temporary. In this example, we set the expiry date using the var.token_expiry variable.

4. Key Vault Secret

resource "azurerm_key_vault_secret" "secret" {
  name         = var.scope_map_name
  value        = azurerm_container_registry_token_password.token_pwd.password1[0].value
  key_vault_id = var.key_vault_id
  depends_on   = [azurerm_container_registry_token_password.token_pwd]
}

To make this token easily accessible to the applications and services that need it, we store it securely in an Azure Key Vault. The azurerm_key_vault_secret resource allows us to create a secret in the Key Vault and populate it with the token password.

I created a module from above resources and use that for each team to provision their access scope for the ACR.

For each squad I am creating below two tokens/scopes using above created module.

builder : This will have access to read/write to the repositories. This can be use in your pipeline to push images to the allowed devops repositories.

reader : This will have read access only. This can be use to pull images, and can be use in your deployments to pull the application image.

Builder Scope

In this configuration, we create a scope map specifically tailored for builder activities, granting them access to both reading and writing container images. This scoped token is designed to expire automatically for added security. The following is placed in the main.tf my devops squad terrform codebase.

# Define a variable for devops_apps with a default value
variable "devops_apps" {
  type = list(any)
  default = [
    "app1",
    "app2"
  ]
}

# Define locals for read_repos and write_repos
locals {
  read_repos  = [for value in var.devops_apps : "repositories/${local.prefix-devops}/${value}/content/read"]
  write_repos = [for value in var.devops_apps : "repositories/${local.prefix-devops}/${value}/content/write"]
}



module "acr_scope_map_devops_builder" {
  source                  = "../modules/acr-scopes"
  scope_map_name          = "devops-squad-builder"
  container_registry_name = data.terraform_remote_state.azr-shared-services.outputs.acr_shared_name
  resource_group_name     = data.terraform_remote_state.azr-shared-services.outputs.rg_shared_name
  actions                 = concat(local.read_repos, local.write_repos)
  key_vault_id            = module.akv_devops_squad.id
  token_expiry            = "2024-03-22T17:57:36+08:00"
}

Reader Scope

In some cases, you might need to provide read-only access to container images for auditing or monitoring purposes. This is where the reader scope comes into play.

module "acr_scope_map_devops_reader" {
  source                  = "../modules/acr-scopes"
  scope_map_name          = "devops-squad-reader"
  container_registry_name = data.terraform_remote_state.azr-shared-services.outputs.acr_shared_name
  resource_group_name     = data.terraform_remote_state.azr-shared-services.outputs.rg_shared_name
  actions                 = local.read_repos
  key_vault_id            = module.akv_devops_squad.id
  token_expiry            = "2024-03-22T17:57:36+08:00"
}

Once you apply above terrform snippet you will get scope maps like below, and have access to the repositories mentioned.

Tokens for above scope maps

Since we instruct to store token to AKV, you can retrive token from the devops-squad key vault.

Verify scope-based access

Lets login to ACR with reader token. Now you can fetch the tokens from the devops squad Azure Key Vault

READER_TOKEN=<token>
docker login prageeshatechblog.azurecr.io --username devops-squad-reader --password-stdin $READER_TOKEN

Now lets try to push using reader login, and this should be failed

Now lets login using builder token and try to push the same image

You could see now that the image is pushed to ACR

Conclusion

Managing access to Azure Container Registry can be complex, but with Terraform and Azure Container Registry Token Management, you can simplify the process. By creating fine-grained tokens and securely storing them in Azure Key Vault, you can grant temporary and scoped access to your container images without exposing sensitive credentials. This not only enhances security but also streamlines access control for your containerized applications.

Incorporate this Terraform module into your infrastructure provisioning workflow to ensure secure and efficient access control for your Azure Container Registry.