Manage Access Keys

Lab Objective

In this hands-on lab, you will learn how to:

Understand Azure Storage access keys and their role in authentication
Manage storage account access keys including rotation and regeneration
Test key rotation scenarios without service disruption
Implement secure key management practices for production environments
Configure applications to handle key rotation gracefully
Monitor and troubleshoot access key related issues

Scenario: Your organization has applications using storage account access keys for authentication. You need to implement a key rotation strategy to maintain security while ensuring zero downtime for production applications during key updates.

Pre-Provisioned Environment

The following Azure resources have been pre-deployed in your environment:

Resource Overview

Resource Type	Resource Name	Configuration	Purpose
Resource Group	`AccessKeys-Lab-RG`	Contains all lab resources	Logical container
Storage Account	`accesskeyslab[unique]`	General Purpose v2	Primary storage resource
Blob Container	`application-data`	Private access level	Application file storage
Blob Container	`backup-data`	Private access level	Backup file storage
App Service	`webapp-[unique]`	Web application	Simulated production app
Function App	`funcapp-[unique]`	Serverless functions	Background processing
Key Vault	`keyvault-[unique]`	Standard tier	Secure key storage
Test VM	`TestVM`	Windows Server 2019	Key rotation testing client

Application Architecture

Production Applications
├── Web App (webapp-[unique])
│   └── Uses Key1 for storage access
├── Function App (funcapp-[unique])
│   └── Uses Key2 for storage access
└── Test VM (TestVM)
    └── Manual testing client

Storage Account (accesskeyslab[unique])
├── Key1 (primary)
├── Key2 (secondary)
└── Containers
    ├── application-data
    └── backup-data

Key Vault (keyvault-[unique])
└── Secure storage for rotated keys

VM Details

VM	Private IP	Operating System	Purpose
TestVM	10.0.1.4	Windows Server 2019	Test key rotation scenarios

Lab Exercises

Part 1: Examine Current Access Key Configuration

Step 1: Navigate to Storage Account Keys

Navigate to AccessKeys-Lab-RG resource group
Click on the Storage Account (accesskeyslab[unique])
In the left menu, click Access keys
Observe the two access keys:
- Key1 (primary)
- Key2 (secondary)

Step 2: Document Current Key Information

Record the following information:

Storage account name: accesskeyslab[unique]
Key1 name: Copy the key name
Key1 value: Copy the first few and last few characters (for identification)
Key2 name: Copy the key name
Key2 value: Copy the first few and last few characters (for identification)
Connection string: Copy both connection strings

Step 3: Test Current Key Access

Click Show keys to reveal the key values
Copy Key1 completely
Open a new browser tab and go to Azure Storage Explorer online
Test connection using Key1:
- Select Storage account or service
- Choose Account name and key
- Enter storage account name and Key1
- Verify: Should successfully connect and show containers

Part 2: Test Application Dependencies

Step 1: Check Web App Configuration

Navigate to webapp-[unique]
Click Configuration in the left menu
Look for storage-related settings:
- Find STORAGE_CONNECTION_STRING or similar
- Note which key is being used (Key1 or Key2)
- Do not modify at this stage

Step 2: Check Function App Configuration

Navigate to funcapp-[unique]
Click Configuration
Examine application settings:
- Look for AzureWebJobsStorage and other storage settings
- Identify which key is configured
- Record the configuration for later reference

Step 3: Connect to Test VM

Navigate to TestVM
Click Connect → RDP
Use credentials:
- Username: azureuser
- Password: LabPassword123!

Step 4: Test Storage Access from VM

From TestVM, open PowerShell and test both keys:

# Test Key1 access
$storageAccount = "accesskeyslab[unique]"
$key1 = "[KEY1-VALUE]"
$ctx1 = New-AzStorageContext -StorageAccountName $storageAccount -StorageAccountKey $key1

try {
    $containers = Get-AzStorageContainer -Context $ctx1
    Write-Host "Key1 Test: SUCCESS - Found $($containers.Count) containers"
    $containers | ForEach-Object { Write-Host "  - $($_.Name)" }
} catch {
    Write-Host "Key1 Test: FAILED - $($_.Exception.Message)"
}

# Test Key2 access
$key2 = "[KEY2-VALUE]"
$ctx2 = New-AzStorageContext -StorageAccountName $storageAccount -StorageAccountKey $key2

try {
    $containers = Get-AzStorageContainer -Context $ctx2
    Write-Host "Key2 Test: SUCCESS - Found $($containers.Count) containers"
} catch {
    Write-Host "Key2 Test: FAILED - $($_.Exception.Message)"
}

Expected Result: Both keys should work and return the same container list

Part 3: Implement Zero-Downtime Key Rotation

Step 1: Identify Current Key Usage

Before rotating, document which applications use which keys:

Application	Current Key	Configuration Location
Web App	Key1 or Key2	Application Settings
Function App	Key1 or Key2	Application Settings
Manual Scripts	Both	Various locations

Step 2: Plan Rotation Strategy

Zero-downtime rotation process:

Identify the unused key (if web app uses Key1, then Key2 is free to rotate)
Regenerate the unused key
Update applications to use the newly regenerated key
Test new key functionality
Regenerate the old key (now unused)

Step 3: Regenerate Key2 (Assuming it’s unused)

In Storage Account Access keys
Click regenerate for Key2 (three dots menu → Regenerate)
Confirm regeneration in the dialog
Copy the new Key2 value

Step 4: Test New Key2

From TestVM PowerShell:

# Test newly regenerated Key2
$newKey2 = "[NEW-KEY2-VALUE]"
$ctxNew = New-AzStorageContext -StorageAccountName $storageAccount -StorageAccountKey $newKey2

try {
    $containers = Get-AzStorageContainer -Context $ctxNew
    Write-Host "New Key2 Test: SUCCESS - Regenerated key works"

    # Test file operations
    $testContent = "Test content - " + (Get-Date)
    $blob = Set-AzStorageBlobContent -Context $ctxNew -Container "application-data" -File "C:\temp\test.txt" -Blob "key-rotation-test.txt" -Force
    Write-Host "New Key2 Write Test: SUCCESS - Can write files"

} catch {
    Write-Host "New Key2 Test: FAILED - $($_.Exception.Message)"
}

Part 4: Update Application Configurations

Step 1: Update Web App to Use New Key2

Navigate to webapp-[unique]
Click Configuration
Find the storage connection string setting
Update the key value in the connection string:
- Change from old Key2 to new Key2
- Or switch from Key1 to new Key2
Save the configuration
Restart the web app to apply changes

Step 2: Update Function App Configuration

Navigate to funcapp-[unique]
Click Configuration
Update storage-related settings:
- AzureWebJobsStorage: Update to use new Key2
- Other storage settings: Update as needed
Save configuration
Restart function app

Step 3: Test Applications After Update

Test Web App:

Browse to the web app URL
Verify: Application loads and functions normally
Check logs for any storage-related errors

Test Function App:

Trigger a function (if possible)
Monitor execution in the portal
Check logs for successful storage operations

Part 5: Complete the Key Rotation

Step 1: Verify All Applications Use New Key

Confirm applications are working with the new key before proceeding:

From TestVM, test the old key to confirm it’s no longer needed:

# Test if old Key1 is still being used somewhere
$oldKey1 = "[OLD-KEY1-VALUE]"
$ctxOld = New-AzStorageContext -StorageAccountName $storageAccount -StorageAccountKey $oldKey1

# This should still work since we haven't regenerated Key1 yet
try {
    Get-AzStorageContainer -Context $ctxOld | Out-Null
    Write-Host "Old Key1: Still valid (not yet regenerated)"
} catch {
    Write-Host "Old Key1: No longer valid"
}

Step 2: Regenerate Key1

In Storage Account Access keys
Regenerate Key1 (the previously used key)
Confirm regeneration
Copy the new Key1 value

Step 3: Test Complete Rotation

From TestVM PowerShell:

# Test the newly regenerated Key1
$newKey1 = "[NEW-KEY1-VALUE]"
$ctxNewKey1 = New-AzStorageContext -StorageAccountName $storageAccount -StorageAccountKey $newKey1

try {
    $containers = Get-AzStorageContainer -Context $ctxNewKey1
    Write-Host "New Key1 Test: SUCCESS - Both keys have been rotated"
} catch {
    Write-Host "New Key1 Test: FAILED - $($_.Exception.Message)"
}

# Verify old keys no longer work
try {
    $ctxOldKey1 = New-AzStorageContext -StorageAccountName $storageAccount -StorageAccountKey $oldKey1
    Get-AzStorageContainer -Context $ctxOldKey1 | Out-Null
    Write-Host "Security Issue: Old Key1 still works!"
} catch {
    Write-Host "Security Confirmed: Old Key1 properly invalidated"
}

Part 6: Secure Key Storage with Key Vault

Step 1: Store Keys in Key Vault

Navigate to keyvault-[unique]
Click Secrets in the left menu
Add new secrets:

Secret 1:

Name: storage-key1
Value: New Key1 value
Content type: application/x-storage-key

Secret 2:

Name: storage-key2
Value: New Key2 value
Content type: application/x-storage-key

Set appropriate expiration dates (e.g., 90 days for rotation reminder)

Step 2: Configure Applications to Use Key Vault

Update Web App configuration:

Navigate to webapp-[unique] → Configuration
Add Key Vault reference:
- Name: STORAGE_KEY
- Value: @Microsoft.KeyVault(SecretUri=https://keyvault-[unique].vault.azure.net/secrets/storage-key2/)
Update connection string to reference the Key Vault secret

Step 3: Test Key Vault Integration

From TestVM PowerShell:

# Test Key Vault secret retrieval (requires authentication setup)
$keyVaultName = "keyvault-[unique]"
$secretName = "storage-key1"

# This would typically be done with managed identity in production
try {
    $secret = Get-AzKeyVaultSecret -VaultName $keyVaultName -Name $secretName -AsPlainText
    Write-Host "Key Vault Test: SUCCESS - Retrieved key from vault"

    # Test storage access with Key Vault key
    $ctxVault = New-AzStorageContext -StorageAccountName $storageAccount -StorageAccountKey $secret
    $containers = Get-AzStorageContainer -Context $ctxVault
    Write-Host "Key Vault Storage Test: SUCCESS - Key works for storage access"

} catch {
    Write-Host "Key Vault Test: Authentication required - $($_.Exception.Message)"
}

Part 7: Monitor and Automate Key Rotation

Step 1: Set Up Monitoring Alerts

Navigate to Storage Account → Monitoring → Alerts
Create alert rule for key usage:
- Signal: Transactions
- Condition: When authentication failures > threshold
- Action: Email notification

Step 2: Document Key Rotation Schedule

Create rotation schedule:

Key	Current Rotation Date	Next Rotation Due	Responsible Team
Key1	[Today’s date]	[90 days from now]	Security Team
Key2	[Today’s date]	[90 days from now]	DevOps Team

Step 3: Create Rotation Checklist

Key rotation procedure:

Identify unused key (Key1 or Key2)
Regenerate unused key
Update Key Vault with new key value
Test applications using Key Vault reference
Monitor for errors for 24 hours
Regenerate the other key
Update documentation and schedule next rotation

Troubleshooting Guide

Common Access Key Issues

Issue	Symptoms	Possible Cause	Solution
Application authentication failures	403 Forbidden errors	Key regenerated but app not updated	Update application configuration
Intermittent connection issues	Some operations fail	Mixed key usage during rotation	Ensure all connections use same key
Key Vault access denied	Cannot retrieve secrets	Missing permissions	Configure managed identity or access policies
Applications won’t start	Startup failures	Invalid connection string	Verify connection string format
Storage Explorer connection fails	Cannot connect	Wrong key or account name	Verify account name and key values

Key Rotation Validation

Check	Method	Expected Result
New key functionality	Test storage operations	All operations succeed
Old key invalidation	Test with old key	Authentication failure
Application health	Monitor app logs	No authentication errors
Key Vault integration	Retrieve secrets	Successful secret access

Additional Experiments

Try these optional exercises to deepen your understanding:

Automated Key Rotation: Use Azure Functions to automate key rotation
Managed Identity: Replace access keys with managed identity authentication
Key Rotation Scripts: Create PowerShell scripts for automated rotation
Multi-Region Scenarios: Test key rotation across geo-replicated storage
Disaster Recovery: Practice key recovery scenarios

Key Takeaways

After completing this lab, you should understand:

Azure Storage provides two access keys for zero-downtime rotation scenarios
Key rotation requires careful planning to avoid service disruption
Applications should be updated before regenerating currently used keys
Key Vault integration provides centralized and secure key management
Monitoring and alerting are essential for detecting key-related issues
Regular key rotation is a security best practice for production environments
Managed identities are preferred over access keys when possible

Access Key Management Best Practices

Security Practices

Rotate keys regularly (every 90 days recommended)
Use Key Vault for centralized key storage
Implement monitoring for authentication failures
Document rotation procedures and assign responsibilities
Test rotation process in non-production environments first

Operational Practices

Use one key for applications, keep one for rotation
Update applications before regenerating active keys
Monitor applications after key updates
Maintain rotation schedules and documentation
Consider managed identities as alternative to access keys

Access Keys vs Managed Identity

Comparison Table

Feature	Access Keys	Managed Identity
Security	Requires key management	No credential management
Rotation	Manual process required	Automatic rotation
Complexity	Simple initial setup	Requires RBAC configuration
Audit Trail	Limited tracking	Full Azure AD audit logs
Scope	Full storage account access	Granular RBAC permissions

Migration Path

Identify applications using access keys
Enable managed identity for Azure resources
Configure RBAC permissions for storage access
Update application code to use managed identity
Test functionality thoroughly
Disable access key usage once migration complete

Manage Access Keys

Lab Objective

Lab Environment Ready!

MFA Verification Code:

Lab Environment Ready!

MFA Verification Code:

Pre-Provisioned Environment

Resource Overview

Application Architecture

VM Details

Lab Exercises

Part 1: Examine Current Access Key Configuration

Step 1: Navigate to Storage Account Keys

Step 2: Document Current Key Information

Step 3: Test Current Key Access

Part 2: Test Application Dependencies

Step 1: Check Web App Configuration

Step 2: Check Function App Configuration

Step 3: Connect to Test VM

Step 4: Test Storage Access from VM

Part 3: Implement Zero-Downtime Key Rotation

Step 1: Identify Current Key Usage

Step 2: Plan Rotation Strategy

Step 3: Regenerate Key2 (Assuming it’s unused)

Step 4: Test New Key2

Part 4: Update Application Configurations

Step 1: Update Web App to Use New Key2

Step 2: Update Function App Configuration

Step 3: Test Applications After Update

Part 5: Complete the Key Rotation

Step 1: Verify All Applications Use New Key

Step 2: Regenerate Key1

Step 3: Test Complete Rotation

Part 6: Secure Key Storage with Key Vault

Step 1: Store Keys in Key Vault

Step 2: Configure Applications to Use Key Vault

Step 3: Test Key Vault Integration

Part 7: Monitor and Automate Key Rotation

Step 1: Set Up Monitoring Alerts

Step 2: Document Key Rotation Schedule

Step 3: Create Rotation Checklist

Troubleshooting Guide

Common Access Key Issues

Key Rotation Validation

Additional Experiments

Key Takeaways

Access Key Management Best Practices

Security Practices

Operational Practices

Access Keys vs Managed Identity

Comparison Table

Migration Path

Additional Resources