Skip to content

Blog

How Lab Deployments Work Part 3 - Durable Starter Function

Now let me acquaint you with how a POST request at the front door to the Azure Durable Functions workflow. Below I’ll focus on the Durable Functions Starter function and move onto the app’s orchestrator and activity functions in the next post. The role of the starter function is to accept the POST request and trigger the rest of the workflow. In order to do this successfully, it must:

  • Read the POST request it receives with its JSON payload
  • Validate that the data it receives is in the correct format
  • Pass that data to and activate the orchestrator
  • Create status URLs that will act as query endpoints for checking on the status of the function apps orchestration
  • Return a 202 accepted response to the client with the query URLs
  • Perform various error handling So although the starter function, in essence, only acts as a trigger for the rest of the orchestration, it must perform many tasks in order to fulfil that role.

Starter Function Operations

Request Body Parsing

An HTTP POST request lands at the Azure Function endpoint something like this:

Content-Type: application/json
{
"subscriptionId": "12345678-1234-1234-1234-123456789012",
"templateUrl": "https://example.com/template.json",
"roleDefinitionName": "Contributor"
}

However, that request body can arrive in many forms depending on how it was sent. For example, for testing purposes, I will usually use cURL, PowerShell’s invoke-webrequest or the Azure portal’s test panel. However, in production, the website’s own endpoints will send the request with JavaScript’s fetch command. This means that the payload could arrive already as a PSCustomObject or as a JSON string or even other formats which I have used in the past (direct Bicep file uploads would need to be parsed as streams for example). For safety, a number of possible formats are expected so therefore parsed, as per the below examples.

Body Type Detection - PSCustomObject or IDictionary

This is the simplest format to deal with as it is essentially ready to pass to the rest of the workflow. In the testing phase, this gave me great relief as I was using PowerShell to send my invocations. Essentially, payload is already parsed before it reaches the starter so no further parsing is even required in this circumstance.

Terminal window
if ($Request.Body -is [System.Collections.IDictionary] -or $Request.Body -is [PSCustomObject]) {
Write-DetailedLog -Message "Request body is already an object" -Level "DEBUG"
$inputData = $Request.Body
}

Body Type Detection - JSON String Processing

If the body is a JSON string, it will be parsed as a PSObject by converting from JSON with ConvertFrom-Json. This is so we can access its properties with dot notation (e.g. input.subscriptionID). However, during this process PowerShell will attempt to infer the type of each value; meaning that the subscriptionID may become an integer if the JSON value is numeric, and templateUrl will be a string but could cause issues if not handled as a string in downstream functions (such as serialization or type mismatch errors). To guarantee type consistency, we need to be able to control how the types are parsed. We do this by then converting the PSCustomObject to a hashtable and explicitly converting each property value to a string (performing type coercion).

Terminal window
elseif ($Request.Body -is [string] -and $Request.Body.Trim().StartsWith('{')) {
# Parse as regular PSObject first
$jsonObject = $Request.Body | ConvertFrom-Json
# Convert to hashtable, ensuring string properties remain as strings
$inputData = @{}
foreach ($prop in $jsonObject.PSObject.Properties) {
# Ensure all URL values and IDs are explicitly converted to strings
if ($prop.Name -eq "subscriptionId" -or $prop.Name -eq "templateUrl" -or $prop.Name -eq "roleDefinitionName") {
$inputData[$prop.Name] = "$($prop.Value)"

Body Type Detection - Stream Handling

For reference, stream-based requests are handled as such:

Terminal window
elseif ($Request.Body -is [System.IO.Stream]) {
# Reset stream position if possible
if ($Request.Body.CanSeek) {
$Request.Body.Position = 0
}
# Read the stream
$reader = New-Object System.IO.StreamReader($Request.Body)
$bodyContent = $reader.ReadToEnd()

Required Parameter Validation

Certain parameters are essential. This excerpt focuses on the subscription ID, but there are several parameters that must be included in the request such as the RBAC permission level, template URL and more. Note the use of the Push-OutputBinding command. This is part of a set of specialised PowerShell commands designed for Azure Durable Functions runtime. This provides the HTTP response back to the client - in this case an error response stating it received a bad request.

Terminal window
if (-not $inputData.subscriptionId) {
Write-DetailedLog -Message "Missing required parameter: subscriptionId" -Level "WARNING"
Push-OutputBinding -Name Response -Value ([HttpResponseContext]@{
StatusCode = [HttpStatusCode]::BadRequest
Headers = @{ "Content-Type" = "application/json" }
Body = ConvertTo-Json @{
error = "subscriptionId is required"
message = "Please provide a valid Azure subscription ID to deploy the lab environment"
}
})
return
}

The Orchestrator Is Activated

The starter function can now fulfil its true task which is to launch the orchestrator. A log is written here to note that orchestration is starting which helps a lot with debugging. Next an instanceID is generated by invoking the orchestrator function by it’s name and providing the parsed and validated input data. The returned $instanceId uniquely identifies this orchestration instance, which can be used to track its status or interact with it later.

Terminal window
$orchestratorFunctionName = "`Lab_Orchestrator`"
Write-DetailedLog -Message "Starting orchestration" -Data @{
orchestratorFunction = $orchestratorFunctionName
inputDataKeys = ($inputData.Keys -join ", ")
}
$instanceId = Start-NewOrchestration -FunctionName $orchestratorFunctionName -Input $inputData

Status URL Creation

The function creates management URLs for the orchestration using the InstanceId generated in the preceding code block. There are three endpoints generated here; statusQueryGetUri is for checking the progress and final result of the workflow; sendEventPostUri is for sending external events to the orchestration, for situations where external input is needed; terminatePostUri is for terminating the current orchestration, useful for aborting the workflow. Note that defining these URLs is part of the custom function CreateCheckStatusResponse.

Terminal window
$statusQueryGetUri = "$baseUrl/runtime/webhooks/durabletask/instances/$InstanceId`?taskHub=$taskhubName"
$sendEventPostUri = "$baseUrl/runtime/webhooks/durabletask/instances/$InstanceId/raiseEvent/{eventName}?taskHub=$taskhubName"
$terminatePostUri = "$baseUrl/runtime/webhooks/durabletask/instances/$InstanceId/terminate?reason={text}&taskHub=$taskhubName"

Structured Logging Throughout

You may have noticed the Write-DetailedLog command used in the above code snippets. These refer to an PowerShell function which is defined at the beginning of the Starter Function. Every step is logged with detailed context. This is only a small excerpt from the code but gives you an idea of what is logged:

Terminal window
function Write-DetailedLog {
$logEntry = @{
timestamp = $timestamp
level = $Level
message = $Message
functionName = $TriggerMetadata.FunctionName
invocationId = $TriggerMetadata.InvocationId
}
}

Response to client

Finally, the custom function CreateCheckStatusResponse generates the URLs and defines the $response, returning the data to the client with the Push-OutputBinding cmdlet.

Terminal window
# Create a response with a 202 Accepted status and orchestration status URL
Write-DetailedLog -Message "Creating check status response for orchestration" -Level "DEBUG"
$response = CreateCheckStatusResponse -Request $Request -InstanceId $instanceId -InputData $inputData
Write-DetailedLog -Message "HTTP Starter function completed successfully" -Data @{
instanceId = $instanceId
statusCode = 202
}
# Return the response
Push-OutputBinding -Name Response -Value $response

The End of The Starter

That about covers the role of the starter function in the workflow of the Durable Functions app. These code snippets are simplified and much of the code is left out due to its length, but the spirit of the starter is captured here fully. If you read the Microsoft documentation on the starter function (sometimes called client function) you may be deceived into thinking that this function would be the simplest part of the Function App to implement. In my experience, it was probably one of the hardest. So much validation and logging was needed along the way that it ended up ballooning to 500 lines of code at one point. The custom functions for logging and generating the status response helped, but a lot of my personal confusion came from dealing with data types in PowerShell that I just was not that familiar with. Serialisation errors were in abundance due to how PowerShell parsed the JSON input values and that required a lot of research on how PSCustomObjects and hashtables worked and how to manipulate them in PowerShell. It was not fun, but it was a learning opportunity and a good way to learn about data types in PowerShell which I had not thought enough about until that time. The orchestrator’s job is essentially to pass data between the functions constantly, which was very painful given the learning curve I was on. Join me in the next post wherein I detail the torture of implementing the orchestrator!

How Lab Deployments Work Part 2 - API Endpoints

Website API endpoints

This web application uses the Astro framework. It is a static site generator that supports server-side generation. It is essentially perfect for this website which is document heavy but requires some small “islands” of dynamic content. It also has an amazing and easy to use API endpoint architecture based on files - for example, src/pages/data.json.ts will generate a /data.json endpoint. The endpoints can also be static, as in generated at build time, or dynamic - which is essential to the functionality of this website as the endpoints handle dynamic data. The theme I use is called Starlight which is Astro’s own documentation website theme. It is a pretty basic theme looks-wise, but comes with a world of extensibility as well as some great built-in components.

The Lab Deployment Endpoint

This endpoint initiates Azure lab deployments by retrieving configurations from Cosmos DB and calling Azure Durable Starter Function. This endpoint is activated when the user of the website clicks the button component on the lab’s respective webpage. That component provides the endpoint the ID of the relevant lab.

1. Initialise CosmosDB

The first thing this endpoint does is initialise a Cosmos DB client with connection validation:

// Initialize Cosmos DB client only if connection string exists
const connectionString = process.env.connectionstring;
if (!connectionString) {
console.error('connection string is not defined!');
throw new Error('connection string environment variable is required');
}

I had a good bit of trouble when setting this up due to the firewall configuration as well with the format of the CosmosDB connection string. Rookie mistake on my part, but, needless to say, the input validation was very useful here.

2. Request parsing and lab config retrieval

The endpoint extracts the lab identifier from the request body it received which contains the lab ID:

//request parsing
const body = await request.json();
const labId = body.labId as string;

and retrieves the lab config from CosmosDB using both the item ID and partition key (which I’ve left generic):

const { resource: config } = await container.item(<itemID>, <partitionKey>).read();

3. Azure Function Authentication

The endpoint uses function key authentication for secure Azure Function calls:

const functionKey = process.env.AZURE_FUNCTION_KEY;
if (!functionKey) {
console.error('AZURE_FUNCTION_KEY environment variable is not set');
throw new Error('Function key is required for authentication');
}

4. Azure Function Invocation

The deployment is triggered via HTTP POST to the Azure Function:

const response = await fetch(config.functionUrl, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'x-functions-key': functionKey // Add function key for authentication
},
body: JSON.stringify(config.payload)
});

The Deployment Status Polling Endpoint

Whereas a standard Azure Function performs its function and return a response before exiting, Durable Functions are a different beast due to their asynchronous, stateful nature. I will go further into this in part 2 of this blog series. Essentially, though, Durable Functions return a status URL which your application can use to programmatically monitor the orchestration and execution status of activity functions. The purpose of this endpoint is to monitor Azure’s built-in deployment status endpoints for the Durable Function and return lab credentials when complete.

1. Receiving The Status URL

The status URL is received from the initial call to the Azure Durable Starter.

const body = await request.json();
const { statusUrl } = body;
// Parse and prepare the status URL
const url = new URL(statusUrl);
console.log('Calling status URL:', url.toString());

2. Poll Until Response Indicates Success

The polling endpoint is triggered periodically to make a request to the Azure Functions endpoint.

// Make the request to Azure Functions status endpoint
const response = await fetch(url.toString());

3. Processes Successful Response

A successful response is forwarded back to the website component (the client) that initiates the API endpoint.

// Process successful responses
if (response.ok) {
const data = await response.json();
return new Response(JSON.stringify(data), {
status: 200,
headers: { 'Content-Type': 'application/json' }
});
}

The success response will contain a huge amount of information. The client component that triggers the endpoint sifts through this JSON data to provide the relevant details to the website user so they can access their lab environment. Errors in the deployment are also captured by the polling endpoint and provided to the front-end to be displayed to help with debugging. I haven’t included the error handling as that will be a larger topic later in the series. The client component’s code needs to sift through a tremendous amount of JSON data to find the relevant details - if that’s something you’re interested, let me know. Otherwise, I’ll see you in the next post where we focus on the Durable Functions themselves.

How Lab Deployments Work Part 1 - Backend Overview

Intro

The lab platform started as a proof of concept in a Python Flask app with a simple app route to launch a Bicep template into a resource group (which was hard-coded into the application’s source code no less). In its nascent form I was positive that the project would remain a simple, self-contained workflow that would not possibly require much further overhead than figuring out this core aspect of the platform - which was launching the environments themselves. My theory at the time was that once the relatively simple lab deployment workflow was complete, I could move onto the more standard aspects of a website such as authentication, hosting, front framework, etc. This was the first of many times on this journey I was completely and utterly wrong in the most unmitigated fashion. As such, my intention in this blog post is to document the backend of the platform, at its now more mature state, for anyone else interested in the eventual approach I came to find is robust and reliable for launching fully customisable lab environments to Azure. This will be a high-level overview of the Azure services that are responsible for the deployment of the lab environments, plus a little on how the website’s API endpoints construct the HTTP request that triggers the lab deployment and then polls to find the status of the deployment. In later posts, I will detail other aspects of the platform such as how labs are cleaned up after use, the website framework itself, the Azure DevOps pipeline that launches the site and other details.

A broad overview

Before elaborating on each stage of the deployment process, I will provide you a brief overview of how the whole deployment works from start to finish. Deployment Architecture

API endpoint which triggers deployment

Essentially, when a user presses the launch lab button on the website, that button component provides a labID to the website’s launch-lab API endpoint and triggers it to look up the correct lab configuration from Azure CosmosDB for NoSQL. The JSON data it extracts from this database contains contains the location of the relevant Bicep template, the RBAC permissions that the lab user will require over the launched resources in order to complete the lab tasks and the subscription ID of the Azure subscription in which the deployment will take place. The URL endpoint of the Azure Durable Function, which is the destination of the HTTP request, is also in the CosmosDB database.

Azure Durable Function app architecture

The lab config information is now extracted and sent to the Azure Durable Function’s HTTP trigger - also called durable starter function. The starter function extracts and validates the payload to ensure the relevant information it needs to pass to the orchestrator is present then triggers the entire orchestration which is responsible for deploying the lab environment.

Activity Functions and a note on Table Storage

The orchestrator first starts the ‘Get and Assign Lab Account’ activity function. The role of this function is to query the lab pool for an available lab environment. This pool is configured as Azure Table Storage. Table storage is essentially a very simple database. The function looks for specific rows to find a user and resource group that are not in use and have been fully cleaned up already since their last lab session. Once a free lab environment is extracted, the next activity function is triggered by the orchestrator: ‘Assign RBAC Permissions’. This is the function which assigns the correct permissions to the lab user account over the resource group in which the lab resources will be deployed. This is an essential security step which ensures that the lab user has no more permission than necessary to simply complete the lab operations necessary for this lab. Once this operation is complete, the last activity function deploys the correct lab resources to the resource group extracted from the lab pool via a Bicep template stored in a private GitHub repo the Function App has access to through a personal access token.

Returning lab info to the user

At this point the lab deployment is complete. The website’s polling endpoint will be able to detect this so will at this point provide the user login credentials to the lab environment. At this point the website component will provide a button to generate a TOTP code for the chosen lab environment’s user account as 2FA is required for azure logins.

Not so simple

As you can see, my vision of a simple, self-contained workflow launching a Bicep template to Azure was pretty naive. But, the journey to creating a robust solution was incredibly edifying. In the subsequent blog posts to come, I expand on some of the finer points. In next post, I will expand on the website’s API endpoints and how they interact with both CosmosDB and the Durable Function App. See you in the next post.

The first post

Welcome to the blog

The purpose of this blog is to detail the inner workings of the Build Cloud Skills lab platform, which we like to call the Cloud Skills Dojo, as well as keep you abreast of what direction we are going in, changes we are making and anything useful or interesting regarding the technical or business aspects of the journey. Getting the lab platform to a state where it can launch labs reliably by clicking a button and then clean up those labs automatically ready for the next launch was a huge journey - far more complicated than I realised. I want to document here what went into that journey. Mostly that is making assumptions about a certain process being simple and then having that assumption destroyed. That is a major theme.

What to expect from the blog

So expect to find here a pretty detailed breakdown of how the lab platform works in the back end, some interesting technical details as well as just notes on why things work a certain way and how we intend to improve things going forward.

Welcome to the dojo

Since this is the first blog post I want to tell you why Build Cloud Skills exists. It seems like if you want to learn cloud computing, you either need to bear through a torturously long video series or heed that age-old advice found ubiquitously in IT forums, “just make a lab and start breaking things”. OK thanks, that’s super helpful. It’s hard to find a structured approach to learning the cloud that is lab and practical-learning heavy and which doesn’t inundate you with hours of video or reading. It feels like there should be a middle-ground between the two where you can learn the technology in detail, but focus on practical learning. This website is the beginning of an attempt to find that.

What is the dojo?

At the moment, it exists as more of a reference guide for Azure technology that provides a lab for the Azure solution you may be studying. But, the intention is to create a fully developed, engaging lab platform that provides a practical approach to learning cloud technology. Instead of a 40 hour PowerPoint presentation, here’s all the labs you need to understand how this thing works. That’s the idea. So stay tuned and watch this space. There is a lot more coming and we want to be as receptive as possible so that we evolve with the needs of like minded learners - so please send over your suggestions, critiques, etc. and don’t hold back. We want to hear it all so we can make this a great learning platform.