How Lab Deployments Work Part 1 - Backend Overview
Intro
The lab platform started as a proof of concept in a Python Flask app with a simple app route to launch a Bicep template into a resource group (which was hard-coded into the application’s source code no less). In its nascent form I was positive that the project would remain a simple, self-contained workflow that would not possibly require much further overhead than figuring out this core aspect of the platform - which was launching the environments themselves. My theory at the time was that once the relatively simple lab deployment workflow was complete, I could move onto the more standard aspects of a website such as authentication, hosting, front framework, etc. This was the first of many times on this journey I was completely and utterly wrong in the most unmitigated fashion. As such, my intention in this blog post is to document the backend of the platform, at its now more mature state, for anyone else interested in the eventual approach I came to find is robust and reliable for launching fully customisable lab environments to Azure. This will be a high-level overview of the Azure services that are responsible for the deployment of the lab environments, plus a little on how the website’s API endpoints construct the HTTP request that triggers the lab deployment and then polls to find the status of the deployment. In later posts, I will detail other aspects of the platform such as how labs are cleaned up after use, the website framework itself, the Azure DevOps pipeline that launches the site and other details.
A broad overview
Before elaborating on each stage of the deployment process, I will provide you a brief overview of how the whole deployment works from start to finish.
API endpoint which triggers deployment
Essentially, when a user presses the launch lab button on the website, that button component provides a labID to the website’s launch-lab API endpoint and triggers it to look up the correct lab configuration from Azure CosmosDB for NoSQL. The JSON data it extracts from this database contains contains the location of the relevant Bicep template, the RBAC permissions that the lab user will require over the launched resources in order to complete the lab tasks and the subscription ID of the Azure subscription in which the deployment will take place. The URL endpoint of the Azure Durable Function, which is the destination of the HTTP request, is also in the CosmosDB database.
Azure Durable Function app architecture
The lab config information is now extracted and sent to the Azure Durable Function’s HTTP trigger - also called durable starter function. The starter function extracts and validates the payload to ensure the relevant information it needs to pass to the orchestrator is present then triggers the entire orchestration which is responsible for deploying the lab environment.
Activity Functions and a note on Table Storage
The orchestrator first starts the ‘Get and Assign Lab Account’ activity function. The role of this function is to query the lab pool for an available lab environment. This pool is configured as Azure Table Storage. Table storage is essentially a very simple database. The function looks for specific rows to find a user and resource group that are not in use and have been fully cleaned up already since their last lab session. Once a free lab environment is extracted, the next activity function is triggered by the orchestrator: ‘Assign RBAC Permissions’. This is the function which assigns the correct permissions to the lab user account over the resource group in which the lab resources will be deployed. This is an essential security step which ensures that the lab user has no more permission than necessary to simply complete the lab operations necessary for this lab. Once this operation is complete, the last activity function deploys the correct lab resources to the resource group extracted from the lab pool via a Bicep template stored in a private GitHub repo the Function App has access to through a personal access token.
Returning lab info to the user
At this point the lab deployment is complete. The website’s polling endpoint will be able to detect this so will at this point provide the user login credentials to the lab environment. At this point the website component will provide a button to generate a TOTP code for the chosen lab environment’s user account as 2FA is required for azure logins.
Not so simple
As you can see, my vision of a simple, self-contained workflow launching a Bicep template to Azure was pretty naive. But, the journey to creating a robust solution was incredibly edifying. In the subsequent blog posts to come, I expand on some of the finer points. In next post, I will expand on the website’s API endpoints and how they interact with both CosmosDB and the Durable Function App. See you in the next post.