Automated On Demand High Performance Compute
About
This solution provides automated on-demand provisioning/deprovisioning of high-GPU VMs to autonomously run flood modeling. Model results are stored in Azure Storage and a Cosmos database for viewing via an Azure Static Web App. The solution supports multiple models to be run in parallel to improve flood prediction.
Solution resources and automation are secured with OAuth 2.0 and designed for regional resilience.
Key Components
- Azure ARM template for the VM provisioning
- Azure Functions (active/passive) to trigger the automation elements
- Azure Static Web App to host the modelling front-end and display model results
- Azure Storage Accounts to store model results and automation scripts
- Cosmos Database with regional redundancy
- Azure GPU-enabled VMs
- Application Insights for diagnostics and telemetry
- Azure Key Vault for solution operational secrets
- Azure Traffic Manager for regional resilience
- PowerShell Automation scripts for orchestrating the model run on the VM
- Azure Backup for solution configuration recovery
- PowerShell automation of URBS model parameters upload
Business Requirement
Flood modelling is a compute intensive process involving high-end desktops and manual steps to gather the model input data, update configuration files, run models, and process output. All in all, the process was time consuming, and constrained by hardware.
The business required an automated process that leveraged high-end GPU compute in the cloud with results automatically available via an web application.
The main drivers were leveraging PAYG highend GPU compute over expensive desktop hardware, and ability to run multiple models in parralel to improve flood prediction.
CLOUD PARTNERS TECHNOLOGY