Using Terraform to Create VM & Infrastructure
ARM Templates can also be used to create Azure Resource groups, virtual machines and network related resources.
I have chosen to use Terraform becaue ARM Templates are horribly unreadable and spin out simple tasks into hours of tedious syntax corrections.
To create Azure resources with Terraform you simply write some scripts, which are held in source control. The Terraform tool, now installed on my build agent, takes these scripts and creates or updates everything in Azure.
Terraform for Azure also requires a storage account and container in which it stores its own configuration. My pipeline creates or updates this before running the Terraform steps.
There are no real gotchas with Terraform, the only issue which tripped me up a bit was retrieving the IP address of my new VM once it had been created in Azure. You need to tell Terraform to wait until the provisioning is finished.
To allow Ansible to work with Terraform I generate the Ansible hosts file from Terraform, it contains only the IP Address of the VM.
You can get Ansible to generate its own host files from Azure, provided you have tagged your resources in some way, I didn’t get that to work, but didn’t play with it for long as its overkill for this use case.
Both Terraform and Ansible support variables and I use these so I that I can deploy the system to different subscriptions or resource groups. There are a number of ways of passing the variables from DevOps to these systems but I have used a combination of setting ENV variables on the Build Agent and using a re-writing step to add my variables in to their variables files.
Once the Terraform scripts have been successfully executed within the pipeline the following things will have happened
- A resource group will have been created for my Live Helper Chat virtual machine
- All necessary network related resources and configuration will have been created within that resource group
- The Bitnami Live Helper Chat virtual machine will have been created within that resource group and configured for login with my pre-generated ssh key
- An ansible hosts file will have been created on the Build Agent with the IP Address of the Live Helper Chat virtual machine
Using Ansible to Provision a Bitnami VM
Like Terraform, Ansible is a series of scripts which are ran on the target machine to perform post installation configuration.
Ansible is an agentless system, in that you do not need any “ansible agent” on the target machine. It runs, in this case, on the build server and uses ssh to log in to the target machine. In my scenario both the Build Agent and Live Helper Chat are running on Linux which helps a bit. Ansible also supports WinRM for Windows servers.
The post installation changes we required were all straightforward amendments to apache config files and the creation of some MySQL setup to allow us import and export the database.
The only gotcha with Bitnami VMs is that once they are powered on for the first time they spend a long time configuring themselves. If you attempt to alter any of the configuration they are going to use things go wrong.
Luckily, once Bitnami finish their configuration they write a file out to indicate it is complete. Ansible can monitor for the presence of this file and only apply the rest of its configuration once it appears.
Bitnami Virtual Machine Pipeline
Above is the complete pipeline used to deploy a Bitnami virtual machine to Azure and apply some post deployment configuration.
A brief description is
- Insert some of pipeline variables into the Ansible variables file so that I can use this pipeline for multiple deployments into different resource groups/subscriptions etc
- Create the back end storage account for Terraform, using an ARM Template
- Grab the storage account name to be passed to Terraform
- Install my ssh key onto the build agent to be used in the VM and by Ansible
- Initialise Terraform
- Apply the Terraform scripts
- Copy my baseline database file to the VM
- Use Ansible to configure the VM
- Deploy my test website to check the chat widget works when embedded into another website
Source Code & Scripts
At some point soon I hope to share these on github or somewhere, just as soon as I get that set up.