2 May 2019

Terraform Import (Manually)

Terraform’s all great and all, but what if you already have infrastructure and you can’t just burn it all down and build it back up again using Terraform? Well, terraform import to the rescue.

…except it’s not as simple as that: you have to import individual resources by specifying their type and ID, one by one. But that’s not a big deal and the plan is to support a more robust import in the future.

…except the future is the future and import may not be supported by the Provider you’re using. In my case, OpenStack. So I have three options to get my existing infrastructure under Terraform:

  1. Don’t.

  2. Implement that functionality in the Provider.

  3. Fake it out.

The first isn’t really an option; I only included it because I’d already typed “three options” and didn’t want to go back. The second is very non-trivial because I have only started experimenting with Go. The third seems like a decent option, given that the Terraform state files are JSON (which I can’t stand but anyway) and that Terraform is a pretty solid tool I feel I can confidently play jokes on without it getting angry.

How that went: the first VM

I started with a pretty simple server (one attached volume, no floating IP, default security group). I had an older terraform.tfstate file for this OpenStack project (‘tenant’ in older nomenclature) from earlier experimentation and used this as a base, copying old stanzas, updating them for new VMs, and commenting out the old ones. After some fiddling and experimentation, I got a successful terraform plan where the Terraform configuration and state matched such that Terraform noted:

No changes. Infrastructure is up-to-date.

This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, no
actions need to be performed.

Note that since terraform plan does not make any changes, it does not affect the state file, so I’m free to continue editing that file without it being affected externally to the editor. This will be important later on when I get things synced up and then want to make a change.

How that went: the second VM

I sadly ran into a wall on the second node I tried to import this way, which took some time to figure out. I would get the message:

* openstack_compute_instance_v2.node2: openstack_compute_instance_v2.node2: Resource not found

If you know Terraform, this is pretty vague because a lot of things are resources, and there are dependencies between them. Right off the bat I suspected it wasn’t the node itself that couldn’t be found, but I wasn’t sure what. I set TF_LOG=DEBUG and carefully read through the resulting reams, and developed a suspicion Terraform was having trouble sorting out the network configuration, because that was the last thing the debug logs showing being examined before an exception is thrown.

It wound up being the flavour, which I knew was an older flavour discontinued by our OpenStack team. For whatever reason, the OpenStack provider for Terraform, given one of flavor_name or flavor_id, queries for the other so it has the pair. The older flavour still exists in our OpenStack–otherwise my node could not–but it’s not listed or visible to users so they don’t have it as a choice in creating new VMs.

Fortunately, we have new flavours which are, for my purposes, identical. I used OpenStack’s resize to resize it to a newer flavour, which required a reboot, I adjusted the Terraform confiration and state file to reflect the new flavour’s ID and name, and I was able to proceed.

How that went: floating IPs, security groups and rules

Updating other elements went more or less fine. The problem isn’t that it’s difficult, it’s that it’s tedious as anything. Which leads me to: I could write a script to do this. Which in turns leads me to: maybe I should just learn Go and update the OpenStack provider.