Cloud Migration as a Service
Migrating existing workloads between clouds is a necessity for a large number of use cases, especially for user moving from traditional virtualization technologies like VMware vSphere or Microsoft System Center VMM to Azure / AzureStack, OpenStack, Amazon AWS or Google Cloud. Furthermore, cloud to cloud migrations, like AWS to Azure are also a common requirement.
Project Coriolis(R) addresses exactly those requirements, in particular migrating Linux (Ubuntu, Red Hat / CentOS, SUSE, Debian, Fedora) and Windows virtual machine, templates, storage and networking configurations.
There are some tricky scenarios where Coriolis excels: to begin with, virtual machines need to be moved between different hypervisors, which means including new operating system drivers and tools, for example cloud-init / cloudbase-init in the OpenStack use case, LIS kernel modules on Hyper-V and Azure and so on.
The project is largely using Oslo libraries with an architecture meant to be familiar for devops used to OpenStack, as shown in the diagram below. A clear separation between stateless microservices and proper usage of queues allows scalability and fault tolerance from the start. For PoCs and small environments, all components can be deployed on a single host / VM / container.
Authentication and endpoint discovery is based on Keystone (in the typical OpenStack way), which means that passing an X-Auth-Token header from an existing Keystone session allows an easy integration with other components, e.g. in Horizon's console. The same token is passed to other components along the pipeline, which also implies that importing virtual machines and other resources to the same OpenStack infrastructure doesn't require further authentication.
Authentication to external clouds (Azure, AWS, etc) or virtualisation solutions (vSphere, SCVMM, etc) in order to export virtual resources requires credentials that can be saved in Barbican, thus avoiding the need to pass secrets directly to the API.
Cloud resources that can be migrated:
- Virtual machines
- Virtual Machine templates
- Storage
- Network configurations
VM disks are converted to the desired target format and drivers / tools are automatically added where appropriate during the process (e. cloud-init on OpenStack, KVM Windows VirtIO drivers, etc).
The migration jobs are split in import / export tasks with a scheduler taking care of choosing a worker node where this can be executed. Each task contains progress update info that the client can poll to follow the progress of the operations. Tasks can be relatively long running, depending on the storage size, so proper status reporting was included in the design from the start.
Here's an example Keystone service and endpoints configuration:
openstack service create --name coriolis --description "Cloud Migration as a Service" migration ENDPOINT_URL="http://hostname:7667/v1/%(tenant_id)s" openstack endpoint create --region RegionOne migration ` --publicurl $ENDPOINT_URL ` --internalurl $ENDPOINT_URL ` --adminurl $ENDPOINT_URL openstack user create --password-prompt coriolis openstack role add --project service --user coriolis admin
The API is also very straightforward, here's a complete example available on Postman: https://api.postman.com/collections/2414861-766ace3d-b46c-4510-bbaf-e1f001a8be75?access_key=PMAT-01GJTF9HV7MY7YMHZW7Q79N5V2
Create a migration job:
POST http://server:7667/v1/%project_id/migrations
Example request body:
{
"migration": {
"origin": {
"type": "vmware_vsphere",
"connection_info": {
"secret_id": "ebe69d82-da6f-451e-a0f6-3551d0f7ef85"
}
},
"destination": {
"type": "openstack",
"target_environment": {
"flavor_name": "m1.small",
"network_map": {
"VM Network": "private",
"VM Network Local": "public"
}
}
},
"instances": ["CentOS 7", "RHEL 7.2", "Ubuntu 14.04", "WS 2012 R2"]
}
}
Note: here's an example secret stored in Barbican with vSphere connection info:
{
"host": "10.0.0.10",
"username": "[email protected]",
"password": "Password",
"allow_untrusted": true
}
List migrations:
GET http://server:7667/v1/%(project_id)s/migrations GET http://server:7667/v1/%(project_id)s/migrations/detail
Get a migration job info:
GET http://server:7667/v1/%(project_id)s/migrations/%(migration_id)s
Cancel a migration job:
This API allows the user to interrupt any running job.
POST http://server:7667/v1/%(project_id)s/migrations/%(migration_id)s/action
Request body:
{ "cancel": null }
Delete a migration job:
DELETE http://server:7667/v1/%(project_id)s/migrations/%(migration_id)s
Note: only completed, failed or cancelled jobs can be deleted.
To build the API documentation, while in the repository root directory, run:
sphinx-build -W -b html coriolis/api-refs/source $DOCS_PATH
A reference Python client library implementation is available at: https://github.com/cloudbase/python-coriolisclient
The official Web-based GUI for Coriolis is available at: https://github.com/cloudbase/coriolis-web