The Azure Cosmos DB Desktop Data Migration Tool is a command-line tool that provides data import and export capabilities for using with Azure Cosmos DB. The tool helps with both migrating data from several data sources into Azure Cosmos DB, as well as exporting data out of Cosmos DB too! The tool is an open source project from Microsoft that is built and maintained by a team of developers from Solliance; including Build5Nine’s own Chris Pietschmann! The Solliance team is able to bring their extensive data migration and Microsoft Azure expertise to building the best Azure Cosmos DB migration tool possible!

The Azure Cosmos DB Desktop Data Migration tool is built using a plugin style architecture so it has an extensibility model with several built-in plugins. This plugin model makes the project extensible to support additional database services in the future.

The built-in plugin for the project currently include:

• Azure Cosmos DB
• Azure Table API
• MongoDB
• SQL Server
• JSON

As you can see the most common sources and destinations (aka Sinks) for migration data with Azure Cosmos DB are already supported. With the plugin architecture, you will be able to develop your own plugins, if you dare to dig into the programming side of the tool, to perform your own custom data migrations. If you don’t want to built a plugin for the tool, as I’m sure most of us may not have the bandwidth to, you could export your data to JSON format and then use the JSON plugin to import that data into Azure Cosmos DB.

The following is a diagram that shows a high level of how the plugin architecture works:

The workflow for using the Azure Cosmos DB Desktop Data Migration Tool, as shows in the previous diagram, is as follows:

1. The Azure Cosmos DB Desktop Data Migration Tool command-line interface (CLI) is used on your local machine to kick off a data migration.
2. The CLI tool reads in your migrationservices.json that is configured with the specific Source and Destination (aka Sink) plugin settings, including connections strings to services and other Source and Sink plugin configurations.
3. The tools then makes calls to the appropriate plugins for Source and Destination (aka Sink) data services, and it performs the configured data migration.

The following is a sample that gives a general idea of how the migrationservices.json configuration is specified using Cosmos DB as the Destination (aka Sink) and MongoDB as the data Source:

{
    "Source": "mongo",
    "Sink": "cosmos-nosql",
    "SourceSettings": {
        "ConnectionString": "mongodb://...",
        "DatabaseName": "myDb",
        "Collection": "myCollection"
    },
"SinkSettings": {
    "ConnectionString": "AccountEndpoint=...",
    "Database": "myDb",
    "Container": "myContainer",
    "PartitionKeyPath": "/id",
    "ConnectionMode": "Direct",
    "MaxRetryCount": 10,
    "WriteMode": "Upsert"
    }
}

The use of the migrationservices.json file for configuring the tool helps to make data migration configurations reusable. This enables you to run and re-run the data migration process as many times as you need with the exact same configurations each time. When migrating data to production you generally only need to run the migration process a single time, but when testing the migration to get everything correct in dev/test environments you’ll likely run it multiple times. The migrationservices.json file will help with this test/debug, then migrate to production process.

Running the Azure Cosmos DB Desktop Data Migration Tool from the command-line operates by calling the dmt.exe executable. And when calling the tool, you will use the --settings argument to tell it the location of the migrationservices.json. Keep in mind that with this argument, you can name the migationsevices.json file what ever you want that fits your workflow.

The following is an example of calling the tool from the command-line:

dmt.exe --settings "c:\settings\mongodb-to-cosmosdb.json"

The tool also supports additional command-line arguments that enable some configuration of the tool without using the migrationservices.json if that’s how you wish to integrate the tool into your own data migration workflow.

The Solliance team was a frequent user of the old version of the Azure Cosmos DB Desktop Data Migration Tool, and approached Microsoft volunteering to take on the development of this new version release of the tool. This is a volunteer, open source development effort to built and maintain this tool with the goal to help provide the data community the best Azure Cosmos DB data migration tool possible!

The AzureCosmosDB/data-migration-desktop-tool project over on GitHub is the official home of this project, and contains documentation on using the tool to perform your own Azure Cosmos DB data migrations!