- Published on
Drupal 7 to Drupal 8 migration with configuration entities
- Authors
- Name
- Christophe Jossart
- @colorfield
Outdated content
Most content from this article might still be relevant but some sections can be outdated.
Since Drupal 8.5.0, the migrate system is stable, so this is a great time to review the migration tools provided by the community.
This post does not cover migration from the UI, it focuses on partially customized migration that are runned with Drush. In most cases, you will probably want to review the content model a bit (you know, that content type or field machine name that does not actually reflect the truth since its creation, ...).
The idea behind is to delegate the heavy lifting to Migrate Upgrade for migration template generation, then apply content model changes if needed.
Prepare your setup
Require dependencies
The needed tools are quite obvious, the only tricky part for the moment is getting the right versions.
# Require Drush 8 https://www.drupal.org/project/migrate_upgrade/issues/2915726
composer require drush/drush:~8.0
composer require drupal/migrate_upgrade:~3.0
composer require drupal/migrate_tools:~4.0
composer require drupal/migrate_plus:~4.0
Then enable them.
drush en migrate_tools migrate_upgrade
Add Drupal 7 database references
Edit your Drupal 8 settings file, so in this example, you will have to import your Drupal 7 SQL dump in a separate d7 database.
// Database entry for `drush migrate-upgrade --configure-only`
$databases['upgrade']['default'] = array (
'database' => 'd7',
'username' => 'root',
'password' => 'root',
'prefix' => '',
'host' => 'localhost',
'port' => '3306',
'namespace' => 'Drupal\\Core\\Database\\Driver\\mysql',
'driver' => 'mysql',
);
// Database entry for `drush migrate-import`
$databases['migrate']['default'] = array (
'database' => 'd7',
'username' => 'root',
'password' => 'root',
'prefix' => '',
'host' => 'localhost',
'port' => '3306',
'namespace' => 'Drupal\\Core\\Database\\Driver\\mysql',
'driver' => 'mysql',
);
// Database entry for `drush migrate-import`
$databases['drupal_7']['default'] = array (
'database' => 'd7',
'username' => 'root',
'password' => 'root',
'prefix' => '',
'host' => 'localhost',
'port' => '3306',
'namespace' => 'Drupal\\Core\\Database\\Driver\\mysql',
'driver' => 'mysql',
);
Get the Drupal 7 files
A good idea would be to import files on your development server with rsync, so you can get the changed files only before rolling a migration update.
Example
# Get files on your local dev environment from the production server
rsync -avzh my_user@drupal_7_server:/var/www/my_d7_site/docroot/sites/default/files/ /var/www/my_d8_site/web/sites/default/files/d7/public_files/docroot/sites/default/files/
Get the migration templates
This step will generate migration configuration entities in the active configuration store.
You can make a database backup first, as it could be needed before enabling your custom migration module (to avoid a PreExistingConfigException). Note that restarting from a backup is the "hacking around" method, you may want to rename each migration template instead, after having exported them in your custom module. See below.
drush sql-dump > my_drupal_8.sql
drush migrate-upgrade --configure-only
Create a custom module and the configuration directory.
# Follow Drupal Console steps, naming the module my_migrate in this example.
drupal generate:module
# Create then the config/install directory in your module.
mkdir -p my_migrate/config/install
Export site configuration generated by drush migrate-upgrade --configure-only
drush config-export --destination=/tmp/migrate
Then copy it in the custom migration directory. Do not include migrate_plus.migration_group.default.yml
cp /tmp/migrate/migrate_plus.migration.* /tmp/migrate/migrate_plus.migration_group.migrate_*.yml /path/to/my_migrate/config/install/
Remove then unnecessary templates (entities that will not be part of your migration). When removing templates, make sure that they are not required in the migration_dependencies section. If you remove dependencies, make sure that they have been migrated in another way.
Example
The migration template migrate_plus.migration.upgrade_d7_node_article.yml
has the following requirements:
migration_dependencies:
required:
- upgrade_d7_user
- upgrade_d7_node_type
For some reasons, you could have created content types manually and want to get rid of the upgrade_d7_node_type template.
Optionally customize your migration with machine name changes, source / process plugins or hook_migrate_prepare_row()
/ hook_migrate_MIGRATION_ID_prepare_row()
.
Restart then from your Drupal 8 database backup, or just change your migration templates file name, uuid, id and group for each yml file. Just keep in mind that while renaming your migration templates (configuration entities) id, you will have to rename the required and optional dependencies as well.
Run migration with Drush
Once done with your migration templates, enable your new module.
drush en my_migrate
Start to test your migration by id, with dependencies that are required by other entities first (role, user, file, taxonomy term, ...).
# Check the migrate status first, by id
drush ms upgrade_d7_user_role
# Migrate import by id
drush mim upgrade_d7_user_role
Update
Include changes that were made to entities (new or modified ones).
drush mim upgrade_d7_user --update
Limit
drush mim upgrade_d7_file --limit="10 items"
Display migration messages
drush mmsg upgrade_d7_file
Errors will be logged in these messages, so it could be nice to redirect them into a log file.
drush mmsg upgrade_d7_file > sites/default/files/migration_logs/upgrade_d7_file.txt
Rollback
drush mr upgrade_d7_file
Examples
Users and roles
# Migrate status of roles and users
drush ms upgrade_d7_user_role
drush ms upgrade_d7_user
# Migrate import for roles then users
drush mim upgrade_d7_user_role
drush mim upgrade_d7_user
Content and translation
# Migrate content type article + i18n translation
drush mim upgrade_d7_node_article
drush mim upgrade_d7_node_translation_article
Files
Public and private files
If you have imported your files in /var/www/my_d8_site/web/sites/default/files/d7/public_files/docroot/sites/default/files/
modify the /my_migrate/config/install/migrate_plus.migration.upgrade_d7_file.yml
file by setting the source_base_path to /var/www/my_d8_site/web/sites/default/files/d7/public_files/docroot
# For a legacy site that has /docroot/sites/default/files
source_base_path: /var/www/my_d8_site/web/sites/default/files/d7/public_files/docroot
Do the same in the /my_migrate/config/install/migrate_plus.migration.upgrade_d7_file_private.yml
file according to your private files path (ideally outside of the docroot).
Then
drush mim upgrade_d7_file
# And optionally
drush mim upgrade_d7_file_private
Applying changes after having enabled the custom migration module
Use the Configuration Development module or drush config-import --partial
Troubleshooting
Before running migration import, and depending on the database size, you may consider increasing memory_limit in your php.ini
Documentation
Migrate API
- Chapter Three [06/04/2017] Drupal to Drupal 8 via Migrate API
- Drupalize Me [26/04/2016] Custom Drupal-to-Drupal Migrations with Migrate Tools
- Drupalize Me [31/10/2017] Create Migrations from Core Templates
Drush
Users and Roles
- Drupal.org Migrating users
- Deninet [06/07/2017] Building a custom migration in Drupal 8, Part 3: Users and Roles
- Metal Toad [30/11/2014] Drupal 8 Migrations, part 2: Migrating users from a Drupal 7 site
- Jeff Geerling [13/07/2013] Migrating Drupal 7 users from site to site while preserving password hashes
Nodes, merge entities, translation
- Evolving Web on GitHub [01/07/2017] Drupal 8 I18N / Translation Migration Example
- Lullabot [24/07/2017] Merging Entities During a Migration to Drupal 8
File Entities and Media Entities
- Previous Next [24/3/2017] Migrating Drupal 7 File Entities to Drupal 8 Media Entities
- Jeff Geerling [04/10/2016] Migrating 20,000 images, audio clips, and video clips into Drupal 8
- Evolving Web [30/1/2017] Drupal 8 Migration: Migrating Files / Images (Part 3)
- Deninet [13/06/2017] File migrations