Drupal 8 JSON custom migration

After spending some time on experimenting custom migration for Drupal 7 contents at the early stages of Drupal 8.0.0, I decided to have a look again at what the core and contrib migrate modules have out of the box for custom cases: let's say data structure change from another CMS, or even from a previous version of Drupal.
I was especially interested in the JSON and XML formats that are quite popular and easy to produce as output with Rest or even RSS.

Things have evolved quickly, we have now a huge bunch of helpers since 8.2.x in core for Drupal to Drupal migration: Migrate, Migrate Drupal and Migrate Drupal UI.

When we look at contrib modules like Migrate Plus that comes with loads of examples for Rest / JSON / ..., and Migrate Tools for drush commands and UI we are now really empowered for custom migration.

Based on the article from Jeff Geerling (Migrate a custom JSON feed in Drupal 8 with Migrate Source JSON) + the numerous examples from Migrate Plus, I decided to roll a minimal JSON test to get started quickly.
I took the 'Product' example from Jeff Geerling that explained the migrate_source_json which is now part of the migrate_plus module, with the slight differences mentioned hereunder for migrate_plus.

1. Create the module structure

migrate_test.info.yml

name: Migrate Test
type: module
description: Migration tests
core: 8.x
package: Migration
dependencies:
  - migrate
  - migrate_plus
  - migrate_tools

config/install/migrate_plus.migration.product.yml

# Migration configuration for products.
id: product_json
label: JSON feed of products
migration_group: product

source:
  plugin: url
  data_fetcher_plugin: http
  data_parser_plugin: json
  # The URL of your JSON datasource
  urls: http://drupal82.dev/sites/default/files/migrate_test/products.json
  item_selector: product
  fields:
    - name: upc
      label: 'Unique product identifier'
      selector: upc
    - name: name
      label: 'Product name'
      selector: name
    - name: description
      label: 'Product description'
      selector: description
    - name: price
      label: 'Product price'
      selector: price
  ids:
    upc:
      type: integer

destination:
  plugin: entity:node

process:
  type:
    plugin: default_value
    default_value: product

  title: name
  field_upc: upc
  field_description: description
  field_price: price

  sticky:
    plugin: default_value
    default_value: 0
  uid:
    plugin: default_value
    default_value: 0

migration_dependencies: {}
dependencies:
  enforced:
    module:
      - migrate_test

2. Place the JSON datasource

To keep it simple, I put it under sites/default/files/migrate/products.json

{
  "product": [
    {
      "upc": "11111",
      "name": "Widget",
      "description": "Helpful for many things.",
      "price": "14.99"
    },
    {
      "upc": "22222",
      "name": "Sprocket",
      "description": "Helpful for things needing sprockets.",
      "price": "8.99"
    }
  ]
}

3. Define the content type

Create a content type with product as machine name and the following fields :

• upc (Integer)
• description (Plain text)
• Price (Float)

4. Run migration

# Enable migrate_test
drush en migrate_test
# Get the migration status
drush ms
# Migrate the product group
drush mi --group=product

5. Going further

This example is quite minimal, the readme from the Migrate Plus examples submodules gives plenty of explanations, there is also a brief example of how to define a migration configuration form, from the post mentioned above.

Resources

Wunderkraut Migrate Source Example : contains example of content migration from external database tables, CSV files, XML files, JSON resources. One of the most up-to-date example.