configuration management

Configuration split: deploy subsets of configuration in Drupal 8

From the Configuration split project page: "The Drupal 8 configuration management works best when importing and exporting the whole set of the sites configuration. However, sometimes developers like to opt out of the robustness of CMI and have a super-set of configuration active on their development machine and deploy only a subset. The canonical example for this is to have the devel module installed or having a few block placements or views in the development environment and then not export them into the set of configuration to be deployed, yet still being able to share the development configuration with colleagues."

This small utility module for Drupal 8 allows you to exactly achieve this use case (and many others). I will cover two common scenarios, explaining how to configure your site and which commands you need to run:

  1. Have (development/ui) modules installed on your dev environment, uninstalled on production
  2. Have modules installed on your production environment, but not on development

I've also recorded a screencast in which I configure and run all the commands explained underneath. The best way in the end is to play around with the module itself of course.

Configuration split 101

Configuration split exposes a configuration entity which controls what you want to split off. Currently you can

  • blacklist modules: any configuration that this module owns will automatically be blacklisted too.
  • blacklist configuration: settings or configuration entities. These will be removed from the active sync directory.
  • graylist configuration: settings or configuration entities. These will not be removed if they are in the active sync directory, but also not exported if they are not there yet. I won't cover this functionality in this article post, that will be for another time.

After you configured one or more configurations split entities, you can use the drush commands to export and import configuration, based on one or more of those config entities. When it comes to exporting configuration, you can not use the existing drush core command (config-export / cex). Using drush config-import (cim) or the UI to import may still be used, but this depends on your setup. Drupal Console commands are under revision. Important: since feb 2017, due to https://www.drupal.org/node/2855319 and https://github.com/drush-ops/drush/pull/2471, you can use the standard drush commands without any problems!

Each config split entity defines the directory in which the splitted configuration will live in case there are some. Not all modules define configuration, so there's a possibility that this directory is empty after you do an export.

An important technical aspect is that the module does not interfere with the active configuration but instead filters on the import/export pipeline. What simple happens is this: just before the actual writing, it will check the configuration and remove any entries that you don't want to be there. Then your active configuration is written away. Config that doesn't belong into the active configuration will be moved to separate directory. On import, the opposite happens: it will merge settings back in and set modules to the installed state just before core configuration then starts importing.

Last, but not least: you can swap the config.storage.sync service so that the synchronize screen will use the config split config entities for importing. More information is in the README file and in the video.

Scenario 1: modules installed on dev, not on production

Step 1

After you installed the module, go to 'admin/config/development/configuration/config-split' and click on 'Add configuration split'. Naming is important, so we'll enter 'Dev split' as the name for this configuration. We'll also create a directory called 'sync-dev-split'. Now toggle the modules that you don't want to have installed on production. In this case, we're going to blacklist Devel, Devel Kint, Field UI, Views UI and database logging. Additionally, also toggle system.menu.devel. This configuration is owned by the system module, so there's no dependency on the devel module. There's no problem having this config on your production site though, so it's not required to blacklist it.

Optionally, you can also blacklist the configuration split module because it doesn't necessarily have to be installed on production. We're not doing that here, because in the second scenario, we're building further on this one and we want to have it installed on the production environment as well.

Step 2

Go to your command line interface and run following command:

swentel@dev:/home/drupal/drupal-core$ drush csex --split=dev_split

The split option is not required, but when starting to work with the module, it helps you to know which config split will be used for this command. It's possible to override the status of any config split config entity so that eventually, you can omit the split option. Sadly enough, you don't get any feedback (yet), but after the command has run you should see various files in your sync-dev-split directory:

swentel@dev:/home/drupal/drupal-core$ ls sync-dev-split/
dblog.settings.yml  devel.settings.yml  field_ui.settings.yml  system.menu.devel.yml

Step 3

You can now commit your active sync and go to production. You don't necessarily have to put the sync-dev-live directory in your version control because production doesn't need to know about these files at all. Once you have pulled on production, go to the synchronize screen to verify what is staged. You will see that the setting files will be removed and core.extension will be changed uninstalling the development modules and installing configuration split. Since we only have one config split configuration, you can hit 'Import all' here or run drush config-import, (which is the drush core command). Note that if you don't use the UI, you can blacklist the configuration manager module as well.

You might wonder why we don't use the 'config-split-import' command from the module itself: this would import the active sync directory and also include the modules and settings again that we have blacklisted. And that's not what we want. This seems confusing at first, but ultimately, if this is your setup, you just keep on using the core / drush commands to import your staged configuration on production.

# This command can be used now, but not anymore further on when we will add scenario 2.
swentel@live:/home/drupal/drupal-core$ drush cim

Scenario 2: modules installed on production, not on dev

Step 1

This scenario builds further on the first one. Config split is now installed on our live website. Create a new config split configuration called 'Live split'. You will need a new directory for this second config split, so make sure it's there. On production we still want to log events and for that we're going to use the syslog module. Install the module, so that we can now blacklist it for the live split configuration.

Step 2

Go to your command line interface and run following commands:

swentel@live:/home/drupal/drupal-core$ drush csex --split=live_split

Again, no feedback here, but after the command has run you should see the syslog settings file in your sync-live-split directory:

swentel@live:/home/drupal/drupal-core$ ls sync-live-split/
syslog.settings.yml

Doing exports and imports on dev or live

From now on, if you want to import new settings whether it's on your dev or live environment, you can not use the drush core commands anymore. Use following commands:

# to export on your dev environment
swentel@dev:/home/drupal/drupal-core$ drush csex --split=dev_split
# to import on your dev environment
swentel@dev:/home/drupal/drupal-core$ drush csim --split=dev_split

# to export on your live environment
swentel@live:/home/drupal/drupal-core$ drush csex --split=live_split
# to import on your live environment
swentel@live:/home/drupal/drupal-core$ drush csim --split=live_split

The future

Please join us in the issue queue because there's still some work:

  • Optimize the user interface and allow wildcards
  • Add confirmation and feedback in the drush commands

Ultimately, this functionality should live in core. A core issue to discuss this is at https://www.drupal.org/node/2830300.

Help out with Field API for Drupal 8

Last week, I attended the Field API pre Drupalcon sprint in Munich. My major motivation was to start working on the conversion of Field API to CMI and little than a week later, we have a patch that is green. While there's still a lot of work on it, it's still a good feeling after one week of hacking and discussions. After driving 8 hours home after a fantastic week, I decided to step up as a co-maintainer of Field API. But me stepping up won't make the big difference, we need more people joining us!

Anyone can help in many different ways

  • Gardening the queue: go through bug reports, ask for more info when something is not clear or close duplicates.
  • You can work on an issue by creating patches. Assign an issue to yourself so we know who's working on it.
  • Help by reviewing and testing patches to see if they actually work as intented.
  • Come bring us coffee and cake :)

Where should I start ?

There's a lot of work, that's for sure: CMI, widget and formatter plugins, Field API vs OO and so on. We have a site which lists the most important issues: http://realize.be/field-api - Bookmark that if you're interested!

We are also available on freenode.net on #drupal-fieldapi in case you have any questions.

How much time should I invest in this ?

That is totally up to you. Remember, this is voluntary work, so we do not expect anybody to work 40 hours a week - although that would be more than fantastic of course :) As an example, this is roughly what I think my weekly schedule will look like:

  1. Continue working further on the CMI patch. I have sponsored time from Wunderkraut so I can focus hard on getting this critical piece of functionality in. Goodbye field_config and field_config_instance tables!
  2. Go through the queue twice a week for about an hour and review existing patches from other people.
  3. Go through the queue twice a week for half an hour and close duplicates or ask for more info.
  4. Pick one 'quick win' issue per week to create a patch from it, either a bug or feature request.
  5. Work on Display Suite issue queue, this usually happens once a week for about 3 hours. Been like that since forever.

So wait, I thought you didn't sleep?

Yes I do, and I do other things as well. It's not worth spending all your nights on coding, you'll get burned out before you know it. But wouldn't it be great to go to sleep and someone else was doing some great work ?

So, who's helping along?

Subscribe to RSS - configuration management

You are here