Custom blocks cannot be properly exported and imported

This is how it was designed to work.

If you deploy your content with rest endpoint or deploy module, entity pilot module or use an update hook to create the block entity with the same uuid- it will repair itself.

I'm using this feature on several production sites and loving it.

Sorry but I don't agree this can be closed. Clearly its a usability issue since it results in Broken/Missing dependencies. I don't think solution is that user should know they need to use another contrib module (which seems to be in alpha!) in order to get this working.

Have to agree with @tamerzg here.

I think there would be two solution to this problem, either :
- Remove the export of the block placement (instance) from the config export
- Export the content of the block since it's so tightly coupled with the configuration

It's a real pain for developers new to Drupal 8 to figure out a way to deal with this situation when your first deployment miss the content and you get broken blocks all over the place. The documentation is clearly not cutting it and everyone seems to have a different way to deal with that. Maybe it's designed like that, but is it a good one ? There is room for reflexion here, I truly feel this is a bad DX. Maybe there is other solutions, maybe a clear documentation (cookbook style) would help ?

But in this state it's just plain frustrating and does not really feel 'finished'.

Thanks for weighing in @larowlan. I understand that content can't be part of the export > import workflow, but which of the solutions you outline would you recommend as the easiest and most foolproof? Do you have a simple example of an update hook which creates a block entity and adds some content to it?

To answer my own question, it's actually not too complicated to create a custom module which adds a block with some text. Just follow the instructions at Create a custom block.

Here's a working "Hello Block" module example. NOTE: I am not sure what happened, but you need to rename the file to hello_world.tar.gz to be able to extract it.

1. Enable the "Hello Block" module
2. Assign the custom "Hello Block" block to a region
3. Export configuration from site A
4. Import on site B
5. The block appears on site B, with the content from Site A's "Hello Block" module

Perhaps you want https://www.drupal.org/project/simple_block then, which uses config to store the blocks instead of content.

Some discussion in #2730605: Improve the SBX of Configuration in Block Module

One other possible approach is to use the Default Content module to export the content. It's built for the content to be exported to an installation profile's 'content' folder, and then the module, if enabled, automatically brings the content in when the site is installed. Presuming that reinstalling the site would be a drastic step for getting one block's content (aside: we do find it useful to use installation profiles with default content for automated testing), it's also possible to import the content one item at a time, such as in an update hook, with the below code in your example.install or example.profile:

/**
 * Import a piece of content exported by default content module.
 */
function example_import_default_content($path_to_content_json) {
  list($entity_type_id, $filename) = explode('/', $path_to_content_json);
  $p = drupal_get_path('profile', 'guts');
  $encoded_content = file_get_contents($p . '/content/' . $path_to_content_json);
  $serializer = \Drupal::service('serializer');
  $content = $serializer->decode($encoded_content, 'hal_json');
  global $base_url;
  $url = $base_url . base_path();
  $content['_links']['type']['href'] = str_replace('http://drupal.org/', $url, $content['_links']['type']['href']);
  $contents = $serializer->encode($content, 'hal_json');
  $class = 'Drupal\\' . $entity_type_id . '\Entity\\' . str_replace(' ', '', ucwords(str_replace('_', ' ', $entity_type_id)));
  $entity = $serializer->deserialize($contents, $class, 'hal_json', array('request_method' => 'POST'));
  $entity->enforceIsNew(TRUE);
  $entity->save();
}

Export a custom block with an ID of 8:

drush dcer block_content 8

And used in your example.install file:

/**
 * Add the footer block content.
 *
 * Implements hook_update_N().
 */
function example_update_8001() {
  example_import_default_content('block_content/136efd63-021e-42ea-8202-8b97305cc07f.json');
}

See add content from the default content module in an update hook.

The 8.1.x branch is closed for bug fixes as described above.

To be able to place easily editable content via the block system within a dev > stg > prod workflow using drush config-export and drush config-import, you could create a "Text blocks" view with several blocks, and filter on "Title: starts with" and "Content type=Basic Page".

For example, to place a block on a Subject lists Views page, put some helper text in the blocks' "Empty view" field, like Create a page with Title "Subject list", and this in the Body field: "SOME CONTENT FOR THIS BODY FIELD". This requires of course that only a single Basic Page starts with the title "Subject list". Place the block in the region, and decide on which pages it should be shown.

Only show the Body field and an edit field, so that the editors have a link to edit the content of these texts.

This way a dev > stg > prod workflow is possible, keeping the content placed in blocks AND giving editors the option of updating the content, which will not be overwritten by a staging process like fx:

  # DEV
  drush @DEV-alias config-export
  git add -A :/
  git commit -m "added som stuff"
  git push

  # STG
  git pull
  drush @STG-alias config-import --preview=diff

I admit it is a bit clunky, but it seems to do the job. Also, this is really only an option if there are no more than a handful of pages.

One approach is to create update hook in which you create custom block programmatically. First, you must get uudi of your custom block and then on the creation of the custom block in the update hook you manually set value for uudi (you can also set values for other fields in your block). On site B custom block will be created with the same uuid as it is on the site A and you will get no error for "Block description Broken/Missing". Custom block on the site B will then be editable and nothing will be overridden in the next configuration import.

My only concern here is how safe it is to manually set uuid of the custom block and what are the chances that this will result in an error of duplicated uuid.

I agree with tamerzg and morsok above: this issue needs to be fixed here in core.

I know how to programm custom blocks, but the point is that these blocks here are created using core's custom-block GUI.

So, like morsok said, either drop this placement-config from the sync altogether, or copy the content, or maybe create some block with a content-standin (which then could be changed manually on the target site) so that at least the block does exist -- but don't break the target site by placing a non-existing block!

I'll try one of the contrib modules for now, but I see these as workarounds to a fundamental flaw here.

This is probably the number one minor annoyance I've had with a pure CMI workflow in Drupal 8. It just feels wrong to have to either:

1. Create a block on prod.
2. Pull back the prod database.
3. Place the block locally.
4. Export local config.
5. Push config to prod.
6. Import config on prod so block is finally placed and won't get un-placed on future CMI full imports.

Or:

1. Create a block locally.
2. Place the block locally.
3. Write update hook that creates the block again, but with a custom UUID (this just feels wrong).
4. Export local config.
5. Push config to prod.
6. Run database updates on prod (this creates the block).
7. Run config import on prod (this places the block).

I feel like we should either completely exclude custom blocks from CMI, or take the approach of the Simple Block module in core, for custom blocks...

An entity (block) could be created automatically when importing a config (which may be not a trivial task),
or we could add an option to set uuid while creating an entity (block) manually, so after importing a config the consistency could be restored.

Should this issue not be elevated to 'critical' to get some more love from the powers to be?

By definition:
https://www.drupal.org/core/issue-priority
ciritical is when there is no workaround.

And in this case, the suggested "workaround" drags on major production environments, let alone it is not really feasible for extremely large DBs ( do you really want to install a 1GB+ DB to just fix a block? Try selling that to an enterprise customer, Drupal).

To quote more fully, to be critical an issue needs to "Render a site unusable and have no workaround".

This issue does not render sites unusable. Much as it's a big PITA for everyone trying to have good deployment workflows, it doesn't met critical criteria and escalating to that would just annoy people.

Custom blocks conflict with config synchronization at this point, which creates a fundamental usability problem:
option 1: if using config-sync, then disable custom blocks from the site (or remove permissions so they cannot be created)
option 2: diligently manually config_ignore all newly created custom blocks (unsustainable on an active site with many users)

Perhaps a module could be written that would automatically add any custom block into config_ignore settings, which might make option 2 above more workable.

The problem: add and place a custom block, and it will be added to config on config-export. When importing into production, the new config will create a broken block in production, unless this block config is ignored through some manual workaround.

The only way to avoid this is to fully sync the database (eg: stage to production.) For most sites, this won't work.
Alternatively, we might back-port the site and database backward from production to stage or local, make config updates, sync back up to production. This still will not work because any new custom blocks created in this interim period on production will now be deleted on import.

To test a work-around, I tried to first create a custom block in local, immediately create the same custom block in stage (so they have the same machine name). However, when you run config-import, the config sync takes priority of the machine name. My newly created stage block is re-assigned a new machine name.
When I sync the config from local to stage I now have two configs for different blocks in stage, and the first one is a broken block placed into the site.

Here is a specific example:

Create and place a footercopyright custom block in both local and stage by copy-pasting the title and body.
Import the config into stage, and the footer block is replaced with a broken block.
Manually fix the block config by deleting the broken block created by config and re-place the originally created custom block.

Now, the config export from stage:
block.block.footercopyright_2 create
block.block.footercopyright delete

or re-import the original config:
block.block.footercopyright create
block.block.footercopyright_2 delete

Using another module, like simple_block, doesn't work for my current needs, and ideally core components such as these should not conflict.

In short, I think this should be upgraded to critical, because until it's fixed, it doesn't seem like we can use both config synchronization and custom blocks at the same time. If you have both of those things enabled on a site, then only a highly experienced Drupal developer should be implementing custom blocks, at which point it's probably best to write custom blocks in custom modules.

@bdanin - While it is a major issue, and highly annoying, it doesn't meet the criteria for being 'Critical' (e.g. it's not bringing down people's production sites right now), especially since there are many ways to work around the problem for now (though none of them are very fun).

Well, short of going to critical, there now is a wish list for Drupal 8.4:
https://www.drupal.org/node/2858592

If this matters to you, you may want to chime in there.

If this matters to you, scream there. I did.

As I said in comment #3 this is 'works as designed'

When I was designing the block content to block module integration, my major bug bear was the way D7 tied it to the serial entity ID, e.g. the block equivalent of NID. I fought hard to retain IDs driven by UUID, knowing that you can deploy/create an entity with a UUID.

Deploying content between environments is not a core issue.

We got the 'broken' handler in so that your site doesn't blow up, there are several ways to deploy content in contrib and failing those, you can use an update hook like so (copied from an actual D8 site, works for *any* entity type:

/**
 * Creates the required block for the Footer contact details.
 */
function my_install_profile_update_8001() {
  $default_content = [
    // Keyed by entity type.
    'block_content' => [
        // Then by UUID.
        'b990e653-c29a-48d8-b90b-70e2676f6c6e' => [
          'info' => 'Contact us',
          'type' => 'contact_block',
          'field_something' => 'put your field values here',
        ],
      ],
    ];
  my_install_profile_create_content($default_content);
}

/**
 * Creates install content.
 *
 * @param array $content
 *   Content keyed by entity-type and UUID.
 */
function my_install_profile_create_content(array $content) {
  foreach ($content as $entity_type_id => $items) {
    $storage = \Drupal::entityTypeManager()->getStorage($entity_type_id);
    foreach ($items as $uuid => $item) {
      $entity = $storage->create($item + ['uuid' => $uuid]);
      $entity->save();
    }
  }
}

Finally

they are taking the wish list for Drupal 8.4 right now

- comments like this make me sad. There is no us vs them, there is no 'core devs' vs 'users', we are all drupal developers. By commenting here, you are taking part in the process and I resent the implication of there being a they, it implies there is a special class of users, there is not.

I note there is also a ConfigImportEvents::MISSING_CONTENT event fired, which you could also react to instead of using an update hook, work is afoot in default_content module to support that.

But again, the presence of this event in the config API support my assertion that this is works as designed, and up to contrib to craft solutions for it.

I agree with @larowlan in #3 and #26.
Drupal core already gives all you/contribs need to deal with the problem, and ConfigImportEvents::MISSING_CONTENT is a perfect example (of many, i.e. update hooks).

Besides I don't understand the wonder. D7 (+features) works exactly in the same way and you need other contrib modules (i.e. FE Block, uuid) or update_hooks to create your missing blocks if placed, for instance, in panels or contexts.

It's clear there is a missing piece of the puzzle, but that shouldn't be found in core. And may not be code too.
In fact this thread has a lot of useful suggestions which require only a documentation page. :)

+1 for closing this issue.

My issue is not that a custom block cannot be exported/imported, but that at present the resulting target system shows a broken block after import.

Now, if everybody can program hooks, and we want to write a new update_N hook for every custom block (do we?), we can fix that in code. However, as we are all aware, not all users can do that; for them the out-of-the-box solution impairs usability.

So if we define a custom block as being content (fine), as #3 and other argue, AND we want to keep custom content out of the configs (fine), then the question is: why is the placement info for that block exported at all? Why export ANY info about something that is deemed to be content?

If you really want to be consistent here as far as semantics, at least to me the logical conclusion would be to not write that info into the config at all. I'm happy with that.

But since core does include that config at present, one could argue as well for the other solution: you could say that the content is only what is really inside of the block; and the existence of the block is structural info that is part of the greater system. In that case, the solution would be, as presented above, to create a stub block.

Now, we can belabor this point forever, where do you draw the line between content and structure. I just want to be pragmatic, and get this going.

For that reason, I'd vote for just dropping the config -- it seems cleaner -- and push everything else dealing with content into the contrib space.

I agree that new custom blocks should not be exported into config by default.

For now, I'm using a custom (in code) block similar to #8, along with config_pages, and then rendering fields with config_pages_config()
see specifics on rendering at stackexchange

When I export all my new config, fields in config_pages are created as empty, and my new block from the module code is enabled and placed in the right place. Then content authors can go into the config_pages UI and add content as expected.

Anyway, this method gives me exactly what I need, but it's not very straight forward nor easy for a novice. However, I'm not sure there is a simple core-based solution other than simply removing custom blocks from config-export, which supports the originally suggested solution. I now agree this is best:

Suggested solution

Don't export custom blocks through Configuration management

I'm grappling with this problem in my workflow as well. I'm very unsatisfied with the "works as designed" response as well as all the suggestions to write custom code to handle block creation or to omit blocks from the config export. Very simply, why can't empty block content be created when a new block's configuration is imported? I have not yet dug deep enough down into the import mechanism to know how to do this, but it seem obvious that this is far more desirable than either creating broken and unchangeable blocks or omitting blocks entirely and therefore omitting a very significant portion of a site's configuration. This isn't even any different from the behavior for existing blocks where content changes are not copied.

I'm sure the "works as designed" responses never intended to imply that creating broken blocks was an intentional design choice. I'll probably dig into this fix for my site, but if anyone wants to supply some advice to save me some time, I'd happily accept it and offer the patches back to community. Naturally I'm about to go on vacation so this will have to sit for a while, but perhaps we could mull over the implications of this change?

Why can't empty block content be created when a new block's configuration is imported

This actually seems like a more reasonable solution to the problem, while keeping things the way they are.

The one difficulty with that approach is that there are myriad possibilities for fields, required fields, etc., and whatever system would create an empty block would need to account for that (akin to Devel Generate).

That would then create the problem of people not knowing why a block is empty on prod when they had content in it locally, though :/

Adding this just for reference https://www.drupal.org/project/recreate_block_content. Haven't tried it by myself yet.

Thanks @kpv, that's an interesting find.

It's a very simple module, it uses

          $block_content = BlockContent::create([
            'type' => $type,
            'info' => $block->id(),
            'uuid' => $uuid,
          ]);

so it doesn't attempt to account for field content in any ambitious way.

My dream for something more ambitious would be to serialize the content blocks on export, and then on import creating a new block from the serialized block, if a block with that uuid is not already present.

Even just exporting a label or other hint of what that block's content should be would be helpful.

Thanks for sharing @kpv, works perfectly

Thanks @kpv, module suits my use case.

Based on comments above, and my expectation, +1 to close

I agree that we should not export content at all, contrib modules (like recreate_block_content) should deal with it. +1 to close the issue.

Let's not close the issue but instead repurpose the issue into: document patterns for exporting custom blocks
To capture some of the strategies discussed in these comments.
It's clearly an important and difficult issue, even if the solution is not a core code patch.

I agree that it should be handled as a contrib module. recreate_block_content looks promising. The only reason I hesitate to use this module is that it is not covered by the security advisory policy. That module does have a stable release and as far as I can tell it does satisfy the criteria to be covered. I have entered an issue on that queue to check why it is not covered yet. https://www.drupal.org/node/2888090

I'd like to bump that this is something that should be considered for core. Core provides the engine for exporting configuration and this is just that. This is a major hinderance in moving to Drupal 8. In Drupal 7 you could work around a lot these issues by using panels, mini panels, and features, but we don't have mini panels anymore and panels is severely lacking functionality. Blocks are now used in far more contexts than they were before, so I think this is an issue that needs to be solved by community and belongs in core. If Drupal wants to continue to be competitive in the CMS market it needs to address this. My team is spinning right now because we can't commit and deploy code from one site to another because of this. Saying that people need to write updates hooks or any code at all is poor solution.

I wasn't going to say anything because I wasn't sure if I was just ignorant, but I completely agree with pwaterz in #40. If core is exporting block placement, then it makes no sense in my opinion that a contrib module would handle exporting block content.

Trying to make the case to potential clients that we can build out sites simpler with d8, and then running into the same 'create blocks in two places or we have to program that for you' -issue that we had in d7, just adds to the vacuum that is sending people to wordpress.

I am sympathetic to both sides but generally fall into the camp that this needs to be "solved" in contrib (via code and best practices) before a solution is ever proscribed by core. As mentioned in #25 we're all contributors and recent history suggests core contributions have a much better chance of making it in if they have proven effective in contrib. Also so far I haven't seen any patches :-)

There are two very general use cases I've seen articulated:

As a site builder or developer, I'd like to create a custom block in a non-prod environment and export/deploy its placement and content (and maybe its custom fields, etc) using a config/code deployment workflow.

Some contrib modules that might help with that:
https://www.drupal.org/project/default_content
https://www.drupal.org/project/yaml_content
to help create content from code (not to mention the good ol' update hook approach, see #25).

and

As a site builder or content administrator, I'd like to be able to create and place content blocks in production and not have their placement overridden the next time site-wide config is imported.

Contrib modules that might help with that.
https://www.drupal.org/project/config_ignore
https://www.drupal.org/project/config_filter

The fact of the matter is I've seen these dual, sort of conflicting cases on the exact same project so I don't want to minimize the complexity of the problem or the pain it can cause. None of the approaches I've seen in contrib are great. They don't, for instance, solve the problem I came here for, a variation of use case #2, where'd I like to "config_ignore" just custom content blocks. However, I haven't seen any evidence that core precludes some reasonable approaches in contrib or custom code.

This is quite annoying behavior, but maybe this can be solved by adding link to recreate block entity with the same uuid? This should be quite easy to add.

I've found a good solution to this using paragraphs and block_field (note, you currently need this patch: https://www.drupal.org/node/2865189#comment-12109649)

You can create custom blocks, and then place them on the page through a paragraph (entity reference revisions) block-reference, and it never interferes with the config-sync system.

This appears to work very well, and allows all these components to be useful out of the box. Ideally block_field would have a stable release soon, which would probably help this feel more "plug and play", but it still works with the dev version plus the patch.

Lately I often meet this problem. And it seems to me that it's crudity drupal core. Because if I use fresh drupal installation without any contrib and I make export/import on fresh system using only core functionality -> I get this problem.

Ok, now I have few lines custom code for my use cases, clearing missing block.block.THEMENAME_.. rows inside 'config' table mysql (not production ready). But I think it's a bad way.

This is quite annoying behavior, but maybe this can be solved by adding link to recreate block entity with the same uuid? This should be quite easy to add.

Changing: " This block is broken or missing. You may be missing content or you might need to enable the original module."

To "The content of this block is missing (custom block). Recreate a new content".

Would not fix this issue, but will help drupal user a lot! And I agree this is a critical issue for user even it's trivial for hard core dev.

Re #48 - this is a good idea - but - we can't know in advance the block type

So we'd need a form to allow the user to select the block type (if there were multiple) and then submitting that creates an empty block with the expected uuid and selected type, then redirects to the edit form.

So not insurmountable - but a little nuanced. Please open an issue for that, as it is an actionable task.

Their #2906919: Allow users to create a block content entity with a specific UUID if it is a missing dependency of a config object

@here - there is a patch over on #2906919: Allow users to create a block content entity with a specific UUID if it is a missing dependency of a config object would be great to get some testers and reviews

Re #48 - this is a good idea - but - we can't know in advance the block type

It is possible to get the block type from the configuration, I'm doing that on recreate_block_content
The link option is ok too, but is there a reason to not generate the block content as soon as the configuration is imported? I see no reason to not just create the block with empty content.

The block could have required fields, such that it cannot validly be generated empty.

Also in answer to "is there a reason to not generate the block content as soon as the configuration is imported"? that would interfere with a workflow in which the block content is created with default content or some other way after the configuration has been brought in. Providing a link rather than a broken block message is a good compromise.

No takers to review #2906919: Allow users to create a block content entity with a specific UUID if it is a missing dependency of a config object?

Also in answer to "is there a reason to not generate the block content as soon as the configuration is imported"? that would interfere with a workflow in which the block content is created with default content or some other way after the configuration has been brought in. Providing a link rather than a broken block message is a good compromise.

Good point, indeed a link is the way to go.

From a custom module, developed for a website with the same scenario described here, I have created one that I have just added as a contributed module at:

https://www.drupal.org/project/fixed_block_content

The module provides a type of block based on configuration (the fixed block) that wraps a custom block (the content block). If the content block does not exist, it is created with a default content or empty if no default content was set.

Everything is done through UI, no special files or custom module are needed.

There is a lot of discussion in here that seems to resolve about different issues. The real problem in this issue is that exporting custom blocks leaves one with a broken installation since you can see a block but it is unusable (only solvable with contrib modules or custom hooks).
The best practices that content export and configuration export should be separate is completely valid and does not stand in the way of resolving this issue. The solution proposed in #2906919: Allow users to create a block content entity with a specific UUID if it is a missing dependency of a config object is the right way to go. It solves the broken installation since the user (usually content creators) can use the block again by adding it and also prevents any content being exported/imported when exporting/importing configuration. People still wanting to get their custom's blocks content from one environment to another will have to follow the usual way of exporting/importing content or use one of the earlier mentioned contrib modules.

I updated recreate_block_content to work with any module that uses blocks now.

@manuel.adan I don't know how your module works but mine didn't work with blocks from panels for example, you can check what I'm using on my module if you want, feel free to copy/paste if you want.

@joao-sausen fixed_block_content does it by re-creating the missing custom block "on demand", when the build method of the fixed block is called, so it works in any scenario. Either way, all missed custom blocks could be restored walking the existing fixed blocks.

The goal is to break the dependency on the content that comes from creating instances of a custom block. With the fixed block, we are creating instances of a config-based block that is always present, then the fixed block creates the custom block when needed with a default content also stored (serialized) in config.

A possible workaround solution until this gets resolved, the Structure_Sync module lets you export blocks w/associated content to config. It also allows import/export of Menu and Taxonomy structure+content to config. It's a little tricky in that once the changes need to be imported/exported into the database separately from the importing/exporting of the config.

In case a lot of content has to be imported I'd use the default_content module, but when only one or two things have to be imported using the entity API like proposed in #2756331-25: Custom blocks cannot be properly exported and imported looks OK and avoids an external dependency.

In my case I had to import only one block to add as "Additional info" to the contact page, but I had to do it in multiple languages, so I modified the code in #2756331-25: Custom blocks cannot be properly exported and imported:

function my_install_profile_install() {

  ...

  // Create some default content.
  $default_content = [
    // Keyed by entity type.
    'block_content' => [
      // Then by UUID.
      '5b42bdd0-4399-4a55-a67b-e3309d4a4715' => [
        'original' => [
          'type' => 'basic',
          'info' => 'Contact info',
          'body' => [
            'value' => 'Additional info.',
          ],
        ],
        'translations' => [
          'it' => [
            'info' => 'Info contatti',
            'body' => [
              'value' => 'Ulteriori informazioni.',
            ],
          ],
        ],
      ],
    ],
  ];
  my_installation_profile_create_content($default_content);
}

/**
 * Creates install content.
 *
 * @param array $content
 *   Content keyed by entity-type and UUID.
 */
function my_install_profile_create_content(array $content) {
  foreach ($content as $entity_type_id => $items) {
    $storage = \Drupal::entityTypeManager()->getStorage($entity_type_id);
    foreach ($items as $uuid => $item) {
      $original = $item['original'];
      $entity = $storage->create($original + ['uuid' => $uuid]);
      $entity->save();
      $translations = $item['translations'];
      foreach ($translations as $langcode => $content) {
        $translation = $entity->addTranslation($langcode, $content);
        $translation->save();
      }
    }
  }
}

As I see the question/problem is still open :-(
Here is my contribution. I created another simple module, which stores blocks definition in variables (D7) or configuration (D8). So you can create block on DEV or local, place it in some region then deploy to STAGE/PROD. Content can be changed on other environments without conflicts.
Link to module: https://www.drupal.org/project/conf_block

I have another workaround

I would do import as usual, ignore the error message during / after importing blocks.

After that I used content_sync module to export / import all contents (including block) from other site and it brought block content.

At least this worked for me

I was really looking forward to configuration management in Drupal 8, but this behavior kind of ruined it for me. What a bummer that this doesn't just work out of the box. How about an option to export custom block "content" along with the rest of configuration, and then on import make sure these bits are created before they are added to the block layout. This should happen in core and basically just give people an option about whether or not they'd like their config semantically "clean" and not contain content so they can use one of the other contrib modules or their own hooks if that's their thing.

Same feeling, Suzanne. The block placement exports just fine, but not the custom block config. Hm.

I see we are still showing error messages to anonymous non-admin traffic: "This block is broken or missing. You may be missing content or you might need to enable the original module." At the very least we should be able to agree that anonymous users shouldn't see error messages. Right?

edit: And to chime in on the other comments, I can understand how we wouldn't want block content overwritten on config import, but in my case my block was just a view, not really even content, and even my view-block didn't get transferred through config export/import. And of course when I created the block on prod and assigned it to it's correct regions, next config push it once again was looking for the old block ID numbers and was showing "This block is broken or missing" to anonymous users. I guess I will grab the prod database and then recreate the blocks locally so their ID's will match. I can understand why this issue isn't cut and dried, but lets at least limit errors to admin users, right?

Sorry, have to vent, this functionality is a bit of a disaster. Migrated my prod database to devel. Create blocks, and assign blocks on devel. Create the same blocks on prod. Verify they have both the same machine name and block ID #. Export config, import to prod. "This block is missing" shown to anonymous users. Seriously? Drupal can't match config even if the block ID's match and machine names match? Ridiculous.

Just to second what diamondsea said in #61 - structure_sync is a very dependable workaround for those who are struggling with this, if you can adopt it as part of your config export/import routine.

#72, the UUID would be different in that case.

We're using Recreate Block Content which is working very well. It (re)creates the block after you've cleared the cache. You then recreate the content yourself but the block config doesn't get overwritten or generate errors in subsequent imports.

There's some fun history here: #2181631: Use custom block id instead of uuid for plugin derivative id

Thanks IMclean, that clears things up, really appreciate it.

edit: But why oh why are we still showing errors to anonymous users?

@imclean, that seems like a pretty terrible problem where you have to recreate the content every single time. That isn't a workaround nor a fix.

@pthurmond it doesn't recreate content it every time. It only creates empty content for blocks which are missing content, so it happens once per import of new block(s). This matches the blocks on dev and prod environments so future imports don't cause any errors or overwrite existing blocks.

So I just read through this entire thread. Found really no real answers. I am going to do my best to recap and give my opinions.

1. I agree that config import/export is NOT the right place to store content.
2. I agree that this issue poses problems for a dev, stag, prod workflow (this conundrum already exists though).
3. I agree that we shouldn't have to rely on a separate module to do this in core (multiple are listed here).

So this problem has existed forever in Drupal. How do I get content from dev / staging to production...? Unfortunately, I still have yet to find a consistent elegant way of doing this. What I did in Drupal 7 was create content on production and move it back. This is the exact opposite of what we should do but, seemed to be the quickest (in most cases) however often what your developing needs to be used in the content so... it ends up creating more work.

I vote for a content:import / content:export functionality. In core... to close this gap. Hopefully, this will save someone else the time from reading through this entire thread. I know things like https://www.drupal.org/project/content_sync exist. But I feel this functionality should be in core. Drupal is one of the best CMS/CMF's on the planet and yet we STILL can't do the first 2 words very well Content Management. We are a great CONTENT STORAGE SYSTEM or SCHEMA MANAGEMENT SYSTEM.

@dobe

What I did in Drupal 7 was create content on production and move it back.

I think that's the defacto standard on Drupal 8 too, code goes dev -> stag -> prod (through version control) but content on prod gets migrated the opposite direction to keep the dev and staging environments as close to prod as the real thing. I do this by migrating the entire database from prod to dev and staging about every month or two. Is everyone else doing this more or less the same way?

@plato1123, that's our general process too. We copy the database from prod/live as needed, especially if the client has done a lot of content work and are asking for some major changes.

The issue of blocks as both config and content we've resolved using the module I mentioned in #74.

Another approach to this could be to view blocks as equivalent to content types. Importing config imports the block type but doesn't populate the block with content and doesn't place it in a region. The placement of the custom block could then be considered content instead of config. The same as positioning text or images in a WYSIWYG editor. But I guess it would depend upon what the custom blocks are being used for.

For a simple use case, where I needed to have a custom block populated with default content after drush config-import, YAML Content seems to do the job. Thanks for making me aware of it @krlucas.

You can also be truly horrible and edit the uuid column of your block_content table and change the uuid on production so it matches the one in config (and then clear cache), which is what I ended up doing as I just have config to use on this particular site, and can't add extra modules just for one user editable block.

Would be lovely if the config import pulled in a completely empty block to populate, then the end user could edit it and add their content.

And please hide the broken handler error message from anonymous users.

My solution was to treat custom blocks really like content and use Config Ignore. At least the fields are sent over in config, then I add the content, place the block, and add block.block.MYCUSTOMBLOCK to the ignore page.

In my opinion, without diving into the question of its even possible, is to have a simply check box on each custom block placement that asks if the content should be added to config.

FYI, this issue contains a core patch that fixes publicly displayed "This block is broken or missing.." message: https://www.drupal.org/project/drupal/issues/2918149

I encountered this myself, using modules or patches did not work for me.
Doing the Database fix route, thinking this has something to do with UUIDs or IDs. Everything seems fine.
What worked for me to fix this is to modify the block_content_field_data table and made sure all the existing blocks for the field reusable is not NULL

Then do a cache clear for the site and you'll see the missing custom blocks. So far this workaround worked for me.

Oh and i'm at 8.6.13 at this time.

Thanks to Joao Sausen (#58) and his helpful module recreate_block_content.

My context was as follows.

1 - In my dev site
- A custom block created from a custom type + one instance of it placed into the content region of my bootstrap subtheme.
- Another custom block created from another custom type + two instances of it, one placed into the top region, the second one placed into the footer region.
- drush config-export, git add, git commit, git push.

2 - In my staging site
- git pull, drush config-import, drush cr.

Result -> the frustrating error message This block is broken or missing. You may be missing content or you might need to enable the original module displayed 3 times on the front page, one in content, one in top, one in footer region.

I didn't want to export the block contents into my versionning workflow. I didn't want either to use contrib modules like structure_sync or config_ignore (already used in other projects). So after reading this whole thread, I decided to test the recreate_block_content module.

1 - In my dev site
- composer require drupal/recreate_block_content
- drush en recreate_block_content
- drush config-export, git add, git commit, git push

2 - In my staging site
- composer install
- drush config-import, drush cr

Result -> The 3 error messages disappeared and 2 custom blocks were listed again in the custom blocks admin page (admin/structure/block/block-content).

Then :
- I edited the first one, re-added content similar to what I did in my dev site, then saved it -> Bingo ! Block is there at the right place (content region) in front page.
- I edited the second one, re-added content and saved it -> Superbingo ! The top and the bottom blocks are there !

What else ?
Have a fresh beer, maybe.

3 cheers for recreate_block_content module (#58) !! What a simple and elegant solution. https://www.drupal.org/project/recreate_block_content

Glad to see #2756331: Custom blocks cannot be properly exported and imported. Also, if Recreate Block Content is a good solution, it should be part of core. Sorry, but this is a core issue, especially with new tools like "layout builder" that are encouraging using blocks.

Can anyone speaks to what would be a non-yet-another-module solution to this issue? Will deleting the block from each environment then recreating each before config sync will bring a matching config for that block?

Edit: As stated by Plato1123 on #73, this approach is a big waste of time don't try it.

If you want a front end hack for this issue try printing the block onto your twig template like so:

<div class="YOUR CLASSES HERE" {{ attributes.setAttribute('data-quickedit-entity-id', 'block_content/BLOCK_ID') }}>
                    {{ drupal_entity('block_content', 'BLOCK_ID') }}
                  </div>

The attributes.setAttributes part if necessary to avoid the "Uncaught Error: Quick Edit could not associate the rendered entity field markup" error.

You will lose the ability to quick edit the block but gain control of your layout. It seems that this issue affects custom blocks created with the block layout feature and with Nodeblock gone and Bean swallowed by core roll-your-own block entity might be the way to go.

Greetings,
Nobody talk about the following aspect of this issue:
It reduce greatly the potential of theme creation, because only the native block can be placed at theme installation.
If a themer will be able to create/place custom at theme installation, we could have out of the box very sophisticated themes.
Thus, in the end, this problem reduce the popularity of drupal in general - and disappoint developer...

Any update on this after 6 months?

+1: Put the functionality of recreate_block_content into core!

Maybe there should be an extra notice added to each custom block which was recreated, saying that this block has been recreated programmatically, but fields have to be manually re-filled with content.

I think I got a similar issue with Layout Builder :

• Enable Layout builder on a content type
• Add some custom blocks in the layout
• Make the layout customizable for each entity
• Export your config
• Import it in another environment

Since the content is not part of the config for that block, it creates a list of issues, either while accessing the page (Lightning Layout assuming that this block has content) or by trying to configure it (Core expecting the content to exists).

Allowing people to recreate the block would at least make it easier to both figure out what happens, then fix it in an easier way than deleting part of their config in YAML files.

Notes that the module default_content allow the creation of content at distribution or that module installation by using HAL/Json core module, a GUI is available.

But as i suggest in #91 we should have that functionality at theme installation thus without non-core module.

Thanks to jabberwooki (#86). This workaround worked for me.

Okay wonderful drupal peeps, so having read 4years of comments on this issue, and seen the wonderful contrib modules, I'm lost.

What is the correct way to synchronise content between environments (custom blocks and layout designer content). My use case is a basic theme with layoutdesigner content and few custom bits, that will be rarely updated. I'd hoped to version control the content, but as the design evolved erratically I had originally accepted only the theme in VCS and now wish to do the content as the design has settled. We have a test and production environment as well as local dev setups.

IMHO there should be a workflow that doesn't involve manually recreating each blocks content when using multiple environments like IT professionals do.
The closest I've come is to create a custom theme module with default content, or programmatically created nodes.

Drupal is starting to feel not like the editor friendly content management system I was sold on, as I can't backup and transfer my content, (dont get me started on this node is locked/in use - courtesy of the back button or multiple tabs/users) ignoring the wonderful community who do painstakingly fix everything, and the pretty damn good documentation, it's still feels like an unguided ship with contrib left to pickup the pieces.

Is there any guidance on drupal the right way? Lullabot seems to be the goto apart from the official docs (summer updates are great BTW) or billions of issue pages.

Don't worry though I'm not losing faith, happy to contribute any way possible, and I'd love to see more testing/TDD in contrib modules fwiw, so thanks for reading this as I'm mostly just thinking aloud :sigh:

Tyeth, default_content is the way to go but you will not be able to update content this way (you can with custom code).
There is no correct way to synchronize data between environments.
On Drupal 7 I used this https://www.drupal.org/project/deploy

+1: Put the functionality of recreate_block_content into core.

Configuration sync with common site tasks and out of the box modules shouldn't break production sites. This is really a minimum requirement.

A better solution IMHO would be to extend the Theme system to allow custom blocks to be created directly inside a theme by simply adding block content as a twig template, and using a simple yaml file (blocks.yml) to reference them with all the expected block configuration.

This way Drupal themers would be able to control the design and layout of the site, take advantage of version control, and deploy through config, all without needing to touch custom module code.

Since there aren't patches, the correct status is Active.

I am very surprised that this problem has not been solved in any way so far.
In the context that using drupal without a composer and importing/exporting configurations will soon become almost impossible

There is another module that syncs the configuration of the blocks. https://www.drupal.org/project/structure_sync

#57 (fixed_block_content) Seems to have resolved an issue we were having.

Drupal 9, Custom Block Type, Layout Builder used to display an Article with an instance of a custom block type block.

Didn't export / import as expected.

Of the suggestions listed above, I didn't try `structure_sync`, but this module seems to work well since it allows us to export the configuration, which is important since this custom block type has several fields.

Thanks for the feedback and wanted to give another recommendation to that module.

Looks like the Fixed Block Content module doesn't work anymore.

I agree with @hoporr @tamerzg and @morsok above: this issue needs to be fixed here in core.

however @hoporr @jonathanshaw & @geerlingguy — yes it does render a site unusable the second you accidentally delete all blocks on production because of a config sync. Or worse yet doing a structural sync without importing everything, it will delete everything else.

@bdanin — clearly discusses why this is an issue, an issue open for more than 8 years, is the reason why many still bring this up. It has never been resolved, the addition of modules that make up for cores inability to get it right after 8x, 9x, and 10x, do we expect this to be resolved in 11x after 9 or 10 years of asking the same question? please help me understand how this is helpful to the community at large.