[meeting] Migrate Meeting 2020-09-24 [#3171562]

Hello all, it’s time for the weekly migration initiative meeting. The meeting will take place in slack in various threads
This meeting:
➤ Is for core migrate maintainers and developers and anybody else in the community with an interest in migrations
➤ Usually happens every Thursday and alternates between 1400 and 2100 UTC.
➤ Is done on the #migration channel in Drupal Slack (see www.drupal.org/slack for information).
➤ Happens in threads, which you can follow to be notified of new replies even if you don’t comment in the thread. You may also join the meeting later and participate asynchronously!
➤ Has a public agenda anyone can add to here: https://www.drupal.org/project/drupal/issues/3171562. See the parent issue for an idea of the typical agenda.
➤*Transcript will be exported and posted* to the agenda issue. For anonymous comments, start with a :bust_in_silhouette: emoji. To take a comment or thread off the record, start with a :no_entry_sign: emoji.

Core migration issues:

0️⃣ Who is here today? Have you noticed that the days are getting shorter?

damienmckenna	I'm barely able to get time to go for an evening walk after dinner, it's almost fully dark by 7.
benjifisher	Pretty much the same here. Sometimes I walk in the dark, sometimes I go out earlier.At least we are not so absorbed in our screens that we do not notice.
mikelutz (he/him)	Sadly yes, they are getting shorter.
quietone	Not at all. More daylight hours here and kowhai in bloom.
dinarcon	Hi everyone :wave: no noticeable change in the length of days close to the equator, but timezones continue to be quite confusing :sweat_smile:
benjifisher	My loss is @quietone's gain, at least when it comes to daylight. https://en.wikipedia.org/wiki/K%C5%8Dwhai
dinarcon	Is this worth its own topic? The DrupalSqlBase plugin reads a constants configuration. When set, it looks for the keys entity_type and module. This might collide with "source constants" if the plugin used extends DrupalSqlBase and the user happens to set constants with those names. Probably an edge case, but it could happen. The behavior is mentioned in the base plugin definition so documentation already exists. (edited)

1️⃣ What should we talk about today? Suggest topics here and I will add threads.

damienmckenna	@quietone and I have some README files for Commerce Migrate, are there other things people might like to see documented in these files?
damienmckenna	Should core require that menus be migrated before blocks? I say "yes" :wink: #3171980: Menu migration should happen before block migration
quietone	@damienmckenna I need to add that there is a script to copy the files the config/install to all the READMEs. I will look at those before the end of the weekend.
damienmckenna	Thanks!
quietone	#2713327: Provide a way to remove migration tables (ID map etc.) Now handles migrat tables that no longer have a migration.
dinarcon	Some time ago I mentioned that it would be nice to have a glossary of terms. I wonder if there is any recommendations from the maintainers. I can create a documentation page to get ball rolling.

2️⃣ Action items. To be added later.

quietone	Review and/or commit Commerce Migrate recent documentation patches.
dianarcon	Add glossary to https://www.drupal.org/docs/8/api/migrate-api/migrate-api-overview.

3️⃣ Statistics

benjifisher	Issues Fixed since last week's meeting: 4 (not counting meeting issues)
benjifisher	RTBC: 22, two of which have been waiting more than 3 weeks. Of the 22, 2 are Major.
benjifisher	NR: 10, including one that has been waiting (for me) for more than 3 weeks. Other than that one, this list is in good shape.
benjifisher	The number of RTBC issues is exactly the same as last week. More or less, that means we are moving issues from NR to RTBC about as fast as the core committers are fixong them or sending them back to NW.

4️⃣ Documentation for Commerce Migrate

benjifisher	@damienmckenna is there an issue link?
damienmckenna	#3167817: Add README for Commerce v1
damienmckenna	#3167818: Add README for Ubercart
damienmckenna	#3167821: Add README for Magento
damienmckenna	#3167819: Add README for Shopify
damienmckenna	#3167820: Add README for Woocommerce
damienmckenna	The question is: What would be useful for an initial pass on the documentation for each one, and are there other things we should aim to include longer-term?
damienmckenna	@quietone and I have patches available for each one that gives a quick intro, along with some detail about the fixtures for Commerce and Ubercart.

5️⃣ Remaining multilingual migration paths

benjifisher	#2208401: [META] Remaining multilingual migration paths#comment-13828609
benjifisher	Catch moved this meta issue from NW to fixed, with this comment:Just realised #3008028: Migrate D7 i18n menu links and #3030939: D7 user profile translation migration are the last open issues here, both should haves, one RTBC - I think we can close this? Going to go ahead and do that (adding some triage credit in the process). Please re-open if I've missed something.
benjifisher	Maybe that issue got his attention when I bumped it by fixing a typo in the title. :slightly_smiling_face:
benjifisher	@quietone Are you happy with closing the meta issue? Last I checked, #3030939 was still Active. I think there are a couple of other child issues that are not mentioned in the issue summary.
quietone	I would have preferred to wait until the D7b i18n menu links was committed and to have the pleasure of closing it myself but am OK with it being closed.
benjifisher	I will defer to you on this.

6️⃣ Migrate UI - review help text

benjifisher	#2939328: Migrate UI - review help text
benjifisher	When I reviewed the RTBC issues, I noticed that the patch for this one is not getting re-tested every 2 days, even though the issue is RTBC. Any idea why that would be? I tested that the patch still applies.
lendude	Queued a retest for #24 that usually moves the 2-day retest to the right patch (it was testing #17)
benjifisher	Thanks.

7️⃣ Menu migration should happen before block migration

benjifisher	#3171980: Menu migration should happen before block migration
benjifisher	My initial impression is that this is a good idea. That reminds me ... (another thread coming up)
damienmckenna	The question is - it's such a minuscule change that I can't imagine having any regressions, so what might be needed for test coverage?
benjifisher	I think the general rule is that when you fix a bug, you should add a test so that future work does not undo your fix.

8️⃣ File migration's "owner" user reference should use migration_lookup

benjifisher	#3156733: File migration's "owner" user reference should use migration_lookup
benjifisher	This one is harder than the issue in 7️⃣ because there is a circular dependency: File entities have owners (user IDs), so the User migration should come first. But User entities have user pictures, so the File migration should come first.
benjifisher	I think my suggestion in Comment 7 should work: Maybe the simplest solution is to use default_value in the d7_file migration to set uid = 1 for all files. Then add d7_file_depends_on_d7_user that only has two fields in the process section: fid (use migration_lookup based on d7_file) and uid (use migration_lookup based on d7_user). If we skip any rows, then at least they will have a valid uid.We just need someone to write a patch that does that. @wimleers (he/him), are you interested?

9️⃣ Migrate Drupal 7 node/user reference fields

benjifisher	#2814953: Migrate Drupal 7 node/user reference fields
quietone	Working on it the reroll now
benjifisher	This issue was RTBC, but the patch does not apply. Maybe it is an easy reroll.
benjifisher	Oh, I was going to try that if the conversation here died down. :wink:

1️⃣0️⃣ Provide a way to remove migration tables (ID map etc.)

benjifisher	#2713327: Provide a way to remove migration tables (ID map etc.)
quietone	Needs some feedback on the new delete page now that it lists both migrations where tables exists and tables where no migrations exist.
benjifisher	One of the "Remaining tasks" is "Usability review". Should we add the corresponding issue tag? No, it was already reviewed. I will update the issue summary.
benjifisher	Sorry, x-post. Should we add back the issue tag?
quietone	Yes
benjifisher	Commented.

1️⃣1️⃣ Documentation: glossary page

1️⃣2️⃣ Wrap up

benjifisher	I would say that we could start by adding a section to https://www.drupal.org/docs/8/api/migrate-api/migrate-api-overview, right before "Furhter reading". (Maybe we should add links as "Related content" instead of having that section. We do believe in structured content, don't we?) If the Glossary section grows so large that it dominates the page, then we could split it off onto a separate page.
dinarcon	Sounds good. I will start there. A few things that could benefit from an official name...
dinarcon	Pseudofields. Not only the term might be misleading, but what is the definition. I see them as "temporary values/placeholders to be used in the process section/pipeline." Also, I define them as anything that is not an entity property (base field definition) or field (regular attached field?)
benjifisher	A quick search turns up two other glossary pages in the docs:https://www.drupal.org/docs/user_guide/en/glossary.htmlhttps://www.drupa... second one is so large that it has in-page navigation using the initial letter.If we want to be consistent with those, then the glossary should be at or near the end: (next to) last section on the page or last page in the guide.
dinarcon	"Code migrations" refering to migration files that lived under a /migrations folder. I think the proper term is a "migration plugin" but isn't a migration defined as configuration (via Migrate Plus) a "migration plugin" as well?
dinarcon	On that note, is there a better way to distinguish between the two approaches then referring to them as "code vs configuration migrations"
quietone	In my head one is migrations as plugins and the other is migrations as configuration
benjifisher	Pseudofields: I think all of my migrations use the Entity destination plugin (or ERR, which is derived from it). So fields are what get saved by the destination plugin. But there are also config migrations, and conceivably other destination plugin.Anyway, I think the important thing about a pseudofield is that it is something that will be ignored by the destination plugin. I think "not an entity property nor a field" only makes sense in the context of content migrations.
dinarcon	Also, sometimes I refer "source constants" are "variables" that can be used later. I guess "placeholders" would make more sense. I guess my confusion comes from var vs const from the JavaScript world.
dinarcon	@quietone that makes a lot of sense. And as people learn about the subsystems, things become self explanatory. Like a migration plugin requires (the plugin) cache to be rebuild while a migration configuration requires syncing/importing the configuration again.
dinarcon	@benjifisher yes, 99% of the time I deal with content migrations. hence the bias regarding not a property/field. Any suggestion for an alternative term?
benjifisher	I think "pseudofield" is already in use. As long as the glossary explains what it really means, I think it is OK.
benjifisher	Configuration migrations: unless I am really confused, you need to update the configuration and then clear the plugin cache. (edited)
dinarcon	As far as I know, updating the configuration is all that is needed. You can execute the migrations right away and they work. No need to clear the plugin cache. (edited)
benjifisher	1️⃣2️⃣ Wrap up
benjifisher	Thanks for participating! I will update 2️⃣ . Please continue to add comments in the threads. In 1-7 days, I will post a transcript for today's meeting.
dinarcon	Thanks for facilitating Benji!
quietone	Thanks Benji.
quietone	@benjifisher. Tomorrow, I will document the minutes.