system_requirements() calls install_profile_info() when $runtime. This means that on admin/config it does a file_scan_directory(), which can take around 78% of the page request.
Patch adds an optional third parameter to install_profile_info(), which skips searching for and loading dependency information when set to FALSE. This speeds up the initial request to install.php by 50% (according to firebug) along with the admin/config issue.
carlos8f mentioned using system_get_info() instead of install_profile_info() in system_requirements(), that might be a less intrusive option.
Marking this major because it's going to completely kill admin performance for upgraded D6 sites and any D7 site not using 'standard'.
Comment | File | Size | Author |
---|---|---|---|
#14 | 1006714-14-file-scans-are-bad.patch | 2.81 KB | David_Rothstein |
#13 | 1006714-9-file-scans-are-bad.patch | 2.64 KB | catch |
#9 | 1006714-9-file-scans-are-bad.patch | 2.45 KB | catch |
#6 | 1006714-6-file-scans-are-bad.patch | 1.67 KB | carlos8f |
#4 | 1006714-4-file-scans-are-bad.patch | 1.74 KB | carlos8f |
Comments
Comment #1
catchWithout xhprof cruft in .htaccess
Comment #2
bfroehle CreditAttribution: bfroehle commentedWell, leave this out, of course.
Edit: Nevermind... you beat me to it.
Comment #3
carlos8f CreditAttribution: carlos8f commentedThis one's a bit simpler :)
I don't think micro optimization during the installer is super important. Also, doesn't require an API change.
Comment #4
carlos8f CreditAttribution: carlos8f commentedLet's try this one then, here we also optimize system_modules().
Comment #5
catchDiscussed this with Carlos in irc, I'm happy leaving the installer alone and just optimizing these two places given where we are in the cycle. Marking RTBC, please wait for the test bot prior to commit, however this works just clicking around.
The system_modules() fix removes a full second from that page in webgrind.
Comment #6
carlos8f CreditAttribution: carlos8f commentedThis patch gets more to the source of the problem: drupal_required_modules() does a full .info scan, which trickles down to any function that uses install_profile_info(). This patch is more API friendly and less hacky.
Comment #7
catchThis will also fix the issue in contrib, mollom modules calls drupal_profile_info() from _mollom_get_version() which also triggers this full system scan, just to get the distribution name.
webchick wanted a third reviewer here since Carlos and I have been working on this together in irc, so hopefully one will appear.
Comment #9
catchSo the reason this fails is because module.test actually uses the $info['dependencies'] bit of install_profile_info() during runtime - where this would now be taken from enabled modules rather than the full list from file_scan_directory(). While it's likely to only ever occur in this test, that's a sign that we shouldn't change behaviour here during the same week that 7.0 gets released.
So... I'm re-uploading #4 for the bot, let's open a new issue to clean this up more drastically for Drupal 8, but until then we have a four line patch that removes 78% of the time spent building admin/config.
Only change in this patch is I added a code comment to install_profile_info() documenting the performance issue.
Comment #10
catchForgot the status change.
Comment #11
catchD8 issue #1029390: No API function for getting install profile info without a full file directory scan.
Comment #12
David_Rothstein CreditAttribution: David_Rothstein commentedThe test failure in @carlos8f's patch looks like it's actually just due to a simple bug in the patch, which is easy to fix. The previous code guaranteed that drupal_required_modules() always returned the install profile as its first array element, but the new code doesn't. So when install_profile_info() calls array_shift() on it, it winds up removing the wrong element from the array. We could definitely fix the code in the patch so as to keep that assumption valid.
However, I guess there is still one API-affecting thing in the patch; by getting the info file dependencies from the database rather than always scanning the filesystem, it means that if you use hook_system_info_alter() to affect the 'required' status, that will now start affecting the return value of drupal_required_modules() too, but only post-installation. That's not necessarily wrong (in fact, it might really be a bugfix on its own), but it is a bit of a change.
So overall I think @carlos8f's approach is preferable, but if we're worried about that for D7 then the patch in #9 is good too. However, with this code:
I think it's really ugly to have the calling code responsible for that. Could we add some code to _system_rebuild_module_data() to ensure that the default
'distribution_name' = 'Drupal'
winds up stored in the {system} table for profiles that don't specify their own? (OK, one could argue that's an API change too, but I think the argument would be pushing it.) There's already lots of special-case code in _system_rebuild_module_data() to handle the install profile's .info file, so adding one more there seems much better than having the calling code do it.Comment #13
catchHmm, I'm split between the two approaches, for now I re-rolled #9 with the suggested change from #12, that is indeed a lot cleaner.
Comment #14
David_Rothstein CreditAttribution: David_Rothstein commentedLet's go with this one for now, since it's safest. We could always do the other approach in #1029390: No API function for getting install profile info without a full file directory scan and possibly backport it.
The new code comments in #13 had some typos, so I rerolled to fix those.
I also realized that in the case of drupal_install_profile_distribution_name(), it already has separate code paths for the installer and everything else, so why not just fix the performance issue internally to that function and not have to change the calling code? That seemed better to me, so I made that change in the attached patch also.
This looked fine to me otherwise and testing it out with custom distribution names everything seemed to work fine. So feel free to RTBC if the new changes look good to you.
Comment #15
catchThis version looks great. I agree it'd be good to get #1029390: No API function for getting install profile info without a full file directory scan fixed and backported as well.
Also a note I keep seeing people popping up in #drupal/the forums complaining of extremely slow loading times for admin pages with D7. I haven't been able to actually get enough data out of them to figure out if this issue is the actual problem, there's certainly other possible candidates, but I'd really like to see this one get in as soon as possible. We're talking about a full second or more of admin page load times.
Comment #16
andyposthttp://drupal.org/cvs?commit=502220
Comment #17
mfbBTW, found a related issue, #1081266: Avoid re-scanning module directory when a filename or a module is missing: file_scan_directory() is called for each missing module or install profile, this was continuing to slow down admin/config for me.
Comment #19
q0rban CreditAttribution: q0rban commentedThis has introduced PHP Notices, as there is data being added to install_profile_info() that doesn't get added in system_get_info.
Comment #20
catchSee #1170362: Install profile is disabled for lots of different reasons and core doesn't allow for that.
Comment #21
q0rban CreditAttribution: q0rban commentedHmm, the patch itself looks like it accounts for this though. I'm going to put back to fixed, as this was actually something rfay ran into, so I can't replicate it.