Coming out of discussion at #295697: Increase PHP Memory Limit for Simpletest and probably required in order to do #198053: GHOP #67: Shows how much memory is in use versus your server configuration's maximum memory , processor-intensive modules such as SimpleTest and Views would be well-served if they could specify a minimum memory requirement before being enabled, similar to how they can specify a minimum PHP requirement.

If the system supports it, it should auto-bump the value so no manual work is required.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

aaron’s picture

Great idea. Until #198053: GHOP #67: Shows how much memory is in use versus your server configuration's maximum memory makes it, maintainers might also be able to guesstimate their memory requirements. For instance, if the base requirement is 16M, and on a basic installation + your module, you realize you need 32M, you could state 16M as your req. Then Drupal would add all those modules together (giving you a whopping 128M or whatever). Though if it starts getting to that, I agree with chx that we need to work collectively on seriously reducing the footprints of our modules.

maartenvg’s picture

Status: Active » Needs review

Ok, let's get this started.

Attached is a patch that adds the ability to add "memory_requirement = 32M" to an .info file. This value is compared with ini_get('memory_limit') and installation is prohibited if the current memory_limit is lower than the required memory_limit.
This is conform webchick's description: the 'memory_requirement' in the info file is the needed memory_limit of PHP. Bumping the memory_limit is not (yet) implemented.

I doubt it will be straightforward to sum the actual memory requirements of each module, per aaron's wishes. I believe the actual TOTAL memory requirement well be dependent off which modules are present, how much content there is and whether the modules perform their memory intensive labor at the same time. Because 2 entirely independent, but memory intensive, modules might require both 50M on top of the base requirements, because their independent this does not mean that the total should be 100M + base. Which is a huge difference and becomes more and more apparent if you have a lot of modules running.

Discussion time :)

webchick’s picture

Because 2 entirely independent, but memory intensive, modules might require both 50M on top of the base requirements, because their independent this does not mean that the total should be 100M + base.

Completely agreed.

SimpleTest needs XXMB of memory, but only on admin/build/testing. On every other page, it might need a few K at most, for whatever's in the .module file. This is why I wanted to separate it from the other issue, since I don't see us solving the "figure out Drupal's overall memory requirement" issue anytime soon, and this is a simple and straight-forward improvement for "heavier" modules.

webchick’s picture

Status: Needs review » Needs work

Oh, also? No patch. :(

maartenvg’s picture

Status: Needs work » Needs review
FileSize
2.31 KB

Good point :)

Dries’s picture

- I don't think we should sum the value, I'd simply max() them.

- Maybe system.info should specify the memory_requirement then?

Anonymous’s picture

Status: Needs review » Needs work

The last submitted patch failed testing.

maartenvg’s picture

Status: Needs work » Needs review
FileSize
1.87 KB

Chasing head.

Status: Needs review » Needs work

The last submitted patch failed testing.

lilou’s picture

Status: Needs work » Needs review
mr.baileys’s picture

Status: Needs review » Needs work
Issue tags: +memory limit, +PHP, +admin/build/modules, +modules, +.info, +module management, +ini
FileSize
1.88 KB

Chasing head.

Code needs work though: I set my PHP memory limit to "unlimited" as per instructions on memory limit, and this disabled all my modules with the message This module requires the PHP memory limit be set to 16M or greater. For more information on how to change the PHP memory limit see the online handbook.

memory_limit integer
This sets the maximum amount of memory in bytes that a script is allowed to allocate. This helps prevent poorly written scripts for eating up all available memory on a server. Note that to have no memory limit, set this directive to -1.
mr.baileys’s picture

Fixed coding style error.

mr.baileys’s picture

Status: Needs work » Needs review
FileSize
1.84 KB

New patch to take the "memory_limit = unlimited" situation into account. Back to CNR.

David_Rothstein’s picture

A quick review of this patch - it looks nice, but I see a couple potential problems:

1. I wondered what would happen if someone installs Drupal with an installation profile, and the installation profile contains a module in it that requires a larger PHP memory limit than they have.... So I tried it, and the answer is that the Drupal installation script will ignore the memory limit and install the module anyway, but then give you a big red X on the module administration page that prevents you from disabling it. Oops :)

2. I also wondered what would happen if you already have a module installed and then a new version is released which ups the memory requirement... again, same thing, after you update to the new version of the module, you get a big red X that warns you about the memory limit being too high, but since the module is already installed, the effect is that you can't disable the module!

Both these seem like more general bugs (not specific to this patch) that maybe should be fixed elsewhere, but it seems hard for this to go in without that... right now, the .info system just doesn't quite seem like it's designed to deal with things as variable as memory limits/requirements can be.

I do think the idea of this patch is really good, though. Earlier I proposed a different approach that would use hook_requirements() to do the memory checking instead.... That would avoid the problems above, but on the other hand, the user interface with the current approach is cleaner - it's much nicer to be informed about the requirements problem before you try to install the module (as with this patch) than to go through the process of enabling it and getting a red error message only after you submit the form (with the hook_requirements() approach).

Is there any way to merge the two approaches?

Status: Needs review » Needs work

The last submitted patch failed testing.

sun’s picture

Status: Needs work » Closed (won't fix)

WTF? This is utopian.

How do you measure a module's memory consumption? Exactly those modules that lead to memory issues are modules that have dynamic memory requirements. The more data they need to process, the more memory they consume.

And then we have the other modules that may not depend on the amount of data, but instead on the "size" of a single operation. Scaling an 12 megapixels photo shot down to a thumbnail and applying some cropping while being there. The memory consumption of the module? Down to zero. The required memory for this single process? Megabytes, unlimited, totally depending on the raw, uncompressed image data of the file that has been uploaded.

Pancho’s picture

Title: Allow modules to specify required memory in .info file » Allow profiles to specify required memory in .info file
Version: 7.x-dev » 8.x-dev
Category: feature » task
Status: Closed (won't fix) » Active

For modules neither adding nor maxing really helps. With outlier robust statistical methods it might be possible to get closer, but we certainly open a can of worms if every single module may define it's memory requirement.

But in the case of distributions, I wouldn't say that this is a won't fix. Their maintainers have an overall view of all modules installed by the profile. They have a clue of how much the individual modules are expected to be pushed to their edge. And usually they have the know-how to do meaningful tests.

That's why some of them are doing it anyway: manually adding a requirement to their install, see for example Panopoly.

Given that this workaround is necessary in order to avoid crashs, this IMHO is a task.

mgifford’s picture

Issue summary: View changes

This would be a good addition!

andypost’s picture

Version: 8.0.x-dev » 8.1.x-dev
Category: Task » Feature request
Status: Active » Postponed
Issue tags: -memory limit, -PHP, -admin/build/modules, -modules, -.info, -module management, -ini +Needs issue summary update

it's a feature with a tons of tags

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.0-beta1 was released on March 2, 2016, which means new developments and disruptive changes should now be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

alexpott’s picture

Howe about doing #2309731: drupal_check_profile() does not invoke the profile's hook_requirements() so a profile can have it's own hook_requirements implementation and it can do what it likes.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.