Feature to subscribe all users in a role to a newsletter

I don't think simplenews is meant to do this kind of thing. Simplenews is made to have users subscribe, which might could be limited to certain roles, but the user of the certain roles to receive the newsletters would have to sign up. At least that's how I see simplenews.

Hi Steve,

I've given your idea a lot of thought and consideration, and maybe I should have posted a comment a little earlier (but at the moment I'm very busy writing my PhD, and unfortunately, I don't have a lot of time for these things for the moment).

For now, I decided not to implement your feature request, I'm sorry. Definately not now (not in the 4.7 release), and possibly never, but let me explain my thoughts first.

These are my arguments for not committing your code:

1. The simplenews concept is different from what you are trying to achieve. As bradlis7 wrote, simplenews is designed to allow users with the appropriate permissions to signup for a newsletter, and not for an admin to decide who receives the newsletter. The 'import subscriptions' option is only there to allow migration from an old newsletter system to simplenews. Furthermore, the main reason why I started to write simplenews, is to allow anonymous users to subscribe as well (as all other mail modules were authenticated role-based). So the simplenews module has two principle goals: 1) free choice of users to subscribe, no automatic subscriptions, and 2) minimize the impact of user roles on functionality. Your idea "violates" both of these principles. That's not necessarily a problem, but let's say it's definately not the primary goal of this module.
2. Actually, there is a third simplenews principle: it is meant to be ... simple. Simple in several ways: easy to use for an admin, easy to subscribe/unsubscribe for users, simple database, ... In other words: to be transparent. Your code makes simplenews untransparent, for example because things are happening without users knowing it. I agree, you could send them an e-mail, saying they were automatically subscribed. But it's less transparent anyway.
3. Furthermore, in the light of these first points, you must admit that your database changes are quite heavy: one new database table, and extending an existing table with two columns... for a relatively minor feature which is not simplenews' primary business. During the development of this module, I tried and tested a lot of different database schemes, in order to keep the number of tables and the complexity of the tables as simple as possible, but without loosing functionality of the module. I think that your solution is overkill for what it tries to achieve.
4. Also, the code changes are quite drastic, although it may not seem so at first sight. The simplenews logic is complex, because of the many ways (possible combinations) a user can subscribe, unsubscribe, register at a site, subscribe while being be logged in, logged out (with or without an existing account), the user account can be deleted by an admin, a user can register through remote authentication, and so on... It took a very, very long time before all possible situations (orders of events) were fully covered and worked as expected. We found amazingly well hidden and unexpected bugs. Your code changes/corrupts some of that logic, which is sprinkled throughout the entire module. You couldn't know that, because you really have to know the simplenews code by heart to forsee this, and even then it's a spider's web. I installed your patch and gave it a little try, and I found several of these type of bugs. I can guarantee you that your code will cause more bugs, which may only be detected after a while, whereas I've worked very hard to get simplenews as stable as it is now. Can this be overcome? Sure, with _a lot_ of time and testing, it can be fixed. But then I repeat myself: I don't think it's worth to do this for a function that simplenews was not designed for, and certainly not for 4.7. Which brings me to my next points (3 more to go).
5. This is a short argument: simplenews is a popular module, and there are _a lot_ of feature requests. I have to set priorities, and for the moment, this isn't one.
6. Furthermore, as the author and maintainer of simplenews, I have to set a direction where I want simplenews to go in the future. A general picture, kind of a road map. In the short term, the priority is not to add a lot of new features, but to improve the code (so in some 'feature request' issues I will refer to this long explanation).
   On the top of my todo list is to test and implement the new mail API which was written by killes (Gerhard Killesreiter). This is related to the core business of simplenews, and it will allow the implementation of more urgent feature requests, such as embedding images, handling attachments, embedding stylesheets in html mails, ... So that's what I want to do before anything else.
7. And finally, I'm not sure why you are trying to achieve this with simplenews. Why restrict sending mails to certain roles to the simplenews node type? Maybe other modules are better for this (e.g. 'email this page')? Even a new module doing only this would be relatively simple to write, because now you had to 'force' the simplenews mechanism in a way it wasn't designed for, but in a new module in its simplest form, you wouldn't even need an extra db table to mail a node to a role. Maybe you can even re-use parts of the api of the enewsletter module?

The conclusion is that simplenews has to remain a simple, stable, and easy to use module, and that there are other, more urgent things to do now (mail api on top). I'm inclined to setting this to "won't fix", but I'm setting it to "postponed", as in "postponed to re-evaluate the idea", which I am willing to do after some important code changes have gone into simplenews. But this will take months.

I know how frustrating it can be to write code that doesn't get committed (I experience the same when I write patches for Drupal core sometimes), but I hope you understand and appreciate my point of view.

Cheers, Dries

In its current form (i.e. changing that much code, and extending the db), I don't think I can get convinced of adding this to simplenews, because I think it is too much of a change for a relatively minor (and not much asked for) feature, which will complicate things too much. Changing the db structure, for one, means writing high quality upgrade scripts, and experience learns that this isn't easy, and users are quite reluctant to apply this kind of changes to their sites. Therefore, db changes should add major functionality only. And as I wrote before, this can definately not be implemented before other, more urgent issues have been addressed.

However, there may be another way we can do this. I want to consider adding this functionality through an addon module, much like the addon modules in the ecommerce package. That way, we can keep simplenews.module simple (both in code, db, and ease of use), and people can decide themselves whether to enable this extra module or not. An addon module would be committed in a subdirectory of simplenews, and would be downloaded with the simplenews package, but would have separate database tables to install (in this case, two), not affecting existing tables. However, an addon module can act on the simplenews db tables (without changing their structure), and can use simplenews functions.

Much of simplenews' popularity is because of its "simpleness". People really like straighforward, small and transparant modules. A minority of users are power users, and those people want a little bit more. Although simplenews was not written for power users, I agree that we have to satisfy them as well. So I really want to keep simplenews basic functions what they are now, without extending it too much. Again, this is only one feature request! On itself, it isn't a huge change (conceptually, not considering the code and db changes), but if I would blindly implement all requests, they wouldn't gracefully merge with each other, and simplenews would become a monster module, which is totally bloated and untransparent, both for users and admins.

By using addon modules, simplenews can remain simple, but it allows extention of functionality for those who want it. And adding functionality would be selective: an admin can choose what he wants to add and what not. But it'll be an active choice, by enabling an extra module on the admin page. That way, as simplenews grows, other feature requests can be implemented in this way as well, extending simplenews and building upon it, without compromising simplenews 'core'. That's the way I see it.

Looking at your code, and thinking about your idea, it shouldn't be too hard to write this in an addon module. You'll have to change some of the logic, try some other approaches, but it can be done (even with re-using parts of your code). However, if you do so, take great care in covering all possible orders of events in which (un)subscription can take place. For example, when a user registers through remote authentication, initially, there is no e-mail address for that user, but he _will_ be an authenticated user.

As for the bugs I found: I would have to re-test to be able to give detailed descriptions of my findings (exactly in which order I did things), and I really don't have time for that now. But within a month or two, I'd be happy to thouroughly test an addon module for role subscriptions!

I expected exactly these two questions in your previous comment already. But a 10 minute delay is ok :-)

Answers:

1) Most likely

2) Yes, on the following conditions: a) they should not break the current logic, and b) they should be as generic as possible (meaning that they are general enough for future addon modules to use them as well, so that not all addons need too much separate hooks. In other words, intelligently crafted.)

Thanks Steve, for this patch!

I think there is an error in the patch, however. Line 22 should read:
+ $roles_with_subscriptions = db_query('SELECT DISTINCT (rid) FROM {sn_role_subscriptions} s WHERE 1');
instead of
+ $roles_with_subscriptions = db_query('SELECT DISTINCT (rid) FROM {sn_role_subscription} s WHERE 1');

...using plural in the rest of the script for the database name...

Take your time. I'll only have time within a few weeks to take a serious look at it. I'm still very busy right now. But I do appreciate your efforts! Thanks a lot!

Hi guys

I can't help out here much or anything like that but I'd like to commend you on the very reasonable approach to resolving this issue and allowing the add-on module to continue development.

I would really like this feature for my site. Its just faster than having all the people of a certain role subscribe individually. We have a significant heirarchy of roles (with significantly different permissions) and we want to keep the process of ensuring everyone of a certain role knows about relevant changes/developments as simple as possible. The role subscription is exactly what we need.

Keep up the good work and I'll gladly test the add-on as a non-core-level developer type, if it works for me it'll work for anyone ;)

Cheers,

Mark

Hi Steve

Just wondering if you had an idea of a timeline for that add-on module to be available?
I am just very interested in getting to use it.

Cheers

Mark

Hi Steve

I've installed it an it working so far without any problems. I made a little mysql file encapsulating your changes and the final "module" I installed is attached.

I appreciate you making it available for me and so will my users (even though they don't know it yet!). I'll keep you updated on any events/bugs/features etc.

Cheers

Mark

Just to report that I've installed this patch and additional (sub)module, with simplenews 1.47 and it all works great.

The extra functionality is really useful because I needed to set up a separate 'announcements' mailing list for very infrequent but important messages that all users need to receive from me as the site admin.

Thanks for the patch, and also thanks for the great simplenews module too.

I've just commented on a number of different posts elsewhere where people are saying (along with me) "what do we replace Mail module with if we go to 4.7?". I don't know what counts as a frequent or common feature request but as I followed up the search results from the word "mail" I thought I came across quite a few people wanting this functionality of admins being able to send out emails (or newsletters) to all users or selected groups of users.

Simplenews looks great - I haven't used it before. But combining it with this add-on module really seems to meet the requirements of all those who like me want to communicate easily with users via mass mail-outs.

So:
a) my hope and plea is that this feature will be fully implemented - I gather it is currently in cvs (whatever that is) and I think that means it is close to being finalised
b) it might be useful for there to be some pointer from the entry in the Module listing for Mail in 4.6 to this thread and/or Simplenews (it currently refers to the Send module) (I will see if I can add a comment or something)
c) thanks to both Steve and to Dries for working together on this

Um. Doesn't seem to be any way I can add a comment to the module entry for Mail - maybe someone who knows how to do this could point the way or take it up with whoever might be able to do it. Thanks, sorry etc.

Wow dudes... I am using simplenews as well. And I used to subscribe my users via MYSQL insert 1/2 yr ago because of the lack of this feature...

I read the teaser, saw the long back and forth but am happy that at least with a plugin-structure for this plugin this will be possible in a clean way in the future...

ok - but if I want to modify that again - to I need to discuss a plugin-of-plugin-of-plugin architecture :-)

no prob - you are forcing great architectural standards, which is why I chose Drupal the first place... but there has to be a good line to stop generalizing IMHO

christoph

Just a note to say that while the patch/module/mysql modification did what it was supposed to for me, it also seemingly broke the user-registration process on my 4.7.3 test site (simplenews 1.42.2.5). That is, after adding the mysql tables, patching sn & enabling the sn_roles module, I was able to subscribe roles to newsletters as advertised. However, part of the user registration process is now broken: if I visit admin/user & add a new user to the site, assigning this user to the same role that I have subscribed to the newsletter, I get a blank page back upon hitting submit. The next page that I visit gives me the following error:

user warning: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'FROM sn_subscriptions WHERE s.uid = 135' at line 1 query: SELECT FROM sn_subscriptions WHERE s.uid = 135 in /var/www/html/includes/database.mysql.inc on line 120.

When I go through an identical process of adding a user & changing her role on a fresh install of the same version of the simplenews module, everything goes smoothly.

Also, since I never got far enough to find out for myself, I wonder if you could answer two questions for me:

Is this patch/module designed to automatically add newly registering users to the subscription list (provided they're assigned to the one of the roles chosen in the module settings)? Does this module still allow users to opt-out?

I guess what I'm looking for is a kind of auto-subscribe (similar to the behaviour of the subscription module) that would allow me to create a newsletter to which new new users are subscribed automatically, but to which they could unsubscribe in their user-profile if they want. Plus I want to add all my existing users to the subscription list for the first mailout.

Thanks for all your hard work on this, Steve -- it's very much appreciated.

Thanks Steve! I'll try it out right away.

Yippee! Excellent work.

The module/patch works perfectly. Here's hoping for its inclusion in the next release!

For all those coming after: here's the updated (patched) simplenews_roles module. You'll still need to patch the simplenews module (using the included simplenews_4.patch) & install the mysql tables (also included).

although this is work for other people isn't it clear that the role subscription is a wanted feature now and integrating it properly will solve lots of problems in the future?

This is eactly what I needed, thanks for your work! I have seen a lot of posts for this feature.. simplenews is great and it only makes sense to have this option included in this mod!

hmm... I think I responded too soon above. Being new to Drupal, I thought I new how to apply a patch, but obviously I don't, so now I am stuck asking this stupid ques.. :( SORRY

How do I get this to work? :D I have simplenews installed and dowloaded the latest patch and ran the mysql file. What do I do with the patch file? I thought I should just copy and paste into original?

Can someone provide the patched version of the module? I put in all the changes and
I still get errors...

Thanks in advance.

-B

I still don't know how I got it to work, but all I did was upload the latest patch that included the mysql file as well and added the database. I then went into admin-newsletter and there was a new tab for admin to add by roles. Very cool! Thanks again!

For all those coming after: here's the updated (patched) simplenews_roles module. You'll still need to patch the simplenews module (using the included simplenews_4.patch) & install the mysql tables (also included).

Just trying to wrap my head around this highly useful addition:

1.) I need to install the patched simplenews_roles module provided with comment #40
2.) That file includes a simplenews_4.patch which I should use to patch the regular simplenews module
3.) The mysql tables should be installed.

Some questions:

Should I manually install the mysql tables?
Should step 2 precede step 1 or is the sequence irrelevant?

Thanks in advance.

1.) I need to install the patched simplenews_roles module provided with comment #40
2.) That file includes a simplenews_4.patch which I should use to patch the regular simplenews module
3.) The mysql tables should be installed.

Some questions:

Should I manually install the mysql tables?
Should step 2 precede step 1 or is the sequence irrelevant?

Yes, you should manually install the mysql tables. I'm not sure if step 2 needs to precede step 1 (or if the sequence matters at all) but it seems (to me) like it probably should, just to be safe. I can't remember which order I did things in, sorry!

In response to #44, here's a bundle of patch-free friendliness, including:

- the patched simplenews module
- the patched simplenews_roles module
- the simplenews_roles mysql file

So, here's some instructions:

1. backup your database
2. unpack the .gz file
3. upload patched simplenews module to your site (overwrite the old one if you need to)
4. enable patched simplenews module (installs the sn db tables)
5. install simplenews_roles.mysql in your database
6. upload patched simplenews_roles module
7. enable simplenews_roles module

There should now be a 'role subscriptions' section at /admin/newsletter/users/roles. Yay! Good luck!

Thank you two2the8 for the clarifications and the patched modules.

Today I found a bug that seems to be caused by the simplenews_roles module. When the sn_roles module is enabled, I can't edit the roles that existing users belong to. Here's how to duplicate the bug:

1. Enable sn_roles module
2. Navigate to admin/user
3. Click 'edit' beside an existing user
4. Add this user to a new role
5. Click 'Submit'

If you're like me, you'll see the ol' blank page of death now. If you repeat the process, but disable sn_roles, it'll work. Anyone smarter than me care to take a look at the code and see if they can figure it out?

Thanks guys for this module! I am using it with this paypal module http://drupal.org/node/78002 to allow users to subscribe and pay for a newsletter.

The problem two2the8 is on line 238 of simplenews_roles.module

Line should be this:

<?php
$individual_subscription = db_result(db_query('SELECT snid FROM {sn_snid_tid} s WHERE tid = %d AND role_subscription = 0', $role_subscription->tid));
?>

Spelling mistake with 'db_query'.

Am now getting these errors on this page: ?q=admin/newsletter/users/roles when I click import.
user warning: Duplicate entry '0-57' for key 1 query: INSERT INTO sn_snid_tid (snid, tid, role_subscription) VALUES (0, 57, 4) in /home/alternat/public_html/site/includes/database.mysql.inc on line 120.

Will look into it on Monday.

Cheers

James

Thanks to all you guys working on this. For me, this is a *really* important feature. Almost make or break.

Am now getting these errors on this page: ?q=admin/newsletter/users/roles when I click import.
user warning: Duplicate entry '0-57' for key 1 query: INSERT INTO sn_snid_tid (snid, tid, role_subscription) VALUES (0, 57, 4) in /home/alternat/public_html/site/includes/database.mysql.inc on line 120.

Changed the mysql table sn_subscriptions so that snid is now set to 'auto_increment'

Thanks for these fixes, 9802008. I'll try 'em out ASAP.

By the way, have y'all seen this thread, about the mail module's recent 4.7-ification? I'm very interested to try it out, too!

Just joining this thread. Can someone post a proper patch to the simplenews module? The only way this change is going to get in is if there is a clean patch so people can easily review this. Check out http://drupal.org/diffandpatch for more info.

Also, any database changes should be covered in the simplenews.install file under both the simplenews_install() and a simplenews_update_X() function.

I'm interested in this and will take the final steps to build this out and try to get this committed so hopefully someone can step up with an update patch.

Thanks.

I really would like to install this module, but I don't want to read through all the 50+ posts. Can someone post the final module for 4.7 with all bugs fixed???

I just installed from the file in two2the8's comment #48 and then changed line 238 as per 9802008's comment #51. Everything seems to work as it should, thanks!

I do have one concern though: I have set one newsletter to add all authenticated users to the list, but it doesn't add new users as they sign up to the site. Is it supposed to or should I just re-import the role everytime I want to send one out to make sure that everyone is in it?

thanks
alynner

I will try my best to roll an appropriate patch. I'll keep you posted.

Since Simplenews has been cleaned up for D5 and now uses FAPI correctly, this can easily be done as a contrib module. I have an alpha version and will create a new project in the next couple weeks. If I don't, harass me and I will. :P

does "won't fix" mean that this will only be built for 5.0 now?
thanks,
Maria

It means that this functionality must go in another module.

Has this gone into a new module yet? Thanks...