Release Path and Roadmap for Feeds 7.x-2.0

Would be great to get some kind of confirmation or affirmation from phase 2 on this direction. Anyone?...

I dont have any comprehensive plan for Feeds 7.x-2.x just yet, there are too many outstanding issues/patches to focus forward just yet. I have mobilized a few guys from my team to help tackle the long list of outstanding patches and once we make some good headway there we can start to formulate more of a plan for moving forward. All that to say we will dig into outstanding patches including the "Definitely In" section above.

As of this morning I have all unit tests running cleanly (please verify)

I also have a group of us spending our lunch hour or so right now reviewing patches.

added simpletests

Will the generic entity processor ever be rolled in to this module? The patch seems to work well...

We should stick these in there as well:

• #1033202: [Meta] Generic entity processor (as mentioned above)
• #1362378: Provide entity post-save hook
• #1369874: Don't roll own CSV parser; use PHP's native one

...as other issues either could or do depend on them. We should add any others that are framework-ish.

I think the biggest job that has to happen (ideally before we get this thing out), is cleaning out the issue queue. I'm finding tonnes of duplicates, and those are metric tonnes! We could probably have a summer student work on this exclusively. :)

Way ahead of you - been doing that for the past week at least. ;) Thanks for testing patches - wasn't sure if anyone was still maintaining this!

Marking #1400042: Create stable release of Feeds for 7.x as a duplicate of this issue.

May I suggest that someone who has the appropriate authority (febbraro?) tags the important issues with 7.x-2.0 blocker?

Gives quick entry for anyone wanting to give a hand.

From greggles' duplicate:

It would be nice to have a stable release to provide site builders more certainty that the module will get an SA for any security issues. Please feel free to update this issue summary to include links to issues to fix prior to that release.

As of this post, there are 6 critical bugs in the queue.

You can use the tag D7 stable release blocker to help others identify issues that are blocking a stable release. Currently there are zero issues with that tag.

Xen: I think anybody can add tags; extra permissions not required.

Making sure tag can be added.

EDIT: I was able to add the tag to some issues (including this one), and I don't have any extra permissions. So other folks can help with the ones I didn't get to.

@colan
Wasn't so much a question of permissions, but more to get the stamp of approval from the maintainers on the selected issues. If everybody just adds their favorite issue, we'll never get anywhere.

Might I add:

#996808: Update existing doesn't reset targets that have real_target set.
#712304: Batch import does not continue where it left off, instead starts from the beginning

I actually think there are multiple issues here, but I've been getting a lot of complaints from this one:
#625196: Fatal errors (Unsupported operand types) and warnings (Argument is not an array) in FeedsConfigurable.

@twistor
That's why we need a maintainer to draw out a roadmap.

I think drush integration is a must, and it is quite there #608408: Drush integration for Feeds

Xen: That's a good point, but is might save the maintainer(s) work if we tag what we think is appropriate, and then he/she/they edit it.

@colan:
Well, if someone would gather together all the contenders and make a list for the maintainers to give the nod, it might.

Would be nice if the maintainers would show up and give their input.

Given the fact that the issue queue is so huge, I don't think any one person will have time to evaluate all of the things that should go into a stable release. We would certainly appreciate any support in identifying all of the critical issues for a 2.0 stable release with the 'D7 stable release blocker' tag. These need to be real critical issues though, not just that something specific (or obscure) does not work for a particular use case.

In any case, tagging the issues will help us focus our efforts and allow us to get there faster. Thanks.

@febbraro
Having everybody and their dog tag their favorite issues with the 'D7 stable release blocker' is just going to make another big list of issues. Somebody has to play hardball and decide which of those issues is *really* needed for a stable release, and what'll have to wait.

There's been issues mentioned here that's not really needed to get a stable D7 release out the door, and the more that's added to the list, the longer it'll take.

Having a clear "these are the issues that the maintainer says needs to be fixed" list rather than a big "these are the issues that some random thought should be fixed" list is a bigger motivator for anyone that wants to dig in.

@Xen, Issues can be un-tagged just as easily as they can be tagged. A maintainer just asked for assistance finding the critical issues as this queue is huge.

Might I suggest a few guidelines:

• No feature or support requests. It's valid that they get in, just not that they be tagged blockers.
• Current functionality that doesn't behave as intended/expected should be the main goal.
• Typos. While not a real blocker, they are simple and make for a more polished release.

@twistor
I'm just asking for someone to take a bit of charge here. It's been six months since the last alpha, there's still major functionally issues, and everybody agrees that we need to get things moving and prioritize.

So why is #1369874: Don't roll own CSV parser; use PHP's native one considered a blocker? I'm all for using the standard PHP parser, but considering the problem of too many issues, is it really needed for the first D7 release? If the old code works, let it be and refactor it in a later release.

It's fine if people can help out pointing out the issues, but someone has to decide whats considered a blocker, because we can't all do it.

@Xen: Then this one is something that can be dropped from the draft list once its complete. But let's add a refactor tag to this type of issue, so that these don't get lost.

@febbraro: Do you want more co-maintainers? I (and probably others) would be happy to help, as it seems like you're overwhelmed.

that would be awesome (a stable release) - though 'job scheduler' is still in alpha. could this be configured to work with elysia cron instead? http://drupal.org/project/elysia_cron (seems that it would present potential for more granular control in large import scenarios or predictable intervals at bigger scale)

Hi guys. I'm the lead for Feeds module here at Phase2, so with guidance from febbraro, and your input, I'm trying to get this stuff moving. I've been committing smaller changes over the last few months, while trying to get more familiar with the code base. But anyway, just wanted to chime in here so you know where to direct your questions and concerns.

thanks for the intro - it sounds like the floating question is where things stand for a stable 7x release, whether it's soon or later this year, any insights would be greatly appreciated!

emackn, I'm also in for co-maintainer role, if it's needed.

I don't think #1033202: [Meta] Generic entity processor, #1362378: Provide entity post-save hook and #1369874: Don't roll own CSV parser; use PHP's native one should be listed as blockers at all. New features need to be pushed into 3.x, and we should work hard on fixing the bugs at this time, as we all need a stable realease of 2.x. I'm tired of using -dev release or patching already committed fixes in the old release, so maybe we should setup a sprint or something to get this done.

@franz, I agree that those issues shouldn't be blockers. Although, they don't need to be pushed back to 3.x either. New features, as long as they are backwards compatible, can be considered.

Count me in for a sprint.

I think we should add a new alpha release at this time. Already a handful of bugfixes in.

It's 11 months since the last alpha - makes a ton of sense to me.

Got a new release rolled, I hope beta doesn't take another year.

We could definitely use a sprint here, how can we schedule it? Please ping me on irc #drupal-contribute if you're interested.

Added items from my comment & fixed typos, formatting.

I am with you guys.

me too!

Adding issues

SPRINT@MUNICH?

hey, it's late but not too late: i can imagine getting quite some stuff done at an in-place-sprint around drupalcon munich, alongside the multilingual folks. (Drupal 8 Multilingual Initiative sprint before/after DrupalCon Munich 2012, needs sponsors | Drupal Groups)

it's about this weekend 18./19. and maybe the next one 24./25./26.8.

anyone out there who likes that idea too?

Added one more ticket

deleted fixed issues

reordered issues

removed fixed issue

added issue

OK, i'm at the sprints now and donating spare cycles to clean up this.

As i'm not maintainer i opened up a sandbox.
This is alsy my application for maintainer status.

EDIT:
* cleaned up description, removed already fixed issues.
* propose that we remove more issues from "blocking" as soon as the needs-review-patch in #1033202: [Meta] Generic entity processor is committed. so folks go test that thingee!
* now looking after release blockers
* i'm sticking to this issue's description and reordering it so it is clearer why something is in and out.
* we need manual test of #955236: Handle fatal errors for entity create with better error handler function to fly it in.
* added #1739842: Deprecate some processors in favor of generic entity processor to decide direction

And here's the list of patches i consider ready from minisprint branch:
* #1364116: Option to skip hash check on re-import
* #1698076: Force update for TermProcessor should default to FALSE.
* #1739704: Node lookup by title in Nodeprocessor does not respect nodetype
* #1110762: Feeds does not catch file exceptions properly in file mapper and FeedsParser.inc getFile() function

These need more thorough review:
* (i needed this myself and use it in prod, but not all mappings, so tests needed:) #1689374: Map feed Title and feed URL
* this is a monster patch so i don't trust it until bit more testing: #1127696: Attach multiple importers to one content type (in D7)

#1827164: Integrate Feeds into core

feature request

fixed html

the request to integrate feeds into core has been pushed to D9.

Can we get an update on either a plan of action here or if this is just generally considered unmaintained at this point? We really need a stable release so that the security team will address any security issues with this module.

Updated issue summary.

I don’t have right to complain about releases and bug fixes, as I didn’t commit anything, but it has been a while since the last alpha8 release (13 months).
May I ask if there is a plan for a new alpha or, even better, a beta release (for D7)?

Just release a 2.0 version.

There haven't been a commit for 3 months, it's not like there's a flurry of development at the moment..

This is a four year old module that haven't had a *single* stable release yet. People are using the alpha in production and have done for years.

Yes, there's known issues and things that could be improved, but it's not like the 2.0 version is supposed to be the final one.

Else change the maintenance and development status to reflect reality.

Plans for a new alpha release: #2290465: [Meta] Create beta 1 release

It's really time to just cut a release. All software has bugs. All software is missing some features. Even if there are things that would be nice to fix prior to a stable release they can wait for the 7.x-2.1 release.

There are about 90,000 sites using feeds. They should not be exposed to having security issues reported in the public queue like #2495145: Possible XSS in PuSHSubscriber.inc.

@greggles, I agree, I plan on getting it done this weekend.

Although I agree Feeds is in high need of a stable 7.x-2.0 release, I personally would prefer another alpha release first because I'm a bit concerned about how secure Feeds currently is. For example, an user with the permission to administer Feeds but without the permission to administer users, has still the ability to import users (and he/she can even control the roles these users get).
An other concern is that there are still some issues open that are marked as stable release blockers. At least one of them, #1183440: Multilingual Feeds - Make field import language-aware, requires an API change: #2333029: Extend mapping API to allow for defaults and multiple callbacks. (I see twistor is currently actively working on that, that's great :)). I think we should investigate first if these issues are still valid blockers and if they are not, remove the tag.

@greggles, twistor
What do you think about my concerns? Would fixing these be worth enough to postpone a stable 7.x-2.0 release?

Please understand how much dependency there is on patches. Date ical requires alpha8 + patch to import dates! : #2237177: Apply this patch to Feeds 2.0-alpha8 to fix broken date import.
I'd suggest a beta version - why not - feeds is already in so many production sites. I realize there are guidelines but consider the poor mortal system builders too :).

@MegaChriz, yes that's closer to what I had in mind. I want to get rc1 out the door, and try to shoot for stable in a month after that.

We shouldn't be too scared of a stable, we keep backwards compatibility anyway.

I just filed another security issue: #2502419: Log messages XSS attack vector

Please release Feeds 2.0 so that we can finally have security team support and fix stuff like this in private.

@klausi
I agree that Feeds needs a 7.x-2.0 release and we are doing our best to get there. I feel that we are very near to get one of the last stable release blockers in: #1183440: Multilingual Feeds - Make field import language-aware, though it is a tough one to test. I hope I can find enough time to get through that one quickly.

That issue looks like a complex change/addition, so I would NOT release that with Feeds 2.0. Security support is more important right now and we should not throw even more changes at users with the 2.0 release (which will be a security release everyone has to apply).

+1 for @klausi - how do you determine that this is a requirement for a stable 2.0?

@klausi, the security release isn't going to be 2.0. It's going to be alpha-9. There are already too many changes in dev to make a security release, so we're going to have to cut a separate branch just for it.

OK, so I'm now in the uncomfortable situation that I cannot downgrade to the alpha9 security release because it does not contain all fixes of the dev version.

If I install the most recent dev version I still get an email from sites that I maintain that security releases are available. Can you do a fake commit to the 7.x-2.x dev branch to make the dev release newer than alpha9? Or can you release 2.0 now?

The dev release should have been more recent, I'll find something.

@MegaChriz wrote in #2786487: Create stable release:

I agree that a stable release for Feeds is needed. There are a couple things that are holding me back to do that soon:

1. I know there are security issues in Feeds. Some of these issues are currently discussed in private on security.drupal.org.
2. There are a few major bugs that likely need API changes to fix them. If we would release a stable now, then 7.x-2.0 to 7.x-2.1 would potentially introduce a BC break.

All the conversations I'm seeing for a stable release are well over 6 months old. Where is this at for a stable release?

See #2826403: Plan for Feeds 7.x-2.0-beta4 release for plans for the next beta release. After that I'll have to investigate what still needs to be done. #1995728: [META] Cron import not working on 7.x-2.0-alpha8 and later is marked as critical.