Development Seed (specifically yhahn, tmcw, rsoden, jmiccolis) has been doing a lot of work with OpenLayers both independently and with our module. Their needs have been for fairly large sites and so they have some use cases that I don't think we (or at least I) have thought too much about. So, they have "hacked" up our module, and have allowed us to see their code and are asking if we want to use it and take it in the direction of our 6-2.x (and probably 7-2.x branch).

The Code

You can checkout the code here:
https://devseed.svn.cvsdude.com/sandbox/robert/openlayers/

Please note that it is not complete by any means.

Some key changes:
* CTools dependency to use the plugin system
* Makes layers, styles, and maps ctools plugins
* Use of Drupal behaviors for maps on JS side
* Other JS tweaks in general rendering
* Combing controls and behaviors
* Views has been split so that there is a OpenLayers Data plugin which will then create layers based on those views
** Then there is an OpenLayers Views plugin which actually displays a map

My Thoughts on the Code

* Overall, the changes seem positive.
* I think providing Views as layers is cool, but I am not sure how the user will react (choosing between plugins)
* I have my reservations about CTools and dependencies, but going forward, I think it is a good move
* There is a few function name changes, and the render function seems to be split up (and I am not sure why)
* Using Drupal Behaviors and streamlining the JS is a great direction

My Thoughts on this Direction

EricG and I talked last night about a lot of stuff, and I support this direction. My initial reservations, that we have already talked about:

* Releasing a 1.0. This is a major focus of mine. This does not conflict with a 2.x branch, and they can be done in parallel, but I would like to see a 1.0 happen and not have it sit in Beta forever. Eric said that devseed could probably help get 1.0 out.
* Maintaining an open process. One of the joys of this project for me has been the great open process we have taken to discussing and deciding on where to go with this. Eric is a big supporter, and I think devseeds projects and involvement in the community shows that this has and will be true moving forward.
* Conflict of interest. I was initially concerned that accepting devseed's code meant that we would be accepting their specific organizational goals and that might conflict with Drupal's goals. But in reality, Drupal's goals are the community and devseed is a part of that. Devseed should benefit from any good decisions we make along the way, so there is not really a conflict of interest. Sweet, sweet open source.

So, I think we should move forward with this.

The Plan

The plan as I see it:

* Take on rsoden as a co-maintainer
* Do a quick sprint to make the changes that devseed has done and put into our code, and create the 2.x branch.
* Stabilize 1.0!
* Continue this discussion and make some more concrete goals for the 2.x branch.
** Some of these are already issues, and I have some in my head as well.
* Start to flesh out the 2.x branch. Even the code that is provided is far from complete.

Your Thoughts

The OpenLayers Drupal module is a democracy, so this is not solely my decision and I would like the developers, and users thoughts. Patrick (phayes) and I have been the most invested in this and probably have the most say. I still would like Tom (tmcw) and Bryn (brynbellomy) to chime in because their opinions are valuable.

CommentFileSizeAuthor
#17 drupal-geo-20090923.colloquyTranscript.txt141.22 KBzzolo
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

phayes’s picture

phayes CreditAttribution: phayes commented

Thoughts:

1. Looking at the code by far the biggest change is moving everything to ctools. I think this architectural change is pretty big, and since I don't fully understand what ctools is doing, i'd like to learn a little more about the advantages of why we should use ctools.

2. All map controls are behaviors. I'm not so sure about this. It makes some sense from a simple-architecture perspective (less types of data objects), but I think it also means that maps won't have basic controls by default - which is a problem, and makes the map array longer. I'm open to moving everything to a behavior - but it needs some discussion.

3. Exposing views as layers - this is something I was wanting to do already - so i'm glad someone did it for me. :-) Does this belong in the views module, or the layers module?

I'd like to stabilize 1.x before starting 2.x. A lot of the bugs that are in 1.x will also be present in 2.x so it makes sense to squash these first. Also, I think concentrating on squashing bugs in 1.x will also allow all of us (phayes, zzolo, yhahn, tmcw, rsoden, jmiccolis, brynbellomy) to get used to working together accomplishing things that are not so controversial as major architectural changes and a new branch. This will also allow time for us to have a full conversation about the major changes we want to see in 2.x before actually starting to code for it. I'm hoping that this conversation / stabilization of 1.x won't take more than a few weeks.

rsoden’s picture

rsoden CreditAttribution: rsoden commented

zzolo, phayes,

Thanks for your notes here. You raise a few interesting points that I'm looking forward to diving into over the next few weeks and months. A few quick responses:

1 - Agreed that a stable 1.0 release is the absolute first priority. I think zzolo talked with ericg about this at Drupalcon but I'm happy to do what I can on this front.

2 - I know module dependency is sort of a bummer, but Ctools is a huge win in terms of making Drupal objects exportable, pluggable, and overridable. With bits of this approach moving into core for 7, I really believe that this is the future and we do ourselves a huge favor by getting out in front of it. In the shorter term, it has also made building custom behaviors much much simpler for us.

3 - RE: 2 views plugins. We need to do a bit more exploration of how Ctools would handle this, but I'd really like to discuss removing the map style plugin and instead letting map presets provide their own blocks and page callbacks. I also think phayes's suggestion about moving the exposition of views as layers into openlayers_layers is a really good one. In this way, any module could then provide data that openlayers could treat as a layer. At any rate, there are some interesting things to talk about here.

4 - Combining controls and behaviors. We did this for two reasons. One, the separation of controls and behaviors seems entirely artificial given how the OL api works (but maybe I'm missing something?). Two, by making all of the controls Ctools objects, we can easily extend them and override them as needed. phayes, I'm not sure what you mean by "maps won't have basic controls by default." Map presets can still be set up to have whichever behaviors people want enabled and activated by default. Again, maybe I'm not getting something.

Really excited to be working with you all on this. Looking forward to more conversation and collaboration.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

Just a quick note on behaviors and controls. I think phayes' comment about not having default controls was more because how behaviors are in their own module. So, if behaviors and controls live in the same module which is separate form the default, then this would be true. But it is a simple solution of the base OL module providing the needed behaviors/controls, and pushing anything that is not-default to the behaviors module.

phayes’s picture

phayes CreditAttribution: phayes commented

Thanks for the comments.

1. Controls as Behaviors: My concerns around controls as behaviors was mostly around making sure they are exposed to the UI and that some are turned on by default. But we building behavior and layer UI for 2.x anyways, so controls as behaviors will be fine.

2. Ctools: From what i've heard about ctools it's awesome and "full of win, lots and lots of win". I've just got to put aside some time to learn it so I actually understand what's being proposed!

3. Views: rsoden, could you go into a little more detail about your opinion on how we should be handling views? Specifically what do you mean by "letting map presets provide their own blocks and page callbacks"?

Thanks!

tmcw’s picture

tmcw CreditAttribution: tmcw commented

Hey phayes, just to explain a little more on the unification between controls & behaviors: we saw that a lot of the default OpenLayers controls were hardcoded into the OpenLayers module, but the reality is that they aren't really hardcoded into the OpenLayers module, but rather you can build an OpenLayers javascript library with only the controls you want pretty easily (not as easily as you can build jQuery, but I think that's something to look for in the future). So, given that those hardcoded controls might not exist in the actual js library, and other controls may exist (we're using a modified version of LayerSwitcher which we'd like to get into OpenLayers itself), it made sense to make controls into pluggable components.

In terms of Views interactons (your #3): Ctools is going to be handing the whole page/block callback system that is currently part of Views 2. Since, currently, map presets basically contain everything you need in a map - there is no need for views for some map presets - it doesn't make sense to have them run through the views stack. Instead, we'd like to have page/block callbacks for the map preset, and for that preset to call Views in order to get map data. Therefore, there is all of the map configuration in the map preset, and only data and possibly data formatting in the view. Also, for people using stuff like the KML module for their data, it will no longer be a hack that requires a 'null view' that produces no data.

phayes’s picture

phayes CreditAttribution: phayes commented

1. Behaviors as Controls: We are in agreement. Let's do it!

2. Page and block callbacks for presets: It was never our intention to make viewing maps dependent on views. We just hadn't actually gotten around to coding any alternatives. :-) The easiest way to make a map is to do this:

$map = openlayers_render_map($raw_map_array);
// OR
$map = openlayers_render_map('preset_name');

print $map['themed'];

All we need to do is basically wrap this in various UI's and make it available to the user. We definitely want to expose this as a block and a page. I can also see us creating a openlayers_node module that basically injects the preset editing UI into a node editing page, and let's users create maps as nodes. I'm sure there's lots of other ways to allow users to create maps....

Personally I'd like us to keep both ways of getting views data into maps: Exposing views as layers that can be plugged into any preset, *and* keeping the style map plugin so users don't have to create a new preset for every view. Keeping the style map also lets users set the path in views, and use arguments etc. without a preset acting as a 'go between'.

Keep up the discussion, this is good stuff!

zzolo’s picture

zzolo CreditAttribution: zzolo commented

It may be good to start a list of new features, architectural changes, etc. we all see for 2.x, and maybe we can start branching them off into new issues.

rsoden’s picture

rsoden CreditAttribution: rsoden commented

Just a quick note to let you know I updated the code in the sandbox: https://devseed.svn.cvsdude.com/sandbox/robert/openlayers/

Nothing major, just a few small bug fixes and added in some new preset options: datelinewrap, number of zoom levels, and a layer visibility control like the one requested here: http://drupal.org/node/556134

phayes - good points re: arguments and exposed filters as reasons to keep the ol map display. i'll try to make time to see what could be done with letting presets create blocks and page callbacks - unless this is something you want to work on as part of your ctools exploration? :)

zzolo - a feature list sounds smart. i know you've got thoughts on this already so maybe its best if you kicked this off? one of the things that i'd really like to see is a more end-user friendly way of setting max/minimum resolutions (or scales or zoom levels or whatever) and restricting panning.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

So, here is my list of new features or major changes for a 2.x branch from what the discussion and from what's in my head.

* Any kind of performance improvements
* Integration with CTools
* Presets will become CTools plugins
* Layers will become CTools plugins
* Behaviors will encompass controls and become CTools plugins
* Layers and Behaviors will get separate interface to manage them, and will allow for form options
* OL will provide a Views data layer which will be managed like the other Layers above
* Preset interface will provide very basic interface to create pages and blocks
* A Views Display plugin will still exist?
* JS rewrite
* More streamlined hooks, such as a hook_openlayers, like Views. CTools might be able to handle most of this.
* Maybe a rethink of how styles are handled.

If you want to add more, copy the list so that the end one will be a complete lists (instead of having to read all the comments).

zzolo’s picture

zzolo CreditAttribution: zzolo commented

So, as far as making this 6-2.x branch happen here are my thoughts:

  • Take the 1.x branch and push it back to HEAD (HEAD is a pretty old version). Unforunately, in CVS this is a pain in the ass which basically requires getting rid of everything in head, then putting 1.x on top of it.
  • Make a DRUPAL--6-2 branch
  • Start making changes to the new 2.x branch in discreete commits. (Starting from 1.x branch, not the code that is provided above)

I can make the CVS stuff happen this weekend if this makes sense to everyone. Just a note for development environments: it will probably make sense to have two dev sites, one with each branch.

Various thoughts on the process of the 2.x development:

  • The 6-1.x still is in development! Do not forget about that. I will be very sore to see anyone putting a lot of time in 2.x and not taking even a little time to address bugs in 1.x. Everyone's help is appreciated, of course.
  • Ideally, we would want to have 1.0 out and squash any existing bugs, but even a 1.0 will have bugs we will have to port across.
  • I do not want to just take the 2.x version from above and put it in CVS. We need to work from what we have in 1.x and actually discuss the changes we make. We have a lot of hands in this pot now (tryng to get rsoden's hands in as well), and though we cannot work like Drupal core and make patches for everything, I think it is really important going forward that any changes should be discussed in some fashion.
  • Anything significant changes to be discussed up front.
  • Anythign semi-significatn needs to be announced at least after the commit, and reasoning and examples provided.
  • All commits need to have descriptive comments.
  • This is becoming a fairly big piece of code, and so I don't feel like we are all able to know the whole code base, making the need to have open discussion even more important.
  • We all have great ideas and directions we want to go with this, and they might not always be in agreement, and that is why it is very important to discuss things openly.
  • The 2.x branch is a different beast than the 1.x. With 1.x, it was a really organic process with just two people, and somehow we managed to put this together. With the 2.x branch, there are a number of people able to commit and we all have a base to start from and our own ideas on how to take it.
  • I think we have done a very good job so far, and I have been really happy with how things are, and so my concerns are how to keep it up, not how to change it.

I hope all this makes sense and is reasonable.

zzolo’s picture

zzolo CreditAttribution: zzolo commented
Version: 6.x-1.x-dev » 6.x-2.x-dev

So, I have created the Drupal-6--2 branch! Please make sure to make discrete, taskable commits, and there it is better to talk about it too much on the issue queue than too level. And dont forget about 1.0.

One thing I forgot to mention above is an upgrade path. We need to support an upgrade path. This means that any data structure changes need to have an hook_update_N implementation for it. Also, it means we need to document any API changes.

rsoden’s picture

rsoden CreditAttribution: rsoden commented

Alan,

Glad the 2.x branch is started. It was my understanding that it would be based off the code above and we've been actively developing what's in the sandbox with that in mind.

At this point, taking a piecemeal approach to merging this work back into the 1.x branch would not make sense given 2.x makes architectural changes that don't function as isolated patches. I simply don't have the time/resources to do this. Going back to the 1.x branch as a base for future 2.x work would a major time-sink and in my eyes, a setback for the project.

We've been completely open about the changes we made and why and everyone has had access to the code since the week after it was written. I'd like to keep working together on this but we'll need to start from the codebase above and I'll need to feel comfortable to continue running with some of the architecture and api changes in the 2.x branch necessary for this to become a developer-friendly module.

Apologies if there has been confusion here, I thought that after Drupalcon we were on the same page about 2.x development. Looking forward to hearing your thoughts/concerns as well as those of the other developers.

Cheers,
Robert

zzolo’s picture

zzolo CreditAttribution: zzolo commented

This is a tough spot, but I think we can work this out. Here are some of my reasoning for the approach that I have started.

In the initial post above, I obviously was not clear, but I meant to use this approach from the beginning when I said: "Do a quick sprint to make the changes that devseed has done and put into our code, and create the 2.x branch." As far as your approach and putting your version directly into the repo, this was never stated by anyone. It is unfortunate for the confusion.

I know this may seem like a time sink for you, but putting in your version directly into the new branch is a time sink for this project, in my opinion. There are many changes that have been added since you have forked your version. All these changes need to be ported or reviewed to put into your version. Also, there are a number of things about your version that I don't personally like and would like to remove (after discussing), which again will take time. I cannot say which direction is more time-consuming as I do not have any statistics or real analysis. But, I believe my direction is actually more beneficial to all because there are multiple hands to port your version to the projects 2.x branch.

But more importantly, I think the value of my direction is that we are able to have more open discussion and open process about each individual change. I know we have discussed some of the larger changes above, but we have not had any significant discussions about individual parts. Also, if just push your version in, we have no repository history which is valuable.

If you did not have this version available, we go about it the same way, of taking our current branch and creating a new one, then making, documenting, and discussing each change (large and small). Personally, I don't want to circumvent this process, but I have been trying to speed it up to address your version and efforts.

I think we are still on the same page and my discussion with Eric, and our discussion on this thread are still very valid. I think we all want to make this happen, value your version, and don't want to waste time and work that anyone has done, and I think this is all apparent. I think the confusion is how we actually going about making this happen.

I'll try to ping phayes and brynbellomy and see if they want to comment.

phayes’s picture

phayes CreditAttribution: phayes commented

I think fundamentally this is all about time and resources. If we all had oodles of free time to work on this stuff we could all just sit down for three days and merge the two branches. Regrettably this isn't the case. So we've got to start one way or another. Also clearly it's advantageous to zzolo and I to start with the current branch and merge in robert's chages (since the code will be more familiar to us this way). And it's obviously easier for robert if we did it the other way, since we would be starting from a place that he is intimately familiar with.

Given all this, I still think it would be best to start from the 1.x branch and move from there. This sucks from robert's perspective as it means all the code he wrote goes into patches for review, he'll needs to justify his way of doing things, some things might get changed in a way he doesn't like, and some of his changes might not even survive. This is not only a painful process of watching your code being nit-picked apart, but is also a huge time consumer.

From a project health and code quality perspective I think this process is necessary.By browsing 1.x and 2.x development threads, you can clearly see that *often* the main developers of this module don't agree on the best way to do things. But we manage to work through all of it and I think that is what has made this such an awsome module. Now we have rsoden, who is a very skilled developer, throwing his ideas into the mix - and by looking at the changes his code will be bringing to 2.x it's clearly going to make this module that much more kick-ass. However, I still think the peer review process is critical. Hopefully we'll only have to do this *once*, and when we are done all of us will be working from the same 2.x codebase, and then the real fun will begin!

That's my two cents.

Patrick

zzolo’s picture

zzolo CreditAttribution: zzolo commented

I think it may help to quantify the time that has been put into each version since it has been forked. I will try to quantify what we have done since you have forked your version. My estimate is that your version is from July 31, 2009, but this is not very accurate as there is no CVS info; If we put it at a release it may be roughly 6.x-1.0-alpha4. Here are some rough estimates:

* 60+ commits. This is a very rough metric as to the significance of each commit. You can start the messages from here: http://drupal.org/project/cvs/177400?nid=177400&page=6&nid=177400
* This helps describe what happened with each release: http://drupal.org/node/177400/release
** There are dozen or two bug fixes in there
** Code standardization
** Documentation
** Tests
** Major features like CCK, clustering, Geo integration, etc.
* Personally, I would estimate that I work anywhere form 5-10 hours a week on this module (unpaid in this analytical span), which comes out to be anywhere from 35-70 hours of my time since fork. Not to mention Patrick's, Tom's, or Bryn's.

This is what I feel has been done since then. I do realize that all this work has been done, and it is not fair to say that it takes the same amount to port it to another code base. It would be interesting to know how much time and what specific changes have been made to the forked version.

Even with concrete numbers on times and changes, I still believe in the open process and incremental changes, but I think it helps gives us some better perspective.

tmcw’s picture

tmcw CreditAttribution: tmcw commented

Hey Alan & Patrick,

I understand your frustration and the fear that this could be a completely alien codebase, but merging changes forward from 1.x to 2.x is a vastly different picture than merging post-revision-244450 changes into 2.x. There are a bunch of serious reasons for this:

* Rolling patches that result in a constantly working codebase will be extremely difficult, especially for big design decisions and huge code changes like the use of ctools and the switch to behavior-based behaviors and controls.
* A lot of the changes since 1.x would be implemented much differently in 2.x. For instance, geo integration as it stands in 1.x would basically need a complete rewrite. This actually goes for many of the commits, since many of them were entirely about behaviors and openlayers_views, both of which are revamped.
* I can't speak for others but some of the bugs I committed to OpenLayers in the time between branches have actually been backported from 2.x originally. Several of the bugs have been fixed in 2.x independently, or the code that propagated them was rewritten.

It's clear that the x factor here is that starting with 2.x makes it seem like your work is disregarded. As much as I think that it's vital that everyone on the project has a strong say in the direction, hacking together patches for 1.x to become 2.x isn't a reasonable way to do this. Branching with the 2.x code and porting documentation, tests, and new features from the gap makes sense and would integrate all of the improvements in the main branch in much less time. If there are any serious changes that should be made to the 2.x branch, we should discuss them on the issue tracker and make those changes. I've already been adding issues for 2.x modifications and we're going to add issues that summarize main changes, so that there is no information vacuum in the transition.

zzolo’s picture

zzolo CreditAttribution: zzolo commented
FileSize
drupal-geo-20090923.colloquyTranscript.txt141.22 KB

Just a quick update. tmcw, rsoden, brynbellomy, and I all had a discussion in #drupal-geo. I am attaching a Colloquy Transcript (I'm sorry, I don't know how to get in just plain text).

We discuss a lot of the same things as above, and we're still not able to come to a decision about how to proceed on the 2.x branch.

But, to address rsoden's and tmcw's needs for their Development Seed's clients, we agreed on creating a 0.x branch where they can put their code and proceed with their project. rsoden also got commit access to the project.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

A well-written email by Robert (rsoden):

In terms of how to move forward with 2.x, that still seems like an open question. If you really think you can do it all in a day, then feel free to take a stab at it. At the least, you'll be able to get more intimately familiar with our branch. But I don't see how this gets us any closer to the open or transparent process you claim is the reason you want to do this in the first place. And I would be (happily) floored if it only took a day.

Like I said in the post, I am really worried about the time this approach will take. OpenLayers is just one of the many projects we contribute to and maintain and redoing the 150+ hours of work we've put into this branch when we already have something that meets our needs is going to be tough to justify. Obviously we want to help out as we can, but I'm imagining this process taking weeks or months before we can move off of our branch if we do it this way. From the IRC chat with Bryn and talks I've had with Tom, none of us really think that backporting the architecture changes will be the sort of thing that could be done on a Sunday, or even two Sundays.

As I said in IRC yesterday, I think we can work with you and getting the 1.x changes ported into our branch and make a lot of effort to document the changes in the issue queue and openly discuss why we made them. If they don't cut the mustard, we can work with you to alter them in ways that make everyone happy. I think this gets us very close to the open process you are describing, and will make the merge a much less painful and extended process for everyone. And we'll sooner get to where we all want to be, collaborating and pushing forward the same version of the module.

We were very close during the spring to adding OpenLayers support to NIceMap. When we first looked at the OpenLayers module, the architecture and js handling seemed like they wouldn't work at all for us. Somewhere around then you started to take an active lead and we read through the issue queue and the patches you an phayes were making, we got excited and so we used it on a few projects and started contributing patches as we could. But despite all the changes you and the rest of us were making, the underlying architecture of the module still had too many problems and we were forced to branch mid-sprint in August. Our goal now is to just move those improvements, which everyone seems more or less on board with the ideas behind, back into the mainline asap so we can all start working together again.

I think we can do this quickly and with a level of openness that should be acceptable to everyone through the process Tom, Bryn, and I have been advocating.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

So, here is my plan to help make a compromise.

This weekend, I will *attempt* to take the existing 2.x branch and port over the soon to be 0.x branch (the code from the svn). If it does take too long and I am unable to make significant progress, then I will give up and we can start with the 0.x branch and bring in the 1.x changes.

The only real loss here is my time. Though I will be spending it getting to know the new code, so it's not a real loss at all. If anyone has any objections, please let me know.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

Patrick said that he might be able to help out on Sunday, so I am gonna lay out my very basic plan of porting from what I know form this thread and have looked at the code:

  1. Replace preset backend with CTools plugin
  2. Refactor main JS to use behaviors
  3. Replace behaviors with CTools plugins
  4. Make behaviors 9includes making controls behaviors)
  5. Replace Layers with CTools plugins
  6. Views refactor
  7. Make interface tweaks in Preset UI and other administration

Feel free to let me know about any other major changes.

rsoden’s picture

rsoden CreditAttribution: rsoden commented

This sounds good and the list seems to hit all the big points. One of the main sticking points we ran into during the switch to Drupal behaviors-based setup was how IE handles onload vs documentready. I'll try to be around and in drupal-geo tomorrow to help out.

Good luck! Excited to hear how things go.

tmcw’s picture

tmcw CreditAttribution: tmcw commented

Hey zzolo, I'm seeing some recent commits that are making changes that definitely aren't just merging in 2.x code but kind of rewriting the changes in significantly different ways. Will there be tickets posted about these changes? I know that one of the main objectives of this porting weekend was to increase your knowledge and familiarity with the code (and to open up the process), but it seems like it will hurt the rest of the developers if the merged code is like neither 1.x or 0.x in specifics (and there won't be any discussion about the validity of these decisions).

zzolo’s picture

zzolo CreditAttribution: zzolo commented

I am curious as to what commits or changes you are referring to? Also, it kind of still a work in progress. The UI module still needs a lot of work. In IRC if you want to discuss.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

Well, I am out of steam. It's been a long day. It definitely took some time. I would estimate between Patrick and I, it was about 20 hours total to get to where we are.

Where are we? Well, pretty much all things on that list are reasonably done. There is definitely lots of testing to do and small things here and there. I'd say the two things that are not pretty similar to the the 0.x and that should use some discussion, as well as requiring other things to fall into place:

* The JS rendering process. I added the behavior stuff, but this still needs some streamling
* The Preset UI. I made a fair amount of changes here, but it could still use some love and discussion.

2.x and 0.x are not compatible. This was never my goal. But, it's all pretty close, and a good in-between, and a much better place for us to move forward with.

The following are the list of issues that were addressed, start, closed, etc and where you can find a lot of basic information:

Some things learned in the process for me:

* The architectural changes were not that complicated. Proliferating them is the more difficult part. I think robert and co only wnet as far as they needed, which makes sense for them. There is still a lot to do in getting them everywhere.
* It was not as easy as I would have thought to slice things up into chunks and specific commits. But not horrible either. It's hard for me to not fix a lot of other things long the way, like syntax.
* I am still not convinced that porting this way would take significantly longer. I definitely still think that this process is more valuable in the long run.

As far as moving forward, it would be awesome if you guys can take a look at all this stuff and let us know what you think and maybe we can start moving forward together on the 2.x.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

Hey all. I am sure everyone is busy, but just wanted to see how everyone was feeling about this direction and what direction we should be going with the 2.x branch.

tmcw’s picture

tmcw CreditAttribution: tmcw commented

It looks like the majority of everyone's time right now is devoted to making 1.x stable (and I'm trying to help out when I can on the issue queue). The only incoming issue I can see if custom layer handling - I quickly implemented this in the 0.x branch and it's rolled out in alpha4, but my implementation is clearly quick and needs more work. Ideally layer types would be ctools plugins and would have exportable per-layer settings and form fields. Whatever happens in 6.2 I hope is building off of that kind of idea to make custom layers really nice and clean.

Ping me if you want to talk about the architecture there more, I'm on gchat as macwright, but am not on IRC when on other projects.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

Hey All,

It's been pretty quite on the OpenLayers development front. I have been getting ready to move in a couple weeks to Switzerland, and don't see myself having much time fore the next month.

But, I just had a brief thought, considering all the Drupal 7 talk. It doesn't seem too realistic to make the Drupal 6-2.x version stable before Drupal 7 is out. So, I want to throw out the idea of scratching the 6-2.x branch, and focusing on a completely new version of OpenLayers in Drupal 7. Core is pretty close to stable-ish and CTools is in the process of being upgraded (#589636: Port to Drupal 7). I think it would still be a lot like where 6-2.x is going now.

I know this negates a lot of work phayes and I did, but it kind of seems like a good way to go given the timeframe of Drupal 7 and the push to have modules ready for Drupal 7 when it comes out.

What do you think? Is this is crazy?

--
zzolo

tmcw’s picture

tmcw CreditAttribution: tmcw commented

I talked to zzolo on IRC, and I (and jerdavis) really oppose that idea, because it would tie any OpenLayers improvements to the uncertain timeline of Drupal 7. Core is 'pretty close to stable-ish' means several months (judging from all past releases). Given the fact that there would be nothing to build off for quite a while and likely no stable release for months, it just doesn't make sense. I'm going to try helping out with the 6-2 branch, because apparently zzolo and phayes ran out of extra time to devote to it, and bryn, robert, and I haven't felt comfortable pitching in on that effort.

tmcw’s picture

tmcw CreditAttribution: tmcw commented

My last few commits have been pretty big to the 2.x branch - this one finally puts a lot of the layer type-related code into that branch, and the following one tries to add a layer-type implementation for openlayers_views, but is woefully incomplete. This is a big task and a big change in methodology for layer storage & creation.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

@tmcw, thanks for kicking ass on the development stuff. Things are still in flux as far as the move, but I hope to be settled real soon and back to giving this some real time.

tmcw’s picture

tmcw CreditAttribution: tmcw commented

Today openlayers_layers got ported over to OpenLayers 2.x: Google, Yahoo, VirtualEarth, and CloudMade layers all work. In the process I added a new function called settings_form within layer_type objects. Basically, you can use options_form for per-layer settings (like TMS urls, CloudMade styles, etc.), and settings_form for layer_type settings, like Google API keys.

tmcw’s picture

tmcw CreditAttribution: tmcw commented

If you have a chance, check out this issue about switching the default map preset, I'd really appreciate some input on it.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

Tom. Good point. I have put my thoughts up. I am mostly settled in Geneva and should be able to actually start work up again on the module tomorrow or this week. Thanks again for all your hard work; I am pretty much expecting to get blown away when I finally look at the 2.x code.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

Hey all,

Since there has been a couple threads [1] [2] about what to include in this project, I would just like to take the opportunity to see what people think about what should be in this project and what the scope is. I don't think we need to define a mission statement or anything, but I think it would be helpful to keep things defined so that we are all on the same page going forward.

Personally, I see this project has having three major goals:

1) Creating an API that bridges Drupal and the OpenLayers library. The main goal of this being that a PHP array can be seamlessly translated to an OpenLayers map.
2) Offering a system to maintain and reuse maps. The preset system.
3) Integrating with Drupal's most popular inputs and outputs. This includes Nodes, Blocks, Views, CCK, etc. Hence why I support the filters integration, even though I do realize it is not all that spectacular.

Geo integration is a tough spot for me. It does not fall into my main goals. But I also like the idea of Geo and want to support Drupal as a GeoCMS. Still, Geo is alpha and not yet popular, and it is probably not in the scope of this project to support integration with it, as it holds back some overall stability.

tmcw’s picture

tmcw CreditAttribution: tmcw commented

Hey,

I agree with these points for the most part - I think that Geo integration is definitely a problem area and isn't a good idea for a project that already has serious scope creep.

The one part that I'm not quite sure about is the goal to have a PHP array -> openlayers map conversion. This can be an internal API certainly, but I don't think that creating arrays representing entire configurations is the best way to recommend use of this module, since it just isn't the best way to do it; maps are composed of parts that can be installed and uninstalled, exported, and edited on the web. Creating a clear internal API for this kind of thing is a great idea, I just don't think it should be the recommended route to building maps.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

My bad. I should have said "array/object". I agree with what you are saying. I was just trying to point out that at its basic, the goal is to create a simple way to translate a Drupal-friendly structure to an OpenLayers map.

Phayes, I know you actually use Geo on your projects and have a better understanding of Geo, so I would love to hear your thoughts on Geo integration (and overall, of course).

zzolo’s picture

zzolo CreditAttribution: zzolo commented

Git?

I fairly new to Git, but its all the rage. I just read http://more.zites.net/git_drupal_contrib_cvs which is evern more stuff on Drupal and Git. Unfortunately I am still unclear about details.

The point I have always argued for is that keeping commit history and issues on drupal.iorg is important. And I still feel that way. So, my question is:

Is there a way that we can do development with git that supports a relatively seamless way of keeping commit history and messages correctly in the drupal cvs repository?

tmcw’s picture

tmcw CreditAttribution: tmcw commented

From what I've heard, there is a potential for Drupal.org to switch to Git within the long-term future, not in the short term. In the short term, yeah, I think that there's a potential path to manage merging and committing with git and then push out decent CVS commits, but for my purposes, I think CVS isn't really killing my workflow that much, since the volume of activity and the developer pool on this project isn't gigantic.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

I am always open to bringing new people onto this project. We can always use more heads/hands to help out, and creating patches can be a real barrier to get people to actually help.

Will White has recently been providing patches for the 2.x branch, and I would like to see if anyone has any objections/concerns/questions about giving him CVS access to this project.

I have already expressed my concerns to him in an email which includes the following:

1) You are actually going to continue development into the future (past whatever you are working on now).
2) You appreciate and can actively take time to discuss your changes and ideas in the issue queue.
3) You will take time to support the module, as in help users out.

I know that none of us can really guarantee time towards this project, but it's important to have co-maintainers be people that are somewhat invested. Anyway, let me know what you think.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

I have updated the project page to better reflect the state of the branches, features, and upgrade path. I don't think it described well how the 0.x and 1.x are parallel versions and that they will both go to 2.x (there is not an upgrade path from 0.x to 1.x). The 0.x and 2.x sections could be filled out more.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

Hey All. I don't think I am going to have much time to work on this module for awhile and into the foreseeable future and just wanted to let you know. Things are real busy and I have other projects that need attention. It has been a pleasure working with everyone (through thick and thin) and I hope we will get to share a drink or ten sometime in the near future. Don't be a stranger.

tmcw’s picture

tmcw CreditAttribution: tmcw commented

Sorry to hear that, Alan, and I hope the site upgrade isn't too rocky a course. Keep us posted on how things are going with your other projects, and have a fun time in Geneva.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

Fate has other plans for me, apparently. I probably will be around a little to do bug-fixes on 2.x; I am hoping to switch to OpenLayers as part of our upgrade. And I said I would present on OpenLayers at DrupalCamp Spain. So, don't expect too much from me, but I'll help out where I can.

tmcw’s picture

tmcw CreditAttribution: tmcw commented

I just tagged an alpha1 release of the 2.x branch, which should appear soon enough today. This isn't a huge end-user release, but rather just a base so that I can get this out and tested on various platforms and be able to have all of those at a certain patchlevel, so we can drive improvements & bugfixes even faster there.

phayes’s picture

phayes CreditAttribution: phayes commented

I've just finished a cursory review of 2.x, and i've posted my comments in the issue cue. There are a lot of changes in there, overall I really like the changes, and only the removable of the event system really got my goat. Can't wait for 2.x to mature. Thanks all.

tmcw’s picture

tmcw CreditAttribution: tmcw commented

I've documented basically my mental roadmap as task tickets in the issue tracker, but am currently focusing on picking off the remaining bugs in 2.x before doing anything whiz-bang.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

tmcw's broader ideas: http://groups.drupal.org/node/48453

But concerning an actual roadmap to getting out of alpha and into beta and beyond, I just want to throw out some questions?

* Do we have major features that need to be put into 2.x?
* Do we have smaller features that need to be in?
* Now that 1.x and 2.x have strayed, are there features that should be ported that haven't?
* Do we want to ensure that all bugs are squashed or just the major ones?
* Does documentation need to be complete?
* Should we have a more complete set of tests?
* Should upgrade path from 1.x be complete for a beta?

I'll do some research on all these tonight or this weekend to get a better picture of where they are, so that we can have a more detailed roadmap moving forward.

tmcw’s picture

tmcw CreditAttribution: tmcw commented

Personally I think that we should aim for lots of bugfixes, tests, documentation, and UI improvements before adding any more features to the module. When the user base grows, feature requests will come in and we'll get a better idea of what's needed generally, without the risk of building things that only increase the codebase and thus increase potential bugs. Right now documentation and tests are admittedly minimal, and cause us to have support requests for even easy use cases.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

@tmcw, this is a good direction. I would like to do just some assessment on where things are and stuff, and will aim for this weekend to do that. This will be good for my presentation as well.

I think the only feature I would like to see before beta is the preset displays that has been loosely discussed. @tmcw mentioned that ctools allows for displays like views does. I have yet to look into this, but would like to. I feel that the double views method is too convoluted and given that we have already had a number of people being confused, I think users feel the same. But who knows, maybe the ctools displays thing would be the same. Id like to think there's a better way that doesn't require us writing a custom display system for presets.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

So, where are we and where are we going? Here are my thoughts:

In #700626: 2.x: Preset Form Improvements it looks like there was some discussion on what it is we are doing, and it seems to me fairly obvious that we don't have much of a clear, agreed-upon plan at the moment. In my opinion, we have two way of going about this: 1) a detailed roadmap, or 2) as @phayes suggested doing a data based approach.

I don't necessarily want to do the date based approach, but it does actually provide a solid way to combine disparate developing efforts. This is done with core, because with core you don't have a outline of what features will go into it, the community says "you have until this time to get what you want in". Then at that point, there are no new features and the release is simply stabilized.

I would much rather see a roadmap. We are only really 3 developers and I don't see why we can't create a collaborative document to outline our goals and approach to this module. This will both help us focus and understand what we each do and think, and will give our users a set of expectations and knowledge. Please note, that a roadmap does not need to be set in stone, it is a living document.

I have created a roadmap (wiki page) on groups.drupal.org. I think it would great to fill this in and each put in our thoughts. I think the approach should be to only add at the moment, and discuss to subtract.

To me, there is talk of adding features and stabilizing at the same time. I very much support both, but I find them kind of contradicting if we want to get out of alpha. I think we can do both, but we need to start to define what those features are now. A Beta should be a module that has all the major features in and just needs to fix some bugs and write documentation. If we don't define the roadmap now, I think we'll end up in Beta with unhappy developers and a shaky future for the module.

Like @phayes, I don't really want to even think about a 3.x version, and in reality we should not put any thought into it now. But we do need to manage scope for 2.x, and that may mean cuttting out some features and pushing them to 3.x. On the flip side, smaller features can be added to point releases as well.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

I just updated the Roadmap some.

Given how we are all able to work on this at varying degrees, I propose the following path:

1) Allow anyone to introduce any major features by March 31 and add to the Roadmap. This gives everyone a chance to review what is currently going and propose new features, as phayes and I don't have as much time towards this as tmcw at the moment.
2) In April have discussions about the new features in case anyone feels that they disagree about any of them
3) Starting in May (or sooner depending on discussions), end any new features for 2.x and get this thing to a final 2.0. Once we can define major features, it should be fairly straightfoward, to stabilize and document.

So, this is kind of a combination of date-based and roadmap. I think the problem right now is that we are not in agreement as to when we want to move out of alpha (or how to) and what features should go into the 2.x branch, and this should create a solid way to do this. I don't see too many crazy new feature proposals, but we should just make it more concrete. Of course, a roadmap is not set in stone, and we can always discuss and change things as they come up.

tmcw’s picture

tmcw CreditAttribution: tmcw commented

I oppose the idea of a checklist for new features that need to be implemented for certain releases. Currently I'm working in free time, with no time alotted by work for this, but I enjoy even the bits that I'll never use (implementing cck). But if it turns out that you and phayes continue to have very little time to devote to development of OpenLayers, but you have the majority influence in the direction of the project, then I will not be inclined to do 'assigned tasks' in my free time. I'd prefer a system in which we have easily-agreed-upon core things, like testing, bug fixes, and usability improvements, that we all work on, and then voice our opinion about the rest of it by actually writing code.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

I do agree on the point that creating a checklist of features is bad, IF no one has taken responsibility for those features.

To me, we need to achieve the balance between allowing everyone to define and try out new features, while still keeping the branch on track to releasing a stable 2.0. It is not a good idea for anyone to say we need these things, and not be able to work on them in a reasonable time, but its hard to say what is reasonable as we don't have much direction or plan for the release. But I don't think it should be expected that code is the only way to get new features to be considered. Planning goes a long way, and sometimes time just does not permit code to be written right away. Talking about a feature first can save a lot of time coding.

So I still want to push the idea of getting together all the features we want to see in the module, then we can discuss on what we can actually do individually and what we really want to do for the project overall. Right now, I just see that roadmap page as a draft where we can just put our ideal features, and then we can fix it up according to capacity. I think this is the only way we can actually make a full release of the 2.x branch, otherwise we will have no way of really knowing when we can get out of alpha.

I do think we can all agree on the importance of tests, documentation, bug-fixes, etc, and if not please discuss or add notes to the roadmap accordingly.

Overall, I have always wanted to ensure that we all have the ability to influence the project equally even if someone does not have the most time to give to the project right at this moment. Given the past year and the time that has been put in, I think @tmcw, @phayes, and I should all have an equal voice in directing the project. But we have to create some sort of limitation and plan to achieve a 2.0, IMO.

tmcw’s picture

tmcw CreditAttribution: tmcw commented

I've updated the roadmap with the 'big tasks' that seem to be the most essential for alpha releases. We need to agree on a way to deal with stability and changes - what was in the roadmap was your opinion that I clearly disagreed with. I think that, with patches and careful coding, we can keep the codebase stable from day to day. This really isn't that hard, and it's how every other big development project works (like drupal itself). Unless there is a reasonable alternative (like developing with test-branches), I think that it will be a total pain to work on this module if the code breaks often, and it will make rolling releases so much worse when there is half-broken code that not everyone knows about. Patches or bust.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

@tmcw,thanks for updating the roadmap. I do want to re-emphasize that the roadmap is just a draft right now and should not be considered something we have all agreed upon yet (I have updated the page to say this).

I do agree with what you have written in the roadmap about stability and changes. I think my point of argument was more just based on the fact that I think its unrealistic to ensure that every commit will be perfect at this point, but I don't think you are asking for that.

@phayes, if you have anything to add, that would be appreciated when you have time.

I think moving forward, if we give this draft a few more weeks to settle in and allow for changes, we can then push through to the 2.0! I should have some time to put towards some actual coding this weekend or end of week!

tmcw’s picture

tmcw CreditAttribution: tmcw commented

As we talked about a little while ago, I'm going to roll an alpha3 release of the 2.x branch, so that we have a tagged release with all of the post-alpha2 bugfixes before taking the plunge of the change in hooks (#723530: Change name of Style and Layer hooks)

Before this tag, I want to fix

tmcw’s picture

tmcw CreditAttribution: tmcw commented

I've rolled an alpha3 release which is the new recommended release for the 2.x branch.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

@tmcw great work. Just a quick thought. I have had a number of questions about what is really the recommended release. It's a tough decision at this point and still based on opinion and circumstance. But, I wanted to suggest we change the recommended release to 2.x when it goes to beta, as I think it will be stable enough for recommendation at that point (still mostly is now). I added it to the RoadMap.

tmcw’s picture

tmcw CreditAttribution: tmcw commented

I've released alpha4 of 2.x. I'm interested in making it the recommended release; there are no bugs outside of OpenLayers CCK, and about as many bugs as 1.x. Also, since the upgrade path is rocky and lots of tickets have people who really would benefit from upgrading to 2.x, recommending 1.x is hurting the user experience of the module.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

I don't see how "recommending 1.x is hurting the user experience of the module", or what that means really?

tmcw’s picture

tmcw CreditAttribution: tmcw commented

I think that recommending 1.x results in a lot of people installing 1.x without reading what the version entail, and a whole lot of those users post issue tracker queries that basically ask about things in 2.x - speed / caching, more flexibility as far as styling, layers, views, layer types, etc. And then when they need to switch to 2.x, it's that rough upgrade process. This seems to be a negative thing.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

Well, I think a "whole lot" is inaccurate, but overall it makes sense. Go for it.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

I made a slight mistake and hit the wrong bash command. Basically I committed something I wanted to, but with the previous message. It was a small commit to just address some strict warnings. If anyone think this is really bad, I can either try to contact Drupal infrastructure, or revert it, commit again, then commit the changes with the correct message.

http://drupal.org/cvs?commit=360320

tmcw’s picture

tmcw CreditAttribution: tmcw commented

Can you revert? These changes seem a little weird anyway - dropping in defaults for important settings at the last minute (in a theme function) is kind of a bad idea, and the width: auto; setting on map divs is dangerous (when maps are hidden + auto width, OpenLayers has a very serious bug until post-2.8). And then in openlayers.module this introduces the practice of type-checking (making variables arrays when they are previously null) while you're calling the function... even though the same kind of task is performed a different way about four lines up.

And the double-check for arrays in openlayers_cck.module would be valid in Javascript, but isn't necessary at all in PHP; the isset construct won't throw an error if you run it on nested arrays with higher-up keys unset.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

Reverted:
http://drupal.org/cvs?commit=360344

Started new ticket:
#784092: More in-depth/sane structure checking is needed for map rendering

zzolo’s picture

zzolo CreditAttribution: zzolo commented

Just a quick note to all committers. It would be really helpful to use the standard commit message for commits:

#123456 by username: This commit did this etc.

This means that we should start putting in our own usernames for commits that we make write ourselves, as we are getting more patch contributions.

The main reason I mention this is that creating a meaningful release (and managing a CHANGELOG.txt) become much easier if this standard is used. It took a while to format this release so that it was consistent and helpful. Also, if this convention is used, we can automate the release node with this script.

For reference, please see: http://drupal.org/node/52287

Thank you.

tmcw’s picture

tmcw CreditAttribution: tmcw commented

Since we are moving up in alpha numbers and working on getting possibly a beta eventually, I wanted to point to the issues I'm seeing as necessary for a beta release - not an exclusive list, but basically things that are very important that just haven't gotten done.

strk’s picture

strk CreditAttribution: strk commented

May I suggest cluster support in popups/tooltip to also be on this list ?
http://drupal.org/node/805264#comment-3139610

If we allow attaching clustering behaviour, that shouldn't break popups/tooltips.

strk’s picture

strk CreditAttribution: strk commented

Also, I'd add: fix PHP warnings. I have one on preset editing, which didn't file a ticket for (that I remember)

strk’s picture

strk CreditAttribution: strk commented

Filed the warning I was talking about in comment #69 : http://drupal.org/node/841354

zzolo’s picture

zzolo CreditAttribution: zzolo commented

Hi @strk, your interest is really great, but this specific issue is for high level discussion on the development process of the 2.x branch. Specific issues should be brought up in other tickets.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

So, to help organize the push for 2.x beta now that we have given sufficient time to allow for any new large features, this is how I would like to organize tickets:

* Bugs as critical
* Features/Tasks with the "beta blocker" tag

zzolo’s picture

zzolo CreditAttribution: zzolo commented

OK, so I marked all things as I see fit (and the issues @tmcw mentioned above). This makes 4 bugs and 10 features/tasks.

tmcw’s picture

tmcw CreditAttribution: tmcw commented

I think we're due for a new release around the beginning of September, given that there have been numerous fixes and changes since te last release at the end of June. This'd be an alpha9 release.

Remon’s picture

Remon CreditAttribution: Remon commented

subscribe

zzolo’s picture

zzolo CreditAttribution: zzolo commented

Just releases ALPHA9! Awesome work, everyone! There are lots of good things in there.
http://drupal.org/node/909144

Not that it isn't obvious, but I think we should aim for the next release to be the first beta, then feature freeze and try to move to 2.0 quickly. (then add features)

Just to reiterate, it is extremely helpful to format all commits like the following (include your own username). It makes creating the HTML for the release node a lot easier and more meaningful for users.

#123456 by username and otheruser: This commit did this etc.

Or, if there is no issue associated with it:

by username: This commit did this etc.

As far as descriptions in commits. It is important to make it a meaningful message, as these releases become a change-log and a place for users to easily understand the changes in each release. So, commits should be a complete idea that enough of the meaning of the change can be derived from it. (Don't get me wrong, I am not that great at it, but we should try).

Also, commits like the following are extremely unhelpful. I am assuming these are directed at my comments, which is fine, but lets discuss it, not just put messages in commits.

"And why would anyone make comments that just repeat what's said in code right below them?"
"Enough with comments that just reiterate the code below them. jsdoc is good, this isn't"

tmcw’s picture

tmcw CreditAttribution: tmcw commented

ALPHA9 is broken: the last-minute backwards compatibility code that was put in for OpenLayers Data views was broken, and breaks all OpenLayers Data views. I'll make a release tomorrow. In the future, commits need to be thoroughly tested if they directly precede releases.

zzolo’s picture

zzolo CreditAttribution: zzolo commented

Please note that a new release has been made, but should be improved for the next release.
See: http://drupal.org/node/895082

(Also note that the code put in did not break existing Views, while the original/reverted code did)

zzolo’s picture

zzolo CreditAttribution: zzolo commented
Status: Active » Closed (fixed)

So, I think 6.x-2.x development should probably be stopped besides backwards compatibility and if anyone wants to back port new features. Closing. Shoudl probably have a discussion around 7.x-2.x development soon.

Awesome work, everyone!!