I'm trying to implement http://gaganb.wordpress.com/2012/02/15/tile-feature-strategy-in-openlayers/ to prevent our Drupal site to die when requesting a BBOX containing 60K points.
Using tiles would allow Drupal views to cache data better as the BBOX is user dependent where tiles are afaik tile server dependent.
We should add a new argument handlers similar to the BBOX having the zoom level.
What do you think?
See related #1877072: Allow BBOX in other display format. and #1839554: BBox argument handler fixes
| Comment | File | Size | Author |
|---|---|---|---|
| #7 | POI BBOX GEO json.png | 185.63 KB | clemens.tolboom |
| #6 | views-geojson-1877902-6.patch | 5.06 KB | clemens.tolboom |
| #4 | views-geojson-1877902-4.patch | 3.32 KB | clemens.tolboom |
| #3 | views-geojson-1877902-3-bad-solution.patch | 6.34 KB | clemens.tolboom |
Comments
Comment #1
dasjoif i understand correctly, a tile based argument would imply fixed bounding boxes (based on tiles) and therefore enhance caching over the standard, user-driven flexible bounding boxes. also, you are expecting the tiles to be smaller which leads to multiple requests but they should be a lot cheaper as they can be efficiently cached.
this is an interesting approach. in order to bypass the multiple-requests disadvantage, you could also accept a standard bounding box + zoom level argument and issue sub-requests on the server side. depending if you want to trade network traffic for responsiveness.
we have also a similar feature request in the leaflet_geojson queue: #1868982: Add bbox strategy variations for better performance and caching
Comment #2
clemens.tolboomFollowing the TFS strategy the generated URL like http://.../TFS.php?z=4&x=23&y=10
I need to find the meaning of xyz ... there is no stragety http://dev.openlayers.org/releases/OpenLayers-2.12/doc/apidocs/files/Ope...
But http://dev.openlayers.org/releases/OpenLayers-2.12/doc/apidocs/files/Ope... has the z/x/y.png construct.
Do you know how to calculate the bounding box from z/x/y ?
I hope to add the argument handler today/tomorrow.
Comment #3
clemens.tolboomMy initial thought were to add a new argument handler but I guess BBOX is similar to tile that much we better could merge these.
Here is my bad patch first as I guess it points some puzzles.
- the new handler doesn't show up: I guess I missed a step
- debugging by using http://api.drupal.org/api/views/views.api.php/function/hook_views_plugin... (hook_views_plugins_alter) reveils a weird argument 'default handler': our querystring handler from views_geojson_views_plugins ... I'm not sure why we provide a default handler
Next comment will contain a changed BBOX argument handler.
(put this on needs review as I'd love to know what these two points mean)
Comment #4
clemens.tolboomIt removed the bbox logic from the ->query
This patch changes views_geojson_bbox_argument by adding a few more settings.
It has some todos and it should have a conditional field set for the zxy selection.
Please give it a quick review :p
Comment #5
clemens.tolboomThe algorithm zxy is I guess done here in reverse.
https://github.com/openlayers/openlayers/blob/master/lib/OpenLayers/Laye...
This uses some extra settings which we probably need too
Furthermore the x,y are left,top oriented not center.
Comment #6
clemens.tolboomNew patch has
- some reordering
- options_from to choose what strategy to use
As I'm struggling with the client side for tiles (http://gaganb.wordpress.com/2012/02/15/tile-feature-strategy-in-openlayers/) seems to have bug(s) :( I still haven't implemented the inverse of #5.
Comment #7
clemens.tolboomReattaching image
Comment #8
clemens.tolboomShould we focus on this feature only for GeoJSON or a new project like views_geo_filter? See my comment #2 in #1877072: Allow BBOX in other display format..
Comment #9
dasjoi'm a bit skeptical regarding a zxy filter/argument. from my understanding it allows you to split one big request into several parallelized sub-requests for each tile, right? you really need to do caching right then because otherwise the server-load would be immense.
implementation-wise, i'd see this a two different argument handlers, where zxy might build upon / extend the bbox argument handler.
Comment #10
clemens.tolboomDoing a views caching is possible with ZXY but almost futile for BBOX when users are allowed to drag the maps.
I must investigate on the views caching algorithm whether it is possible at all as ZXY has one argument handler but 3 arguments :(
But we probably take varnish as the cache engine processing ZXY request.
I'm not sure how we could sub class a default argument handler. That requires a module? A did not find docs on it :(
Comment #11
dasjovarnish sounds reasonable.
regarding sub classing - that would just mean leveraging PHP Object Inheritance:
views_geojson_bbox_argument extends views_handler_argumentnow you might want to create a
views_geojson_xyz_argument extends views_geojson_bbox_argumentmy idea was that the xyz handler builds upon the bbox handler, thus it would reuse all code that can be applied for both cases but overload functionality that has to be implemented differently from / in addition to the base bbox handler. this of course might lead into refactoring the bbox handler a bit what we might want to to anyways as of #1839554: BBox argument handler fixes. that's just an idea and maybe we can get feedback from the maintainer, but jeffschuler seems not be having much resources available for views_geojson atm.
Comment #12
clemens.tolboomBut I still don't know how #10
should be handled. I'm tempted to create the views_geo_filters project to provide all views BBOX and/or ZXY filtering capabilities. Maybe I'm on the wrong thought track? It also depends on my customer mapping wishes with aren't still that clear :(
Comment #13
dasjoregarding sub classing, i tried to explain that in #11. it's just the way that i would imagine it implementation-wise, as zxy filtering seems to build upon the concepts that the bbox filtering provides -> a zxy filter internally translates to a bbox filter.
sure you can create a separate views_geo_filters project in order to decouple the filtering capabilities from views_geojson. see #1877072: Allow BBOX in other display format.
Comment #14
clemens.tolboomThis http://wiki.osgeo.org/wiki/WMS_Tiling_Client_Recommendation states one should publish capabilities using ZXY ... that is probably true for BBOX too?
Comment #15
dasjouh, well exposing capabilities is yet another nice to have. what benefit do you expect? WMS is about XML data, while GeoJSON is about well, JSON. i'd recommend focusing on the most important issues at first hand :)
Comment #16
clemens.tolboomThere is code in Openlayers.Map.js doing it another way. Maybe that helps
Comment #17
clemens.tolboomSo we want to reduce our geojson request by transforming request like
http://.../geo/feeds/restaurants?bbox=4.6663614729841,52.285497251916,5.1078745270159,52.453194957342
which is not cachable into several ZXY requests based which are.
http://.../geo/feeds/restaurants/?z=12&x=?&y=?
But the algorithm should use 'resolution' similar to #16?
I just realize I haven't tested the layer request yet :( as it prob looks like
http://.../geo/feeds/restaurants/12/4/7 instead of it's bounding box sibling.
Comment #18
clemens.tolboomThis is my reference http://dev.openlayers.org/docs/files/OpenLayers/Layer/XYZ-js.html where the request going to Drupal is constructed. That is the getBounds() is converted to one or more ZXY request.
On the receiving part (views_geojson) we need to translate the Z/X/Y arguments back to bounding box arguments. As the combination of Z/X/Y is limited we can enable caching for it which would boost performance.
So I expect a function in PHP like this.
Comment #19
dasjoon a general note, it might make sense to refactor the argument handling to avoid too much hard-codings.
geofield has done so #1469956: [Meta] - Improve Views-powered Geofield proximity searches and there are some discussions regarding search api location #1798168-30: Major module modifications
Comment #20
polGuys, could you give me an update on this ?