I'm not sure if this issue lies within the shp2sql code or my shapefile itself, but I get an undefined geotype error when importing a shapefile containing linestrings. This shapefile was created using arcgis, and also renders fine in qgis (and I tried a version of the file which was re-exported from qgis). Basically, the first 2 lines are read with the correct geo type index (3), but then the bytes are somehow shifted out of order, and I get a large negative number (-1806811483) for the geotype index.

I'm merely posting this here to see if other people have run into the same issue; I'm not really sure which direction to continue investigating in at this point.

Also, I am unfortunately unable to post my shapefile due to NDA restrictions.

CommentFileSizeAuthor
#5 shp2sql.inc_.txt10.43 KBapolzon

Comments

allie micka’s picture

I'm thinking that this may be related to http://drupal.org/node/429256 - Bec patched the code in geo.wkb.inc , but the same treatment should probably be applied to the shp2sql code.

phayes’s picture

I ran into this issue before and can confirm that this is an issue with shapefile parsing in shp2sql.inc

phayes’s picture

Category: support » bug
apolzon’s picture

Version: 6.x-1.0-alpha1 » 6.x-1.0-alpha2

This issue still exists in 6.x-1.0-alpha2. I looked into applying the wkb changes to the shp2sql file, but my attempts all ended in failure; admittedly I don't know a ton about how to properly use unpack and byte_order.

apolzon’s picture

Status: Needs review » Active
StatusFileSize
new10.43 KB

A coworker and I (scottschulthess, at OHO Interactive) were able to update shp2sql.inc and successfully import both polylines and polygons. The bulk of the clues we found were from this article: http://www.easywms.com/easywms/?q=zh-hant/node/78
Here are the things I believe we changed:

1. _shp_get_linestring was not pulling the numparts (4 bytes) out of the file, so the bytes were off when loading a polyline into the database. This was causing the geo_type to come up as unsupported. (I'm not sure if a polyline/linestring can ever have more than one numpart, so right now we're assuming its only one)
2. The unpack parameter 'i' was changed to 'V'. 'i' is machine-dependent, but 'V' will always read in little endian byte-order.
3. in _shp_get_linestring, the wkt is built but never added to the $data array which is returned. This results in a null geo object.
4. There seems to be some issue with the way the unpack method is used in a couple places. in _shp_get_linestring, it pulls 4 8-byte elements from the file (as one 32 byte segment). When we changed this to 4 separate 8-byte reads, we were able to successfully import. I don't really understand the difference between what we wrote and what was already there, so I've pasted it here in case anyone has any ideas:

Original: (broken)
  $data = array(
    'bbox'  => unpack('dmin_x/dmin_y/dmax_x/dmax_y', zip_entry_read($fp, 32)),
    'point_count' => current(unpack('i', zip_entry_read($fp, 4))),
  );
Original (patched): (also broken)
  $data = array(
    'bbox'  => unpack('dmin_x/dmin_y/dmax_x/dmax_y', zip_entry_read($fp, 32)),
    'part_count' => current(unpack('V', zip_entry_read($fp, 4))),
    'point_count' => current(unpack('V', zip_entry_read($fp, 4))),
  );
New code: (working)
  $min_x = loadData('d', zip_entry_read($fp, 8));
  $min_y = loadData('d', zip_entry_read($fp, 8));
  $max_x = loadData('d', zip_entry_read($fp, 8));
  $max_y = loadData('d', zip_entry_read($fp, 8));
  $numparts = loadData('V', zip_entry_read($fp, 4));
  $numpoints= loadData('V', zip_entry_read($fp, 4));
loadData func:
function loadData($format, $data) {
  if (!$data) return $data;
  $tmp = unpack($format, $data);
  return current($tmp);
}

I've uploaded our modified shp2sql.inc and would be happy to commit it to CVS if the maintainer approves.

apolzon’s picture

Version: 6.x-1.0-alpha2 »

I've also just realized that 'd' is also a machine-dependent format specifier. If anyone is on a different architecture (powerpc most likely) and having issues, I'd try using 'V' in place of 'd'.

apolzon’s picture

Status: Active » Needs review

Changing status to needs review

allie micka’s picture

Status: Active » Needs review

There are now 3 issues that suggest similar problems. Please try and see if there is already a similar issue before filing a new one - that saves everyone a lot of time!

alpozon, thanks for your patches and followup! I am referring #483094: Geospatial data - Cannot import Shapefiles and #394678: Geospatial Data - Shapefile import here. Technically, _this_ issue is probably a duplicate of those, but I have selected the issue with the most input to be the main one.

Thanks!

allie micka’s picture

Status: Needs review » Fixed

Thanks apolzon,

After revisiting the specfications and fixing up some obvious code issues, I was able to get the linestring data working properly. I suspect that Endian-ness is still an issue. It may be necessary to revisit bec's work in #429256 for this functionality.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

khalligan’s picture

I am having this issue as well. I downloaded the modified version of shp2sql.inc from apolzon but this did not solve the problem. I am receiving the following error:

"Unsupported geo type found in file."

When I try to upload a shapefile with polyline data type created in ArcGIS. Point and Polygon shapefiles work fine. I am running Ubuntu 8.10 (intrepid) on an Intel based virtual server, so I don't believe it is an endian issue.

blattmann’s picture

Version: » 6.x-1.x-dev
Status: Closed (fixed) » Needs review

The fix suggested in #5 doesn't seem to make a difference for me, and uploading a polyline continue to result in the error Unsupported geo type found in file.

Any ideas?