We need to create tests.

Comments

zzolo’s picture

Title: Tests » Create SimpleTests for 1.0
Version: » 6.x-1.x-dev

I would not feel comfortable releasing a 1.0 without a few SimpleTests. The JS stuff becomes a little hard to test, but there is still a fair amount of PHP side we can test.

zzolo’s picture

I have create the first SimpleTest for the module: http://drupal.org/cvs?commit=265428

This includes a test module, which is hidden from the interface with a "Hidden = TRUE" in the .info file. At the moment, it simply provides a preset, and the test looks for it in the interface.

You can technically erase the "Hidden = TRUE" line and install the module as it is. This might be a good place to set up some examples to manually test, since there is a fair amount we cannot test with the SimpleTest framework, as it is more for the PHP side.

SimpleTest documentation: http://drupal.org/node/395012

zzolo’s picture

Title: Create SimpleTests for 1.0 » 2.x: SimpleTests!
Version: 6.x-1.x-dev » 6.x-2.x-dev

Begrudgingly conceding for 1.x (since 2.x has already started).

I implore that everyone start to add tests. Either actual tests or just adding pages/or maps to that module that will make it a lot easier to go to a single place to test things. There's no reason we can't help save ourselves sometime.

tmcw’s picture

So test should include something like

  • Adding a typical preset, testing for the fact that the map works, editing it, testing the same thing
  • Editing layers, testing that they work
  • Editing styles, same deal.
  • Would make sense to have this test the code with OpenLayers 2.8 and dev, and onsite/offsite javascript

We should have some tests within the module itself that simply ensure that the map processing functions behave the same way, as to ensure that the API is stable when we want it to be.

Probably something like Selenium is best for writing the Javascript-heavy tests, because obviously Drupal's testing framework falls short there. Also, we could test the javascript code with something like QUnit to make sure that its behavior is correct.

tmcw’s picture

Refreshed the tests already included, and added a new test that creates a new style and checks to see that it was properly saved. Tests for layers and presets will be the same pattern.

http://drupal.org/cvs?commit=326350

tmcw’s picture

As of today's commit, there are 165 SimpleTests which all pass, and every layer type is lightly tested. What is the standard by which we're going to mark this as 'complete'?

zzolo’s picture

Awesome work, @tmcw. As far as my initial vision, it was just to get some basic tests in. Because this is all JS heavy, we can't really test everything with SimpleTests. My thought would be to ensure that all parts of the PHP side of the API has tests and the other large parts. So, this would include the following:

* OpenLayers settings (including JS and preset default)
* Layer types work via the API
* Layers work via the API
* Styles work via the API
* Behaviors work via the API
* Map presets work via the API
* The main Map Preset UI
* Views functionality
* CCK Functionality
* Filter Functionality

I don't feel like every instance of these objects, need to be tested, just the core functionality part. Also, I have not looked at the tests yet, so you could have all this done.

tmcw’s picture

Priority: Normal » Critical
tmcw’s picture

If you could clarify:

* Layer types work via the API
* Layers work via the API
* Styles work via the API
* Behaviors work via the API
* Map presets work via the API

What this really means in practice, then I'd appreciate it; I'm testing most of this stuff via the ui, but am not sure what exactly is the API anymore; this module no longer includes a lot of array transformation, etc., so there's a bit less to test there. Tests for the load functions I guess?

On the javascript side, I've tried decomposing openlayers.js into more independent functions and then writing tests for those, in test/js/index.html, there's a qunit test set up.

Updating this list with the one that's already in:

* OpenLayers settings (preset default)
* Layer types work via the API
* Layers work via the API
* Styles work via the API
* Behaviors work via the API
* Map presets work via the API
* The main Map Preset UI
* Views functionality
* CCK Functionality
* Filter Functionality

zzolo’s picture

Currently when running the tests (which are awesome, btw), I get 136 exceptions. These are not failures, but PHP warnings for such things as undefined variables/indexes, etc.

Please note that when one installs SimpleTest and patches core, it sets the error warning level higher so that all these issues show up in the interface as well.

Some examples:

Undefined offset: 5	Notice	menu.inc	576	_menu_translate()
Undefined property: openlayers_layer_type_osm::$data	Notice	openlayers.module	693	openlayers_layer_type->__construct()
Undefined index: resolutions	Notice	osm.inc	57	openlayers_layer_type_osm->options_form()

I don't think these things are critical, but they should be resolved at some point.

tmcw’s picture

Ah, now I remember - my PHP error level had to be bumped down, because I use 5.3, and Drupal is not actually really supporting the stable release of PHP yet. I should bump it up again to start chipping away at these.

tmcw’s picture

Committed a minimal test suite for the OpenLayers CCK module

http://drupal.org/cvs?commit=340244

tmcw’s picture

This commit reduces the warnings by quite a bit, but it's not zero yet. I'm getting an undefined index in the menu system for which I don't have a clear answer yet.

http://drupal.org/cvs?commit=340658

zzolo’s picture

* The common menu title warning is a known bug in Drupal: #676010: diff_menu() issue - undefined offset in includes/menu.inc on line 576 function _menu_translate()
* I added a test for Filters.
* Fixed some minor exceptions.
* There are a few exceptions popping up in the tests in the CCK test, but after spending a little time on it, I was still not able to figure out why.

zzolo’s picture

An update. The following parts that should have tests:

* Views functionality
* Presets
* JS Tests (haven't broken this down yet)

I have never used qUnit; @tmcw, if you have any links or tips that might help someone get up to speed quicker, that would be very appreciated.

zzolo’s picture

Issue tags: +beta blocker

tagged as beta blocker

zzolo’s picture

Status: Active » Closed (cannot reproduce)

Yes, tests are good. But as the number of people working on this module are pretty low now, tests are still important, but this issue is just not useful.