Automate testing as much as possible

I had a thought about how to make an 8.x test: Make a test module that would include configuration for a node type with a text field, with a view mode having the maps formatter on it.

Then in the test module you could verify that the maps formatter is outputting the right thing. Then update the config for that view mode to tweak the settings, and verify again. Etc.

I think this would be the easiest way to set up the content type and view mode (using config/install directory exported config rather than writing out the code to add a content type, add a field, etc.).

I had some thoughts about this recently... Given that the Simpletest UI exists (although deprecated probably) in both Drupal 7 and 8 (for now at least), we could potentially write a mostly automated test for both 7 and 8. It could run through the steps in #2543032: Steps to manually test the module., and then to test a patch, we would run the test in Simpletest UI with verbose output, and take a look at the verbose output at a few key points corresponding to what we do in our current manual process. We could output something into the verbose record like "LOOK AT THIS ONE" at those points to make them easier to find.

I don't think we can have a fully automated test very easily, because this is Google Maps, and we kind of need to look at the screen and verify that the output is what we expect... but we could at least make a partially-automated test, and take some of the human variability out of the testing process by automating it, and make it easier so the procedure is just "run the test and look at 5 verbose output screens".

I think I'll assign this issue to myself to explore this idea's feasibility... in a week or two I should have some time.

My thought regarding the Simpletest UI is that part of the test is "Look at the resulting web page and verify it is showing the right map". Using the Simpletest UI, we can simply click the link for the verbose output for those pages, and look at them. The UI is not going away for Drupal 8, and I imagine if it is scrapped in 9 (whenever that comes), someone will possibly spin it off as a contrib module.

I went ahead and wrote/committed a test for the Drupal 7 version of the module. Besides making the output page where you can verify the maps are working correctly, it also automates testing that:
- The formatter can be applied to a text field.
- The settings form allows you to display the text and set the link text.
Obviously, it can't really test that the maps and links to Google actually work well in terms of displaying maps, but at least it does some testing, and if you run the test in your own browser there is just one link to click to see the results at the end, and I also put the instructions for what to look for in the page, as verbose output.

So, I think this is a success. I'll port it to Drupal 8 sometime... if I have time today I'll do it today; if not, it might be a few weeks.

Note that you need to edit the test file locally and add a static maps API key in order to run the test and see the static maps.

Well, that's annoying. The test passes locally, but fails on the test bot in D7. Weird, the way it's failing it is as if the field_ui module is not there. I'm not going to attempt to fix it right now though. The test is still useful for local testing of patches.

The D8 port turned out to be pretty quick. I'll go ahead and commit it, since it doesn't affect the operation of the module itself. Try it out sometime if you like.

To use:

a) Enable the Testing module (simpletest in Drush, or Testing on the Extend page).
b) Edit the src/Tests/SimpleGmapTest.php file -- add an API key around line 24. (Or don't, but the static maps will not work.)
c) Go to admin/config/development/testing. [Verify settings once: you'll want to make sure that "Provide verbose information..." is checked.]
d) Run the SimpleGmapTest class from that page.
e) When the test completes (it should pass), expand the output. Scroll to near the bottom. You'll see a line that says:

"Open the previous link. In the first field, verify: (1) Both the static ...."

f) After reading the 3 instructions for what to check, click on the previous "Verbose output" link. Note: the page title is at the bottom of the page. But you should be able to see the 3 map fields and check the output as described.

Committing now... and I'll see if the test bot can run this one. ?!?

The Drupal 8 test passes on the testbot. So, I've set it up to run weekly (in case Drupal 8.x changes under us), plus on commit and for issues (any patch set to Needs Review will run the test, which will at least verify it didn't totally break the module).