Support for Asynchronous Tracking

This sounds like a must have to improve page load times, which are by the way becoming a new SEO ranking factor:
http://analytics.blogspot.com/2009/12/google-analytics-launches-asynchro...

More on this by Steve Souders: http://www.stevesouders.com/blog/2009/12/01/google-analytics-goes-async/.

I have changed the code on my non-drupal sites already and the pages load about 3-times as fast! With the current implementation, I can see that the analytics code slows my drupal sites down by about 2-3 seconds per page load. It would be great do this with this wonderful module. Seconded.

Seems like an easy patch for new installs. What's involved in updating existing installations?

Cool stuff... we should re-introduce the radio we had in 5.x for switching the tracker types and we need a special googleanalytics.push.js for the new code. This beta code should not enabled by default.

Great news. Subscribing.
I also changed the code manually and it does look much faster now.

subscribing. can help with testing.

This is needed ASAP as Google stated clearly it will rank search results with page speed:
http://searchengineland.com/google-releases-page-speed-report-in-webmast...
http://outspokenmedia.com/seo/seos-guide-to-page-speed/

No, but you could provide a patch.

I will give this a try now.

Someone of you aware what will happens with the adsense integration? Maybe they better integrate it as in past?

  drupal_add_js('window.google_analytics_uacct = ' . drupal_to_js($id) . ';', 'inline', 'header');

Subscribe

subscribing

subscribing

subscribing

subscribing

I took a quick stab at this but it kinda gets sticky around the fact that FF 3.6 is the only browser that supports this. So we'll need to serve things up both the old way and the new way and do browser side detection to trigger which one runs. Seems like this might not be something ideal to support till more browsers actually support it(or at least one is out of beta).

That said testing some patches might not be bad since someday this could be a pretty cool feature to support in the future. Very interested in seeing it happen.

I took a quick stab at this but it kinda gets sticky around the fact that FF 3.6 is the only browser that supports this.

This is not the way I understand the documentation:

most browsers will load the tracking code in parallel with other scripts on the page, thus reducing the web page load time. Note here the forward-looking use of the new HTML5 "async" attribute in this part of the snippet. While it creates the same effect as adding a

element to the DOM, it officially tells browsers that this script can be loaded asynchronously. Firefox 3.6 is the first browser to officially offer support for this new feature.

Subscribing.

yhager is correct: it will work now, in all browsers, asynchronously. The 'async' attribute is just there for the future and will improve performance even further. neclimdul, see this post for a clear explanation: http://www.stevesouders.com/blog/2009/12/01/google-analytics-goes-async/.

So can anyone post a patch? When will this thing be committed?

I'll continue working on my patch then if it works fine. One thing to consider, this is a fundamental change to the way things are added to the javascript object so it might be well served in a different branch. I think this is echoing hass' feelings in #12

subscribe

subscribe

I think there is an argument here for changing the way drupal loads javascript in general. You can load all javascript asynchronously - not just analytics, using pretty much exactly the same code.

The only catch is if something like an onClick gets called before the javascript has finished loading, in which case it obviously can't work...

Anybody want to rewrite the way $scripts works feel free!

@chrism2671: Good idea. Not without problems, but sounds like an interesting path to explore.
However, I recommend to decouple the general solution from moving analytics by itself to be asynchronous.

Let's not divert the issue here - open another one on Drupal isssue queue - I am sure it will draw a lot of opinions from the community.

I think the general solution to Drupal JS loading could use this GA fix as a model... So let's make this one work first.

Here's a quick seems to be working implementation.

• The link tracking stuff does not work and will show drupal_set_messages on every page when its trying to be added.
• If you've got previous before and after scripts, its going to blow up. I made no effort to convert them and I'm not sure there's a good way to. Also, I'm not sure the after is that useful.
• Docs are probably all pointing to the wrong place with no reference to asynchronous api's.
• I'm not sure about the segmentation support and its not tested. It might work? I couldn't find any examples online.
• Any number of other bugs... Its really not well tested.

re @hass in #12. I see what you mean now but I think that's completely separate from the analytical code other than providing the value in the global for adsense to use. In that respect I think we're fine rolling forward without worrying about it. I've been wrong before (in this thread even *grin*).

Hi neclimdul!

I gave it a try, and I get the following message:

Not sure link settings actually work yet. They probably do not.

Request:
http://code.google.com/apis/analytics/docs/tracking/asyncUsageGuide.html...

Could you please add support to this function too!

Thank you!

*Points to first bullet and issue status*

Hi,

is there any chance we'll see this function in the official release soon? Although this is a great module, using new and faster code seems like an attractive alternative...

subscribe

I subscribe the idea, also should be great the options to add the code of this web site to improve the google's report:
http://briancray.com/2009/12/29/understanding-user-behavior-google-analy...

Oskar

Here is my implementation based on neclimdul's patch.

• The link tracking and the segmentation support does work.
• You can turn on/off asynchronous tracking by configuring JavaScript scope in admin/settings/googleanalytics.

Shimizu

:-.
Couple questions.
Why wouldn't I want the asynchronous code to also being in the footer?
What if I want the old code in the header?
Why can't we use any of the other features of the module like local file caching.
Honestly, If we're going to support both, I'd say it'd be better so separate some of the logic maybe into separate functions rather than just kludging it together with if's. It'll make the flow a little better. But really it's going to be kinda confusing since $after and $before code is going to be between the to. People might think they can just switch back and forth and try both but wouldn't really be correct.

Nice use of function scope in the javascript though.

I actually had some additional work on the patch that I was just watching for a couple days to make sure the Analytics results looked right. I've gone ahead and attached it here as it looks right so far. Link tracking should be working. No dual support like mak00s but other then that they're pretty much identical.

Hi neclimdul,

I have had some error during patch:

www analytics_mod # patch -p0 googleanalytics.js googleanalytics.js.diff
patching file googleanalytics.js
Hunk #1 FAILED at 15.
1 out of 1 hunk FAILED -- saving rejects to file googleanalytics.js.rej

googleanalytics.js.rej:

***************
*** 15,46 ****
        // Expression to check for download links.
        var isDownload = new RegExp("\\.(" + ga.trackDownloadExtensions + ")$", "i");

-       try {
-         // Is the clicked URL internal?
-         if (isInternal.test(this.href)) {
-           // Is download tracking activated and the file extension configured for download tracking?
-           if (ga.trackDownload && isDownload.test(this.href)) {
-             // Download link clicked.
-             var extension = isDownload.exec(this.href);
-             pageTracker._trackEvent("Downloads", extension[1].toUpperCase(), this.href.replace(isInternal, ''));
-           }
-           else if (isInternalSpecial.test(this.href)) {
-             // Keep the internal URL for Google Analytics website overlay intact.
-             pageTracker._trackPageview(this.href.replace(isInternal, ''));
-           }
          }
-         else {
-           if (ga.trackMailto && $(this).is("a[href^=mailto:],area[href^=mailto:]")) {
-             // Mailto link clicked.
-             pageTracker._trackEvent("Mails", "Click", this.href.substring(7));
-           }
-           else if (ga.trackOutgoing) {
-             // External link clicked.
-             pageTracker._trackEvent("Outgoing links", "Click", this.href);
-           }
          }
-       } catch(err) {}
-
      });
    });
  });
--- 15,43 ----
        // Expression to check for download links.
        var isDownload = new RegExp("\\.(" + ga.trackDownloadExtensions + ")$", "i");

+       // Is the clicked URL internal?
+       if (isInternal.test(this.href)) {
+         // Is download tracking activated and the file extension configured for download tracking?
+         if (ga.trackDownload && isDownload.test(this.href)) {
+           // Download link clicked.
+           var extension = isDownload.exec(this.href);
+           _gaq.push(["_trackEvent", "Downloads", extension[1].toUpperCase(), this.href.replace(isInternal, '')]);
          }
+         else if (isInternalSpecial.test(this.href)) {
+           // Keep the internal URL for Google Analytics website overlay intact.
+           _gaq.push(["_trackPageview", this.href.replace(isInternal, '')]);
          }
+       }
+       else {
+         if (ga.trackMailto && $(this).is("a[href^=mailto:],area[href^=mailto:]")) {
+           // Mailto link clicked.
+           _gaq.push(["_trackEvent", "Mails", "Click", this.href.substring(7)]);
+         }
+         else if (ga.trackOutgoing) {
+           // External link clicked.
+           _gaq.push(["_trackEvent", "Outgoing links", "Click", this.href]);
+         }
+       }
      });
    });
  });

I'm using the latest stable of google_analytics modul, and of corse I'm not tried to patch an already patched file.

Its late so I haven't tested but the patch was against -dev. I never make patches against a stable release since that really doesn't make much sense. The maintainer isn't going to commit to the stable release but to the current working version :)

Thank you for your answer. As you suggested, I was able to apply the patch to the latest 6.x-2.x-dev (2009-Nov-10) version. The patch works like a charm.

Thanks for the good work!

Thanks for the patch. I applied it successfully and it seems to work well, but now ubercart transactions aren't being reported in Analytics. Does anyone have an idea of why this might be.
Thanks,
Christian

This cannot work with Ubercart

Yeah, Ubercarts tracking code would need to be ported to the new asynchronous API as well. This is why I think it would be useful for this to be on a separate branch.

subscribe

subscribe

subscribe

cool!

re #42, you mean some work needed in Ubercarts?

i think it does not need to be a new branch, but a checkbox with details of conflicts, while asyn default is off.

Pretty sure I can set this as at least Needs review. Don't know the final steps to getting this in yet though.

Subscribe

Subscribing

Subscribing

subscribe

subscribing

Is there any progress updates on this?

Just a question would things like event tracking (for onclicks and such) fall into the same category as ubercart not working, if anyone knows off the top of their head?

Thanks.

The patch from #36 works fine for me. I have been using it for about a month. Only drawback: It seems the communication with the GA server is always at the end, irrespective of header/footer loading. For 99% of my sites this is perfect in terms of performance, but sometimes I would like to measure the immediately leaving visitors.

_scribe

@JeebsUK Actually, I think the pageTracker variable ends up getting created so if you have custom analytics code its possible it will work without any changes. It depends on when your script runs and what its doing.

@digi24 Not sure what you mean here. The way its run asynchronously I guess does mean that it could be run later in the page load but that should mostly be because its not blocking the rendering. That's kinda the point.

As a side note, I have this patch installed on a handful of sites that share a multisite install. Its worked great so far including adding a couple custom bits of code tracking code. The download tracking is being used on one site as well and that's working quite well too.

@neclimdul
I was referring to the non-blocking loading and script execution. I have to do the debugging of the Google examples first, but I thought it would be possible to have the script execute in parallel to the regular page rendering.

To quote from the Google code blog

The second half of the snippet provides the logic that loads the tracking code in parallel with other scripts on the page. It executes an anonymous function that dynamically creates a

element and sets the source with the proper protocol. As a result, most browsers will load the tracking code in parallel with other scripts on the page, thus reducing the web page load time.

I profiled in chrome and it definitely looks to be running parallel. Also, this patch with a few exceptions implements the exact reference implementation for loading provided by google(http://code.google.com/apis/analytics/docs/tracking/asyncTracking.html). It uses pretty much the exact implementation provided by stevesouders' blog post and the googlecode blogpost from early in the queue. the differences are negligible and shouldn't make a difference.

It should be noted that the script is appended to the body dom element in all implementation so it is likely to be loaded after any inline code provided on the page. It will also be downloaded last because of this which may be what you are seeing. That's however by design and any analytics javascript should be pushed onto the _gaq object to be run later as documented in the asynchronous api documentation.

thanks for the explanation.

@neclimdul Thanks for the detailed explanation.

Based on these successful reports does anyone know if there are plans to commit the changes to the next dev version or if there are still issues to be worked out?

I think it will go into a new 6.x-3.x and 7.x-2.x branch. So we have a clear cut between the versions. Otherwise we need to add a radio like we have in D5 to distinguish between the tracker version... I'm only reluctant to create a release for BETA tracker codes... always keep in mind the current stable tracker is not the async tracker and works pretty well. There is no real need to use this patch except your are an early adopter who like to have issues. :-)

We could always mark the 3.x branch as beta with a note and leave the 2.x branch as the suggested download.

That aside, it is for early adopters but I don't actually see anywhere that it says the code is unstable, buggy, beta or otherwise dangerous to use. I don't see a reason that it shouldn't be actually supported in a release as the only way we're going to get better support is through actual use and testing.

Another note, I personally like the idea of using different branches but if requested I will provide a toggle patch along the lines of #35(maybe #35 is actually ready to go, I don't know). I like the idea of the branch better though because the code can begin to be tuned to the specific usage. The downside is its probably going to be harder for modules to detect how to add their own javascript without something like ctools_api_version or some constant they can manually toggle on.

The #36 patch is working for me.

I used to have

pageTracker._setDomainName("www.example.com");

in my Code Snippet (before). The purpose was so that the tracking cookies would be for www.example.com and not .example.com, which is the default. This prevents the wasted effort of sending the tracking cookies for each static.example.com or cdn1.example.com request.

I've substituted

_gaq.push(["_setDomainName", "www.example.com"]);

and it seems to be working fine.

Thanks for the work on this.

you can still get the pageTracker variable in asych mode quite easily when working with older code that needs it. Here is a blog post I found that deals with this issue for AddThis integration:
http://jochen.kirstaetter.name/blog/community/addthis-google-analytics-a...

Why should he use old code? www.example.com is pretty useless as this is the default hostname used by the GA code. You only need _setDomainName if you need to track all sites in a domain and than it need to be ".example.com" otherwise don't us this parameter at all. OT here.

I have found the reason for the issues I experienced in #55, it was a mistake I made, analysing the wrong code. Sorry about that. It works as expected setting the scope to header.

However I had to make a minor adjustment to the source path when local caching is enabled, as the source path is only relative:

Index: googleanalytics.module
===================================================================
--- googleanalytics.module      (revision 1377)
+++ googleanalytics.module      (working copy)
@@ -150,7 +150,7 @@
       "  var ga = document.createElement('script');\n";

     if (variable_get('googleanalytics_cache', 0) && (variable_get('file_downloads', FILE_DOWNLOADS_PUBLIC) == FILE_DOWNLOADS_PUBLIC) && $source = _googleanalytics_cache($url)) {
-      $script .= "  ga.src = '$source';";
+      $script .= "  ga.src = '/$source';";
     }
     else {
       $script .= "  ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';\n";

My cookies default to .example.com.

I am using the extra setting to get www.example.com.

@digi24: this is not a correct fix. The path is fully qualified and _googleanalytics_cache() cares about this.

hopes to see the final version soon

subscribing

sub...

sub...

subscribing

Any updates? Which patch needs review? There are so many above , getting difficult to keep track off.

To anyone who is in a panic over Google's changes to ranking based on site speed, relax. They aren't going to punish anyone with reasonable load times for actual visitors (said something like 1% of sites would be affected by the change) and that calculation is very unlikely to include javascript times at all.

That said, this issue is still important because it has an impact on the user experience for sites.

Even if we put an async loader into core, GA and other analytics scripts would still need special attention because the analytics should always load after the user interface components (jQuery UI, etc.). Otherwise a delay in the tracking results in a delay for the user, and that's unpleasant.

I definitely think async should be the default behavior for GA once it has been thoroughly verified.

It's currently unclear what patch actually needs review. Could you roll a fresh one with all the recent feedback taken into account?

subscribe

subscribe

I'm still for the patch in #36.

Most of the feedback after it has mostly been noise I think. I haven't had the problem mentioned in #55 /#68 though I don't /use/ the cached option I did test it and it looked correct. All the other complaints about it running last are noise as I noted in my explanation in #59. Its almost certainly going to run last in most cases but that's the point of the patch/asynchronous API. If anyone has particular points that need addressing I'll definitely take a look.

Hello Neclimdul,

please ignore #55 and #58, as noted before those problems were not related to your patch. Your comments were very helpful to figure out my local problem there.

However I still think that I had a reason for #68. As far as I remember, the problems occurred when using subdirectories. $source is a local file, the destination is given by: $source = _googleanalytics_cache($url)
The beginning of source is determined by: $directory = file_directory_path() .'/googleanalytics';
file_directory_path uses conf_path(),
and conf_path() has 'sites' hardcoded without a starting dash.

So while the relative URL is absolutely perfect as seen from Drupal root, it caused problems when the script is inserted on URLs with directories.

My resulting javascript looks like this:

...
(function() {
  var ga = document.createElement('script');
  ga.src = '/sites/myconfigdir.tld/files/googleanalytics/ga.js';  ga.setAttribute('async', 'true');
  document.documentElement.firstChild.appendChild(ga);
})();

My suggestion in #68 was still just a quick fix, I think it will not work when Drupal root is in a subdirectory itself.

I am in for testing and using this new asyncronious GA. Which patch is the most usefull now? How can I help improving this?
/Lars

This feature just became stable and the new default in Google Analytics, so we have a stable code fragment to work with: http://analytics.blogspot.com/2010/05/its-now-easy-to-set-up-new-sites-w... They also suggest all existing users to migrate to this new code soon!

subscribe

Fixed the wrong path in the patch. No changes at all. Let's see what the robot says.

Fixed Unix LF

I tried apply the latest patch but received the following error messages:

missing header for unified diff at line 8 of patch
can't find file to patch at input line 8
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|Index: googleanalytics.js
|===================================================================
|RCS file: /cvs/drupal-contrib/contributions/modules/google_analytics/googleanalytics.js,v
|retrieving revision 1.9.2.2
|diff -u -r1.9.2.2 googleanalytics.js
|--- googleanalytics.js	22 May 2010 23:41:32 -0000	1.9.2.2
|+++ googleanalytics.js	24 May 2010 00:00:14 -0000
--------------------------

I used the following Terminal command: patch -p1 < 648284_google_analytics_async_support_3.patch

Am I doing something wrong?

subscribe

Subscribing.

subscribing

The following features are broken with the latest patch:

1. search tracking produces invalid quotes _gaq.push(['_trackPageview', '"/drupal6/search/node?search=test"']). Simple fix.
2. We need an hook_update to update custom javascript settings to the new API. I think we can automate this in 99% as this are simple string replacements and nothing else.
3. custom URL for 403 and 404 tracking is not added to page. The main reason for this must be the removed hook_footer. Some information is only available in the footer! Additional we are not able to set the JS to header if we are in hook_footer. Gabor refused to fix this in D6 with the comment hook_footer is to late, but it has been fixed in D7 as I know and hook_footer is the only place we have all data we need to fully build the JS code.
4. There is a typo in "overide"

#3 also breaks #339235: Add multi/cross domain tracking (not yet implemented). At least we would also need a custom region to add code to $page_top that is also not available out of the box in D6 themes and only in D7 themes if they have been ported correctly. This are some bad news as the API looked like we could solve #339235: Add multi/cross domain tracking together with this patch :-(.

We could add the hook_footer back to get all previous functionality back and remove the ability to assign tracking code to the Header.

Please forgive me if what I am about to say isn't correct, but I thought the asynchronous GA had to be done from the header or it didn't work properly?

I know it's suggested to add it directly after the opening body tag (not in the header!) and the new async code will not block page loading until the ga.js file has been fetched from google - what the previous code have done. The async code allows us to load the ga.js without blocking. Another good reason for adding the code to the top of a page is - it's theoretically possible that the page hasn't finished loading yet if a user clicks on a link on the page. In such a case if GA is added before the closing body tag it MAY not executed yet - what means the page view hasn't been tracked yet when the user navigates to a new page. I guess this may happen sometimes on slow websites, but in our case we break other useful functionality that helps you to find broken links on your website.

You could use the linkchecker module for this functionality, but not so many people use it as the GA module and it's base functionality for statistic modules to provide the 404 tracking in GA

I think we have no real alternative... maybe someone have an idea or...

@Gabor accept a D6 core patch for #212560: drupal_add_js in hook_footer does not add JS files :-)

I'm not sure if splitting the code may help us...

hook_init()

<script type="text/javascript">
  (function() {
    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
  })();
</script>

hook_footer()

<script type="text/javascript">
  var _gaq = _gaq || [];
  _gaq.push(['_setAccount', 'UA-123456-1']);
  _gaq.push(['_trackPageview']);
</script>

Fully untested... but this is the way how it was implemented in past after some people pleased to be able to move the ga.js file download to the top (header).

subscribe

not in the header? I'm confused, Google states otherwise:
the optimal location for the asynchronous snippet is at the bottom of the <head> section, just before the closing </head> tag.
at
http://code.google.com/intl/nl/apis/analytics/docs/tracking/asyncTrackin...
and
http://www.google.com/support/analytics/bin/answer.py?hl=en&answer=174090

but when adding a new site to Google Analytics (Asynchronous code) it claims:
Copy the following code, then paste it onto every page you want to track immediately after the opening <body> tag

Also, I'd like to help creating a Dutch translation for this Google Analytics module if there is no Dutch translation yet.
Can someone point me in the right direction of how to approach this?
Can I just create a po file and upload it here?

Hehe... this is really confusing and inconsistent documentation... I'm also confused why they sometimes write before the closing head and sometimes after the opening body and what is now the place we should really use :-(

For the Dutch translation, please open a new case and attach the .po file over there. You may also like to join translation server at http://localize.drupal.org/translate/languages/nl for the review process and provide the .po file in the case if you got the approvals over there and it definitely save your time as many strings are shared across projects.

How bad... I found this about splitting http://code.google.com/intl/nl/apis/analytics/docs/tracking/asyncUsageGu.... So we end up - with the async code under D6 - we cannot provide the header section any more. So we remove the selectbox in custom settiungs from UI, but keep the setting and if someone upgrades to D7 he automatically will get what we got in 6.x-2.x. How good that we used a new branch for async stuff...

This is the async code that goes into 7.x-1.x. Upgrade from old API to new API still needs to be written or many people end up in a broken tracker if they have customized before/after scripts. D6 patch in #90 have outdated tracker implemented. The JS code have changed at Google.

try again

Commited to D7

D6, http://drupal.org/cvs?commit=375580

Installed 6.x-3.x-dev. Two questions.

1) The config page /admin/settings/googleanalytics does not have any new controls in that regard. Does it mean it is already switched on by default in this version of the module?

2) I would like to verify that the tracking is now asynchronous. How can I do that?

have a look at your sorce code. you must find a splitted version auf GA.

like the example:
http://code.google.com/intl/nl/apis/analytics/docs/tracking/asyncUsageGu...

I had installed 6.x-3.x-dev over the previous version (2.x) and ran /update.php as usual. The list of modules correctly shows 6.x-3.x-dev. Cleared all Drupal caches, and the browser cache.

Still, the page sources contain just one part, and it is different from the example you've referred to:

<script type="text/javascript"> 
<!--//--><![CDATA[//><!--
try{var pageTracker = _gat._getTracker("UA-XXXXXX-X");pageTracker._trackPageview();} catch(err) {}
//--><!]]>
</script> 
</body> 
</html>

NO guys - you are running a 2.x that has been committed to 3.x and does not yet have the async code like 120+ others who have installed a DEV version without understanding that this is 100% 2.x code, too. Install latest DEV OR - what is much better for you - DO NOT RUN DEV VERSIONS IN PROCUCTION without beeing a developer who understand VERY WELL what You are installing and how it works.

That's not fair, hass.

Your post #109 in this thread ("Support for Asynchronous Tracking") states "patch (to be ported) » fixed" for 6.x-3.x-dev.
I am sorry if I, as others, have missed the fact the 6.x-3.x-dev actually contains the old 6.x-2.x-dev code and no asynchronous tracking.

I don't run unstable code in production (that was your assumption); just wanted to help testing the new version.

Sorry, but if something has been committed it can take up to 12 hours until this code is inside the next DEV build. You have downloaded the DEV after the commit, but before the DEV has been re-packaged. As a general rule - if you are not a developer - do not use DEV versions as they can clutter your site and could destroy your data!

Latest DEV (Last updated: June 5, 2010 - 14:07) is the very first 6.x version that contains the async code - but NO other DEV before have had.

For multi domain sites:

Does 6.x-3.x-dev support both new and old analytics codes?

subscribing

subscribe

For ppl interested in Ubercart integration, refer to #922230: Asynchronous tracking for Google Analytics