Google Analytics Premium - Finally what I have been looking for

I am very excited about GA Premium announced yesterday.  I am hopeful that the pricing is reasonable because it seems like a great leap forward for businesses who find GA today a bit too limited in speed, functionality, and especially "fast-access-mode friendly".

Also I am very curious as to when it will be available outside of the US, Canada & UK.  Perhaps we could just signup via the UK and then use it for all of our other sites...?

Great intro commercial for the new service.  The only thing is that it seems perhaps a bit too inspired from a 2009 video from Grasshopper communications.  There sure seems a be a recent wave of videos/motion graphics pieces that all come from a common ancestor.  Mannatecheurope soon after Grasshopper seemed to have done a complete ripoff of Grasshopper the same year it came out. I wonder who was the first to do this type of piece. I think the first video that I can remember with some of this same style-DNA or at least something that could be an ancestor were the end credits from An Inconvenient Truth.

Look at the similarities between these 3 videos and you'll see what I mean.  You even get a nice audio effect if you start the first two videos at the same time (or start video 2, let it run 10 seconds and then start video 1). They complement each other quite well.

If you know of any other similar videos with the same DNA, include them in the comments.

Google Analytics Fast-Access Mode = FAIL-Access Mode

GA is a great service.  Google gives away this great service that other companies charge thousands of dollars. Besides that they have a wonderful API that you can use to write useful data applications.  My main complaint over the years, has been the use of Sampling.  In general, it may be good enough when looking at high level visitor data, but my experience is that sampling dramatically distorts calculations related to goals, e-commerce data, as well as data related to traffic sources which send relatively small amounts of visitors.

Recently, GA has introduced some new changes to its use of sampling. It appears that they have started using it in more cases.  Reports that would never trigger sampling with the amount of visits involved are now using "fail fast-access" mode.  That is Google's euphemism for "this report was based on sampled data and may be totally wrong".  Plus it looks like there may be a bug in the sampling method.

Take a look at this report using normal data for a site I monitor:

Normal graphic (you can see Easter's impact) of Google (Paid/Organic) coming to a site

Everthing above looks fine showing a fairly typical traffic pattern, with the exception of Easter week and an extra holiday in Great Britain (It must have been special hat day in the UK) .

Where did the traffic go?

Now look at the next two images which are the same report as above but with advanced segments applied:


Non-Paid Search Traffic: This should show the Google organic traffic the site received.

Non Paid Search Traffic....I didn't think that I only bought PPC???

Paid Search Traffic: This should show the Google PPC traffic the site received.

Data Dave is sad and confused.  If Google the search engine is sending me this traffic, what is it if not PPC or Non-Paid

What happened here?

OK, wait.  What if I just looked at the same report showing all three segments at the same time. Perhaps, it will start to make sense (or not):

Nope.  If anything, the results look even weirder and more distorted than when looking at the data individually.  There sure seems to be a big time bug in the sampling trigging and calculation system of GA.


In conclusion:

• With no segments applied, the site data looks normal....however:
• With segments applied, 1 month of relatively little data triggered sampling ("fast-access"). This never used to happen for relatvively little traffic in GA. 
• In both cases of sampling, Google sent the site 0 visits during the month of April.
• Where GA showed visits, it was very incorrect (multiple stand.devs apart from the actual observed data)
• With sampling applied, in the first week of May, GA reports that Google PPC sent more way traffic than all of Google combined.
• Non-Paid alone surpassed All Visits traffic for a coupld of days as well.

Data integrity is the foundation for any analytics / decision support tool.  We cannot improve our websites if we cannot rely and believe the data upon which we will be basing our decisions.  Luckily, I am not alone in seeing this latest turn for the worse with GA and sampling. 

 Hopefully, they will fix it soon.

Updated App engine project GA Evolution

Despite the lack of posts.... this blog has not been forgotten.  Just wicked busy at work and home for the last few months.

I have updated gaevolution.appspot.com with a few improvements:

• Fixed bug in Weekly reporting that caused weird placment of data when the weeks in included the middle-end of Februrary (parseInt bug in most browsers was to blame)
• Updated jQuery to 1.6, Flot to 0.7 and fancyBox to 1.3.4 - it really flies now
• Using the latest SDK for appengine

Next up I will need to change how the app handles sampling.  The CI value has been marked for death deprecated by Google, so I'll switch to another parameter they have that indicates if sampling was triggered in the report or not.

Any requests out there? Please let me know through the feedback form or as a comment here on the blog.

Soon, I hope to write a post too about how I won my Nokia E7 in February while in London.

Updated GA Evolution - Now with Weekly Data, Goals, Optional Metrics

Over the last few weeks, I have made some improvements to GA Evolution. The functional changes were pushed last Friday and this morning I added a quick YouTube Video to the login page to better help show users what the app is all about.

Here are the highlights of the changes:

1. Improved performance and session handling (faster is better with less errors)
2. I have included Goal Data which can be reported as either Goal Completions or Goal Conversion Rates
3. The metrics columns are now optional so users can build the report that they really want
4. Report data can now be broken out by week besides by month

As always, enjoy and please pass along feedback on the app. 

Link to original GA Evolution post

Link to GAEvolution web app 

---

Goal Funnel Exit Values in Percentages - New Userscript

In the Goal Funnel Visualization report of GA, it paints very nicely all of the users who flow in and out of the typical conversion path. One thing that I thought it was missing, however, was to show the exit numbers in percentage terms.

It has always shown the top 5 locations to where a user "abandons" a certain step of the conversion funnel, but I am someone who really likes looking at data in % values, not only in absolute values.

So, here is a simple little script that calculates the % value for each of those top 5 exit pages for each step. The % is the number of exits to a specific location divided by the total # of visits to that specific goal funnel step. Enjoy.

I like % values

Instructions for installing the script in Firefox

• You need Greasemonkey already installed
• Download the user script directly or from the Userscripts.org script page

Google Analytics Evolution - Time Series GA Data using the API & App Engine

This project started with 3 main ideas:

• Need to quickly pull visits, bounces, transactions by month for many sites? Check.
• Want to rapidly chart sites' performance against each other? Check.
• Need a simple interface to allow for getting data into Excel that any marketing person could use? Check.

I am happy to introduce Google Analytics Evolution. A new app hosted on Google's App Engine (for Java) using the Google Analytics API and Flot., a powerful jQuery plug-in for charting.

GA Evolution in Action

Reason for the app
I often have the task of slicing and dicing data for the many sites I oversee. at logismarket we operate our 16 sites in 15 countries. Using Google Analytics, the process of pulling data for reports across all of these sites, was a time-consuming and tedious task. Custom reports, advanced segments, and the great greasemonkey script, cleanerGAProfileSwitching, made things much easier, but I wanted to be able to work even faster. Additionally, any time that I wanted to see data over a longer period of time, my biggest pet peeve with GA, sampling, would rear its ugly head. Given the size of our sites and depending on the dimension or segment used, we can trigger sampling with only a few months of data. That makes using GA for measuring trends quite unreliable, especially for transactions, since the margin of error is much larger on the transactions than it is for visits (the basis of sampling in GA).

So when my boss asked me for a review of the last 18 months for all of our sites and to further segment the data into 3 different slices, I decided to start coding instead of pulling data right away. A little investment now in coding to get a big payback over time for me and anyone else who has the same kind of needs with GA.

Enjoy.


Examples of Usage

• Site comparison: Pull data for the same period for 2 or more sites and click a column header to see the results in the chart
• Segment comparison: Pull data from the same site (or more) and choose various segments for the same period and click to see the difference in the chart without the distortion caused by being forced to include All Visits (as it does in the GA UI). Also less likely to trigger sampling.
• Quick data pull for multiple sites: Select all (or most) of your site profiles, get the data, copy and paste to Excel. Only minimal manipulation needed to get the data ready for Pivot Table usage :-D

Check out the free app today

Ideas for Future Expansion
While this app currently meets my needs I have identified some areas that I want to expand and improve the app. I am also open to suggestions. Please add a comment to this post or report an issue on the Google Code project page for GA Evoluation.

• Add export to CSV feature. I am thinking to do this along similar lines as the charting... where you export the data that is already pulled and you select the tables. That way you know what you are going to get (eg, you can check Confidence Interval to make sure sampling was not triggered).
• Make the metric selection dynamic with a set of default metrics. You can greatly expand the reporting capabilities, but I need to make sure that the reporting stays correct (keep apples to apples in charting of the data).
• Add some calculated metrics not in API. Bounce Rate, eCommerce Conversion Rate are not available via the API because that info can be calculated after pulling the data. No need to burn free Google cycles when we can just calculate it ourselves. However, it would be more user friendly to have the option to get the data as we want it, not just as it is given to us by Google.
• Other enhancements to flot charts. These are not fully defined yet, but I am thinking toward making the charts a bit more dynamic... still need some thought here.

Export Google Trends to Google Docs Script

Feeling inspired from the success of the script that I wrote to export from Google Analytics to Google Docs , I decided to see if I could reuse most of the code for other Google CSV exports.  My first sub-project was to tackle Google Trends.

I made the necessary front end changes to put in the new links and collect the correct export URLs.  Generally everything worked on the backend as well after only a few minor tweaks.  However, I kept receiving a lot of junk in the export and the content was not legible.

There seems to be some issue with the format of the CSV file being delivered by Google Trends.  It opens fine is a pure text editor like Notepad++ , but the browser recognized thousands of "invalid" characters in the export.  Basically, the CSV contains a null value in between every other valid character and it included rare form feed characters at the beginning and end of the file.  These invisible characters were blowing up the import to Google Docs so that while I would receive a "success" message for the upload, everything came out as gobbledegoop .

After multiple attempts to try to request a different file format, I finally decided to work a little string magic using RegEx.  I stripped out all of the junk characters before sending it up to Google Docs and voila! Everything works fine now.

Unfortunately, Google Trends only allows exports for keyword comparisons at the moment, but not for website data.  In the meantime though, for anyone doing SEO Keyword research, this could be of some valuable use to you.

Here are the links:
Download GM script directly
Greasemonkey Script Page

Enjoy.

Google Analytics Bug - The Sum of its Parts is More Than the Whole

Google Analytics is a great product, in general, but sometimes I come across things that are totally head-scratching.

Their relatively new feature for Advanced Segmentation will be very very useful once it reaches some semblance of reliability. As for now, though, I cannot rely on it.

The image does not lie.  I have 2 segments applied:

1. All Visits (basically, unfiltered)
2. True Non-Paid (all visits that are not CPC)

GA tells me that yesterday I had about 110 more conversions coming from my non-cpc traffic than all of my traffic combined?????

I know that the little red tag says "Beta", but we have come to expect very reliable behavior from a lot of other "beta" products from Google (eg., Gmail , Google Earth , Maps for the longest time, etc...).

This is something that needs to get fixed quickly.  The worst thing that GA can do is put doubt in the minds of its users about the accuracy of the data.

Updated Google Analytics Export to Google Docs

Google Analytics has updated the layout for the graphs and export links in their reports...breaking the script. Now it is updated to v2.6 and works smoothly.

New links:
Greasemonkey Script Page
Download GM script directly

Cheers.

Updated Greasemonkey Script Integrate Google Analytics with Google Docs

Google seems to have updated their security in Google Analytics for exporting data, so I updated the Google Analytics to Google Docs Greasemonkey Script to accomodate the GA improvements.

Integrate Google Analytics with Google Docs using Greasemonkey

UPDATED 07/31/09: Not sure if the problem was upgrading to firefox 3.5 or if the GDocs API folks changed how they handle uploads, but the script started returning 415 errors from GDocs.  I worked with Google and now we have the problem fixed.  Should work for all again.

Any suggestions? Please add them in the comments.

---

UPDATED 05/04/09: All fixed now. However, I am starting to notice some odd 404 errors coming from the Google Docs API. The file uploads nonetheless, but every once in a while the confirmation message comes back as a 404. Any suggestions? Please add them in the comments.

---

UPDATED 04/30/09: Google just made another change to their code which broke the script. I will work on fixing it starting next week. When it is fixed, I'll make another post. Thanks for your patience.

---

UPDATED: I updated the script with many improvements to reliability and functionality.

---

UPDATED: I updated the script to handle security changes made by Google to Google Analytics

---

Recently I was inspired to start tinkering with Google Analytics using Greasemonkey after listening to a presentation by Avinash Kaushik.

One problem that we have at work is that our version of Excel for some reason does not like the CSV files that GA exports. Our copies of Excel XP (I know its 6+ years old, but we just moved to it last year) only see the graphing data, but they do not recognize the tabular data further down in the CSV files.

After seeing the Juice Analytics integration for seeing new reports in GA, I realized that I probably could solve my export problem too using Greasemonkey. I decided to integrate GA with Google Docs, so I can do a one-click export to create a Google spreadsheet for analyzing the GA data.

Now, on any report that offers a CSV download, the user sees a new option for Google Docs:







Clicking the link will log the user into the Google Docs service and upload the report. Currently I have the # of records limited to 10000 rows of data, but I may make that a configurable option. After the report is successfully uploaded, the user received a confirmation message with a choice to open up Google Docs in a new tab or just to remain in GA:











Greasemonkey handles your data securely and only limits access to values stored by the user script to the user script itself. There is an additional measure of security I added that fully encrypts user passwords. Finally, the export service can only upload to Google Docs using the exact same account that you are currently using in GA. If you want to share, I suggest you share the spreadsheet after it has been created.

Installation Instructions

• You need Greasemonkey & Firefox 2+ installed
• If you have Greasemonkey working you will see a little monkey icon in your status bar:
  
• Now, download and install the user script (click OK/Accept on any message boxes. The file used to be called exporttogoogledocs.user.js, but apparently Userscipts.org renamed the file!)
  
• After opening choosing to open up Google Docs, the pop-up blocker may stop the new tab from launching...if so, you will need to allow pop-ups from the domain www.google.com because of the way Google handles the redirect.
  

Happy Analysis!