Updated App engine project GA Evolution

Despite the lack of posts.... this blog has not been forgotten.  Just wicked busy at work and home for the last few months.

I have updated gaevolution.appspot.com with a few improvements:

• Fixed bug in Weekly reporting that caused weird placment of data when the weeks in included the middle-end of Februrary (parseInt bug in most browsers was to blame)
• Updated jQuery to 1.6, Flot to 0.7 and fancyBox to 1.3.4 - it really flies now
• Using the latest SDK for appengine

Next up I will need to change how the app handles sampling.  The CI value has been marked for death deprecated by Google, so I'll switch to another parameter they have that indicates if sampling was triggered in the report or not.

Any requests out there? Please let me know through the feedback form or as a comment here on the blog.

Soon, I hope to write a post too about how I won my Nokia E7 in February while in London.

Google Analytics Evolution - Time Series GA Data using the API & App Engine

This project started with 3 main ideas:

• Need to quickly pull visits, bounces, transactions by month for many sites? Check.
• Want to rapidly chart sites' performance against each other? Check.
• Need a simple interface to allow for getting data into Excel that any marketing person could use? Check.

I am happy to introduce Google Analytics Evolution. A new app hosted on Google's App Engine (for Java) using the Google Analytics API and Flot., a powerful jQuery plug-in for charting.

GA Evolution in Action

Reason for the app
I often have the task of slicing and dicing data for the many sites I oversee. at logismarket we operate our 16 sites in 15 countries. Using Google Analytics, the process of pulling data for reports across all of these sites, was a time-consuming and tedious task. Custom reports, advanced segments, and the great greasemonkey script, cleanerGAProfileSwitching, made things much easier, but I wanted to be able to work even faster. Additionally, any time that I wanted to see data over a longer period of time, my biggest pet peeve with GA, sampling, would rear its ugly head. Given the size of our sites and depending on the dimension or segment used, we can trigger sampling with only a few months of data. That makes using GA for measuring trends quite unreliable, especially for transactions, since the margin of error is much larger on the transactions than it is for visits (the basis of sampling in GA).

So when my boss asked me for a review of the last 18 months for all of our sites and to further segment the data into 3 different slices, I decided to start coding instead of pulling data right away. A little investment now in coding to get a big payback over time for me and anyone else who has the same kind of needs with GA.

Enjoy.


Examples of Usage

• Site comparison: Pull data for the same period for 2 or more sites and click a column header to see the results in the chart
• Segment comparison: Pull data from the same site (or more) and choose various segments for the same period and click to see the difference in the chart without the distortion caused by being forced to include All Visits (as it does in the GA UI). Also less likely to trigger sampling.
• Quick data pull for multiple sites: Select all (or most) of your site profiles, get the data, copy and paste to Excel. Only minimal manipulation needed to get the data ready for Pivot Table usage :-D

Check out the free app today

Ideas for Future Expansion
While this app currently meets my needs I have identified some areas that I want to expand and improve the app. I am also open to suggestions. Please add a comment to this post or report an issue on the Google Code project page for GA Evoluation.

• Add export to CSV feature. I am thinking to do this along similar lines as the charting... where you export the data that is already pulled and you select the tables. That way you know what you are going to get (eg, you can check Confidence Interval to make sure sampling was not triggered).
• Make the metric selection dynamic with a set of default metrics. You can greatly expand the reporting capabilities, but I need to make sure that the reporting stays correct (keep apples to apples in charting of the data).
• Add some calculated metrics not in API. Bounce Rate, eCommerce Conversion Rate are not available via the API because that info can be calculated after pulling the data. No need to burn free Google cycles when we can just calculate it ourselves. However, it would be more user friendly to have the option to get the data as we want it, not just as it is given to us by Google.
• Other enhancements to flot charts. These are not fully defined yet, but I am thinking toward making the charts a bit more dynamic... still need some thought here.

SymbianNote - Evernote for Symbian - My first WRT Widget

Evernote for S60 v5: hosted on Google Code.

After playing around with the WRT kit and navigating the documentation for the WRT kit and the beta version of platform services 2.0 (you would think these would be more easily available online!), I have created my first WRT App for Symbian v5 phones (eg, 5800XM and N97, etc...).


Version 1 of the app does the following:

1. Provides 1 touch (or run in the background) access to Evernote online
• Add quick notes
• Search and view notes already uploaded to the service
2. Send notes with /  without attachments to Evernote via e-mail.
3. Add your personalized Evernote incoming e-mail address to your contacts list with only 1 touch.  As a contact, it is easier to send files and notes to Evernote from other applications on the phones.
4. Check for updates (in the About view)
   

Some obervations about WRT:

1. The 5800 is not powerful enough or does not have a recent enough version of webkit for its browser to properly display jQuery UI effects.  Effects that are very smooth on the PC and emulator are jerky and incomplete on the actual phone. Hopefully, the new firmware rumored to be coming in October will improve the performance of the browser and JavaScript engine
2. WRT still needs a lot more work before it is a very useful toolkit
• Its APIs are still fairly limited. I wanted to use the Messaging API to send e-mails directly from the application but no such API exists yet. It only supports MMS and SMS (and even the addressing of those messages is limited).
• Users should be able to set security preferences one time for an app and not have to approve every use of an API by an application.  This is a problem with some Java apps as well. Users should have the choice to setup the security preferences one time for each app.  The constant request for approval (just like Vista) is very frustrating.
• The new camera API looks promising, but until the e-mail api exists it will be useless for Evernote (although other apps could work fine).

I hope you enjoy the app and I welcome your feedback on bugs and improvements.  Once I figure out how to upload source code to Google Code and not just final packages, I'll put up the source code as well.

Direct Download link:
http://evernote-s60.googlecode.com/files/SymbianNote.wgz

Link to Google Code Project Page:
http://code.google.com/p/evernote-s60/

I'll try to add some screenshots soon.

Balsamiq Mockups - it Roqs (Web Prototyping Software)

Recently I heard about a piece of software for creating mockups for web apps. After a brief read of the features and nature to the tool it sounded almost too good to be true.

Striking the prototyping balance...
For many years, I have tried virtually every type of prototyping method for web page design.  Each has its own dedicated following (fanboys), advantages and disadvantages.


Examples of prototyping methods used previously

Method	Description	Advantages	Disadvantages
Pencil & Paper	Just as it sounds, sketch it out on a pad of paper.  Other variations include cutting and pasting paper to show interaction	Focused on ideas and functionality (if done with layers) Anyone can have input Less pride of ownership Can do it anywhere more info	Very tedius (especially if you have little to no drawing skills) No scale to duplicating elements
PowerPoint / Visio	Create a background image and start building on top of it with boxes / cut screenshots	Easy to duplicate elements Can show animation (PPT) On most desktops more info	Animation is very tedius Stuck with 1 page size (for all intensive purposes this is the case)
Photoshop	Use the industry standard image softwrare to build your comps	Powerful use of layers Easy to duplicate elements Lots of tools and templates available more info	Not always available Expensive Not easy for a novice Often focuses on "sexiness" instead of usability Animation not easy more info
Fireworks	Similar to Photoshop (and Illustrator) but handles reuse and "animation" better	Powerful use of layers Easy to duplicate elements Good animation tools more info	Not always available Expensive Not easy for a novice Often focuses on "sexiness" instead of usability
Dreamweaver	Use the most popular WYSIWYG tool to write HTML	WYSIWYG click and add elements Quick edits (small changes) Templates more info	Not always available Expensive Not easy for a novice Bad tradeoffs in having semi hi fidelity and inability to reach quickly to big changes (CSS can be a pain)
Straight HTML/jQuery	Grab your favorite Text Editor / IDE and start coding.	Closest approximation to the final product Adds real interactivity Forces designer to think of practicality jQuery adds lots of powerful tools to speed up layout and functionality more info	Strong technical knowledge needed Hard to react quickly / in real time to radical changes Often focuses on "cool factor" instead of usability

And then came Balsamiq Mockups...perhaps the perfect balance

Balsamiq Mockups (BM from now on) is from a micro-ISV called Balsamiq.  The founder is a former Adobe employee who decided to head out on his own and moved back to his home country of Italy to focus on the company.

The product's objective is to give you most of the advantages of Paper Prototyping but with the added value that comes from being digital (quick repetition and duplication, capture state easily, collaboration across long distance, etc...).

The interface is simple and the usability is very high.  Although it is a young product from a young company, the software so far has worked fairly reliably.

Every element acts a lot like a visio element.  They can be resized (I really like the autosize feature), grouped, layered, duplicated, etc... There are a minimal set of properties that many controls have to adjust background color, to add a real image (the sketch it feature is helpful to not lose site of the mockup's purpose), or adjust font sizes (this last feature is limited and something I suggest improving below).

Since I installed the application (it is Adobe AIR based) a couple of days ago, I have used it to mockup some important page changes that we are considering.  It is amazing how fast I was able to whip up the new design ideas using this tool.  Most recently, I have been doing these mockups using a combination of Firebug, Firefox, jQuery, and SnagIt.  For small changes I will continue to use these tools, but Balsamic Mockups was much easier and quicker for me to sketch up big changes to the pages, especially when it came to layout.

Examples of mockups from Balsamiq (I cannot show my mockups due to our corporate policy e.g. trade secrets)

  

 

What is still needed for the software right away:

• Zoom - You cannot zoom in or out of the page.  Needed, especially for us without fancy monitors (or on noteboooks)
• Adjust font size & style for all (or most) controls - font size and styles can only be changed in certain types of text controls but not all (ej, lists, some  text, etc...)
• Personal Library for Custom controls - Lets say that you created an image using an asset from your site.  You should have a simple way to add this to the collection for use in other prototypes.
• Naming Exported PNGs - you can export mockups at any point to a PNG file, but you are never given an option to name the PNG file.

Other items to add in the future...

• Interactivity - add clickable links to navegate between mockups (or PNGs), perhaps add common effect features (slide, fade, lightbox)
• Integration with other services - Balsamiq will focus on creating plugins and small opps for the web office.  It would be great to be able to export mockups or PNGs to photo sharing services like Picasa or collaboration sites like Basecamp.

Conclusion - it Roqs!
I love this software.  Looking over the blog posts, there seems to be a healthy rate of updates to the software, so I can only expect the features to become richer over time.  You can see on their site that they already have a fairly large following.  They can happily add one more (and any one else that I can convince) to that list of happy users.

You can try it using their online trial (benefit of being AIR based) although it is time-limited.

Export Google Trends to Google Docs Script

Feeling inspired from the success of the script that I wrote to export from Google Analytics to Google Docs , I decided to see if I could reuse most of the code for other Google CSV exports.  My first sub-project was to tackle Google Trends.

I made the necessary front end changes to put in the new links and collect the correct export URLs.  Generally everything worked on the backend as well after only a few minor tweaks.  However, I kept receiving a lot of junk in the export and the content was not legible.

There seems to be some issue with the format of the CSV file being delivered by Google Trends.  It opens fine is a pure text editor like Notepad++ , but the browser recognized thousands of "invalid" characters in the export.  Basically, the CSV contains a null value in between every other valid character and it included rare form feed characters at the beginning and end of the file.  These invisible characters were blowing up the import to Google Docs so that while I would receive a "success" message for the upload, everything came out as gobbledegoop .

After multiple attempts to try to request a different file format, I finally decided to work a little string magic using RegEx.  I stripped out all of the junk characters before sending it up to Google Docs and voila! Everything works fine now.

Unfortunately, Google Trends only allows exports for keyword comparisons at the moment, but not for website data.  In the meantime though, for anyone doing SEO Keyword research, this could be of some valuable use to you.

Here are the links:
Download GM script directly
Greasemonkey Script Page

Enjoy.

Yahoo Mail in iGoogle

iGoogle is my homepage and although I have a Gmail account, I still prefer the Yahoo! Mail experience.

Here is a link to the Google Gadget using the mobile interface of Y! Mail

However, the mobile interface has a lot of fluff and does not fit perfectly in the gadget frame. Therefore I created some gm scripts to make it more useful and provide a better fit.

Greasemonkey Scripts
Main Script: http://userscripts.org/scripts/show/33874
Detail Page Script: http://userscripts.org/scripts/show/33875

Chrome 1st Impressions

I have been testing out Google's new webkit-based browser, Chrome, for about 1 day now and so far I am pretty impressed.  The performance is generally fantastic, although I did have some sluggishness switching between open tabs when some CPU intensive activity (loading a lot of JavaScript and playing Flash video) was going on in my current tab.  Otherwise, it seems to be very very fast and lightweight in the memory department.

I wanted to compare Chrome vs. Firefox 3, especially in the area of JavaScript execution.  My favorite js library is jQuery, so I decided to test how well each of the two browsers can rapidly process all of the appear/disappear effects.  Here are the video results:

Updated Greasemonkey Script Integrate Google Analytics with Google Docs

Google seems to have updated their security in Google Analytics for exporting data, so I updated the Google Analytics to Google Docs Greasemonkey Script to accomodate the GA improvements.