a canadian startup

Nov

Thu

User Statistics Overload

For medium sized websites, organizing your user traffic statistics into a useful format can be a challenge. Typically you will have more statistics than you can deal with.

The first source of user statistic data is your Apache (webserver) log files. These logs record every page that is visited on your site and may contain other useful data such as the referring site from which users arrived, IP addresses of users, etc.

The Apache logs will only get you so far. The main problem with them is that Apache logs are based on tracking “hits” — they contain no notion of “sessions.” Sessions are important because they allow you to see the path that a specific user took through your site, rather than an aggregate view of all pages visited.

Above is example awStats data (awStats reads Apache logs and displays them in a graphical format) showing user traffic. We use this to find out which days are busier than others, and to get a rough idea of where our visitors are coming from, what countries they’re in, and general traffic trends over long periods of time.

Okay that’s a start. But now we need to dig deeper

For an e-Commerce store, the next thing you probably want is general sales and visitor data based on accounts, orders, and transactions. To do this at Well.ca, we wrote complicated SQL queries that question our database of all sales and customers.

Designing such a reporting system should begin with these questions — we all sat around a table and whiteboarded a list of 20 or so “questions” that we should be able to ask our server and get back data. Example questions for us were:

How many sales happened in each category of product in the past X days?
How many of our users were from Canada vs. the US in the past X days?
How many orders were sold each month for the past N months? What was the average order size in each of those months?

After a session of whiteboarding, Chris was ready to start writing the queries. We used amCharts to plot the info in real time on our site so we can ask these questions at any moment and get presentable data.

Take a look at some of our real-time data (with some details cropped out):

and:

This data is all nice and useful but still hasn’t solved the problem of how to get useful and readable user data.

For smaller sites that I used to maintain, all we had to do was add in a session-by-session stats counter and then view each session individually. Of course for medium to large sites, this is impossible — there’s much too much data to interpret one-by-one.

Once again, our statistics strategy is to sit down and ask questions. What kind of data are we looking for? If you’re trying to make money from your site, you should probably be asking questions about conversion rates. At Well.ca, we’re working on setting up a series of conversion targets on our site. We also need to segregate our incoming traffic so we can see which visitors came from google organic results, google adwords, paid banner ads, and whatever else.

Now we can ask questions like:

What percent of our google organic visitors proceed to checkout?
What percent of people that begin checkout, actually complete checkout?
What stages of checkout take the longest? Where do we lose customers?
What percent of people that read a blog entry with products in it, actually buy said product?

Once we have these questions, we build a tool that would help us get at this data in the simplest way.

As a demo, Chris took all all of our session-data (data that’s recorded in PHP using session information to divide up visitors) and generate an image (using the very funky Java Processing toolkit) to show what our users were doing last week. This is the graph:

Actually that’s only a slice of the aggregate picture (it’s a big image). Each circle is a different page on our site. Each line is a time when a user connected between the two, more connections makes the lines more solid white. Different colours represent different page types (product pages, brand pages, articles, etc.)

The first thing I said when Chris showed me this data was: AWESOME!

The next thing I said was: This is completely useless. (but still awesome)

The thing is, we have more than 10,000 pages on our site at Well.ca and thousands and thousands of users. The amount of data is overwhelming and really difficult to represent.

The solution may be to return back to the original moral of this article: focus on asking the right questions such as questions about conversion rates, etc. This still leaves something to be desired: we want to know what paths our customers are taking as they visit our site. How can we view this without getting overwhelmed?

The solution we’ve chosen is to use a solution that we’ve traditionally used in User Interface Design. The idea is that, instead of looking at all the user data, we select very few users that are made to represent the majority. Every few months, we at Well.ca will select 3 or 4 users that recently visited our site and we will analyze the path they took. To make this simple, for every order that happens on our site, we design the site to allow us to view this visitor’s path — this lets us see the typical path of favourite customers allowing us to promote and improve the path that these customers take.

Here, for example, is a slice from an image that shows the path of a recent customer. Chris has written a combination of PHP, Python, and other code that displays this in a program called GraphViz.

Check it out:

This gives us a very specific view into the paths that our customer’s are taking when viewing our site — giving us more familiarity with what really happens in our live site. In this example, it took 36 clicks for our customer to go from start to the end.

Of course, this kind of zoomed-in view isn’t good for finding small problem areas (because we’re only selecting 4 customers out of thousands) but it allows to think like our customer, putting us in their head and confirming our assumptions about the processes we’ve designed. It’s a fun and useful process for our design team.

So that’s my post about user data and your website. My main points are:

You need more than Apache logs
Think about conversion targets
Segregate your incoming traffic
Don’t build any software or stats until you’ve decided on a set of questions you want answered.
Visitor session data is overwhelming, look at a small subset to get an idea of paths. Every month, carefully pick 4 example users and, as a team, look at their paths through your site.
Make everything graphical using tools like Processing and GraphViz — your developers will have fun doing this.
Give yourself access to real-time charting and graphs. amCharts and SWF/XML charts are great.

9 Comments »

Updates to Well.ca

User Statistics Overload

More Camps