Now, there is no such thing as un-opinionated article. Everyone will have a view, even if they try to not be biased, so the first and obvious question is: When should opinion count?

What Is an Opinion?

Well, a quick visit to dictionary.com brought 2 out 7 definitions for ‘opinion’ worth considering:

1. a belief or judgment that rests on grounds insufficient to produce complete certainty.
2. a personal view, attitude, or appraisal.

The words I think are important are judgment, view and appraisal. These lead us to right, wrong, negative and positive, so we could perhaps say an opinion, in context of a written prose, is something that is skewed towards negative or positive wordings.

Tim is an utterly amazing and cool guy. I think everyone should follow him on twitter because he is awesome.

Can be safely assumed to be an opinion, as could

Tim is a miserable, boring, depressing person no one should follow!

But it’s not long before Storm gets started:
“You can’t know anything,
Knowledge is merely opinion”
She opines, over her Cabernet Sauvignon
Vis a vis
Some unhippily
Empirical comment by me

“Not a good start” I think….
I resist the urge to ask Storm
Whether knowledge is so loose-weave
Of a morning
When deciding whether to leave
Her apartment by the front door
Or a window on the second floor.

Tim Minchin – Storm

But what about:

Placing the L45 into Warp drive mode will cause it to fail resulting in a negative feedback in the pinky.

Here is a something that, while negative words are used, could not be described as an opinion, assuming the warp drive does indeed fail in those circumstances. Still, if we assume an opinionated piece will have a higher concentration of negative or positive words per word count, we can create a simple rule for determining if text is opinionated or not.

How To Determine Positive or Negativity

Let’s face it, certain words are positive and others are negative. Actually, as humans, we can quickly work out which are which easily enough. Computers, however, need a little more help, so we have to provide them with a set of words and identify if they are positive or negative. Then, we simply process through them and work out if the line is positive, negative, or neutral on a line by line basis. Simples.

Ok, so to do this, we could simply bucket count, but that’s not very efficient. Instead, we’ll use the Bayes Theorem :

P(A|B) = P(B|A)*P(A) / P(B)

Where P is Probability. Where A and B are events and A|B is where event A occurs if B is true.
So, Bayes theorem is calculating the probability that “A” is true or will be true, given a certain set of circumstances “B”. For more information, see Bayes Law in Plain English.

With this math under our belt, we can construct a set of Baysian classifiers. These will sound familiar because they are what a lot of anti spam filters use. In our case, they allow us to process through and look for negative and positive words and phrases. With the classifier in place and a suitable set of positive and negative word lists, we can begin processing through the documents.

While the client was only interested in opinions, we actually stored the total number of sentences with negative or positive opinions. The code for our initial version was heavily influenced by Baysian opinion mining code on PHPIR. However, we did end up rewiting the code in C++ as a php module to speed up the results for our large data sets. In addition, we took a similar method to Darko Romanov and filtered the stop words from our sentences.

Once we processed through document, sentence by sentence, we determined an overall score. This score was then used, along with the total number of sentences, to determine how opinionated a piece was. We also showed if it had a positive or negative bias and compared it to 100 examples of what was deemed to be opinionated and 100 that were not.

The system correctly identified the one hundred opinionated pieces, but also incorrectly identified 12 of the non opinionated pieces. Tweaking of the Lexicon reduced this down to 4, which was deemed a reasonable error margin. However, it also let 1 opinionated piece through. Again, this was deemed acceptable. The goal now is to provide multiple lexicons dependent on the site and author who is writing the piece.

Does It Cope With Sarcasm?

Surprisingly well! Most sarcasm is used when a positive indicator is used. When in fact a negative is inferred, most sarcasm is surrounded by other negative sentences when the system breaks down content sentence by sentence. Thus, while the sarcastic sentence itself will indeed be misclassified, the surrounding sentences will not (hopefully), and so there would be more negative then positive sentences within the piece.

What Other Uses In Search

Well, obviously, in the original client request, they were looking at removing or reducing relevancy of opinionated pieces, but imagine if you did the opposite. Let’s say I run a review website that people can use to search for product reviews. Obviously, I link to products with affiliate links.

Now, imagine if my internal search was designed to show more positive results for higher commission or converting items? Negative reviews would be found further down the list of product reviews. Sneaky, but if you have lots of people using your internal search it could be one way to increase revenue.

Should it be a ranking factor?

Well, I suspect many site owners of products may wish it was a factor. Take, for example, my post on GoParks recently. The owner of the site would probably be quite keen if opinion pieces had reduced rankings. I can also see for Google News or similar this style of ranking could be useful, but on the main stream web I’m not convinced that it would work.

What do you think, should opinion be a ranking factor?

The new API allows:

• Retrieve, create, modify, and delete pages and content.
• Upload/download attachments.
• Review the revision history across a site.
• Display recent user activity.

While at first glance this doesn’t sound breathtaking and to gain multiple Google Page sites you need multiple sites and multiple API keys this is the first time a “free” provider has made it quite so easy to automate parasite sites not hosted by the SEO company. And while Google may well de-list individual Google Pages or ban that user they are not going to ban their entire system meaning that its one of the few sites where you are sure to be able to from the off create nice clean sites in the eyes of Google with no worries about past history.

Ideas for content Network Developers

My first thoughts was to grab all those nice article directories and republish them, or indeed a bit of desert scraping, automate the search, Xpath the content and automate the uploading after the initial setup you could simply insert keywords and let it go, for multiple Google accounts Mechanical Turk provides a simple and cheap way round the CAPTCHAs.

Just food for thought, of course these low quality sites would be terrible for users and near worthless for search engines so expect poker, viagra and other competitive niches to be abusing them soon.

What you want more? Seriously I mean what do I need to tell you?
I will be running a seminar on Profiling website users, looking at life cycles and a huge pile of tricks and ideas to get the most out of your users. Want to know if colour effects genders differently? Want to know how to estimate gender in website visitors?

Then you should be attending this seminar! What’s more its free well sort of, its actually free to Think Visibility attendees everyone else its £50 but of course what this really means is if you buy a Think Visibility ticket at £99 and use my discount Coupon TIMNASH the price will drop down to such a level that you got to ask yourself why not come to both

I’m afraid while this seminar will be filmed it will be only for internal use and not available for purchase, if it’s a success I might run a future seminar and film it.

If you are interested in improving your conversions, or even just curious as to what it is that I find more interesting then SEO (or if your into SEO why what we do is so much more then the normal) then come along for the afternoon it’s free, simply register to confirm your place at the behaviour modelling seminar.

• Several of the best SEOs in the country
• WordPress and CMS experts
• Social Media experts
• A true accessibility expert with hands on experience
• A guy who has been sued by well everyone!
• oh and me

Check out the full list of speakers and the schedule and if you haven’t already buy your ticket, using the discount coupon TIMNASH

Important – would you like a more indepth seminar the day before?

I’m seriously thinking about running a free or very very cheap seminar for attendees on the friday if their was a charge it would be to cover costs for hiring somewhere. If you would be intersted in attending this seminar which would be on the Friday afternoon then please let me know if there is enough interest then I will organise one.

What I would be covering, metrics, human behaviour modelling and identifying patterns (including some nifty ideas for SEOs out there) as I say nothing concrete but either use the comments here or email/contact/tweet me (yes I’m now using twitter @tnash)

This seminar would only be for Think Visibility Attendees basically I’m offering a £500 half day course for free if people want it.

Oh don’t forget to buy a ticket!

Heat/Click maps

Most readers will be familiar with these but if your not basically a click heat map measures the number of clicks occurring on specific parts of a page (normally links) and then creates a pretty density match the more clicks the brighter the colour (normally they look like heat maps hence the name) they are used to see areas of interest for users and where users are leaving. These maps are really quite useful if you know how to use them and you can easily get started using open source software like Clickheat and there are also paid for services out there.

Temporal Click Density maps

So I’m sure the title (time based click maps) has given away what our current project is all about, you see clickmaps give a very two dimensional view of click density by introducing time, we get a much better picture of what areas are being clicked on and some motivation behind them. Unfortunately introducing time into web analytics is notoriously difficult perhaps more so then it would appear on first glance.

Time periods
The first problem is accurately recording time, either you rely on the server or the clients browser to record the time, both come with problems and neither can be considered accurate the solution in this case is simply to probe in time periods in our case we have chosen a near logarithmic scale for determining time periods. It would be unhelpful if 99% of all our clicks were in our first time period because we chose to long a period and likewise we wish to keep processing and data storage overheads down so polling data every second is unrealistic.

In focus or idle?
The next hurdle is determining if they are even looking at the page all modern browsers are capable of tabs and multiple windows, and so users could leave a page open and be surfing for hours on another window never to return to your site. Likewise a user referencing your site can be coming backwards and forwards between tabs or windows. Most browsers now support some javascript focus option which will allow you limited knowledge when combined with blur. However this does require that we first access the browsers capabilities . In reality we actually use compiled action script assuming the user has flash and fall back to javascript only really for iphone browser and similar.

Some Temporal Click Density Map goodies

ok so now for some totally unhelpful yet oddly cool tidbits we have discoverd while running temporal clickmaps on a couple of our sites. Remember now this is just initial findings on a couple of sites.

Greater the density the earlier the click – Afraid so from our initial finding the click area with the greatest number of clicks also had the clicks in a short period of time relative to their arrival. The positive side is a good call to action is likely to be clicked on quickly.

Adverts are clicked late – This was a little surprising I had always assumed advert clicks occurred quite early on but adverts tended to be clicked later, is this showing a subconscious desire to complete the primary call to action? Well possibly and (warning blatant plug ahead) its certainly the theory we have put forward for why newmedias.co.uk Your CAPTCHA advert wordpress plugin works (end of plug)

Further down the page the later – yep your hocked but the lower down the page the longer it takes to get clicked, there is an exception to this and that is pre known call to actions a good example being long sales pages where we saw a split between quick clicks vs long clicks. So some people realising the style of page shot to the bottom while others waited.
Interesting and this is when this sort of mapping comes into its own, those who waded through the long sales page and clicked to purchase were statistically less likely to complete the transaction

Stumblers really do hate you – a couple of years ago I posted about the fact that the most clicked area on a page during a stumble was the top left hand side and theorised that this increase was due to stumblers missing the stumble button. With the new system we can see the different density levels for the top left for stumblers vs non and with a noticable difference between the two the theory still stands to make things worse… nearly all those clicks are in the first time period meaning you really don’t have much chance to impress.

So what do you need to get temporal data going on your site, well clickheat map system is a good start I recommend labsmedia clickheat php class as a starting point. Then you need to measure time and focus we use an actionscript system but I would be interested in seeing or hearing other ideas. Indeed if people want to play with our code I maywell clean it up and throw it up on newmedias.co.uk

• Your Name
• Age
• Shipping Address
• Some basic Credit details.

They can almost certain make educated and pretty accurate guesses to:

• If you are at work
• your income bracket
• Gender

They might even in a few users cases be able to tell you what you had for breakfast all in a few seconds. Scared?

We recent worked with a client who was looking to improve conversions on their alternative payment methods in this case primarily Google Checkout. They were disappointed by the take up on their Google Checkout option (with such low commission for the first year I suspect they were expecting significant savings) we were asked to develop a method to target existing Google Checkout users and present them with using the one click checkout option in favour of the traditional credit card approach.

Solution

We ran a very short questionnaire, of sales made to ask a simple question:

Do you use any of the following Google Services (tick as many as you use):

The results were pretty staggering 55% used one of the above services with Google personalised homepage making up the majority followed by Google checkout!

Making it worthwhile
ok so we now know there is a feasible amount of users accessing Google services and a reasonable (12.4%) using Google Checkout to make targeting while 12.4 of participants to the survey we can normally add an error margin of + or – 2% which means theoretically 14.5% (approx) could be using checkout thats not a small majority. If we could up the number of completed checkout by even a few percent it would make a difference.

The idea was fairly simple instead of presenting a Pay with Credit Card button with a google checkout button as one of the alternate smaller payments, we would show the Google checkout button as the primary option and pay with credit card as the alternate.

This idea could backfire, users may have made a single one off payment with Google Checkout and absolutely hated it! We have no way of knowing we maybe suggesting something they loathe but lets for now assume that paying via Google Checkout is the preferred option for all those who have previously used checkout but if its a new user how do we know?

Iframes, Javascript and the DOM

Using just Javascript to do this sort of XSS is no longer really viable, however there are plenty of alternates using flash and PHP proxies.

If I was to get you to click this link https://www.google.com/accounts/ManageAccount?service=profiles&hl=en chances are that it has just opened your Google profile. Why is that? Well mainly because the people reading this are likely to use one or more of Google services with a fair few using Gmail. Now whats the first thing you do when you get online?

Ok so this page is going to be accessible via javascript for a lot of users, to access it we simply open it in an iframe. We then can use javascript to pull data out, by calling the document in the iframe using something similar to:

window.frames['myframe'].document

where myframe is the id of the frame, once we have access to the document we can use a range of techniques for extracting data from getelementbyid through to implementations of xpath (since Google can’t manage nicely designed web pages the later is needed)

note – Its worth remembering this sort of implementation is subject to the whims of the site your grabbing the data from! If they change the page layout you will need to match it. Also you are reliant on the user a) being logged in, b) their browser playing fair, each browser handles cross domain requests differently for example IE is a lot more strict then say Google Chrome the above code will not work on most browsers without some modification

If you take a look at the above you can see the section marked default payment you will forgive me if I have removed some personal details but it shows the default Card type and last 4 numbers if any card is associated with the Google account. So all we need to do is check to see if there is a 4 digit code there if there is then we can assume the user is a Checkout user if there is not or if we couldn’t log in then we assume they are not.

Once they have made their first purchase we can ask them to create an account and allow them a preference check as to how they wish to pay.

We have only been running the new system a few days but the initial data is positive sadly we can’t do a similar thing for Paypal and would need to rely on CSS history trick (see below)

All your data is mine

Lets go look at that Google account page again, I’m pretty frugal with my details yet on that page there is my Full name, email address, a shipping address (probably my home even) if you go down to the services you would have quickly realised that I use this account for work as some of my services include Google Web Optimiser, Adwords, Analytics.

So what about other sites and other information for example…

MSN / Yahoo
Similar to Google both have a central account location lots of information to grab should you choose to.

Facebook

http://www.facebook.com/home.php?ref=home

Gives basic login info including full name and their Facebook ID via the profile link which allows us to access:
http://www.facebook.com/profile.php?id=FBID&ref=profile#/profile.php?id=FBID&v=info&viewas= FBID

Which gets you the users complete profile, dates of birth, mobile phone numbers etc etc, oh and all their friends information to (well what they shared with your original victim sorry customer)

Twitter
I did say find out what they had for lunch, while Twitter information can be obtained through the Iframe method its much simpler but my friend TheHodge pointed out an alternate method utilising a bit of javascript and Twitter API the downside to this is that like the rest of Twitter you are reliant on the thing being up! Still I did promise we could find out what some people had for lunch and what better place, once you got the username, just pull in their feed and mine.

Profiling and Data guessing

Once we have all this data we can start making some guesses and profiling users, we could go the whole hog and do credit checks after all we have name and current address but in the UK that would probably abusing your credit license plus time consuming etc. We could also pull details from the land registry on their property and then estimate an income bracket, or we could look up their job description. I can’t find the study but the BBC ran a report a while back claiming people were less likely to lie about their Job title and description on facebook then on other sites such as LinkedIn etc always worth knowing.

Identifying if some one is at work
This is pretty easy, verify they have a job, check current time in their location is it between 9 and 5? reverse DNS on their hostname check against a list of common terms such as residential etc if their ISP shows no sign of being primarily for non commercial use, its within normal hours of work and they themselves do work its a reasonable assumption to assume they are at work! Of course they might not be but chances are they are.

What if they use a secure session?
Some sites like Paypal won’t let you stay logged in so you can’t grab information from the browser window unless they happen to be using Paypal within a few minutes of visiting your site. If we want to know if they have visited Paypal we will have to go down an alternate and less reliable route.

CSS Browser History

Your browser can identify visited links, so when a page renders it knows when you have been to certain pages before. Web designers can even style these visited pages with a different colour link, which means its equally easy for us to identify where a user has been. The first step is to generate a list of possible locations, then create a very specific visited colour. When the user visits the page it will show the other pages they visited in your specific colour, we then run a piece of script to identify elements with that specific colour on the page and we now have a history of where you have been.

Note: This is flaky and I mean really flaky I have used the term URL but I should really have said URI it is the exact location you have been for example http://www.example.com and http://www.example.com/#hello are not the same and in some browsers (such as chrome they would not both show in css browser history unless you visited both)

For more information and for the original script that see the Ha.ckers.org CSS History Hack

Final thoughts
If this has worried you a little then remember the solution is simple, log in and out of your sites, if you must stay logged in use a different browser (or in Chrome and new Firefox user their privacy mode)

However for marketers the ability to know as much information as possible about a user is gold but only if you can properly utilise it also keeping in mind data protection and retention regulations means a lot of the data you may find you can’t keep so can only be used for one off judgments (such as assigning a profile group, or changing checkout info around)

So now I will leave you with a simple question, how much are you worth? Your data I mean how much do you think all the data your exposing is worth?

Affiliates are great no seriously they are the online sellers best asset without them many of the big stores would barely make a penny. So its strange how so many of them take their affiliates for granted treating them as one homogeneous block. We were asked to do some work for a client and I thought I would share some of the theory behind it (warning this post contains logic and a sprinkling of basic math) even though its not strictly an SEO post many SEOs are asked to perform this sort of statistical analysis every day.

The problem
Company A sells a diverse range of products it has a large affiliate base and are keen on their affiliates doing the leg work. Their philosophy is to help the affiliates along as much as possible. While they don’t have a dedicated affiliate landing page (they prefer to encourage deep linking to specific products) their “special offer” pages are used within PPC campaigns and as a focal point for affiliates activities. These pages are regularly tested however traditional split testing has caused a problem. Several of their high earning affiliates have low conversion rates on these pages while the PPC campaign is doing well and according to traditional split testers this page is the best performing of all recipes that had been presented over a large dataset.

The traditional solution
We have numerous traffic sources all arriving at one area the traffic is bound to be looking for different things and at different points along the buying process it makes sense therefore to simply split this traffic into groups and optimise each group. So we now have 3 separate experiments running each with the same recipes. The results come in the PPC is up as is the misc organic traffic but conversions while up slightly are still disappointing for the affiliates, time to investigate whats going on.

Interrogating demographics

Eccomerce sites have a wealth of information about their users, their buying practices and with a little bit of data mining we can create a profile of shoppers (we have 12 in this study) on the site. When we match that with where they have come from we can begin to organise what sort of user is coming from our various traffic sources. Obviously there is crossover and sources such as PPC and Organic traffic should produce a diverse set of profiles as will affiliates on the first glance it is only when you break down your affiliate traffic by affiliate that the number of profiles reduces.

Since I really wasn’t interested in Affiliates making a sale a year I quickly discarded a few thousand records and associated bits to a core group of 500 or so, each ones sales patterns were looked at and users profiled. Interestingly the top 5 affiliates had far less cross over in profiles then those earning a little less. This points to the fact that they are using distinct niches and they were tending to not step on each others toes. The next step was to compare these profiles to our conversion information for our landing pages were they way outside of the normal profiles?

Our best performing profiles on landing pages were 3 distinct profiles to keep things simple we shall call them the bargain hunter, the infrequent and the impulse. In reality each of these had 2 or more subset profiles but we can quickly see the special offers landing page are converting people who are looking for bargains, who don’t spend often with the store and those whose shopping patterns are erratic but often put more then one thing in the basket before checkout. Now this is not surprising but its always nice to have stats back up common sense particularly as its not always the case! Our top affiliates users profiles were on the whole not part of our profiling grouping of high conversions though there was some crossover with impulse profile amongst them. Interesting affiliates were producing the highest returning customer group even from this pages (indicating even if they didn’t buy the deal they were remembering the site) . Through out all of this one affiliate was standing out his conversion rate was close to 25% while his traffic stats were similar to the rest of the top affiliates these figures were so high they were hard to believe and our client was suspicious something was afoot. His traffic was almost entirely a single profile and were purchasing specific products and only those products for our client his users were infuriating not responding to future mailings, rarely returning accept through his link to specific products something was clearly going on, could he be part of some secret shopping sect? Is he brainwashing? Or was something darker going on?

Investigating an Affiliate

The vast majority of affiliates get on with their affiliate managers its a symbiotic relationship one can’t exist without the other. Therefore most affiliates don’t mind when their manager calls them for a chat often its going to be over some offer or a rise in commission when someone refuses to communicate it signals if nothing else a deterioration in their relationship. When we were unable to contact this particular affiliate it seemed that we might have a bad egg it was time to unravel his traffic sources. Like most affiliates he used a couple of central redirects in an attempt to both hide the real source but also for stats gathering, a whois and DNS lookup allowed us to find the fact both of these were on a single server and after a few minutes we had a 122 domains mainly .infos to look through passing those through our link mapping software produced nearly 400k backlinks across the network, nice thats a weekend of fun. Thankfully manually scanning through such a large dataset is not really feasible. Taking a small sample of the 122 indicated most could be considered as “made for adsense” style sites scraped or just crud content with links pointing to other parts of the network and a pile of ads. Thankfully the link mapping also provided 6 domains (all hosted on separate servers) where the network was pointing to. So far we still have a suspicion and some dodgey link building methods certainly nothing to damn the affiliate we now also have the main sites he has been promoting.

6 sites some interesting cloaking to provide overly keyword stuff context to Google but on the whole these sites have good landing pages, they are auto generated and producing coupons for every product to get discount. They are also incentivising the offers through a ponzi scheme (the cheeky bugger!) basically incentives were only being issued if a user not only bought the product with the coupon and then recruited 3 other users who did the same. The whole thing was financed through the fact that the affiliate was earning a standard commission regardless of the coupon and so was using the difference to payout in the unlikely case he needed to, I have no doubt when you joined “the club” you would be put on a mailing list and treated to a bombarding of mailshots. The system was ingenious but totally against our clients TOS and depending exactly how he was funding the payouts possibly illegal in the UK. So having ruled out our super affiliate as being the answer we handed over our findings to client for them to deal with and went back to the original question.

With our anomaly ruled out as a way to improve our affiliate conversions it was time to start gathering information on the presale our affiliates were using, this occurred more or less the same way we investigated our first affiliate, in addition each affiliate was sent a very quick questionnaire and a couple of follow up conversations with several of the affiliates and we probably knew more about our clients affiliates then they did!

Our clients affiliates ran into four main groups, the lander, the blogger, the comparer and the mailing list guru. The landers are affiliates using lots of mini sites to push a certain product often very much into PPC and SEO they push very targeted products. The bloggers run blogs in our clients case they were mainly gadget and sport blogs they would be targeting individual products but would often reuse our clients site over and over again. They were the group with the most residual traffic after a launch. The comparison websites are big business for affiliates and with some studies showing 1 in 4 people now using a comparison site for finding online purchases its no surprise that 2 out of our clients top 5 affiliates were primarily using comparison sites to drive traffic. Finally the mailing list such a powerful tool in Internet Marketing was actually the least used but as our client mentioned when we started they always knew when 1 of their affiliates released his newsletter by a small spike, the downside this affiliate brought almost no long term traffic.

With our affiliates traffic sources and profiles in mind we went back to the split testing to provide a unique and adapting page for every affiliate was not practical so instead we took our 4 traffic source groups and then cross referenced their user profiles the result was 7 diverse profile groups each was then run in an experiment of their own with the original recipes. Results were pleasing with all 7 experiments seeing a large leap in conversions it was interesting that out of the 7 experiments none were exact matches with each other yet our comparison website profile experiment matched the PPC.

Explanation – Ok someone is bound to ask, if we have experiments which rely on profile information and we don’t have any user data how are we assigning a user profile prior to gathering the information? The answer is we or rather our testing software makes an educated guess, we then refine the results later and push it back into the correct group as a corrupt test. The actual guess is generated using a genetic algorithm with demographic info we do have available introduced we then use that demographic data as a means to cause mutation within our default population, to provide an initial population in both experiments we include two or more mutators (profile indicators) and we can feedback information from successful sales the result should introduce two or more variants once we have enough feedback information we can remove or scale back our initial mutators.

Alternate Approach – Evolution

While we do rely on the use of genetic algorithm math to make educated guesses about profiling it can actually be used to generate the entire landing page profiling as an alternative to our experimentation. Rather then considering our source as the population we would consider the page to be the population with our testing variables individuals since genetic algorithms are ideally suited to provide a positive or negative result of an individual within a set we can use the same math to “breed” our individual (testing variables) to identify strongest matching pairs which can in turn be tested against one or more pairs or individuals. Their have been several interesting experiments on this type of split testing and its something we have found works well on large data sets. However the additional processing power may not justify its use unless the results show a marked improvement in conversions.

Quick Tip – Statistical data collected is temporal it is effected by time an obvious example is a searcher looking for Funny Christmas Hats today (mid December) is likely to be a different profile to one looking in January the search maybe the same as perhaps the intention but chances are the result is not.

Affiliates are not a single group

The client site now when running a special offer page has 9 experiments running on it reflecting 12 profile groups and 6 distinct traffic sources. The alternate without analysing both the profiles and traffic sources would have been to assume that each traffic source was capable of producing all 12 profiles (which they are but in most cases statistical negligible) resulting in 72 separate experiments. Without profiling we would never have been able to identify personality groups and would have relied on traffic and guess work the key is to remember that testing is only useful if you know what variables are actually controlling the test.

I hope this little study has helped people to see the potential of mining your data and that of your affiliates but its really be very much theory based so here are some handy and a bit more practical tips.

Generating Profiles

You cannot know your customers individually and eventually you have to accept that you are going to have to start grouping them. Using their spending habits along with other demographic information you should be able to split your users into between 6 to 18 distinct groups. Once you have your profile groups you can target and market to these groups.

Pool your data externally

Most sites do not have all their customer data in one place, apart from order and purchase logs, stats will be held elsewhere and questionnaire data in yet another location. Keeping in mind the concept of temporal data it is far easier to take snap shots of your data sources and pool them for analysis while most work is automated initial analysis and some interim work still needs to be compiled so having a stats package handy is always good. While I know most can’t justify a copy of Crystal, Matlab or SPSS there are free alternatives that will work well for you such as the open source PSPP

Is it a gift?

During the purchasing process its a great idea to understand the reason for purchase this simple easy to answer question allows us to discard the user within our normal product profiling, whats more it allows a simple upsell for adding wrapping and alternate delivery at a small cost.

Interrogate your data not your users

Questionnaires are a great source of information though most opinion is subjective the demographic data is not but people who answer such questions will lean into one or more profile groups. When asking questions to users keep it short and sweet you are looking for ways to “enhance” their future experience not put them off ever shopping with you again.

Think Tescos – Unique Coupons

If I have a set of profiles which are unique from purchase history and I have a users purchase history it should be easy enough to provide a unique coupon designed to encourage them back to the store. Tesco are experts at this through their clubcard but very few online retailers use targeted coupons mainly because of delivery methods. But by providing a monthly newsletter with generic material chosen based on purchase history with unique coupons based on purchase history and profile a user can be tempted back.

A simple example 2 users both interested in computers both purchased several items in the past. Customer A is an impulse buyer – his coupon is a discount % of a single product the discount is substantial but the site makes money on the chance that he will add more items to the trolley at the full price. Customer B is a bargain hunter he is presented with a % of all items which is significantly lower then customers A percentage off. However customer B is likely to purchase only a couple of items and so the store wishes to retain some markup on the items.

The downside to this method is the sheer amount of data that is required plus maintaining a high delivery method (assuming email this means running your own mailing list system not using a 3rd party like Aweber)

What methods do you use for profiling users? Do you think we can pigeon hole customers into 6-18 little groups?

For those that are not sure what this is about, Adobe and Google claim Flash is now “indexable” of course since most SEOs are sheep they spent a lot of time cheering even those who were sceptical still clapped liked penguins. For those of us who have an interest in Flash Google announcement was nothing more then a minor update to farcical heap of dung. But lets ask some very basic questions:

• Is Flash indexable?
• Does Flash Rank for terms?
• Is a Flash Link Followed?
• Does a Flash Link pass juice?

So back to the original tests….

Back in February I updated a post (Flash the SEO Devil?) from previous April discussing a set 30 tests over 4 domains to see what information the search engines saw from a Flash file.
It looked like…

Source:http://www.timnash.co.uk/02/2008/swf-seo/

Flash SEO Now

Not pretty basically ignore Ask and they all could manage to do something, but none of them could really deal with anything complex such as converting text into a symbol. Also multiple screens and points on timeline caused lots of problems, and then we got to passing link juice it just didn’t happen.

What’s it like now

So 6 months on and a major announcement later…

Drum roll please
yes you have guessed it nothing has changed!

Actually that is not true Google now doesn’t index 2 of our test files but it does find 3 extra lines of content in one of our files. So lets taker a closer look at Googles implementation of the Adobe Search Indexing Solution:

Flash Indexing Today

Is Flash indexable?

Yes. Generally text elements held in a single screen or scene within a Timeline that uses limited actionscript can be full extracted. Complex actionscript can cause some problems but on the whole textual content can be got at but when it is extracted no positional (within the timeline) information appears to be passed.

Flex Indexable?

Yes, because of a change in the Search SDK to call SWF within a player and allowing it to make its calls first most Flex applications can be indexed this is actually a pretty big thing if the rest of it had been sorted. But Server Side Flash is the one area where improvement has been made.

FLV Subtitles Indexable?

Ok this was never going to happen but we can only hope.

Flash Paper

Well this died when Macromedia were bought by Adobe and was a bit to close to PDF for comfort however if I was given the choice of PDF or Flash Paper, PDFs are indexed and links followed so are probably preferable from an SEO point of view.

Does Flash Rank for Competitive terms?

Not without a serious amount of external links and it never will until Adobe stops pandering to Flash Designers desire not to have to change their habits, for more information why see Adobe You Just Get Search. But Lets look at the factors that help rank pages, excluding external links…

Is a Flash Link Followed?

The original SDK would find all the links in a Flash file and put them in a text file minus anchor text for the search engine to do what ever they wanted to. The new version appears to do the same thing, which means that links do not appear to be followed.

Does a Flash Link pass juice?

Do I have to even comment if the links are not being followed then they can’t pass any link juice, but if the new Search mechanisms work in the same way as the old SDK then the link would be stripped from its context anyway so even if it was followable it wouldn’t pass any serious juice.

Other Flash Ranking factors

Well given Adobe has not provided a mechanism to mimic heading tags etc this is pretty much none. However this is the place where Adobe and Google could work hand in hand and truly make Flash and more importantly Flex searchable but it does require changing designer habits.

Flash SEO Tests next generation

So now its time to think of some of the problems flash will face even beyond those we initially posed (and which it failed) So here are some of the current tests I am running and will return with the results in part 2.

• If an SWF is embedded using the embed tag but not linked to will it be indexed?
• What about the current crop of Javascript Players?
• If in robots.txt file the Flash file is blocked but the embedded page is not is it indexed?

Yahoo and Flash

Always the bridesmaid never the bride they seem to have been missed out of all the major announcements yet they also have access to the new technology (unlike Microsoft) but it would appear have yet to implement the new features as of writing.

I have a whole other post dedicated to the subject of how, but quickly each page was grabbed, all block elements (div,span etc) were turned one colour and stripped of all other info to make life easier the block we wish to identify was turned red. Then the page had a grid overlayed (24×24) and each square was checked for colour of it’s content.

With a working mechanism we then went and pulled a 100 SEO sites (Sites which advertised services rather then just blogs though many did have a blog as well) backlinks (we removed links that were internal and capped it at 200 per site) and mapped to our grid. The result (17680 backlinks later) is the ability to see popular locations for inbound links to SEO websites, which may or may not interest you.

The following is a pseudo heat map, blue is the limited number of links, orange a reasonable amount red is a lot.

As you can see, the first thing is there is a lot of blue, the red areas are primarily in side bars, footer and inching into what would be comment area on blogs. The orange is a bit more evenly spread out, of interest though is the lack of serious link cluster in page content themselves. The left hand side is interesting there has been a trend away from left hand side bars and so comments in a blog may be counted in that left hand side.

Useless trivia, out of the 100 SEO sites we tested:

• 86 had 1 or more inbound link coming from footer area
• 96 had 1 or more inbound link coming from the right hand side sidebar
• 45 had 1 or more inbound link coming from left hand side
• 15 had no inbound links in content area on any sites
• 36 sites had 1 or more inbound links with only image as an anchor
• 98% of inbound links found in footer to our SEO sites site wide
• 99% of right side bar links found to our SEO sites are site wide
• 40% of left had side links found to our SEO sites are site wide

You can read how the experiment was set up on the previous click through article. One thing I should add both sites are in the UK and we only monitored the click through from Google.co.uk and not .com or other regional, this makes the data set much smaller but also more reliable.

Remember this is nothing to do with higher rankings, merely attempting to improve click through rate from results pages once your already ranked.

UPPERCASE EXPERIMENT

So to recap we changed the case of the content in the title tag to be entirely upper case. In both experiments we had clients with 2 sites in the same SERP so that we could provide a norm, and experimented on the lower ranking of the two.

So the top blue line is our higher ranking site for the two weeks the uppercase was in effect (remember this is our norm and hasn’t been changed in any way but ranks higher so will have a higher ctr), the cream brown line is the rate of the two weeks for the site we have uppercase, while the cyan blue/green is the previous two weeks.

The fluctuations may seem large but keep in mind the number of clicks from the highest peek and trough on the blue is just 8.

Clearly our UPPERCASE title has not done as well in the two weeks compared with the previous two and it’s interesting to note its peeks and troughs are in a slightly different pattern to our higher ranking site. All in all for the two weeks it consistently under performed.

UPPERCASE and Click Me

In this experiment we added a Click here for wording before the title with the entire title in uppercase, the idea give some one a direct call to action and they will follow. To make life more interesting the product related to this SERP was featured on a TV program in Canada causing a small spike in CTR even on the .co.uk with a much larger one in google.com.

Like before the Blue is our higher ranking norm (notice the spike), the mocha brown line is the same week as the blue with our call to action and uppercase, while the cyan is the previous two weeks

Our call to action does not on first glance to be performing badly but keep in mind the 25% increase in the period for searches related to our product and the click through is looking less impressive.

What did visitors think

To assist with the experiment we asked a few visitors arriving via google.co.uk if the experiments had made a difference to them, we started by showing them a picture of the Google result for the site from the previous two weeks and the current look asking them which they had clicked on.

• Uppercase results – 90% remembered clicking through the uppercase version
• Click here results – 100% all remembered they clicked through the click here version

Next we showed a generic result, and asked which element first drew them to our site

• Uppercase results – 70% Said it was the title that had inspired them
• Click here results – 60% Said it was the title

Finally we asked if the site was the first site they had clicked on the search page?

• Uppercase results – 50% Clicked the test site first
• Click here results – 75% Clicked the test site first

What about sales?

Sales figures are only half available at the moment but the conversion rate has seen a slight performance increase in the click here experiment, but remember this is more likely to be related to the Canadian coverage.

So at the halfway mark what have we learnt?

• Neither experiment is showing improved results, indeed the click through rate is lower
• Both experiments were distinctive enough for people to remember the “odd titles”
• The survey indicates the titles do make the result stand out
• Far to many Brits watch Canadian TV!

Before I go, no I will not be revealing the keywords, site information or client info. A) the experiment is still on going, B) Both clients have been kind to let me run the experiments and deserve praise but both have asked for privacy. So come back in a couple of weeks for the full results.