The two big releases from Paypal have been Adaptive Payments and Adaptive Accounts, Adaptive Payments is a collection of methods of payments that changes the way you do business online (at least that’s what the marketing department at Paypal said I still think they are pretty useful though) while Adaptive Accounts will allow you to generate your customers with Paypal accounts on the fly (yes you read that right) This article I’m going to focus on Adaptive Payments and in particular split payments, though fear not I will be talking about the other features over the next few weeks and months.

Split (parellel) or Chained Payments

One of the features that is going to appeal to Internet Marketers and Ecommerce developers is the Split and Chained payment feature, this allows someone to make a singular payment but the payment is split amongst several receivers this immediately opens up some amazing possibilities. With Adative payments you have:

• A Purchaser or Buyer - the party who is buying the product
• A Receiver - a party receiving money from the Purchase
• An API Caller/handler - a Party which instigated the transaction and determines where funds are sent, the API caller maybe a receiver but can't be the purchaser.

Split Payment

A singular Payment is split so the money is divided amongst one or more parties, each party is shown on the invoice, in effect while their has been one payment their are several transactions. The purchaser can see each party and interact with them, for example requesting a refund against one of the payments rather then all.

Chained Payments

A Chained Payment a singular party becomes the API Caller the purchase is made in the API caller name and the payment is then split to other parties which could include the original API caller. In this scenario the buyer only sees the API Caller in the purchase even if several transactions have occurred.

Paypal Split and Chained Payment Usage Examples

Trials and Upsells

Using Split payments a merchant can provide multiple payment cycles for different products for example:

A user visits a site and purchases the online version of a product and is also offered access to a Joint Venture Partners product as part of an upsell if he agrees a split payment is made one to the original merchant and the other to the JV partner, while the user has made just one payment his invoice clearly shows two separate transactions. A problem with one of those transaction that results in a refund only effects that transaction. Allowing JV partners to have a direct financial relationship with the customer during the merchants sales person. Transactions do not have to be of the same type, while the original purchase maybe a one off payment a JV transaction might be a pre approval subscription.

Drop Shipping

Dropshipping is where a site actually does not have any stock but purchases directly from a drop shipper on purchase. When the purchase is made the dropshipper is sent the shipping details of the client and the order as well as the wholesale price of the product plus their commission. They then ship the product on behalf of the original site. With Adaptive Payments Chained payments this can be instantly automated into just 2 payments rather then just one. When the purchaser sends payment to the Merchant the merchant then generates a split payment to all drop shippers along with sales information.

The advantage is that within Paypal every item purchased by the purchaser is passed to multiple dropshippers as one job making tracking processing a lot easier and far fewer payments.

Instant Affiliate Payments

Have a direct relationship with Affiliates you can literally split the payment, Why is this better then a monthly pay back, if their is a refund you can refund from both your Merchant account and the Affiliate account, either partially or fully! Allowing Affiliates instant access to their cash as long as they have the understanding it maybe refunded, of course you can choose to take the entire refund or indeed in part.

Is your Ecommerce system ready for Adaptive Payments?

Almost certainly not but over the next few months and years the larger e-commerce software will start to adopt this technology but by then a new breed of payment systems will be in place which can take maximum benefit. For Eccomerce systems to accept split payments they will need to create Adaptive Payments Gateway as a payment provider however to make maximum use then it will require a change in the way eccomerce systems work.

Ways current Ecommerce Systems could use Adaptive Payments?

So you want to use these right now with Zen Cart or OS commerce and similar? Well with a little work you could get something working, here is how I would implement Adaptive Payments Split Payments to notify 3rd Parties.

Let us assume certain products within your shopping cart require paying 3rd Parties, these are treated as regular products we just store their Product ID for future use along with who we have to pay and what call we need to make.

Assuming your Ecommerce system has flexible gateway system, building an Adaptive Payments gateway is very similar to building any Paypal based API gateway their are a couple of strange authentication changes meaning you will need to send Public Keys in Headers rather then in Post data. To generate the Payment details we would need to look at the Product IDs in the basket and check for third party products. Create a Payment button with the correct data and send the chained or Split Payment with the IPN return set to the cart, on return of the IPN request make additional calls as required to notify the third party suppliers, this is not unusual most carts have the ability to allow such calls after purchase for autoresponders. Of course this does require the third party to accept such calls and have some sort of headless Ecommerce system.

Why Headless Ecommerce Systems are the future of Ecommerce!

A headless system is one with no user interface, so no pretty web pages, rather its designed to be interacted with by web services. Think of them as listeners when data is sent to them they process the data and return if the transaction is completed. When combined with split payments these systems can be used to automate the purchase and delivery of digitial products or order of physical product without the user ever having to go to the companies own web site in effect allowing them to expand beyond their own user base. With Headless Eccomerce Systems a web site procurement program can be added directly into other peoples systems.

Examples

Membership site - A headless Application manages membership to a site allowing site owners to subscribe users to the site, after payment the API generates the user with subscription appropriate to whatever was purchased. The Membership can be integrated directly into the original merchants checkout process as a cross sale.

Printing on Demand API - A headless API which accepts print on demand orders from Publishers who as part of their upsell process offer hard copies of their book, on purchase the Publisher makes a split payment with the royalties going to the Author, Printer and themselves, after payment is complete they call the Printers API to fire off the print job.

This was a quick introduction to what split and chained payments can do as well as introducing the idea of headless ecommerce systems, over the next few weeks and months I plan to show more and more of the ideas as well as more practical examples of usage.

Hours prior to to Geocities closure, Reocities was born a project to archive its content, it got a lot of twitter time including myself and they manage to scrape a hefty chunk. However unknown to them Yahoo had been working to save Geocities through archive.org

For legal reasons Yahoo felt they could not wholesale dump Geocities content to archive.org but did provide the site with every assistance to allow them to store and and access all publicly available sites at time of closure, they also allowed their bots unlimited connections and bandwidth to grab as much content as possible.

Why didn't they publicise it?

Good question, but they were quite keen when I asked that the message was made clear they had worked with archive.org and that this was a long term plan, as David said back in June. With archive.org not being indexed however it does mean a large part of the web has still vanished from day to day searches and so potentially their still is a place for projects like reocities.com.

Sometimes watching television can be frustrating and I don't mean because of the silliness of CSI awesome computing and forensic capabilities No what I find silly is the way advertisers try to leverage the web. So I would like to introduce you to a case example.

Park Hampers – Park Hampers are a Christmas saving scheme based in the UK they work by you paying for your hamper in the weeks months prior to Christmas when it comes to Christmas time they deliver your hamper. Its aimed more at low income families or people who are terrible at saving. Their SEO team should be shot!

Their recent TV campaign tells you to visit park11.tv this is of course a redirect to a truly terrible landing page, but it is not the landing page that concerns me but the fact other then this URL there is no mention of Park11 or Park11.tv anywhere on there site this results in a Google Search for Park11 being:

Christmas Hampers? No. Parkisons disease for Christmas? No thanks

What has gone wrong for Park?

Park obviously want to track number of hits their TV campaign is bringing to their website so have created a unique url for that campaign the aforementioned park11.tv this I assume logs the user before redirecting to the landing page. Park have numerous other URLs leading to various landing pages and mini sites so this seems to be part of an overall strategy the problem is that as the unique url has diversified away from the core brand they haven't laid any foundation on their site to cope with it.

So they don't show in a Google search for Park or Park11 so what?

How many people put urls into Google not the url bar on a web browsers? I would suggest quite a lot especially those people who perhaps are not as web savy which are in part the customers Park Hampers are aiming at. So by having a sponsored listing they are simply losing traffic and what traffic they are picking up they are paying for via PPC. Not only are they paying for an advert on tv they are then paying to pick up the traffic from that advert!

In the case of Park they have a small reputation issue as well, while badly researched by them Park11 association with Parkinsons can't be helping as users arrive at Google but find no results for hampers they do read a pile of scientific jargon if that doesn't put the user off they may well visit the wikipedia page on Parkinsons. Which to be fair I think every one should so here is the link again.

How do you solve the problem

First off all sorting out the rest of Park Hampers site so that its pages are ranking, then reference Park11 campaign create a dedicated landing page for it with the keywords in use and at least internal navigation pointing to the site. Just to be safe surely you would optimise for keywords such as Park11 & "Park Eleven"

Park are not the only company not to do this, but since they are on tele right now advertising I thought they were the perfect example.

Update: Myself and Carolyn were bored over the weekend and so have been playing with look at Park Hampers redirect strategy to say its a mess is an understatement, they own park1.tv through to park40.tv and seem to be using most in various tracking, about 20 of the Google results for park1-40 actually have Park anywhere in the top 10, examples of other Park redirects include park5.tv and park15.tv. This itself wouldn't be that bad but in addition Park own another 8 or so domains including myparkmag.co.uk, gopark.co.uk etc these are not minisites but rather badly designed redirects. In short a mess, don't fall for the same mistakes.

At the time I first double checked our own bots don't do anything quite so stupid before suggesting that I thought it unlikely but would happily test it. Dave suggested a wager oddly enough one I never took him up on and I'm glad I didn't!

How we crawl robots.txt file

The need for speed is paramount when crawling a site, a bot is taking up server resources and you want it to complete its required action in as short a point as possible. If your bot follows REP (what's REP See Post notes for details )it's first action should be to download the available robots.txt file, on average these files are 2-4kb in size very small and take no time to download, however a file sent with a 404 is closer to 22-40kb assuming it also sends the associated html. a much larger size given the majority of sites do not have a robots.txt file this means if you are not careful your robot will spend more time downloading a useless file then anything else. The method we use is to simply ask initially for packets if the return is a status 200 we proceed to download the file, anything else and the status is stored and is ignored.

Is the way you do your crawler the correct way Tim?

There is no "official" recommendation within the RFC governing REP that covers how you should treat status codes and which you should follow to only follow Status 200 is by far the most efficient method but it comes at a cost as you could be ignoring the file! It also doesn't totally protect against downloading 404 pages as some servers send out a status 200 not 404 when a page can not be found.
A draft proprosal did suggest that other status codes should be followed including 3xx related to moved documents temporary or permanent it did not explicitly mention dealing with cross domains.
I have started to make changes to our own bots See Post notes for details

How does Google deal with cross domain 301 of a robots.txt file?

It reads the file at least according to webmaster tools, in Bronco follow up post they show Google Webmaster tools accepting bit.ly/robots.txt file nice should we be alarmed potentially though only if your allowing custom urls to your user at a root level on your domain with dots in them so if your running a URL shortener then yes perhaps something to check.

Did you do your own tests?

yes I had already done some tests last night which backed up what they did here is how I tested.

Experiment 1 Cross 301 oh please say this doesn't work!
We created two domains domain A and domain B with a robots.txt on Domain A and 301 to a file on Domain B the robots.txt dissallowed access to /test/ folder, a test folder was put on both domains and index file was put in both, each domain was given a root index cross referencing each and each of the test files.

If Google crawled the robots.txt then Domain A should have 1 indexed page, Domain B 2 when finished.
with a monitor attached to the logs doing reverse DNS looking for a Google IP so we could watch the interaction some links were thrown at Domain A.

Result: Domain A - 1 page indexed, Domain B - 2 page Indexed

In Webmaster Tools a status 200

Experiment 2 - Let's give google the benefit of the doubt
Ok so maybe they have indeed adopted the 1997 draft and are therefore obeying redirects it will ignore a Status 666 right?
Fresh domain this time our robots.txt file will be in the correct location but will send a http status of 666

Result: Domain A - indexed 1 page

In Webmaster Tools - status 200

Experiment 3 - given you a robots.txt file regardless
Ok so what if we tell you our server is broken i.e 500 but we give you a correct robots.txt file?
Fresh domain, correct location but headers sent are http 503 - Service Unavailable we are telling it we are not available the server is buggered in effect.

Result: Domain A - indexed 1 page

In Webmaster Tools - status 200

Tim - If you think about this it actually supports the belief google have actually programmed in the ability to follow 3xx as otherwise it would have for the 3xx returned a 404 or a 200 and blank file

Experiment 4 - I'm not here even though I'm here
Final test send http status 404 but also a valid robots.txt file what you going to do Google!

Result: Domain A - indexed 2 page

In Webmaster Tools - status 404

Only in the final test did Google behave as if it was paying the blindest notice to http status codes, can we assume 404 is hard coded and it will accept anything else?

Why should you care?

While the potential for abuse is small unless you run something akin to a URL shortner what happens when your site is producing an intermittent 500 error. From playing with status codes it would seem Google if shown a invalid or unreachable robots.txt will continue to use the old file could this be a potential for abuse what about a sneaky redirect only google on a 301 from your robots.txt by a very mischievous hacker. food for thought, and I'm glad I didn't take that bet of with DaveN.

What you want more? Seriously I mean what do I need to tell you?
I will be running a seminar on Profiling website users, looking at life cycles and a huge pile of tricks and ideas to get the most out of your users. Want to know if colour effects genders differently? Want to know how to estimate gender in website visitors?

Then you should be attending this seminar! What's more its free well sort of, its actually free to Think Visibility attendees everyone else its £50 but of course what this really means is if you buy a Think Visibility ticket at £99 and use my discount Coupon TIMNASH the price will drop down to such a level that you got to ask yourself why not come to both

I'm afraid while this seminar will be filmed it will be only for internal use and not available for purchase, if it's a success I might run a future seminar and film it.

If you are interested in improving your conversions, or even just curious as to what it is that I find more interesting then SEO (or if your into SEO why what we do is so much more then the normal) then come along for the afternoon it's free, simply register to confirm your place at the behaviour modelling seminar.

Passive Profiling

From the moment a visitor enters a site a wealth of information is provided by them including hopefully where they have come from, their browser, their physical location (of their ISP at least), their language all of this can and is faked by a small minority but for the vast majority this information can be treated as reasonably accurate. Even with this limited information we can make educated guesses about a user for example we can guess if the user is at work or not.

Once the visitor starts to interact with the site we gain even more information for example once a suitable number of clicks have been recorded we can start to use behavioral models to attempt to estimate gender and age See Post notes for details. Return visitors provide even more information how do they return direct or through the original referrer? All of this builds up a picture which helps to determine how to present the site to the visitor and change them from a visitor to a “user” but this sort of profiling requires the user to stay on the site long enough to profile the visitor. 10 clicks may not sound much but that could easily be a completed transaction or a lost sale before passive profiling has had a chance to determine the best route for the visitor.See Post notes for details

Active Profiling

Active profiling in this context is confronting the visitor into a decision making process rather then allowing them to wander around the site on their own. This sort of profiling is not without risk people baulk at being told what to do and especially when a site does not operate in preconceived boundaries of how a site should behave. For many people the website asking questions just isn't cricket!

The key to active profiling it to force the user into making a decision which can then be profiled this is usually done by removing navigations or hiding content behind some sort of form. By giving the user no real choice (other then to bail) but to tell us something we can then use to show them the next step in our profiling. Let's take a really simple example if I am selling software I may want to identify larger corporate customers from small businesses I therefore present visitors with a screen with two options one for small businesses and one for corporate customers. Each leads to a different landing page selling the same software but highlighting different features of the software simple and pretty un-invasive we are adding a step between getting the information and this will increase the number of exits at that point but if those that stay convert higher percentage it's worth losing a few visitors to gain the information needed to improve conversions.

Pros of Active Profiling

• Gather information that you just couldn't gather through other means
• Confirm information gathered through passive profiling
• Can be introduced directly into visitors funneling process
• Much quicker and more reliable data then passive profiling

Cons of Active Profiling

• Higher bounce and exit rates
• Can cause confusion to visitors

The best method is of course to combine the two methods to gather as much information as possible while keeping the active profiling limited and filling only vital gaps.

Landing Pages - Case Study

Lets take an example of a site selling laptops to households it's target demographic is 18-30yrs olds in the UK.

On arrival to the site notices the top of the site has a loading icon in reality the site is well optimised however the site is generating the new arrivals profile to put them initially into one of 6 groups. Male/Female/Neutral UK/Not UK to do this the page is pulling the IP address to confirm country if not able to identify assume not UK, and using css history to determine gender (if this fails pesky firefox user! Then gender neutral)

Once the initial profiling is generated the assets are loaded a “British male” is presented with a female “Kirsty” , “British female” would get “Craig” while other would get “Barry” each have certain charms designed to entice and more importantly distract the visitor similar but more generic characters appear for the other 3. These characters then ask a question with four choices (this is our active profiling) Once loaded the characters have a hover element track on them to help determine how distracting the characters are being so that on future pages they can be brought to the fold or are pushed back into the background. The only two pages where the visitor is not determining the character position (though of course the visitor has no idea they are or it would ruin the effect) are this initial page and the last checkout page where they are always at the front almost egging the user on.

Once the user has “answered” the characters question they are further subdivided into groups from this point the wording, colours and location of buttons are determined by the profile they are in. In effect it determines if “Kirsty” talks tech to you or blue is your colour!

This sort of segmentation is highly effective if you already have a tight demographic you are targeting.

Preventing Comment Spam and Payment Fraud – Case Study

Something we have been working on for a little while is the way to identify users that are most likely to be spammers prior to them actually leaving the comment rather then just relying on identifying the comment as spam. For truly automated spam this is pretty easy to do, for human spammers (those people being paid through systems like amazon turk to leave comments) it's a little harder to identify but not much. By using a first past post point scoring system so no one issue will cause the visitor to be considered a problem you can build up a passive profile and compare it to a typical spammer who will have a specific click pattern, hover over search words and many using blocks of known IP, or indeed from a certain country in some blog cases. Once the passive profile has been matched the active profiling is engaged and forces the user into a simple question “Are you a spammer?” ok so not quite however when the visitor makes a comment rather then an immediate submission the user is presented with a logic puzzle or simply a question this is not to prove they are human as we actually know that already! But to see if they have engaged with the site or the post. If they pass through this active profiling then the normal spam prevention kicks in, if they don't then they can be subjected to what ever torments the webmaster has in store for them (a java applet is always cruel).

Similar techniques can be used to identify potential fraudulent transactions where users passive profiling might indicate their behavior is “suspicious” the active aspect would be used to reiterate part of the transaction process.

Over using Active Profiling

This post was inspired when my friend Andy Beard pointed me to a marketing tool which uses active profiling (though being marketers they added 400 additional buzz words) via a quiz to generate sales pages in what you could consider to be traditional quizes. When he initially pointed me to the program I was presented with a quiz asking quite leading and obvious questions before I could “see the video” after the second question I simply started randomly clicking until it went away it was only then did I realise the thing that was irritating and annoying was in fact what Andy wanted to show me!

So why did it fail? Well over use of active profiling results in “oh get out of the way” mentality where the user blindly clicks or feeds false information to just get rid or past the profiling system. This is a triple disaster you not only haven't gathered useless information, you don't know you have gathered duff information (unless you are confirming passive profiling and are double checking for errors) and you have an upset visitor more likely to bail. The solution is to use active profiling sparingly and only as part of a funnel sales process.

Do you use any of these techniques? If so have they helped?
If you have found this article interesting you might also like to take a look at my recent introduction to this type of user profiling.

<

What is SENSE Tim?

glad you asked SENSE is the uk largest deaf blind charity (note no and) it's dedicated to people who are both deaf and blind normally from birth it is an amazing charity and the work they do is vital to help give some hope to people who live in utter isolation caused by their own bodies. If you want to know more about what SENSE does I really suggest you check out their website sense.org.uk

So back to my day while I appreciate her enthusiasm her timing sucked so I cut her short and said very bluntly what's the website address.

She looked blankly and said well Google SENSE but I think it's .com (it's not it's sense.org.uk) anyway I dashed off but made a mental note to visit the site which I duly did while sitting on my train it's pretty terrible and so I spent some time making notes which I will send them though I doubt anything will be implemented but the whole incident got me thinking.

How do you record an off-line campaign accurately?

The simplest and most obvious route is when handing out information use a separate domain name for all offline promotional info that redirects to your main site. It's cheap to implement and assuming it's just a top level domain name change not going to cause to much confusion. you see larger companies do this on TV adverts normally with the .tv version of their trading name. However lumping all the traffic from your campaign really doesn't help you from an analytics side it also makes working out what promotion is working harder particularly as off line campaigns can have long lag times mail shots going astray for 5 weeks that old poster being seen in the railway station a year after it was put up. Getting any more detailed information and we are going to have to start going beyond domain names.

The problem is simple the longer the url the more typing the person needs to do the less likely they are to complete the whole url but give up and either go to the domain root or worse Google it. Yet to give each campaign a new domain would seem impractical for most businesses and a management nightmare.

Shorturl vs domain name

we are all familiar with the idea of using shorturls for social media campaigns where we are limited to a set number of characters but would they be usable for our offline campaigns.

Pros - Short equal less characters, hopefully means less likely to abandon type-in
cons - Any domain recognition goes out of the window, often short urls will use strange domain extensions and might prove hard to type in.

So the question is a potential customer more likely to type in:
www.mydomain.com/mytrackingurliscool or sen.se/wrt4

Well this afternoon I managed to convince a client to test it, here is the proposed plan.

3 separate mail shots each will include a unique shorturl for that mail shot (indeed we could go to that mail/poster etc) the shorturl domain is purchased and is 7 characters long including the dot. The domain root logs visit and redirects to the main site, the unique urls act like any normal domain shortner.

To test we will be split testing against long url equivalents though with an extra /m appended to the end for tracking purposes.

I will let you know the results but what do you think is this a good idea have you used such tactics? oh and if you happen to know anyone who was canvassing for SENSE in Sheffield today point them at this post and to the nice blonde lass I'm sorry if I was rude to you but I hope I have made up for it.

Her Majesties Privacy document

The British Monarchy web site, with the exception of the subscription system, does not store or capture personal information, but merely logs the user's IP address, which is automatically recognised by the web server.

We do not use cookies for collecting user information, and will not collect any information about you except that required for system administration of our web server.

http://www.royal.gov.uk/Contactus/Aboutthissite.aspx

While the above is technically true it is let down by


< script src='../TemplateAssets/GoogleAnalytics1.js' language='javascript' type='text/javascript'>< /script>


Queens SEO

Patrick from blogstorm who also was wandering through the Queens source code noticed the amusing use of meta tags.

< meta name="keyword" content="Queen" />

The single keyword per tag is down to the ASP content management system they are using, actually it seems to do several odd things but at least no SQL errors (unlike the assumed developers site)

Talking of which I did think the web development company (I assume that's the connection) leaving in their URL was a nice touch

< html xmlns="http://www.w3.org/1999/xhtml" xmlns:Abseil="http://www.coraider.co.uk">


Staying on the SEO front, no h1 or 2 tags, and the h3 tags are nicely keyword rich with headings such as "Quick Links" and "Latest Videos" at least most images had an alt tag though no title tags on either images or links.

Royal duplicate content - yes I'm sure you are not surprised that the site itself is an indexed mess, most articles have at least 3 versions (ignoring the welsh and Gaelic language versions) with no order or priority in terms of indexing so some articles the text only version is showing, others the full content. The URL structure is also confused with some pages appearing as

www.royal.gov.uk/output/Page5140.asp

while others are:

www.royal.gov.uk/OutPut/Page6369.asp

Camel casing on a non unix server is not a disaster but if you have a url structure stick with one, but fear not because the new site has done away with these and now use categorised urls...

Only they are not indexed and the old ones, you guessed it 404s

Anything else? I really only spent a few minutes, to be fair the site is no where near as bad as most government websites but still a total fail. When making high profile launches your site will be picked at but sometimes it pays to actually check for the little things.

Update: Even the telegraph is having a dig six points for Search engine optimisation for the queen let's hope Queen Elizabeth trusted website team corrects some of their faults.

One final thing and its not SEO or privacy based, why when I look at the site do I think it looks like an old Nuke site?

Heat/Click maps

Most readers will be familiar with these but if your not basically a click heat map measures the number of clicks occurring on specific parts of a page (normally links) and then creates a pretty density match the more clicks the brighter the colour (normally they look like heat maps hence the name) they are used to see areas of interest for users and where users are leaving. These maps are really quite useful if you know how to use them and you can easily get started using open source software like Clickheat and there are also paid for services out there.

Temporal Click Density maps

So I'm sure the title (time based click maps) has given away what our current project is all about, you see clickmaps give a very two dimensional view of click density by introducing time, we get a much better picture of what areas are being clicked on and some motivation behind them. Unfortunately introducing time into web analytics is notoriously difficult perhaps more so then it would appear on first glance.

Time periods
The first problem is accurately recording time, either you rely on the server or the clients browser to record the time, both come with problems and neither can be considered accurate the solution in this case is simply to probe in time periods in our case we have chosen a near logarithmic scale for determining time periods. It would be unhelpful if 99% of all our clicks were in our first time period because we chose to long a period and likewise we wish to keep processing and data storage overheads down so polling data every second is unrealistic.

In focus or idle?
The next hurdle is determining if they are even looking at the page all modern browsers are capable of tabs and multiple windows, and so users could leave a page open and be surfing for hours on another window never to return to your site. Likewise a user referencing your site can be coming backwards and forwards between tabs or windows. Most browsers now support some javascript focus option which will allow you limited knowledge when combined with blur. However this does require that we first access the browsers capabilities . In reality we actually use compiled action script assuming the user has flash and fall back to javascript only really for iphone browser and similar.

Some Temporal Click Density Map goodies

ok so now for some totally unhelpful yet oddly cool tidbits we have discoverd while running temporal clickmaps on a couple of our sites. Remember now this is just initial findings on a couple of sites.

Greater the density the earlier the click – Afraid so from our initial finding the click area with the greatest number of clicks also had the clicks in a short period of time relative to their arrival. The positive side is a good call to action is likely to be clicked on quickly.

Adverts are clicked late – This was a little surprising I had always assumed advert clicks occurred quite early on but adverts tended to be clicked later, is this showing a subconscious desire to complete the primary call to action? Well possibly and (warning blatant plug ahead) its certainly the theory we have put forward for why newmedias.co.uk Your CAPTCHA advert wordpress plugin works (end of plug)

Further down the page the later – yep your hocked but the lower down the page the longer it takes to get clicked, there is an exception to this and that is pre known call to actions a good example being long sales pages where we saw a split between quick clicks vs long clicks. So some people realising the style of page shot to the bottom while others waited.
Interesting and this is when this sort of mapping comes into its own, those who waded through the long sales page and clicked to purchase were statistically less likely to complete the transaction

Stumblers really do hate you – a couple of years ago I posted about the fact that the most clicked area on a page during a stumble was the top left hand side and theorised that this increase was due to stumblers missing the stumble button. With the new system we can see the different density levels for the top left for stumblers vs non and with a noticable difference between the two the theory still stands to make things worse... nearly all those clicks are in the first time period meaning you really don't have much chance to impress.

So what do you need to get temporal data going on your site, well clickheat map system is a good start I recommend labsmedia clickheat php class as a starting point. Then you need to measure time and focus we use an actionscript system but I would be interested in seeing or hearing other ideas. Indeed if people want to play with our code I maywell clean it up and throw it up on newmedias.co.uk

• Your Name
• Age
• Shipping Address
• Some basic Credit details.

They can almost certain make educated and pretty accurate guesses to:

• If you are at work
• your income bracket
• Gender

They might even in a few users cases be able to tell you what you had for breakfast all in a few seconds. Scared?

We recent worked with a client who was looking to improve conversions on their alternative payment methods in this case primarily Google Checkout. They were disappointed by the take up on their Google Checkout option (with such low commission for the first year I suspect they were expecting significant savings) we were asked to develop a method to target existing Google Checkout users and present them with using the one click checkout option in favour of the traditional credit card approach.

Solution

We ran a very short questionnaire, of sales made to ask a simple question:

Do you use any of the following Google Services (tick as many as you use):

The results were pretty staggering 55% used one of the above services with Google personalised homepage making up the majority followed by Google checkout!

Making it worthwhile
ok so we now know there is a feasible amount of users accessing Google services and a reasonable (12.4%) using Google Checkout to make targeting while 12.4 of participants to the survey we can normally add an error margin of + or – 2% which means theoretically 14.5% (approx) could be using checkout thats not a small majority. If we could up the number of completed checkout by even a few percent it would make a difference.

The idea was fairly simple instead of presenting a Pay with Credit Card button with a google checkout button as one of the alternate smaller payments, we would show the Google checkout button as the primary option and pay with credit card as the alternate.

This idea could backfire, users may have made a single one off payment with Google Checkout and absolutely hated it! We have no way of knowing we maybe suggesting something they loathe but lets for now assume that paying via Google Checkout is the preferred option for all those who have previously used checkout but if its a new user how do we know?

Iframes, Javascript and the DOM

If I was to get you to click this link https://www.google.com/accounts/ManageAccount?service=profiles&hl=en chances are that it has just opened your Google profile. Why is that? Well mainly because the people reading this are likely to use one or more of Google services with a fair few using Gmail. Now whats the first thing you do when you get online?

Ok so this page is going to be accessible via javascript for a lot of users, to access it we simply open it in an iframe. We then can use javascript to pull data out, by calling the document in the iframe using something similar to:

window.frames['myframe'].document

where myframe is the id of the frame, once we have access to the document we can use a range of techniques for extracting data from getelementbyid through to implementations of xpath (since Google can't manage nicely designed web pages the later is needed)

note – Its worth remembering this sort of implementation is subject to the whims of the site your grabbing the data from! If they change the page layout you will need to match it. Also you are reliant on the user a) being logged in, b) their browser playing fair, each browser handles cross domain requests differently for example IE is a lot more strict then say Google Chrome the above code will not work on most browsers without some modification

If you take a look at the above you can see the section marked default payment you will forgive me if I have removed some personal details but it shows the default Card type and last 4 numbers if any card is associated with the Google account. So all we need to do is check to see if there is a 4 digit code there if there is then we can assume the user is a Checkout user if there is not or if we couldn't log in then we assume they are not.

Once they have made their first purchase we can ask them to create an account and allow them a preference check as to how they wish to pay.

We have only been running the new system a few days but the initial data is positive sadly we can't do a similar thing for Paypal and would need to rely on CSS history trick (see below)

All your data is mine

Lets go look at that Google account page again, I'm pretty frugal with my details yet on that page there is my Full name, email address, a shipping address (probably my home even) if you go down to the services you would have quickly realised that I use this account for work as some of my services include Google Web Optimiser, Adwords, Analytics.

So what about other sites and other information for example...

MSN / Yahoo
Similar to Google both have a central account location lots of information to grab should you choose to.

Facebook

http://www.facebook.com/home.php?ref=home

Gives basic login info including full name and their Facebook ID via the profile link which allows us to access:
http://www.facebook.com/profile.php?id=FBID&ref=profile#/profile.php?id=FBID&v=info&viewas= FBID

Which gets you the users complete profile, dates of birth, mobile phone numbers etc etc, oh and all their friends information to (well what they shared with your original victim sorry customer)

Twitter
I did say find out what they had for lunch, while Twitter information can be obtained through the Iframe method its much simpler but my friend TheHodge pointed out an alternate method utilising a bit of javascript and Twitter API the downside to this is that like the rest of Twitter you are reliant on the thing being up! Still I did promise we could find out what some people had for lunch and what better place, once you got the username, just pull in their feed and mine.

Profiling and Data guessing

Once we have all this data we can start making some guesses and profiling users, we could go the whole hog and do credit checks after all we have name and current address but in the UK that would probably abusing your credit license plus time consuming etc. We could also pull details from the land registry on their property and then estimate an income bracket, or we could look up their job description. I can't find the study but the BBC ran a report a while back claiming people were less likely to lie about their Job title and description on facebook then on other sites such as LinkedIn etc always worth knowing.

Identifying if some one is at work
This is pretty easy, verify they have a job, check current time in their location is it between 9 and 5? reverse DNS on their hostname check against a list of common terms such as residential etc if their ISP shows no sign of being primarily for non commercial use, its within normal hours of work and they themselves do work its a reasonable assumption to assume they are at work! Of course they might not be but chances are they are.

What if they use a secure session?
Some sites like Paypal won't let you stay logged in so you can't grab information from the browser window unless they happen to be using Paypal within a few minutes of visiting your site. If we want to know if they have visited Paypal we will have to go down an alternate and less reliable route.

CSS Browser History

Your browser can identify visited links, so when a page renders it knows when you have been to certain pages before. Web designers can even style these visited pages with a different colour link, which means its equally easy for us to identify where a user has been. The first step is to generate a list of possible locations, then create a very specific visited colour. When the user visits the page it will show the other pages they visited in your specific colour, we then run a piece of script to identify elements with that specific colour on the page and we now have a history of where you have been.

Note: This is flaky and I mean really flaky I have used the term URL but I should really have said URI it is the exact location you have been for example http://www.example.com and http://www.example.com/#hello are not the same and in some browsers (such as chrome they would not both show in css browser history unless you visited both)

For more information and for the original script that see the Ha.ckers.org CSS History Hack

Final thoughts
If this has worried you a little then remember the solution is simple, log in and out of your sites, if you must stay logged in use a different browser (or in Chrome and new Firefox user their privacy mode)

However for marketers the ability to know as much information as possible about a user is gold but only if you can properly utilise it also keeping in mind data protection and retention regulations means a lot of the data you may find you can't keep so can only be used for one off judgments (such as assigning a profile group, or changing checkout info around)

So now I will leave you with a simple question, how much are you worth? Your data I mean how much do you think all the data your exposing is worth?