Having spent a long time as a C++ image processing software engineer and SEO, I’ve always thought I had a good handle on what works best for image search. A background in the field provides insights into the possibilities for image classifiers, such as skeletonization, histograms and more complex image transformations.
So I thought I’d share an experiment, to put some theories to the test. Starting with an experiment with colour.
As with any test in SEO, there can be no conclusion as the rules change (Google keeps updating algorithms), and many SEO factors will interfere as they cannot be isolated adequately for statistical rigour. The following test is subject to competition from images already indexed and the inter-connectivity of the web through time. So this test is pseudo-scientific. That said, I hope it still proves to be interesting…
The SEO colour experiment involves a series of coloured images.
Pink SEO experiment images
pink seo colour experimentpink seo colour experiment
The current results for ‘pink seo colour experiment’ are:
current pink seo colour experiment
Green SEO experiment images
There are two green SEO experiment images
Green SEO experiment imagesGreen SEO experiment images
The current results for ‘green seo colour experiment’ searches are:
current green seo colour experiment
Yellow SEO experiment images
Yellow SEO experiment imagesYellow SEO experiment images
The current results for ‘yellow seo colour experiment’ searches are:
current yellow seo colour experiment
Purple SEO experiment images
Purple SEO experiment imagesPurple SEO experiment images
The current results for ‘purple seo colour experiment’ searches are:
purple seo colour experiment
White SEO experiment images
White SEO experiment imagesWhite SEO experiment images
The current results for ‘white seo colour experiment’ searches are:
current white seo colour experiment
Gray SEO experiment images
Gray SEO experiment imagesGray SEO experiment images
The current results for ‘gray seo colour experiment’ searches are:
gray seo colour experiment
So, the scene is set. A selection of big block colour images, with a ‘gray’ version of each image colour so we can see whether the filename and other descriptors can influence the decision for the classifier, or whether the information in the image histogram overrides this deliberate misdirection.
The number of indexed images is in the order of billions, so the capacity for complex image processing may be reduced. In other words, image classifiers may still be relatively simple to avoid huge processing overheads, hitting only the filter options offered in image search.
I’ll revisit this post when search results change.
One of many holy grails in search is striking the right balance between PPC spend and getting appropriate traffic through SEO. By SEO, in this article, I’m referring to the rather narrow definition of position in organic SERPs.
There are 3 main scenarios involving optimisation, paid search and organic rankings
Scenario A – A small business with limited budget
A client is ranking on page one, and receiving a paltry amount of traffic for a head term because their URL is ranking beneath competitors. The top positions are way out of reach, because the client is small, new to market, or has an unknown brand. Whatever the reason, the problem for the client is that the competitors are simply gaining the click throughs because they are in top spots.
In order to compete, the client turns to PPC and pays to be seen at the top of the page for important keywords, with optimised ad copy and a well organised PPC campaign structure.
Scenario B – A cash-rich business in a competitive market
A client is ranking well on page one, but wants more, much more. In fact, they are bent on annihilating all competitors and are prepared to spend on PPC to achieve this aim. This happens in competitive startups where brands vie for mindshare, or when impressions are all important to boost brand awareness metrics. This scenario is generally a declaration of war amongst marketeers, with PPC adverts littered with brand terms and brand references.
Scenario C – Most medium sized business with in-house SEM teams
Even established campaigns need optimisation (!)
Professionals in search are target driven. Whether the target is to increase ROI, leads or conversions or support marketing efforts in a timely fashion. The overriding objective is often to control budgets and minimise waste whilst optimising PPC campaigns automatically through APIs, through tools such as Kenshoo or Marin or manually via Adwords Editor. The process is likely to involve a pivot table or two. CTR for PPC campaigns is undeniably a very important metric. The best PPC managers will also pay attention to natural rankings for keywords used in PPC campaigns. The noble objective is to marry organic ranking data with that of PPC bids/positions and achieve the targets set.
When budgets are cut, a common tactic is to look at your keywords capable of ranking organically in 1st place, and then trimming the PPC budget accordingly – the logical question asked is “Why waste money on PPC, when we are getting 40+% of the traffic from being in 1st place organically?”
Conversely, if a competitor is ranking in 1st place organically, pushing your business into 2nd or 3rd place, then there is the fear that turning off top-placed PPC slots will be akin to “handing your traffic to the competition on a plate”. That’s the hold PPC can have on your business and this is where achieving a balance between SEO and PPC becomes paradoxical…
Let’s step out of the PPC world for a moment and consider organic search.
Most people in search would agree that there are at least 200 different factors involved in determining where a URL ranks in search results. Hold the front page! Furthermore, as time passes it is a safe bet that the algorithms competing and collaborating to determine the mix of results in the final ‘sort order’ become ever more sophisticated. Nothing too contentious there either I hope!
Amongst the myriad of ranking factors, it is quite likely that CTR is included in the decision taken. For SEO, Google makes CTR available through GA and GWMT and it can provide useful information when addressing poor calls to action in search results.
SEO’s have noticed for some time that new URLs are often ‘tested’ in search results – perhaps to determine an initial CTR score ?
The underlying principle here, is that a URL attracting all the clicks, could easily be the most useful page in the search results, especially if visitors hang around on the site post-click.
So, it makes logical sense that a URL attracting all the clicks should be promoted in the search results. After all, it is likely to be the best result available.
Conversely, a URL with abysmal CTR is unlikely to be kept in 1st place for very long… this seems like a good way to filter out bad sites – URLs in high ranking positions with poor calls to action, poor Ux, or badly laid out snippets will attract fewer clicks and over time drop in search results.
My question is: Do you think CTR may be a contributory factor in the calculations to determine the future position of a URL in the SERP?
My answer: is that there may be a time-weighted relationship between CTR and rank ie. a time lagged causal relationship rather than just a correlation.
Part of the work of an SEO is to optimise snippets to attract clicks (the analogous activity to improving PPC ad copy), one can monitor improvements in GWMT (With Change) statistics showing how the new improved snippet has changed the CTR for the better. Over a period of weeks or months, the URLs with improved CTR often move into higher positions in SERPs. Improving CTRs can help towards an improved rank.
The diagram below shows a typical screen layout following a search on Google. The blocks represent positions of PPC and organic search results that could be representative of any of our PPC scenarios.
In this sample layout, your clients PPC ad is dominating the page in position 1, forcing the PPC advert from Competitor B into position 2. The aim of aggressive PPC bidding may be to counteract the competitors strong organic position #2, in an attempt to attract their clicks.
PPC used to compensate for poor SEO
The paradox here, is that if CTR in organic results is used to determine rank over time, then by bidding through PPC you are harming your organic CTR.
The clicks for your brand will go to your PPC advert at the top of the page, starving your organic result of clicks.
A visitor looking for your company or brand, will see the PPC advert first and the organic result in third place will attract less clicks than it deserves. At least some clicks will be lost to the PPC advert at least some of the time.
What makes this worse, is that the resulting lower CTR for the organic result can be exaggerated by the very presence of the PPC advert whether the ad copy is good or bad…
If a visitor sees the PPC advert for your site at the top of the page but chooses not to click on it, then they may be less inclined to click on your organic result too. The decision ‘not to click’ on the PPC ad could be taken when the ad copy is poor or irrelevant, or when they’ve seen the advert before and it didn’t give them the landing page they wanted.
Whatever the reason, the PPC advert that you place in #1 must be good else you risk ensuring that the click goes to your competitor.
…and…
the better your PPC advert, the less clicks your organic result receives
So, running PPC is not a simple decision. A bad advert can turn people away from your brand, and a good advert will also steal clicks away from your organic result, increasing costs.
If, as is logical, CTR is used in the calculation of position of your URL in the organic SERPs, then by reducing your CTR as a consequence of running PPC campaigns (optimised or otherwise!), you are more likely to drop in search results than to improve.
Therein lies the paradox of SEO and PPC.
Time to test this theory and show some interesting results !
Introduction to test methodology
First, pick a single, highly competitive head term, for which the site ranks sustainably in the top 4 and for which there is a history of considerable PPC spend.
A term attracting many thousands of the clicks per day through PPC was chosen (typically over 3k visits per day), making it an ideal test candidate for a CTR impact study.
Test Stage 1: Does Turning PPC off improve CTR and rank ?
For 2 days at the end of March, we turned PPC for this high traffic term off in all our major campaigns. The impact on PPC traffic, for this term alone, can be seen below.
PPC turned off for 2 days on 30th March to test impact on CTR
Lets compare the PPC traffic graph above to the CTR and Average Position data for this same head term below…. Discussion on the impact turning off PPC has had on organic CTR follows…
Improved organic CTR lead to rank change the following day
When PPC was running, our head term averaged 3rd place organically, with a relatively poor CTR (for third place) of just 5%.
When PPC was turned off on the 30th March, CTR improved instantly to 9.23% (the term remained in 3rd place)
The following day (31st March), PPC campaigns were still switched off. The CTR for this term increased to over 15% and average position had improved to 2.5.
The really interesting day came next because the average position improved to 1.3. The organic CTR is just 12% because PPC had been turned back on.
If PPC had remained off, then the organic CTR in first place would have been higher.
Stage 2: Turning PPC back on
By the 2nd of April, PPC was once again running at near previous levels (2.5k a day visitors) for the head term.
Once again, the average position for the head term dropped back to 3.0 and the reported organic CTR had dropped back to 5%.
Our new aim was to determine if there was a ‘sweet spot’ for PPC alongside SEO, so we decided to aim to get the organic CTR as close to 9% as possible. This was the level at which organic rank seemed to improve…
Stage 3: Optimising organic CTR by dialing in PPC
The theory was that a CTR of 9% had been sufficient to improve the average position from third to first at the end of March within a day… (stage 1).
In order to test this idea, we estimated that by reducing PPC spend by half and waiting for CTR results to come through (as shown above), we might be able to improve organic rank once again.
This test started on 4th April and it took until Monday 9th April before we saw improvement in average position. By Tuesday 10th April (one week later), the chosen head term had an average position of 1.2.
One week later, aiming to support an organic CTR of 9% in third place, average position improved. By the next day, the head term was in first place.
Stage 4: Perturbation testing
Now that the head term is in first place organically and has a reported CTR of 25%, the next step is to increase PPC once more, to see if there is a point at which we lose this organic position and uncover correlations to reported organic CTR.
You can see in the graphs above, that on 13th April, PPC traffic has been increasing and the organic CTR has already dropped back to 12%.
At the time of writing, the head term is still in first place organically and our traffic from organic search looks like this:
Optimising organic search traffic by dialing down parasitic PPC - a test in progress
There is some cannibalisation of search traffic by PPC once more, yet for the time being the head term remains in first place…
In summary
We continue to test running PPC at various levels against top organic results, early results suggest that CTR may lead rank change, so there are two important points to consider:
you could be stealing clicks away from your organic result – harming the ability of a head term to climb in rank organically
with bad PPC copy, you could even be increasing CTR for your biggest competitor, boosting their ability to rank well organically…
PropertyFinder was bought by Zoopla and 301 redirected correctly with a link map that matched appropriate pages to each other across domains.
Until recently, the PropertyFinder website would not show up for the search, as the site had been ‘de-indexed’ in Google. However, looking at the screenshot below (Jan 2012), it is clear that PropertyFinder now has a Phantom URL in first place in Google search results:
Example 2:
Dothomes.co.uk was bought by the DPG Group and redirected to FindaProperty.com
Again, as with PropertyFinder, the DotHomes website would not show up in results until recently. Performing a site: search for this old domain suggests Google has acknowledged the permanent redirect:
However performing a direct search for the old domain (dothomes.co.uk) shows a mixed up result that:
displays the old 301d URL (dothomes.co.uk/)
uses the old 301d URL as destination for the current FindaProperty.com title
the current FindaProperty.com description and sitelinks
Phantom 301s have been appearing more frequently recently (since Dec. 2011), and understanding the full implications of this change in Google search results is important if:
1. You have a website founded on a hotchpotch of old websites that have been 301’d
2. You are considering a redirection project, such as conversion to friendly URLs from unfriendly, parameter based URLs
What is the impact of a phantom 301?
Websites that have been bought out, or taken over and had their pages correctly 301 redirected to new sites several years ago are most affected by this change.
The historic 301d URLs from these old sites are re-appearing in search results, taking up positions they used to occupy before the 301 was put in place. The impact is at least threefold:
There is a new duplicate content risk for the page redirected to (as the redirection has ‘failed’)
The search results show an old entry, with a less optimised call to action (historic titles, URL & snippet)
Equity may not be passed to the destination URL (as the redirection has ‘failed’)
Before you say, ‘I know how 301s work, and what you are saying simply isn’t true’, read on!
How 301s used to work in the ideal world
What used to happen, is that Google would crawl the old URL, receive the 301 header response and new location, then pass ‘some’ link equity to the new URL, index it and de-index the old URL over a period of a few days.
The upshot is that your new URL would rank immediately, and the old URL would be gone forever…
What’s changed ? Is this Panda related ?
What has started to happen, is these old URLs have popped back into SERPs.
There have been no technical changes, these old URLs are still 301’d (no changes made for many months / years in some cases), but the they are being presented in SERPs carrying the old site name and titles, showing the old URLs and often having much lower quality CTAs.
So the smoking gun points to Google tweaking the algorithm…
It is as if 301 redirects are no longer guaranteed to remove a URL from the index.
If Google believes the old URL is the best result for the search, then it will show it.
Digression follows:
My personal hunch, is that Panda is strongly related to CTR testing and bounce rates. Sites that have suffered under Panda have tended to have the sort of content you’d ignore (packed with adverts, poor layout and spelling etc), or the URL in the search results that you’d usually skip over, because the last time you clicked that site, it was full of articles based entirely on spun content.
Panda nailed those sites, they were the sites people didn’t like to click on, or stay on.
The phantom 301 URLs could be the latest extension of presenting and testing the URLs that people prefer to click. They are back in the popularity contest, despite the permanent redirect. An old URL may have had the best CTR. The only way to find out, is to re-present the old URL in the search results and see how it fares. If it receives a lot of clicks, then the phantom menace may be there to stay, whether it is 301d or not. This change may leave Google open to abuse and may change the way we work with old domains
Here are some ideas:
Create a lot of duplicate pages with poor titles and CTAs, then 301 them to competitor URLs in the hope that some of the ‘fresh’ pages create phantoms and devalue the competitors original URL.
Occupy more search positions by creating multiple URLs within a site (such as changing to friendly URLs), to allow ranking of the old unfriendly URL that has been 301d alongside the new friendly URL. Continue to add new URLs and 301 to old URLs to create a 301 farm. Google may show some of the new URLs alongside their duplicates for CTR testing.
Link build to old domains (if they were popular) that currently 301 to your site. These old domains are often recalled strongly and continue to be entered in search results, even when a company’s name has changed. For example, even though Santander bought Abbey, searching for ‘Abbey National’ still has a lot of suggestions, and shows phantom 301 results in the SERPs:
If you have any experience of this phantom 301 phenomenon and how it could be used to advantage, please get in touch.
