Conclusions

So here’s a fun quiz to see how good a communicator I am: What’s my favorite key phrase for my new companies? If you said “data-driven development and user metrics”, you win! Next time we have coffee together, I’ll pour the creamer for you.

Yes, data driven, data driven, say that 10 times fast. After 10 years of experience chaperoning web-based businesses, I can say this for sure: those that succeed have one thing in common: they measure, and adjust, measure and adjust. Have you ever seen those robotic vacuum cleaners looking for their home base? I was just over at a friend’s house, playing with the robot, and I’d push the “go home” button. The robot would orient itself towards the base, drive an inch, readjust, and repeat. As long as it kept up that cycle, it always got back to the base. But sometimes it seemed to stop adjusting, and just drove across the floor until it hit something. And you know what? Once that started happening, it didn’t make it back to base for a long time.

You, Mr/Ms startup entrepreneur, are a little robotic vacuum cleaner. You can drive your product until you hit a wall, and have to change something, or you can measure and adjust constantly, pointing at base. No points for figuring out which one I think you should do. Metrics are going to make or break your business. Back when we all started out, metrics were an afterthought, especially in the dark ages of the Web up to, say, 1994. You built the site first, and then – oh, yeah, those marketing people want metrics! Okay, let’s add that in. Thinks have gotten better – more and more , web entrepreneurs realize that they need analytics. So they turn to Google Analytics because, shoot, you’re using Google for everything else, right?

The reality is, Google Analytics is better than no analytics. But not much. What you really need to know is the answer to that key question “how is my business doing?” Which, in web-entrepreneur land, closely correlates with “is my website doing its job?” There are a plethora of analytics packages out there, some better than others, but I can’t wholeheartedly endorse any one, because none of them are as complete as you need. And now I come to the core of my article:

You need to build your analytics along with the site itself,
as an integral part of the site design and implementation.

Unlike big companies that can afford huge analytic implementations, we scrappy start-up folks need a different perspective. The reporting must be built custom, by the engineering team or one of the two programmers. Smart data-driven people like Andrew Chen have this to say about it: "I generally prioritize this as a peer to the product experience, perhaps even higher, since I often overrule product functionality based on real data." *. You will be well served by building an integral reporting system that allows you to quickly extract the Key Performance Indicators for your site and business.

I try to avoid throwing around buzzwords like “key performance indicators”, but I think that the concept is so important that it can’t be helped. (For a brief overview of KPIs in general, see The Accounting Blog). Indicators are events or aggregate numbers that give us an indication of whether we have obtained our goal. KPI’s are not a web-specific thing, but in our case we need specific indicators for our website. Each website and business will have unique indicators appropriate to our business model. E.g. a KPI for a site that hosts users’ photos might be one of the following:

• The frequency with which a new unique user uploads a photo.
  
• The number of pages a user visits before uploading a photo.
  
• Abandonment data – what percentage of unique visitors start an upload, but quit the site without uploading anything? On average, what path do they take and where do they abandon?
  

To get back to the Robot Vacuum Cleaner metaphor, you have to know where your home base is, if you’re ever going to get there. Your KPIs will give you sense of where base is for you.
It’s crucial that you keep your list of KPIs to a manageable number – no more than a dozen and ideally ones you can take action on. Choosing this dozen will be an involved and evolving task, but it’s more important that you do it, than that you get it completely right. Once you’ve found your set of Indicators, you will quickly realize that few-to-no off-the-shelf analytics packages will meet your needs. You just have to make it part of the development process.

Labels: analytics, kpi, marketing

I’m going to talk about the intersection of two facets of many comprehensive online advertising strategies: Google Adwords, and Long-Tail Keyword Strategies. And most interesting, the obscure and unpredictable outcomes that can occur when trying to put the two together.

So as almost everyone who has ever done *any* on-line advertising knows, Google Adwords is where you start. With roughly 70% of the on-line search audience, anyone buying paid search impressions starts with Google, and their Adwords program. Once upon a time, this was a simple bidding system, where paying more got you more impressions.

However, now that their monopoly status is undisputable, the advertising elite at Google have decided that they’re not just cyber-lackeys who deliver ads to the public at the market price . No, they’ve decided that Google has an ethical duty to try to deliver quality content; and to that end Google Adwords currently rates your campaign based on the quality of the content that you provide. Advertising campaigns that offer higher quality content (according to Google’s monolithic standards) pay less for their ads. Those who deliver junk content may unexpectedly see themselves hit with “penalty pricing”, where they see their minimum bids jump to either $5 or $10 per click -- effectively killing the campaign.

The wisdom of this approach is debated endlessly, and I’m not going to tackle it here. However, it does raise one very significant question: what makes quality content, and how do you discriminate? And most importantly, what do you do if you and Google disagree? Well, the short version is, it really doesn't matter what you think, it matters what Google thinks. As far as this church goes, Google hands down the scripture and the rest of us just read it. You have to run your campaign by their rules.

I’m going to ask you to table the discussion on Adwords Quality Score for a minute while we talk about Long Tail Keyword advertising (can we just say LTKW?). The basic concept of this technique is well described in this article by Matt Baily http://www.searchengineguide.com/matt-bailey/keyword-strategies-the-long-tail.php. Here’s my one-sentence summary: “Rather than trying to find keyword phrases that will appeal to many people, you find hundreds or thousands of oddball phrases that may appeal to just a handful of customers.” So when you use a Long-Tail Keywords campaign, you pick up customers by putting together vast lists of keywords which you hope will attract some of the more unusual search phrases. You then create a Google Adwords campaign where you bid on thousands of these unusual keywords, hoping that your list is long and comprehensive enough to catch and convert customers. Now, precisely because you’re not going after the common search terms, many of the terms might appear less relevant to the content of your landing page. Or hell, they might really be irrelevant! But that’s okay – you’re only trying to capture the stragglers, not the body of the searchers.

However, here comes the rub: When you put together the strategy of advertising using obscure keywords along with a system that enforces relevance, you’ve got a problem. Many advertisers found that LTKW strategies that were exceptionally successful for them – until the Adwords Quality Scoring system decided that the Long Tail Keywords were not relevant enough to the content, and -zing! - the penalty pricing shut it all down. Once your campaign has been tagged by Google as “low quality”, it can be difficult to impossible to recover. You may simply have to junk the domain, and start building a new website and new campaign from scratch.

What’s the solution? Knowledge, Research, Testing (do we have a mantra here?) You can successfully using a LTKW strategy with Adwords. But it requires that you have a much more intimate understanding of how Google ranks your campaign, what the rules are, and how to give them what they want to see. Which would be much easier if they actually told you what the rules are, but that’s the monopolists’ prerogative. However, the LTKW strategy can be enormously successful, and even with the extra work, it can be a highly profitable approach.

For a recent project, the team I was working with was experimenting with different keyword strategies, opened a new AdWords account, and created an 8000 keyword campaign. Within hours all keywords jumped to a $10 minimum bid, effectively shutting it down. We couldn't even create a single-keyword campaign which used the company's name -- which is also the URL! Google won't say how to fix it, but we know it's a combination of account history (new in this case), click through rate (low here), keyword density (again low, we didn't use enough uniquely matched landing pages) and relevance, bounce rate (high), and in some cases, the categorization of the target site. I've built some best practices around LTKW strategies and Adwords, but am still learning. If you'd like to share, please drop me a note.

In my research, I’ve found the following articles, which provide a more in-depth introduction to the field. They talk about the Google Quality Score, and techniques on improving your quality score and recovering from penalty pricing. I recommend reading through them before implementing any campaign.

Can You Recover from a $10.00 Quality Score?

CPAM Series - Quality Score Rd 2 - Adwords Account And Quality Scores

Google Semantically Related Words & Latent Semantic Indexing Technology

How To Check Your AdWords Quality Score and Reduce Click Prices

10 Ways To Increase Your Adwords Quality Score - A Mini Case Study

How to Beat the Google Slap with 5 Simple Steps…

Keyword Research

HitTail: What Is It?

Competitors Use Long Tail Keywords To Hurt Yahoo Search Advertisers

Labels: adwords, google slap, keywords, long tail, marketing, seo

Let's pretend that you've established your customer market, identified a product they want, and have a good idea of how to provide it to them (perhaps even profitably!) You’ve analyzed your surveys and key metrics, everything seems to make sense. You've done it -- you found an ideal product, with a website to deliver it. Now, let the flow of commerce commence! Ah, but... finding the right business to be in is only the first step on the road. Having found the right business, you still need to run that business and run it well. In the case of a business that relies significantly on traffic to a website, that means making sure the website is doing the best it can in converting visitors – that is, getting them to do what you intended. (As an aside, you’re likely wrong about having found the perfect business, as I’ll explain in a later post.)

This might sound like an obvious question, but let's consider it anyway: “What is a website's job?” Well, at a fundamental level, the website has two tasks:

• To attract visitor traffic; and
  
• To get those visitors to do something desirable -- like spend money, click on ads, upload pictures of lolcats . Whatever your site does.
  

I'm going to talk about the idea of optimizing your website’s content, layout and messaging so that you achieve your business goals as effectively as possible. You may not have considered website optimization in your competitive analysis, but you can bet your next 100,000 page views that your top competitor will be optimizing their website. Fortunately, you can too – website optimization technology has become accessible and affordable, and there are a great many knowledgeable people who can help you implement it.

Now, if you Google "website optimization", you will find almost exclusively articles on Search Engine Optimization (SEO). SEO is important too, but it's not what we're talking about here. SEO is about driver optimization -- hopefully it sends more people to your site. But "website optimization", as we're discussing it here, refers to optimizing the user experience to maximize the site’s contribution to your business goals (whatever they are.) Only a few years ago, the techniques that I'm about to mention were not in widespread use. Now, a large percentage of successful companies use them, and those that do have a massive edge over those that don’t.

For the purposes of this article, a website is successful if and only if it furthers your business goals (which can be wide and varied.) We can say that one version of a website is more successful than another if does a better job of furthering business goals – which often means attracting and converting customers. You optimize your site, you run a more successful business. Easy, huh? So here’s Website-Optimization-In-A-Nutshell: On any given page of any website, you (the designer) make thousands of choices of what to present to the viewer (or not to present). Some of these choices will give better results than others, yielding more conversions. To select the best choices, you present different versions of your website to different viewers; then track how successful each version is. If you do this methodically and repeatedly, you will very quickly find that you can improve website performance by 20%-100% or more!

Let’s take a closer look at what this kind of optimizing/testing looks like:

The old guard is A/B testing. This is also known as "split run" testing, or single-variable testing. Under an A/B test, you create two versions of your page, which will be displayed in rotation. One of the two versions is shown to each visitor, often randomly. Based on user response, you establish that one of the pages is more successful. You take the winner, make another version (change something else) and do it again. A/B testing is the aged workhorse of the world of website optimization. It's also arguably obsolete for most applications, although many people still use it. The nice thing about A/B testing is that there are free tools like Google’s Website Optimizer that allow you to jump right into basic optimization. And it’s a heck of a lot better than better than no testing at all.

More advanced is Multiple Variable Testing. Aliases are Multivariable Testing or Multivariate Testing. There are also sub-categories, such as Fractional Multivariate Testing, of which a well-known variant is the “Taguchi Method”. If you're going to play with the big boys, Multiple Variable testing is your big gun. You see, the fundamental problem with A/B testing is that you have to test one variation at a time. Hey, who's got just one variation of their web page? There are millions of possible variations. Can you imagine how long it’ll take to test them all, doing one variation at a time? Uh-uh; if we're going to get results in any reasonable time frame, we're going to have to test multiple variations at ones. Lucky us -- here comes Multiple Variable Testing!

Most commonly, you will actually be using Fractional Multiple Variable Testing. Because there are virtually always too many possible variations of a web page to count, we're going to have to choose a subset of them if we want to get results in time to use them. That's the Fractional part of Fractional Multiple Variable Testing -- we pick a fraction of the possible variations. So just go with your gut -- focus on variations that you feel are likely to give the most bang. Here's what it might look like: you pick out a few basic elements of your web page, choosing the elements that you think are going to be most valuable in attracting and converting customers. For each of these elements, you create several variants. For example if you pick 4 elements with 2 variants each, you have a total of 16 possible web pages. Next, you create a table of the possible variations. Fortunately, experience has told us that you don't need to test all 16 variants; 8 may be sufficient. Don’t worry, about which 8 – the experts have figured it all out for you. Now when a customer lands on your site, they will see one of 8 test pages (rather than just two test pages under A/B testing). This way, you can build up performance data much more quickly, and you can identify successful variations much faster.

By running these tests methodically, systematically and thoroughly you will succeed in honing your website to a razor-sharp edge. One that will translate directly to a business that performs better, however you measure that.

Of course it's much more complicated than this. There are some very sophisticated systems out there that will automatically generate hundreds of variants of your web pages, gather in-depth metrics, and run complex analysis. There are companies that do nothing but this kind of optimization. And in addition to just varying your pages, you have to evaluate dozens of interrelated factors. Some important ones to consider are:

• How long do you test before making a decision? (And should you ever really stop testing?)
  
• You must evaluate the combination of the driver ad (that's sending visitors to your site) and the message on the landing page. This combination is just as important as optimizing each individually.
  
• What are your Short-Term vs. Long-Term goals? Often products that give you good short-term results will give you poorer cumulative results over time.
  
• How do your website and the user experience fit in with your other advertising and business image?
  

By now, you've gathered that website optimization is a complicated process. But you don't have to throw up your hands in despair -- the methods exist to do this systematically; and you can hire the experts to do it. No, you have to hire the experts to do it. It's how the business is run.

Labels: a/b testing, multiple variable testing, seo, taguchi method, testing, website optimization

You should meet my friend Paul. He’s a successful entrepreneur, and one of the principals of the consumer web startup P.au/lWeb. They’ve spent countless person-years of engineering time building out their new web application to track distributed social widgets in 3D. I’d like to say that he should be an inspiration to all of us, except that the site isn’t generating any revenue, or none to speak of anyway. They’re still looking for their customer.

Okay, the truth is there’s no Paul… well, not exactly. But there are thousands of Paul-clones out there, desperately building new startups, with the sole marketing strategy of getting a mention on TechCrunch. In fact, I did it this way myself for a period, believing I should just ”Build it and they will come.” Fortunately, a few years ago, I made a change. I learned to forget about the product. The world may or may not want it. Instead, develop your customers first, then let them tell you what product they want; at that point, all you have to do is give it to them. Think about it – how much more business could you do if you knew exactly who your customer was, what they wanted, and how to reach them?*

* This is somewhat of an over-simplification. In practice, not all customers are the customers you want or should listen to. This is especially true of early adopters. Nevertheless, on the balance, you'll run a far more effective business strategy if you do more than a little analysis to determine who your customer might be, how and whether you can reach them profitably.

The catch-phrase is Customer Development. It sounds odd, doesn’t it – I mean, what the heck is Customer Development? We all know what product development is, but how do you develop a customer? Think of it as more akin to developing an picture on photographic film. The image is out there, and if you just go through the right “developing” process that image becomes clear. Similarly, we don’t create our customers so much as develop a picture of them. To succeed, we need to know all about our customers, especially the ones we don’t have yet, and haven’t marketed to. The ones that we didn’t have a product for, primarily because we didn’t even know that they wanted something. We can find these customers by using a systematic approach - we start with a hypothesis and test its fit with our audience until the image of customer resolves itself. Our tools are surveys and interviews with clear metrics and sufficient depth so that the picture of our ideal customer resolves itself, and our business’s path follows.

I found this concept well-articulated in Sean Ellis’s excellent Startup Marketing Blog. I emailed Sean about his ideas, and was invited to coffee with him to share thoughts on startup marketing. Sean recommended The Four Steps of the Epiphany by Steve Blank, which I ordered and read. The book expounds on Customer Development, (arguably coined the term and is the definition of it) and gives an overview of the critical task of (as Sean writes):

“Finding the right fit between the problem, product, message and target customer … achieved through interviews, surveys, and A/B message testing….”

There is a Customer Development process that is fundamentally the same, yet unique, for every business out there. By researching and surveying, rather than jumping on the hot market drivers, I’ve found that I’m able to make much more effective use of my marketing dollars. It’s a very systematic, measureable approach, with quantifiable results. Stay tuned to this blog for more on this topic.

I want to thank Sean Ellis for sharing his most effective business practices with me, and to pay it forward: if you’d like to meet me for coffee and share thoughts on startup marketing, I’ve love to hear from you.

Labels: customer development, marketing, sean ellis