Some of the topics discussed in this podcast:

• Where should someone new to testing begin?
• Does method matter?
• Is suboptimization a risk?
• Can we trust the statistics behind tools?
• What test result has been the most surprising?

Download the podcast

The panel in this podcast about testing:

Bryan Eisenberg is the co-author of several best-selling books: Call to Action, Waiting For Your Cat to Bark?, and Always Be Testing. He is also the co-founder and chairman emeritus of the Web Analytics Association. Bryan serves as an advisory board member of several venture capital backed startup companies (such as Bazaarvoice, iPerceptions, UserTesting.com, and ClickEquations) and is on the board of trustees of the Direct Marketing Education Foundation.

Lance Loveday is the founder and CEO of Closed Loop Marketing. His company has become highly sought after, serving a wide array of clients—from startups to big brands such as Hewlett-Packard, Brocade, and Lockheed Martin. Lance is co-author of the book Web Design for ROI.

Anne Holland is the publisher of WhichTestWon? and president of Anne Holland Ventures Inc, an online business media company. She is the founder and former president of MarketingSherpa, and has for more than 20 years been conducting tests and research into what works best in marketing.

Chris Goward is the CEO of WiderFunnel Marketing Optimization and an expert in website conversion rate optimization. Chris has been optimizing online and offline marketing for companies such as eBay, Epson, SAP, Getty Images, BabyAge.com, Rudder.com, Outrigger Hotels, and Google.

If you would like to give my podcasts a listen, try:

Podcast with Bonnie Thomas, Marketing Director of Memetrics
Download podcast (MP3)

Interview with Matt Roche, CEO of Offermatica
Download podcast (MP3)

Wine or Gasoline? A Podcast with Mario Fantoni of TaguchiNow
Download podcast (MP3)

Podcast with Mark Wachen, CEO of Optimost
Download podcast (MP3)

I am currently planning a new round table podcast about testing featuring thought leaders.

Two podcasts about web analytics as well:

Web Analytics in Europe
Download podcast (MP3)

Interview with John Marshall, CEO of ClickTracks
Download podcast (MP3)

In this podcast we discuss:

• Different methodologies
• The importance of methodology, technology and people
• Head-to-head tests with competitors
• Which businesses and business verticals are ahead of the game in testing
• Cultural differences
• Surprising findings
• Plans for expansion in Europe and Asia
• How to do testing for multiple markets and in multiple languages

Listen to the podcast

I had the pleasure of talking to Mario Fantoni, founder and president of TaguchiNow.

In this podcast we discuss:

• Taguchi and choice modeling
• Multivariate testing in general
• Methodology, technology and creativity
• Three websites (Sony Ericsson, Volvo and Stardoll)
• Whether small, large, new or old companies are the early adopters
• Small, large and niche markets
• Case studies (Nordax Finans, IntelliQuote, WebEx and Dell)
• Corporate identities
• Focus groups
• The 80/20 rule applied to testing
• Avinash’s 10/90 rule applied to testing
• How Mario’s modified version of Taguchi has evolved over the years
• TaguchiNow’s pricing
• What restaurants have to do with testing
• Which tool is Mario’s favorite
• Head-to-head tests with competitors

Download podcast (MP3)

A correction: Stardoll has nine and a half million users now and had approximately one million users a year ago. I looked at the Stardoll site a few days ago and noticed how membership grew by 5,500 members in just eight and a half hours.

Listen to this recording if you would like to know almost everything about web analytics in Europe. It’s 75 minutes of insights.

Learn about the European edge, what the future is for European vendors and much more. The experts are also comparing Europe to North America and Asia. Some myths about the state of web analytics in Europe are also debunked.

We are considering making it a series so please post some questions you’d like to get answered.

The following people are speaking:

• Lars Johansson (me!), moderator, Sweden
• Steve Jackson, Satama, Finland
• Dennis Mortensen, IndexTools, Hungary (a very kind person!)
• Neil Mason, Applied Insights, UK
• Oliver Schiffers, United Internet Media, Germany
• Aurélie Pols, OX2, Belgium
• Marc Sårde, Creuna, Denmark (sometimes referred to as Benjamin by Dennis)

Download the podcast

Some previous posts of interest:

Dennis
Neil
Aurélie
Oliver

The typical customer of Offermatica is a marketer or head of e-commerce who has been frustrated by the rising cost of media or the ability to market. Matt mentioned, as an example, a merchant who had to spend 75% of her time dealing with production and IT issues.

I asked Matt about the biggest advantages to using Offermatica’s solution compared to their competitors’ solutions. He broke it down to three areas:

• Speed. The burden on IT is less compared to other platforms.
• It’s a marketer-driven tool intended to be used daily by marketers.
• It’s a delivery platform. Sort of like an Akamai for personalized delivery.

The average Offermatica customer runs 13 campaigns simultaneously and Offermatica is handling almost 10 billion impressions a month.

I also had to ask Matt how often he is right when simply guessing which of two options will perform the best. He said half of the time. I’ve found that to be true for me as well (for some reason this reminds me of Hans Rosling’s chimpanzee story). Matt went on to expand on his answer by adding that there is always something new to be found—new copy, layout, or segments. Things that we don’t think are relevant end up being phenomenally relevant.

One case that Matt found especially exciting was a category affinity test. When a visitor through surfing patterns showed an affinity for a subproduct area, they showed offers related to that area when the visitor came back. There was a 65% improvement in revenue per visit. Even more surprising was how it changed the behavior of marketers. There were merchandisers who previously never had the chance to place their products on the first page, meaning that the outcome of the test changed the ownership paradigm and not only copy or layout decisions.

Matt believes that Offermatica sells engagement of consumers and marketers. He mentioned the difference between cycles that are six months or longer to put things on the website to getting an idea, launching it, and evaluating it all in a week.

Matt is quick to point out that they are not just a testing company; they optimize content for the source of traffic and use behavioral profiling.

Matt thinks the remaining barrier to adoption is mainly organizational issues and not a lack of understanding the benefit of testing and personalized optimization. A certain amount of traffic is required to achieve statistical significance, but smaller websites can still use the targeting capabilities to change elements on landing pages based on source of traffic to achieve higher relevance.

It is possible to integrate with web analytics solutions from Omniture, WebTrends, and Coremetrics, and take segments from there and use in Offermatica’s tool. You could, for instance, import RFM scoring from your web analytics solution to Offermatica.

Offermatica has a database of almost 300 million profiles at the moment and uses an open API.

Offermatica is growing by roughly 30 clients per quarter, and Matt says he doesn’t want to make the largest amount of money from the five biggest marketers, but instead help the most marketers. He thinks that Offermatica will see a 200% growth this year as compared to 315% last year.

Offermatica is primarily used in North America where they have some 150 customers, but they do also have some customers in Europe. Skype is a customer, and so are five or six banks, predominantly in London. There are about the same amount of customers in Japan. Offermatica opened an office in London and has an established service partner in France.

Matt’s advice to someone who wants to get started with testing is to just do it. Since you’re not buying software, you could get started in a week. So what are you waiting for?

Press release: Offermatica Hosts Inaugural Strategic Optimization Council
The Merchant Versus the Machine

Parts of the interview are available in this podcast:
mattroche_podcast.mp3

The first issue we talked about was one that is difficult to avoid, even though it has probably been discussed to death already. You probably already guessed it, but I was interested in hearing John’s views about the effect of the new version of Google Analytics. He is convinced that many Google Analytics users would never consider paying for web analytics anyhow. Therefore it doesn’t negatively impact ClickTracks’ sales. John saw an increase in sales right after the free version was announced. He explains that people use a free tool as proof of concept and then graduate on from that.

John views WebTrends as the main competitor of ClickTracks.

I also asked John about the plans for ClickTracks Appetizer. He was keen to point out that the key thing about the solution is the free web analytics classes provided by the company. ClickTracks Appetizer is the vehicle that enables them to teach for free and is supposed to suit someone who is getting started in web analytics. ClickTracks is, however, starting to get into some slightly more advanced topics now—teaching more advanced segmentation, for instance.

Every month ClickTracks also announces a Web Analytics Day when advanced features are made available in ClickTracks Appetizer. That combined with free classes is pretty effective, and I was wondering if they had considered taking Web Analytics Day on the road. John told me they would like to, but they would have to team up with a couple of other companies for it to be doable. An idea is to perhaps combine it with tracks about search engine optimization and PPC management.

Many web analytics vendors are branching out and aiming to offer a complete package covering all aspects of online optimization, not just web analytics. I asked John whether ClickTracks has any similar plans, and he said that they are working on some interesting new things that will probably be announced in a couple of months.

When John was on the vendor panel at Emetrics London, he said that ClickTracks has the ambition of making their solution the iPod of web analytics. He also mentioned that their unofficial slogan is “web analytics that sucks less.” I thought both things were very colorful and asked him to elaborate a little. John went on to say that ClickTracks has always tried to attack problems in a different way. One such example is ClickTracks’ approach to funnel analysis, which strictly speaking perhaps should be named something different.

John explained that iPods were much bigger than competing solutions when they came out, plus they went against the convention that portable music players should use flash memory and instead used a hard disk. But it turned out that other players, which used what was considered more modern technology, didn’t have enough storage capacity for people to use them. In the same manner, ClickTracks is not afraid of taking a different approach if it will result in a better experience for the analyst.

The most meaningless report that John has seen is “top paths through the site.” ClickTracks has consistently refused to provide this report as it doesn’t add any value. People have, however, asked for it many times but they are, in John’s experience, starting to learn that it’s not that useful.

ClickTracks’ biggest challenge in 2007 is integration with other tools, and North America is a bit farther ahead in that process than Europe. About twenty of ClickTracks’ clients have integrated, or are currently integrating, with other enterprise systems.

Generally John thinks that companies in the USA are somewhat ahead of Europe, probably because of a stronger tradition of direct marketing and looser privacy regulation.

Approximately one-quarter of ClickTracks’ customers are located in Europe. ClickTracks has its European headquarters in the UK and will be expanding from there. The next country in line is for an office is probably Germany.

The first thing I wanted to find out was what the key differentiators between Memetrics and their competitors are.

Bonnie listed the following:

• The behind the firewall software deployment means that no information is passed over the Internet. It also makes it possible to deeply integrate within a multitude of systems giving access to rich customer segment data.
• More robust analytics. Possibility to optimize for multiple outcomes simultaneously.
• The methodology used is based upon choice modeling, a demand-driven optimization science. Offermatica, for instance, is focused on Taguchi. Bonnie said Memetrics methodology produces more accurate results.
• Memetrics has been around since 1999. Bonnie said others tend to sell a silver bullet while Memetrics views optimization as part technology, part process.

I also asked what share of Memetrics’ customers integrate with more than one system. Bonnie didn’t know off hand but said that it is a large percent and mentioned the possibility to create experimental rules and export them to another system, in cross-channel initiatives.

Memetrics consider their ASP solution the long tail, whereas the head is mostly the behind-the-firewall, enterprise solution.

The ASP solution is about US$50,000 a year.

The biggest challenge for Memetrics this year is evangelizing the value of multivariate testing. The problem is that businesses are realizing the value but may not have room in their current budget for an investment.

Market adoption has increased significantly since last year, and Bonnie has noticed that the inbound requests from agencies have increased dramatically.

The full interview, 22 minutes in length, is provided below.
bonniethomas.mp3