|
Signal vs. Noise |
Behind the scenes: A/B testing part 2: How we test |
A few weeks ago, we shared some of what we’ve been testing with the Highrise marketing page. We’ve continued to test different concepts for that page and we’ll be sharing some of the results from those tests in the next few weeks, but before we do that, I wanted to share some of how we approach and implement A/B tests like this.
Deciding what to test
Our ideas for what to test come from everywhere: from reading industry blogs (some examples: Visual Website Optimizer, ABtests.com), a landing page someone saw, an ad in the newspaper (our long form experiments were inspired in part by the classic “Amish heater” ads you frequency see in the newspaper), etc. Everyone brings ideas to the table, and we have a rough running list of ideas – big and small – to test.
My general goal is to have at least one, and preferably several A/B tests running at any given time across one or more of our marketing sites. There’s no “perfect” when it comes to marketing sites, and the only way you learn about what works and doesn’t work is to continuously test.
We might be simultaneously testing a different landing page, the order of plans on the plan selection page, and wording on a signup form simultaneously. These tests aren’t always big changes, and may only be exposed to a small portion of traffic, but any time you aren’t testing is an opportunity you’re wasting. People have been testing multiple ‘layers’ in their sites and applications like this for a long time, but Google has really popularized this lately (some great reading on their infrastructure is available here).
Implementing the tests
We primarily use two services and some homegrown glue to run our A/B tests. Essentially, our “tech stack” for running A/B tests goes like this:
- We set up the test using Optimizely, which makes it incredibly easy for anyone to set up tests – it doesn’t take any knowledge of HTML or CSS to change the headline on a page, for example. At the same time, it’s powerful enough for almost anything you could want to do (it’s using jQuery underneath, so you’re only limited by the power of the selector), and for wholesale rewrites of a page we can deploy an alternate version and redirect to that page. There are plenty of alternatives to Optimizely as well – Visual Website Optimizer, Google Website Optimizer, etc. – but we’ve been quite happy with Optimizely.
- We add to the stock Optimizely setup a Javascript snippet that is inserted on all pages (experimental and original) that identifies the test and variation to Clicky, which we use for tracking behavior on the marketing sites. Optimizely’s tracking is quite good (and has improved drastically over the last few months), but we still primarily use Clicky for this tracking since it’s already nicely setup for our conversion “funnel” and offers API access.
- We also add to Optimizely another piece of Javascript to rewrite all the URLs on the marketing pages to “tag” each visitor that’s part of an experiment with the experimental group. When a visitor completes signup, Queenbee – our admin and billing system – stores that tag in a database. This lets us easily track plan mix, retention, etc. across experimental groups (and we’re able to continue to do this far into the future).
- Finally, we do set up some click and conversion goals in Optimizely itself. This primarily serves as a validation—visitor tracking is not an exact science, and so I like to verify that the results we tabulate from our Clicky tracking are at least similar to what Optimizely measures directly.
Evaluating the results
Once we start a test, our Campfire bot ‘tally’ takes center stage to help us evaluate the test.
We’ve set up tally to respond to a phrase like “tally abtest highrise landing page round 5” with two sets of information:
- The “conversion funnel” for each variation—what portion of visitors reached the plan selection page, reached the signup form, and completed signup. For each variation, we compare these metrics to the original for statistical significance. In addition, tally estimates the required sample size to detect a 10% difference in performance, and we let the experiment run to that point (for a nice explanation of why you should let tests run based on a sample size as opposed to stopping when you think you’ve hit a significant result, see here).
- The profile of each variation’s “cohort” that has completed signup. This includes the portion of signups that were for paying plans, the average price of those plans, and the net monthly value of a visitor to any given variation’s landing page (we also have a web-based interface to let us dig deeper into these cohorts’ retention and usage profiles). These numbers are important—we’d rather have lower overall signups if it means we’re getting a higher value signup.
Tally sits in a few of our Campfire rooms, and anyone at 37signals can check on the results of any test that’s going on or recently finished anytime in just a few seconds.
Once a test has finished, we don’t just sit back and bask in our higher conversion rates or increased average signup value—we try to infer what worked and what didn’t work, design a new test, and get back to experimenting and learning.  |
 |
| (Published: Tue, 02 Aug 2011 19:22:00 +0000) |
|
|
|
Real Estate Leads |
Generate real estate leads using direct marketing |
The are endless types of direct marketing that can all be used to generate real estate leads for real estate investment companies. Each type of direct mail is used to attract different types of real estate leads and to market different types of real estate solutions.
Direct mail can create real estate leads fast if used effectively. However it can also be a huge waste of money if you chose campaigns that don’t work. Some common types of direct marketing are: direct mail (post cards, mailings, all in one mailers), direct email, personal selling, and telemarketing.
In order for direct mail to be effective at generating real estate leads it has to make an impact. You want potential customers to look at it, not think of it as junk mail.
Once the homeowner reads it, you need them to take a specific action. The direct mail should make it easy for the motivated seller to respond to the offer. Follow up is integral either by sending additional mailers or telemarketing.
To perfect your direct mail marketing plan geared towards generating real estate leads you need to answer these 5 questions: 1. Who will you target? 2. What do you want to say? 3. What format should you use? 4. How will you follow it up? 5. How will you track the results?
All marketing should be tested in smaller markets to learn the return on investment. If you can’t seem to perfect your marketing system or if you want to save time and money have Wehavehomeleads.com generate quality real estate leads for you.
To learn more visit www.Wehavehomeleads.com. |
 |
| (Published: Mon, 03 Aug 2009 18:37:00 +0000) |
|
|
|
About Dating |
Lying Online |
I've found that a select few singles lie about their height, weight, age, marital status (or other demographic information) online, but that in general most people are honest regarding who they are. Now, that isn't saying that people use photos on their online dating profiles that are truthful (or at the very least, recent), and a recent OkCupid study found that the hotter the online dating photo, the more likely it was an older picture. |
 |
| (Published: Wed, 17 Aug 2011 00:17:41 +0000) |
|
|
|
XP, Vista and Windows 7 |
Create a Windows 7 Password reset disk |
| To create a Windows 7 password disk to help you out of a bind if you forget your user account password.
1: Click On Start,
2: Cick Control Panel,
3: Click User Accounts and Family Safety,
4: Click User Accounts,
5: Click on Create a password reset |
 |
| (Published: Wed, 16 Dec 2009 10:13:09 -0800) |
|
|
|
SEO and Internet Marketing |
Website Designing and Development |
| GetHighRanks is a leading website design and development company with a strong reputation, history, and expansive skill-set. We consistently deliver creative, scalable web designs and solutions that reflect the personality, needs, and vision of our clients. |
 |
| (Published: Fri, 13 Nov 2009 19:53:33 -0800) |
|
|
|
Search Engine Optimisation (SEO) Blog |
How Seasons Affect Search Behaviour |
| Not as many modern industries as in the agricultural past are as aware of the changing of the seasons. Certainly, the odd business may have a heightened awareness of the time of year every so often, and most businesses are sensitive to the end of the financial year. Seasonality on the whole, however, is not |
 |
| (Published: Tue, 08 Dec 2009 16:45:53 +0000) |
|
|
|
Time Management |
A Doctor and a Poet on Work |
| How to Be
I paid a visit to a book store the other day, where I came across an intriguing book written by two Finns – poet Tommy Taberman and Dr. Jussi Koski. It was called “How to Be” (*. I liked part of the insights in it, so I thought I would share some of |
 |
| (Published: Mon, 30 Nov 2009 15:00:54 +0000) |
|
|
|
Financial helpdesk |
Hard Money Loan – Borrowing Made Easy |
 When you require funds in urgency and cannot wait for weeks to search for lenders for traditional loans, your best option is to go for a Hard Money Loan. There are tons of loans available in the market today for real – estate investors, but one common type of loan is hard money loan. Such loan allows investors to buy and fix investment property and if used correctly it can be definitely put money in your pockets. A hard money loan is a specific type of borrowing in which a borrower receives loans based on the value of a specific commercial real estate. The lender approves the loan based on the equity in the assets. The ease for getting such loans has made the interest rates go higher than other categories of loans. Unlike traditional money lenders in hard money loan, the lenders do not spend amount and time in verifying the borrowers’ credentials like his income, tax history or credit history. The lenders borrow the amount totally based on the value of the real assets. Hard money loan is funded very quickly and you do not have to wait for weeks. There is considerable flexibility in the loan terms which can vary from 6 months – 20 years. Hard money lenders restrict their lending to not more than 70 percent of their assessed value of a real estate property. This shows the rest of 30 percent is your goes to your down payment or equity. There are many high costs involved in hard money loan which makes the loan very expensive. The lenders always look for a strong return on their investment and charge you rate of interest in between 10 – 18 percent. There is also another major disadvantage as your loan will not be reported to the credit bureaus. This means that even if you pay your loan in a timely manner, it will not improve your credit report in any sense. There is also the facility of foreclosing properties with delinquent payments through hard money lending. These types of issues are generally avoided by traditional banks. Thus hard money loan gives you the benefit of a large amount of funding without considering your financial income or credit history. You just have to have a real estate property and have enough of equity assets to fill out the application for such a loan. If you solely think that you can get a benefit from hard money lending, then do your research and know your lender well before signing on any documents for the loan. |
 |
| (Published: Fri, 19 Mar 2010 10:28:00 +0000) |
|
|
|
Money does matter |
New Year Debt Resolution |
 It’s New Year again, and YOU have to solve your DEBTS all and for once. How to do it and when to do it? It is very simple but hard to control your expenses that you often do. It is similar to your over-eating and emerging with a huge tummy. You should first realize to why you spend so much in eating or shopping using your credit cards when you can’t realize the hardships to face at the time of filling up those open mouths of the banks and money-lenders. Why you shop at a rate when you cannot repay even the 80 percent of the credit that you take. There are some simple ways by which you can control your life and finances as well.
It is well known that the most common attractive destination is the shopping malls. Try to avoid them as and when it is not required so that you don’t get attracted at the new arrivals displayed on the windows of the malls. Try habiting window shopping when you are bored, this will divert yourself from purchasing each and everything beyond your limit. Start telling yourself that the good one you saw today will be replaced some other good ones the next day, “I should come back another day to get them”. This will help you a lot in understanding your finances.
The next big thing is to get rid of your credit cards as urgently as it is required. Buy anything that you like in simple cash. This will help you in controlling your finances and prevent you in overspending by credit. The situations will come like that you have not enough cash to get that one, so let me take the other cheaper one. This will keep your budget going and you will also be satisfied of getting the stuff that you wanted.
Moreover if you require cash in emergencies then there are always the payday loan companies that you will help you get cash in an urgent basis, even on the same day or in a few hours of time. This will help you avoid the credit card debts and stop destroying your credit record also. |
 |
| (Published: Fri, 08 Jan 2010 22:41:09 -0800) |
|
|
|
The Don Diebel How to Meet, Date, Seduce Women Blog |
Why You Must Take Control with Women |
Observing behavior and actions and not theory has shown us time and time again that as soon as a man gives up the control that he "should have," a woman sees this as a sign of weakness and becomes dissatisfied with his performance in filling the "man's role." Women respond favorably to the man taking control of a situation, from which movie to go see to sexual encounters. They want the door held, the wine opened, the evening planned, and the car driven. They want traditional male/female roles fulfilled. They are happiest in every phase of a relationship when they feel a high level of power and security coming from men leading the situation.
Does this mean a man should become a tyrant in order to control a relationship? No. Filling the man's role can be done in many subtle ways which often amounts to no more than awareness of the situation, making the decisions, and expending the energy to take the lead. In the next chapter on "your image" and throughout the book, you will learn exactly what we mean by "power and leadership" and how to fill the man's role.
This article is an excerpt from our best-selling book, "A Man's Guide to Women."


 |
 |
| (Published: Wed, 17 Aug 2011 16:40:00 +0000) |
|
|
|
Living in Thailand |
How to Send Money to Thailand |
| There are various ways of sending money to a friend in Thailand: interbank transfer (also called a "telegraphic transfer" or "SWIFT transfer"), sending an ATM card, and using ATMcash (if you are in the USA) to send a personalized debit card.These are documented at this website: SendMoneyToThailand.com |
 |
| (Published: Tue, 30 Mar 2010 02:50:20 -0700) |
|
|
|
Financial Debt Solutions |
What Is This Refinance Auto Loan Thing? |
| So, if you have an existing car loan, have you considered auto refinance loans? Many people are looking into refinance car loans today, thinking this might just be the ticket to lowering their payments and interest rate. Prevailing auto refinance rates are driving this move to get a better deal on car payments. Maybe you |
 |
| (Published: Sun, 14 Mar 2010 07:42:26 +0000) |
|
|
|
Loan Modification |
How to Identify Bad Credit Cards |
| What you know about spotting bad credit cards can definitely help you especially if you are in the position of trying to fix or repair your credit. Learn different ways to identify bad credit cards and you will be one step ahead when it comes to your financial future. |
 |
| (Published: Sun, 22 Nov 2009 07:13:50 +0000) |
|
|
|
Air Duct Cleaning Blog |
Beware of Whole House Specials |
| Lately we've been hearing more and more complaints about "whole house specials" and reading about them on the internet too. Whole house specials are advertised in the Toronto area for anywhere from $99.00 to $120.00 or $130.00. It just depends where you look.
Bait and Switch is a common rip-off tactic |
 |
| (Published: Tue, 16 Feb 2010 20:49:29 -0800) |
|
|
|
Movies Music Games & More Blog |
The Best Sites To Download Free Movies And Music |
Where To Download Free Movies And Music
Now that almost everyone on the planet who owns a PC and portable MP3 player of some type are in the market to buy DVD’s and audio CDs has declined tremendously, while the demand for free individual movie downloads and individual song downloads has greatly increased. The main reason for this is because it has just become so much easier and convenient for people to download free movies and music online these days.
The Very Best Sites To Download Free Movies and Music
Without a doubt the very best way to download free movies and music is by joining a free downloads membership site such as The Movie Downloads and Unlimited Downloads Center. For a very affordable one-time only membership fee you can acquire unlimited free movie downloads, free music downloads, free game downloads, free TV show downloads, free music video downloads, free software downloads, free ring tones, free game cheats, free wallpaper, free themes and more. Your free downloads will all be of the highest quality and are completely spyware and adware free and the chances of downloading a virus or other maleware are rare.

>Click Here To Visit The Movie Downloads |
 |
| (Published: Mon, 19 Apr 2010 14:34:52 -0700) |
|
|
|
Top Success Network |
Welcome to our new site! |
|
Hi everyone!
You may notice we have a new look here at Top Success Network.
After a lot of thought we decided it was time to update the site and most importantly, to re-package our books.
As you may remember, our books are pretty pricey, and unquestionably worth every penny too! We had decided the selling price of |
 |
| (Published: Thu, 22 Oct 2009 16:01:00 +0000) |
|
|
|
Sconce Light |
Sconce Lighting |
| When you are designing the sconce lighting in your home, it is vital that you take an overall view first and then take it room by room. When I say an overall view, it is best to keep your style of lighting, fixtures and fittings in style with the type of home you live in. |
 |
| (Published: Fri, 12 Feb 2010 16:43:31 +0000) |
|
|
|
Online Education |
GED High School Diploma – Get Yours |
| Famous celebrities who earned a GED high school diploma are pop singers like Britney Spears, Jessica Simpson, Avril Lavigne, actors like Bill Cosby and Christian Slater, and others. |
 |
| (Published: Sat, 19 Dec 2009 00:10:38 +0000) |
|
|
|
Debt Consolidation Details |
Major insurance policies that you need to purchase |
| Many of you think that insurance policies are expensive and it is a waste of money if you don’t make a claim. However, you can realize the important of purchasing insurance policies when you really need it. Insurance is basically an agreement wherein you purchase a policy and pay the premium on time; in turn, |
 |
| (Published: Mon, 22 Feb 2010 05:49:07 +0000) |
|
|
|
Ten 34 Tips |
Forex Black Panther bonus |
| Forex African american Panther is a robot that will be able to truly help you get started on forex. That is since it is really a extremely sophisticated robotic built specifically for producing cash trading. Many individuals lose money on forex because they allow their emotions get within the way. Forex African american Panther takes |
 |
| (Published: Thu, 11 Mar 2010 10:04:48 +0000) |
|
|
|
Aubenades tips and advice |
An easy way of choosing your next helmet |
| If you are thinking about buying a new motorbike helmet then you may be asking yourself a few questions. You will be surprised to know that there are a number of choices out there for you to choose from.
Your choice of helmet will rely on what you want it for and this will help you |
 |
| (Published: Thu, 11 Mar 2010 10:06:43 +0000) |
|
|
|
DSNTD Information |
key reason why i find panic attacks so daunting |
| If you have previously endured an anxiety attack you will know that it’s very likely the single most frightening experiences you can actually encounter in a lifetime.
Things start to close up in on you, your chest muscles tightens and you also feel just as if you cannot breath, your heart really wants to pound out |
 |
| (Published: Tue, 16 Mar 2010 23:20:00 +0000) |
|
|
|
Shower Doors Installation |
Shower Doors Upland |
| Are You in Need of Shower Doors Upland? Do you have old shower doors Upland serving you for years now? Are these shower doors already in need of some repairs or even replacement? If your answers are all yes, now is the perfect time to do something about your broken shower doors. What We Offer |
 |
| (Published: Fri, 17 Feb 2012 03:04:26 +0000) |
|
|
|
Dating Advice Today |
Tips For Short Men To Look Taller Beautiful Women |
| Many short men are too insecure or not confident around beautiful women, especially taller women. If you are a short man, here are a few tips for you to have success with women:
1)Learn to look confident. Feeling confident and look confident are two different things. Some people always look confident, where as others, should work |
 |
| (Published: Sat, 23 May 2009 04:15:15 +0000) |
|
|
|
Water Filtrations |
How Water Filtration Evolved |
| Water will fall from the sky. Drops through the lakes and streams absorbing the energy and essential minerals. Humans drink it, uses the natural ingredients for energy. It is then passed back to the earth where it evaporates and a whole natural cycle is complete. But-that was then.
As the human population grew, water became industrially |
 |
| (Published: Wed, 12 May 2010 03:08:57 +0000) |
|
|
|
Content Creation - Using My iPhone to Draft Ideas |
Using My iPhone to Draft Ideas |
| topsyWidgetPreload({ "url": "http%3A%2F%2Fwww.richardpargeter.com%2Fusing-my-iphone-to-draft-ideas", "style": "big", "title": "Using My iPhone to Draft Ideas" }); Some people stress over creating content for their websites. And to be honest I was one of those people. I constantly put off writing because I saw it as a job that needed a lot of time to do. I made excuses |
 |
| (Published: Wed, 09 Mar 2011 08:15:20 +0000) |
|
|
|
Discount patio Furniture |
Special From Sunbrella |
| One of the cool things about the Internet is the amount of great news you can find out about if you just know where to look.
I have one of those very cool alerts from Gmail set up on my account which lets me know when there is interesting stuff in the news about particular items. |
 |
| (Published: Wed, 03 Nov 2010 20:00:16 +0000) |
|
|
|
|