In the past few days I've been offering lessons I learned the hard way through managing campaigns.

The biggest lesson was to actually test a message before spending hundreds of thousands of dollars on TV time.

Which brings up the question:  How do you know if a message works?

In the old days it was simple.  There were only a few channels--three to five local TV stations, three national networks, newspapers, radio.

Now there are hundreds of cable networks, blogs, every newspaper can be accessed online.

That creates a lot of noise in the system.  Noise that has to be accounted for.

In the old days you made an ad, focus grouped it and put it on the air.  If it moved poll numbers, you would say the ad worked.

But with all the different channels and all the noise, there are more variables that can move poll numbers.

Therefore, to determine if an ad or message actually works you must control for all the variables.

This can be done in two ways:

A.  Use a series of many random noise signals as a control to determine if you are consistently beating randomness or;

B.  Randomized double or triple blind testing with control groups and test groups.

Since most candidates are not well versed enough in the mathmatics of randomness to trust "A" the only viable option is "B."

(Only the Jacksonville Jaguars understand randomness enough to use the noise signal as a control.)

That requires selecting random samples that are exposed to the message at the appropriate level of Gross Rating Points then comparing the results to a random sample in a control group.

For television, that would require running ads in isolated, cheap markets in non-battle ground states.

In its simplest form it works like this:

Two media markets of similar demographics are selected.

One is the test market, the other is the control market.

A poll is run in both markets.

The ad is then run at 1,000 Gross Ratings Points in the test market.

After the ad run is completed, both markets are polled again.

If the ad really works, if it really converts voters, the test market will out poll the control market by a statistically relevant margin.

This process takes 10-11 days.  It also removes the guess work from deciding on whether to drop a couple million dollars in TV time in a key state.

(When the Obama campaign made buys in some unusual states, I suspected they may have been using them as tests.  Or maybe they really thought they could compete.)

I learned the hard way that if you do not truly test a message, you are just guessing if it works.  You can get on a run of guessing right.  You can also get on a run of guessing wrong.

If both sides are just guessing randomness is in control of the election.  Or, more precisely, randomness and the noise is in control of the election.

So, the next time you look at an ad and say it is good or bad, or think a candidate should run a certain type or ad or use a certain message--are you willing to bet a couple million dollars your hunch is correct?

I'm not.  Which is why I test everything.

---