Feldman's rebuilt dental campaign hits 0.733% qualified interest
Over half the replies on the original run were out-of-office autoresponders, and rescoring flipped which angle won.
Jay Feldman says the rebuilt version of the Spark Study Club campaign is now running at 0.733% qualified interest, against 0.302% when the work started. The earlier numbers covered the full seven week run, 39,834 emails to 15,114 leads off 50 mailboxes across 17 sending domains, returning 274 replies and 54 opportunities, or 19.71% of replies. Feldman puts the lift over the whole period at 1.98x, from 0.302% to 0.598% qualified interest, with the best rebuilt campaign now above that and still live.
The reply count was lying
The part worth copying is the audit. Feldman tested four angles, compliance, licence requirement, recognition and state requirement, then read every reply by hand. More than half were out-of-office autoresponders. When he rescored the test on qualified interest rather than raw replies, the ranking flipped. The angle that had pulled the second most replies turned out to be the worst performer on the metric that mattered.
He cut the two weakest angles and rebuilt around compliance and licence requirement. That is the campaign now sitting at 0.733%.
"Score tests on qualified interest, not reply rate. Otherwise you may keep your worst copy and cut your best," Feldman wrote.
Score tests on qualified interest, not reply rate. Otherwise you may keep your worst copy and cut your best.
Why the client cared
Spark Study Club runs free continuing education webinars for dentists, paid for by dental manufacturer sponsors, and sponsor pricing scales with attendance. They were averaging 200 registrations a session against an industry expectation of 500. Their warm list had poor deliverability, and Feldman notes dentists are among the most pitched people in B2B. The campaign is still running as they push toward the 500 number.
The infrastructure side was unremarkable by design. Fifty mailboxes, 17 domains, 2.49% bounce rate across the run.
For people sending
An out-of-office is a reply in almost every sequencer's reporting. At the volumes most people run per angle in a split test, autoresponders alone can be enough signal to reorder your leaderboard, and the angle you kill is the one that was actually producing interest. Feldman's claim is one account on one campaign in one vertical, but the mechanism is not vertical specific, and it is cheap to check. Pull the replies from your last test, strip the autoresponders and anything that is not a human expressing interest, and see whether the winner survives.
The practical version is that reply rate is a deliverability metric more than a copy metric. If you want to know which angle to scale, you have to count the thing you are actually selling against.
Know what is landing before you burn another domain
One email a day. Under three minutes. What the operators found out yesterday.