Standard Market Research Formulas
Overview
This reference document defines the key performance metrics used in project reporting at MAXimum Research. It is intended for use by Sales, Project Directors (PDs), and Phone Operations staff. Metrics are organized by data collection channel: Phone and Web / SMS. Understanding these rates helps evaluate project health, sample quality, and overall field performance.
A note on manual calculations: When calculating any of these metrics by hand, results will always appear in decimal format (e.g. 0.45). With the exception of Production Rate, all metrics should be interpreted as a percentage -- meaning 0.45 = 45%. Production Rate is expressed as a plain number representing completes per hour (e.g. 2.5 = 2.5 completes per hour).
Phone Metrics
Production Rate
Formula:
Completes / Hours Worked
Production measures how many completed interviews are achieved per hour of calling time. It can be calculated at the project level or broken down by individual agent / interviewer (INTV). Higher is better.
Every project carries two production benchmarks:
- Bid Rate -- The minimum completes-per-hour needed to meet the budgeted cost of the project. Falling below this rate puts the project over budget.
- Goal Rate -- Typically set at between 15% and 20% above the Bid Rate. This is the target agents and INTVs should be actively working toward during a shift. Reaching or exceeding the Goal Rate is profitable for both the company and INTVs as it means less man-hours were required to reach the client's expected # of completes. From and INTV standpoint, exceeding the Goal opens the possibility of earning incentive, a bonus to their payrate while on this project.
- Bid Rate & Goal Rate may change over the course of a project as incidence, survey length, contact/cooperation rates all play a role in assigning/calculating these rates. The rates could increase or decrease multiple times on any given project.
Incidence Rate
Formula:
(Completes + Qualified Refusals) / (Completes + Qualified Refusals + Overquotas + Terminates)
Incidence is the percentage of people who start the survey and actually qualify to complete it. It reflects how well the available sample matches the target population for a given project.
Qualified Refusals are respondents who made it through the screener and qualified for the survey, but then declined to participate in the main portion. Because they did qualify, they are counted in the incidence numerator.
Incidence can vary significantly from project to project and can shift mid-field. Common causes of a lower-than-expected incidence include:
- List accuracy issues -- For example, a project targeting registered Democrats may be reaching a high volume of other party affiliations, resulting in excessive terminates early in the survey.
- Survey design -- A higher number of screening questions gives more opportunities for respondents to terminate before qualifying.
- Quota closures -- When quotas fill faster than anticipated and the sample cannot suppress already-filled records, those calls still go out and result in overquotas rather than completes.
Sales works with the client to establish an initial incidence estimate before fielding begins, and it is common to adjust that rate mid-field as actual data provides a clearer picture.
Contact Rate
Formula:
(Completes + Suspends + Refusals + Terminates + Overquotas + Callbacks + Respondent Hang-Ups + Against Company Policy + Non-Residential/Business/Government) / Total Calls Made
Contact Rate is the percentage of all calls made that resulted in a true contact -- meaning a live person was actually reached. Higher is better.
This metric has a direct upstream impact on both incidence and production. If contact rates are low, fewer people are being reached per hour of dialing, which compresses the opportunity to generate completes regardless of how well the rest of the project is performing.
*This should NOT be confused with the dialer's CONNECT rate, as that will generally be higher. CONNECTS also contain Answering Machines, "Dialer Dead Air" calls, Language Barriers, and other calls that make it through to an agent/interviewer but are not a true "contact".
Cooperation Rate
Formula:
(Completes + Suspends + Terminates + Overquotas) / Total Contacts
Cooperation Rate measures, of all the people actually contacted, what percentage showed enough interest to at least begin the survey -- regardless of whether they ultimately completed it or qualified. Higher is better.
This is distinct from Contact Rate. Contact Rate asks "Are we reaching people?" Cooperation Rate asks "Of the people we reach, are they willing to engage?"
Refusal Rate
Formula:
Total Refusals (Soft + Hard + DNC) / Total Contacts
Refusal Rate is the percentage of contacts who were not willing to participate and refused out of the call entirely. Lower is better.
Elevated refusal rates may point to one of two things:
- Agent / INTV miscoding -- Calls being incorrectly dispositioned as refusals when they may belong in another category. This is worth auditing at the interviewer level if rates spike unexpectedly.
- Subject sensitivity -- Some topics (women's health, taxes, personal medical issues, etc.) generate genuinely higher refusal rates because respondents are simply not willing to discuss them with a stranger over the phone. This is expected and should be factored into project planning.
Disconnect Rate
Formula:
Disconnected Numbers (All Dead Dialer Codes + INTV-Coded Disconnects) / Total Calls Made
Disconnect Rate reflects the percentage of numbers dialed that are no longer in service. Lower is better. This metric is primarily used as an indicator of sample quality.
Expected ranges vary significantly by sample type:
- Client-provided lists / Voter roll files -- Should produce very low disconnect rates. If a client list is running high, the list is likely outdated.
- Consumer sample -- May run slightly higher than client lists, as consumer databases are not updated as frequently.
- True RDD (Random Digit Dialing) -- Disconnect rates can be quite high by nature, since RDD numbers are algorithmically generated and have no confirmed association with a real person or active line.
Web / SMS Metrics
The three Web / SMS metrics below work as a funnel. Each one measures a different stage of the respondent journey. For example: of 10,000 records sent, 9,000 were successfully delivered (90% Delivery Rate). Of those 9,000 delivered, 900 clicked the survey link (10% Click Rate). Of those 900 clicks, 90 completed the survey (10% Completion Rate).
Delivery Rate
Formula:
Delivered / Sent
Delivery Rate measures what percentage of records sent were successfully delivered to a respondent. Higher is better. This is the web/SMS equivalent of Contact Rate in phone fielding.
A low delivery rate may indicate issues with message deliverability, spam filtering, or sample quality -- similar to how poor contact rates on the phone side signal dialing issues or an outdated list.
Response Rate / Click Rate
Formula:
Clicked / Delivered
Response Rate and Click Rate refer to the same measurement -- the terms are used interchangeably depending on context or client preference. Both apply equally to email/panel and SMS-based survey distribution. Higher is better.
This metric is the web/SMS equivalent of Cooperation Rate in phone fielding. It answers the question: "Of everyone who received the survey invitation, how many actually clicked through to begin it?"
Completion Rate
Formula:
Completes / Sent
Completion Rate is the web/SMS equivalent of Production Rate in phone fielding. Rather than measuring completes per hour, it measures what percentage of all records sent ultimately resulted in a completed interview. Higher is better.
This metric gives a top-level view of how efficiently a web or SMS deployment is converting the available sample into usable completes.
Cost Per Complete
Formula:
(Total Records Sent x Cost Per Record) / Completes
Cost Per Complete measures how much it costs, in SMS/email spend, to generate a single completed interview. Lower is better. This metric helps evaluate the overall efficiency and cost-effectiveness of a web or SMS deployment.
Current vendor rates per record sent:
- BBD -- $0.02 per record
- TTH -- $0.03 per record
For example, if 10,000 records are sent via BBD at $0.02 each, the total send cost is $200.00. If that deployment yields 90 completes, the Cost Per Complete is $2.22.