Skip to main content

Reference Material - Survey Response Coding Guidelines

Overview

This document outlines the standard process used by MAXimum Research's Data Processing team for coding open end and Other Specify survey responses. It is intended for use by both coders and programmers, and covers everything from initial information gathering through final file delivery.

The process is broken into four sections:

  • Section I – Gather Information: What to collect from the client and PD before work begins.
  • Section II – Coding Preparation: How to set up your workspace, codebook, and data files.
  • Section III – Building the Lists: How to read responses and develop approved code lists.
  • Section IV – Coding the Responses: How to assign codes, verify your work, and deliver a finished file.

Following this process consistently ensures coded data is accurate, well-documented, and easy for others to pick up if needed.

Section I – Gather Information

Before coding can begin, several key pieces of information must be gathered from the client and/or Project Director. Each item is detailed below, along with why it matters.

1. Coding Percentage Method

Most clients specify a percentage threshold for coding. This can be presented in one of two ways:

a. Percent to Code

This method is most commonly used for Other Specify questions. The client specifies a threshold — for example, 5% — meaning a code should be created for any response that accounts for 5% or more of the total respondents answering that question. This is not 5% of the "Other" responses — it is 5% of everyone who answered the question.

  • If the question was asked of all respondents, use the total quota as the base.
  • If only a portion of respondents were asked the question, use that sub-group as the base. A project marginal will show you this.

You do not need to wait until the job is complete to begin coding with this method. If a response is already at or above 5% in a partial file, it will very likely remain there. You will simply need to re-check uncoded responses in later files to see if any have crossed the threshold.

b. Percent of Other Code

This method sets a target for how small the remaining "Other Mentions" bucket should be when coding is complete. For example, a client may require that the Other code represents no more than 10% of respondents when finished. The smaller the allowed percentage, the more thorough the coding must be.

  • Like the Percent to Code method, the base is either the total quota or the sub-group answering, depending on the question.
  • Unlike Percent to Code, this method typically requires multiple passes through the data as you work toward bringing the remaining Other responses below the target threshold.

2. Check for Existing Codes and Lists

Open End questions typically do not have a pre-built response list. However, some clients may provide a starter list to build from, and new codes can be added to it. Other Specify questions will always have an existing code list attached to the driving question.

Always refer to the QPX file in Textpad — not the hard copy of the survey — when reviewing code lists. The hard copy may be outdated or may show different code/response combinations than what is actually in use.

a. Open End Questions

In most cases, the programmer will anticipate coding is needed and will build a hidden coding question into the QPX. A standard 3-digit code structure is used:

  • 997 = Other Mentions
  • 998 = Don't Know
  • 999 = Refused
  • 000 = No Answer / None / Blank

b. Other Specify Questions

Review the existing list in the QPX to identify where the codes left off, and confirm what values are assigned to Other, Don't Know, Refused, and None. When the programmer knows coding will be needed, they will typically follow the same 3-digit structure above — but when coding is unexpected or the client requests specific codes, verify what was actually used before proceeding.

c. Client Codebooks and Special Instructions

This is also the time to ask the PD whether the client has provided a codebook to follow, or has any specific requirements for the coded responses. An example would be a client requesting that a Positive / Neutral / Negative net be added to an open end asking why something was rated a particular way.

3. Timeline

Timeline is the most critical factor in planning the coding process. You need to know when coded data is due so you can schedule accordingly.

As a general rule of thumb:

  • Allow 1 hour per 100 cases, per question to complete coding.
  • Add 1 additional hour per question when you also need to build the code list from scratch.

If coded data is due the day after a job completes, coding work should begin as soon as data starts coming in — do not wait until the job is finished. Bid sheets and job alerts should have deadline information listed, provided the salesperson and/or PD were informed prior to the job start.

Section II – Coding Preparation

Once you have gathered the necessary information, it is time to set up your workspace before coding begins. There are five steps involved in preparing to code.

1. Create Your Coding Folder

Inside the project's job folder, create a new folder called Coding. All coding work must be saved here so that anyone else who needs to step in can find and continue the work without confusion.

2. Create Your Codebook in Word

Start with a blank Word document. At the same time, open the survey QPX file in Textpad. The QPX can be found at:

Phone Studies: t:\studies\phone\jobname\jobname.qpx
Online Studies: w:\studies\client\jobname\jobname.qpx

In the QPX, search for the text "!TEX" (without quotes). This tag is used for 99% of open end and Other Specify questions. For each instance you find:

  • Look one or two lines above it for the question label, which will appear in the format {Q##: — this is the label you will see in the open end Excel files during coding.
  • Most Other Specify labels end in _os. Most Open End labels end in _oe, or have no suffix.
  • Other Specify questions will also typically have an !if condition line below the label, identifying the base — meaning which respondents are routed to this question. For example: {q12_os: with !if q12(21) means question Q12 is the list question and code 21 is the "Other Specify" code.

Copy and paste each question from the QPX into your Word codebook. You may clean up the formatting to make it more readable. When finished, cross-reference your list against the hard copy of the survey to confirm no questions were missed.

Always follow the code numbering from the QPX, including the number of digits used:

  • If a question uses codes 01–10, new codes must be in the range 11–99.
  • If a question uses codes 001–015, new codes must be in the range 016–999.
  • If a question uses single-digit codes (1–5), new codes can use 6–9, then A–Z.
  • If more codes are needed beyond what the field allows, the programmer must create a new question. They will provide the updated list when ready.

For all Open End response questions, always include the four standard codes listed in Section I, Item 1a (997, 998, 999, 000).

3. Note Single vs. Multiple Response Questions

As you build your codebook, note whether each question is single response (only one code can be assigned) or multiple response (more than one code can apply). This will affect how you code the data later. Typically this can be garnered from the question text, by words like "Select ALL" or "Which ONE", etc. If unsure, DP can confirm.

4. Obtain Sample Data

You will need sample data to begin building code lists. If you know how to set up and run the oe_edit.spx file located in each job's DP folder, do so now. If not, ask the PD or Programmer to pull a file for you.

Best Practice: Do not run the file on data collected the same day you are running it. Always run it from the first day of the job through yesterday's date. This avoids having to strip out already-coded records when you run subsequent files later.

Save the output file in your Coding folder using the naming convention:

openends_mmddyy.xlsx

This prevents it from being accidentally overwritten by later runs. When you run the file again, run it from the day after your last file through yesterday's date, and repeat this process until the job is complete.

5. Begin Building Code Lists

At this point you are ready to start developing your code lists. This step requires uninterrupted concentration — similar to programming a survey. Do not hesitate to send an email to your team letting them know you will be unavailable for the next several hours.

Use Excel filters to view one question at a time and sort responses alphabetically. This helps group identical answers together (such as "None" or "Don't Know"), but it is only a starting point. You must read all responses carefully to truly understand what the data is saying. The full process for building code lists is covered in Section III.

Section III – Building the Lists

This section covers the process of building code lists for Open End and Other Specify questions in detail.

1. Set Up Excel Filters

Filtering your data is the first step, and makes the response review process much more manageable by letting you focus on one question at a time.

  1. With your coding file open, click on the first row (the header row containing Case, Question, and Response labels).
  2. Click the Data tab, then click the Filter button. Small dropdown arrows will appear next to each column header.
  3. Click the arrow on the Question column. Uncheck Select All, then select only the question or questions you want to work on. In cases where multiple questions share the same list (such as stores or hospitals), select all relevant questions together.
  4. Once your question filter is applied, sort the Response column A to Z. This groups similar answers together and makes it easier to identify Don't Know, None, Refused, and other common responses at a glance.

2. Start Building Codes

Do not actually code the data at this stage. The goal right now is to build the list only. Making changes to codes after data has already been assigned is significantly more time-consuming than getting the list right first.

  1. Open your codebook (see Section II, Item 2) and navigate to the question you are working on.
  2. Keep scrap paper nearby. As you read through responses, jot down potential codes and tally how many times you see each one. Once a response meets the coding percentage threshold discussed in Section I, add the response text to your Word codebook — but do not assign a code number yet.
  3. Watch for responses that use different words but carry the same meaning. For example: "They are close to my home," "Convenient locations," and "There are lots of branches in my area" all describe the same concept and should be grouped under one code.
  4. Also watch for responses that sound similar but have opposite meanings. For example, when asked why something was rated positively or negatively: "There are a lot of locations" and "There are too many locations" both reference quantity, but mean very different things in context.
  5. Read the full response before assigning any code. Respondents often ramble. A lengthy response may ultimately be saying just one simple thing. For example: "I am really not sure what the brand was called, all I know is that the store always has it in stock and it has their logo on it" — this is a roundabout way of saying "Store Brand" and should not be coded as Don't Know.
  6. After your first full pass through the file, assign code numbers to the list items that met the threshold and add them to the codebook. Mark them off your scrap paper so you know they made the cut. Hold onto the scrap paper — you will reference it again during subsequent passes.
  7. Submit the completed list to the PD and/or client for approval. Do not begin coding responses until approval is received.

Section IV – Coding the Responses

Once your code list has been built and approved by the PD and/or client, you are ready to begin assigning codes to the actual response data.

1. Filter and Sort Before You Begin

Just as when building the lists, apply your Excel filter and work one question at a time — or group questions that share the same list together. Sort the response column again if it is not already sorted.

2. Assign Standard Codes First

Before coding open responses, assign your Don't Know, Refused, and No Answer codes first. Then apply a filter to the code column and hide those already-coded rows, leaving only the blanks visible to work through.

3. Code the Responses

Work through the remaining responses following your approved codebook. A few important rules to follow:

  • Do not code Other Mentions yet. Leave responses that do not meet any code as blank for now. After additional data files are added, you may be able to create new codes from what remains.
  • If you are coding Other Specify responses and a respondent's answer actually matches an existing code on the driving question's list, you may correct it here.
  • As when building the lists, stay alert for responses that look different but share the same meaning, and for responses that look similar but mean something different in context.
  • You do not need to zero-fill code numbers. The programmer will handle that before adding the data. If the list uses 001 for a code, entering just 1 in the cell is fine.

4. Run a Pivot Table to Check Your Work

After completing your first pass on a question, run a pivot table to verify your coding before moving on. To do this:

  1. Highlight the QLabel and Code columns together.
  2. Click the Insert tab and select Pivot Table. When the dialog box appears, click OK.
  3. A new sheet will open with the pivot layout and field boxes on the right side. Configure it as follows:
    • Drag the QLabel field into the Filters box.
    • Drag the first Code column into the Rows box.
    • Drag the first Code column into the Values box.
    • If the Values box shows Sum of instead of Count of, click the field, select Value Field Settings, and change it to Count.
  4. Use the filter at the top of the pivot table to select the question you just coded.
  5. Review the counts against your codebook to confirm nothing was mistyped or miscoded.
  6. Repeat this process for each additional code column by swapping out the Code field in both the Rows and Values areas, again ensuring each is set to Count rather than Sum.

Once you are satisfied the coding is accurate, move on to the next question and repeat the process from Step 1 of this section.

5. Finishing a Complete Pass

When you have worked through all questions in the file:

  1. Remove all filters so the full dataset is visible again.
  2. Save the file.

6. Adding Additional Data Files

If more data files will be coming in as the job continues:

  1. Open both the master coding file and the new data file.
  2. Copy all data rows from the new file and paste them at the bottom of the master file.
  3. Return to Step 1 of this section and work through the new data.
  4. After adding a second file — or once no further files are expected — go back and re-examine all blank (uncoded) responses. Determine whether any new codes can be created from what remains. PD and/or client approval is still required before adding new codes.

7. Wrap Up

  1. Once all passes are complete and no further codes will be added, go through the remaining blank responses and assign them the Other Mentions code. Every record must have a coded response before the file is considered done.
  2. Notify the PD and Programmer that coding is complete.

Summary

Coding survey responses is a methodical process that requires careful preparation, close reading of the data, and clear communication with the PD and client at key checkpoints. Rushing any stage — particularly list building or approval — creates significantly more work downstream.

When in doubt: slow down, re-read the responses, and ask the PD. A well-coded file the first time is always faster than correcting a poorly coded one after the fact.

Back to top