If you’ve ever been tasked with writing survey questions, you know it’s harder than you might expect. For our March session, Teresa Edwards from the Odum Institute will present on writing effective questions and best practices for web-based surveys.
Date: Thursday, March 5th
Time: 2:00 PM – 3:00 PM
Location: Graduate Student Center
Presenter: Teresa Edwards, Odum Institute
Survey Design Best Practices (pdf)
Presentation Recap and Notes
Presentation Overview
- Writing questions
- Response categories and scales
- Questions to collect qualitative data
- Ordering questions
- Format and layout
Goals for writing
- Each question interpreted the same way by all respondents
- Minimize cognitive burden
- Maximize accuracy
Response Task
- Comprehension
- Recall
- Judgment
- Formatting
- Editing
Example
In the past 6 months, have you used Google or another search engine to look on the internet for information about an infectious disease?
- Comprehend individual words (Google, “another search engine”, internet, infectious disease) and put them together to understand the question, reference period (past 6 months)
- Recall any useful information/experiences (My mother got sick and we couldn’t figure out what it was, we got that note from the school about chicken pox)
- Make a judgment based on information retrieved (Does googling “causes for cough” count? Was mother’s illness within the past 6 months?)
- Format response using response options (Yes, No)
- Edit answer, social desirability
Tips for question Writing
- Keep questions short
- Use simple words
- Define key terms
- Use a reference period
- In the past 30 days
- Be specific, but use examples carefully
- Avoid “double-barreling”
- Would you like to be rich and famous?
- Avoid hidden assumptions
- How old were you when you first smoked a cigarette?
- Avoid negative grammatical structure
- Women with young children should not work.
Tips for Writing Response Categories
- Response categories should:
- Match the format of the question
- Be mutually exclusive (unless check all that apply)
- Cover all circumstances
- Not assume regularity of frequency
- “In the past 6 months, how often did you visit the library” – might be a lot of variation in a long time period
- Unsure/don’t know option
- Don’t assume respondents understand percentages
- Rethink ranking
- only use if you want to know the ranking order really matters
- Beware of anchoring
- Respondents may assume that the “middle” means average or typical and choose/adjust their response accordingly.”Surely I don’t want more tv than the average person”
- Scales
- Label all points on a scale
Open Ended Responses
- Specify the reporting unit
- Size the answer space appropriately for the information being asked
- Punctuate the answer space appropriately (for phone numbers, dates, etc.)
Collecting Qualitative Data
- Be clear what you seek
- What do you think of this webpage? vs. What aspects of this webpage are helpful in your daily work
- Size the answer space appropriately
- If you’re asking open ended questions, ensure that the response boxes are sized appropriately
- Only one question per answer space
- Have a plan for analyzing/summarizing the data