ALA Annual Conference...Saturday, July 10...Designing Effective Research Surveys

The next session I attended was called Designing Effective Research Surveys. I was very excited about this session and thus somewhat disappointed with the simplistic coverage…although I also have to confess that I probably should have anticipated this, given that the audience was likely to be (and, in fact, was) made up of practitioners rather than more experienced researchers. However, not all was lost! I think this would be an excellent program for NASIG and one I will probably propose for their next conference. So I made a lot of notes about what worked and what didn't in the presentation. Here are some of my notes on the session:

(The speaker's name was Regina McBride). A successful survey project begins with deciding precisely what it is you want to know, specifics are very important at this point. One means to zero in on this is to talk to a colleague to who you can explain aloud what you want to know and who can help you clarify your ideas. Knowing what you want to know will help you to decide who you want to know "it" from.

She emphasized the large amount of assistance that an institutional research department can be in both collecting and analyzing data. She provided links to a couple of websites that will calculate minimum sample sizes: http://www.surveysystem.com/sscalc.htm and http://www.raosoft.com/samplesize.html. The second one is a little more involved in terms of knowledge of the statistical terms.

Surveys can be approached both qualitatively and quantitatively. McBride defined both approaches and their strengths: qualitative uses open-ended questions to achieve results that describe feelings and experiences but are more difficult to extrapolate to a population whereas quantitative uses closed-ended questions to achieve results that are precisely calculated and able to be extrapolate a broader population

In designing a questionnaire, it's important to first decide how you are going to deliver the questionnaire to the respondents. Keep the questionnaire simple: short, direct questions, clean simple presentation. The Killer B's: be brief, be objective, be simple, be specific. Use objective language (no shoulds, musts, always, nevers, frequentlys, etc.) when writing your questions. It's very important to pre-test your questionnaire to learn how potential respondents will interpret (or misinterpret) the questions, both among persons with no knowledge of the topic and those with some knowledge of it. Pre-tests will make your results carry much more weight (by increasing their validity and reliability).

[I liked the way she gave us some practice with designing good questions…but it didn't work very well. She gave us about 10 minutes to work on our own or in pairs on them and lots of people left. What might work better is to do the exercise as a group…which might also work better for a smaller audience.]