Survey
A survey is an indirect method of assessment that can measure specific goals or learning outcomes related to the teaching and learning in academic programs. For example, through students’ self-reports a survey can assess:
- the impact of educational programming
- experiences on students’ learning, development and satisfaction
- students’ preparedness for the workface and continuing education
Best research practices indicate that a good survey is short and contains a combination of quantitative (closed-ended) and qualitative (open-ended) questions.
How do I develop a survey?
Below are general steps modified from DeVellis (2012).
Step 1: Decide what to measure. In this step you decide what constructs to measure. For example, you might decide to ask students about their learning experiences in relation to program goals.
Step 2: Generate items. Once you have defined your construct(s) of interest, then you need to write survey items. Typically, you generate more items than you will ultimately use in your survey. There two main reasons for this:
- Upon review of items, you will find that some do not work as well as you like.
- It is easier to find 10 good items measuring a construct if you have a pool of 30 items, then it is to come up with 10 great items from the start.
In creating your survey questions make sure to do the following:
- Avoid exceptionally lengthy items.
- Choose an appropriate reading difficulty level (between the fifth and seventh grades for most scales).
- Avoid ambiguity.
- Avoid double-barreled items where the item could be affirmed or denied for multiple reasons.
- Follow conventional rules of grammar.
- Decide if items will be positively or negatively worded. Remember, negative wording can be confusing for responses and will need to be reverse scored.
Step 3: Determine the response scale. The third step is to determine the response scale you will use to measure the construct(s) of interest. Will you use a dichotomous scale (e.g., yes or no) or a polytomous scale (e.g., Likert agreement scale)? Will you include open-ended items?
Step 4: Review the survey. It is always best practice to have your items reviewed by an outside person to determine if your survey has face-validity, in that is measures what it purports to measure. Survey items should be reviewed for quality too (not too lengthy, negatively worded, double barreled, jargon, ambiguous or otherwise unclear). Make sure to have students review the survey for clarity in language and ease of use too. This could be done through a pilot study or by involving students in the survey development process.
Considerations when collecting data for your survey
Make sure to collect relevant student background information to help make sense of the data. When examining survey data, it is important to disaggregate students’ rating by any variables that might influence their experience. For example, student might differ in their experiences based on their major, gender identity, or enrollment status (full or part-time), to name a few.
How do I make sense of findings?
Make sure to analyze your data at the level of the question scale. This means, that if you used a dichotomous scale (e.g., yes or no) the level of analysis is limited to counts and percentages. If you used an interval-level scale (e.g., Likert-type agreement scale), you can report means and standards deviation.
Survey tools
There is a variety of tools you could use to create and administer your survey. The two most common survey tools used at PSU:
-
Qualtrics: If you are looking for a more advanced data collection this is the best tool to use. This software allows you to both collect and analyze data. This platform is easy to learn and offers a variety of training webinars. Visit the PSU OIT Qualtrics resource page.
Google Forms: If you are looking to obtain quick and easy feedback or suggestion regarding a course or meeting this type of data collection is a great choice. Data is saved to a file that can be exported for more detailed analyses. Visit the PSU OIT Google Forms resource page.
For more information on how to create surveys with Google Forms or Qualtrics, visit the PSU OIT resource page.
When should I consider doing a survey?
As you decide which survey fits your needs it is important to consider evaluating beforehand the length, number of participants, cost/benefits and the time/effort invested in the process (Wise & Barham, 2012)
Types of surveys
Student exit survey/Alumni survey. Student information about their collegiate experiences can be gleaned through surveys and exit interviews. Surveys are often conducted on exit from the university, and then again as an alumni survey at scheduled interval cycles (e.g., six-month or one-year, three-year, five-year, and 10-year follow-ups). Student interviews are topically conducted on exit.
These methods indirectly assess, through students’ self-reports, the impact of educational programming and experiences on students’ learning, development, satisfaction, and preparedness for the workface and continuing education, to name a few.
There are myriad aspects of the students’ university experience one can assess:
- Employment history
- Career development (career preparedness)
- Graduate school preparedness
- Learning outcomes achievement related to campus-wide learning outcomes and/or program-level learning outcomes
- Educational experiences related to
- Reflections on classroom and faculty experiences
- Participation in and satisfaction with academic support services
- Participation in and satisfaction with student services and programs
- Involvement and satisfaction with co-curricular and out-of-class activities and experiences
- Perceptions of and satisfaction with the overall collegiate environment
Looking for more assessment resources and guidance?
The Office of Academic Innovation is here to support you. Don’t hesitate to contact Raiza Dottin, Ed.D., Associate Director of Teaching, Learning, and Assessment at dottin@pdx.edu.