You notice something that isn't working: office hours turn into you debugging students' code, regrade requests pile up after every assignment, nobody answers in section. You have an idea for a fix. A small teaching experiment means trying that one change on purpose and checking whether it helped, instead of guessing. It doesn't need statistics or a paper. It needs one question, one measure, a fair comparison, and honesty about what the result can and can't tell you. This page walks through the steps, gives a one-page plan, six examples for CS TAs, the usual traps, and the ethics, including when classroom data collection needs review by your institution's review board.
Defaults that work. If you only take four things from this page:
- Change one thing, and keep everything else the same.
- Measure something you already see or record: counts, minutes, regrade requests, a two-question survey.
- Get a "before" or a fair comparison. Without one, you can't tell what changed.
- Report what happened honestly, including "no difference." That is a result too.
The steps
This follows the plan, do, study, act cycle used for testing small changes in many fields.
- Notice a problem you see every week. "Office hours turn into me debugging their code."
- Ask a question you can answer in two to four weeks, about one change and one outcome. "Does opening with 'What have you tried?' cut the time I spend at the keyboard?"
- Pick one measure and a baseline. Collect a week or two of "before" data first, or set up a comparison.
- Check the ethics (below), and tell your instructor.
- Make the change. Keep everything else the same, and keep a short log of anything unusual.
- Look at the results. Compare with the baseline, look at the spread and not only the average, and write down other explanations.
- Decide: keep it, adjust it, or drop it. Write one page, including a null result, and share it.
A one-page plan
- Course, your role, dates.
- The problem you noticed, in a sentence or two.
- The one change: exactly what you will do differently.
- The question: change, then outcome, answerable in four weeks or less.
- Your prediction: what you expect, and by about how much.
- The measure, and how you will record it.
- The comparison: before and after, alternating weeks or sections, or two versions in the same section.
- How many: students, sessions or submissions.
- Likely confounds, and how you will reduce them.
- Ethics check: normal teaching? Everyone gets the benefit? Anonymous? Will you publish? Instructor told?
- Decision rule: what result means keep, adjust or drop.
- Results and next step, and who you will share them with.
Six examples for CS TAs
| Setting | Change | Measure | Comparison |
|---|---|---|---|
| Office hours | Open every visit with "Show me what you tried and what you expected" | Minutes per student; how often you take the keyboard | Two weeks before vs. two weeks after |
| Grading | Add a three-level rubric row with one example comment per level | Regrade requests per assignment; minutes per submission | The previous assignment of similar size |
| Discussion section | A three-minute warm-up question on last week's topic | Exit-question accuracy; count of "muddiest point" answers | Alternate weeks with and without the warm-up |
| Course forum | A pinned "how to ask a good question" template | Share of posts with the error and code attached; time to first answer | Two weeks before vs. after |
| Labs | Pair programming with a role swap every 15 minutes | Checkpoints finished by the end of lab; a two-question survey | Your two sections take turns having the change, swapping each week |
| Exams | A practice exam with worked solutions, released five days early to everyone | The kinds of questions you get in office hours; scores on similar questions | A past term's similar questions, cautiously |
Common traps
| Trap | Fix |
|---|---|
| The time of term changes everything (deadline weeks differ) | Alternate weeks, or compare the same point in the term |
| The topic got harder or easier | Compare similar assignments or matched questions |
| Different sections or times of day | Swap which section gets the change |
| You try harder while testing | Run it for more than one week, and note your own effort |
| Students know they're being watched | Measure things you already record |
| Different students show up | Report attendance; avoid "those who came did better" |
| Your baseline week was unusually bad | Use two or more baseline weeks |
| Survey wording or low response | Keep the same questions, and report the response rate |
| Small numbers | Report raw counts and ranges, and call it a signal, not proof |
Reporting honestly
With 20 to 40 students, a difference of a few points can easily be noise. Show the raw numbers, say what else changed during the trial, and describe the result as "worth continuing" rather than "it works." A null or negative result saves you, and the next TA, from a change that doesn't help, so report it the same way. If you want to see what stronger evidence looks like, studies comparing teaching methods across many terms and courses are a good model (two are in the reading list).
Ethics and review
- Improving your own teaching is usually not "research." Publishing is the line. U.S. federal rules define research as a systematic investigation "designed to develop or contribute to generalizable knowledge." Collecting feedback to improve your own section generally falls outside that.
- If you might publish, in a conference paper, a poster or a thesis chapter, you need your institution's review board (IRB) approval before you collect any data. It can't be added afterward. Studies of normal educational practice may qualify as exempt, but the IRB decides that, not you.
- At UMass, the Human Research Protection Office explains when classroom projects need no review, and has a form and a phone line (413-545-3428) for when you're unsure.
- Don't disadvantage anyone. No group should lose something that affects grades. Swap conditions between sections or weeks, or give everyone the better version afterward.
- Protect student data. Grades and submissions are education records under FERPA. Keep data in course systems, report only group numbers, never share names or student work, and keep surveys anonymous, optional and separate from grades.
- Tell your instructor before you start.
Further reading
Doing it
- Institute for Healthcare Improvement, Plan-Do-Study-Act (PDSA) Worksheet: a form for testing one change.
- Angelo and Cross, Classroom Assessment Techniques: about 50 quick ways to see what students are learning.
- James Lang, Small Teaching (2nd edition): small changes worth testing, grounded in learning research.
- Geoffrey Mills, Action Research: A Guide for the Teacher Researcher.
- Illinois State University, Scholarship of Teaching and Learning.
- Carleton College, The Scholarship of Teaching and Learning.
Ethics and rules
- UMass Amherst, Classroom Research Projects and the IRB.
- UMass Amherst, Activities Involving Human Subjects Research.
- 45 CFR 46.102, Definitions and 45 CFR 46.104, Exempt research.
- U.S. Department of Education, FERPA.
Examples of stronger evidence
- Porter, Bailey Lee and Simon, "Halving fail rates using peer instruction: a study of four computer science courses" (SIGCSE 2013).
- Freeman et al., "Active learning increases student performance in science, engineering, and mathematics" (PNAS, 2014).
Related pages: Getting Feedback on Your Teaching, Office Hours FAQ and Teaching Statements and Portfolios.