Syllabus
Updated October 23, 2026.
Course information
| Course | CICS 190G, Generative AI: Evidence, Failure, and Judgment (190 is the University number for experimental courses) |
| Term | Spring 2027 (pilot, experimental course) |
| Credits | 4 |
| Gen ed | R2 Analytic Reasoning (designation proposed). Letter grade only: Gen Ed courses may not be taken pass/fail. |
| Enrollment | About 30 |
| Prerequisites | None. Open to all majors. No programming or math background assumed. |
| Format | Two 75-minute interactive class meetings and one 60-minute lab per week (210 minutes of contact time), in permanent teams of 5 or 6 mixed by major |
| Delivery mode | In person. All class meetings and labs are in the classroom; course materials and submissions are on Canvas. |
| Appointment hours | Two hours a week, times posted on Canvas in Week 1, plus appointments by Canvas message. |
| Instructor | Tim Richards, tdr@umass.edu. Teaches the class meetings and runs the labs. |
| Course staff | None for the pilot. The pilot produces the first undergraduate course assistants. |
| Website | Canvas, for syllabus, schedule and grades |
Catalog description
How do generative AI systems turn words into answers, and when should we trust them? Students learn the core mechanics of language models (tokens, sampling, embeddings, attention, retrieval and agents), then test real systems through structured experiments, document where and why they fail, and contribute findings to a shared record of failures. Written position papers take up understanding, authorship, labor and trust. Open to all majors; no programming or math background required. Substantial direct interaction with generative AI tools is a required part of nearly every assignment.
In plain words. You use AI tools every day. This course is about knowing when to trust them. You will learn how systems like ChatGPT turn your words into answers, run careful experiments to see where they break, and keep a record of what you find. Along the way we take on the big questions: can a machine that produces the right answer understand anything, who owns what it writes, and what should we believe when it sounds sure of itself. No programming and no math background needed. Everything happens in your browser.
Learning objectives
By the end of the course, students will be able to:
- Write and systematically evaluate prompts using few-shot examples, chain-of-thought, decomposition and self-critique.
- Design, assemble and test a working generative AI system, such as a retrieval pipeline or a tool-using agent, and explain what each step contributes and where it fails.
- Explain core language model mechanics (tokenization, embeddings, attention, sampling) well enough to predict where and why a model will fail.
- Design and conduct an audit of a real deployed AI system, including the course’s own tools, identifying specific and testable failure modes.
- Take a position on a philosophical question raised by generative AI, then revise that position in writing when later evidence undermines it.
- Distinguish AI marketing claims from what a system’s architecture can actually support.
- Contribute a documented finding to a shared record of AI failure modes that outlives the semester.
How the course meets R2 (Analytic Reasoning)
The course meets all four R2 objectives through graded, recurring work, not a single unit. Criteria as stated on the UMass Gen Ed R1 and R2 page.
| R2 objective | Where the course does it |
|---|---|
| Advance formal or mathematical reasoning beyond basic competence | Students compute one attention step by hand (query, key, value, weighted sum) before using the visualizer (Unit 05); reason about probability distributions and temperature (Unit 03) and cosine similarity (Unit 04); every lab starts with a written prediction checked against the result. |
| Increase sophistication as a consumer of numerical information | Batch runs of one prompt 20 times, with defaults tallied (Unit 10); consistency checks across five phrasings (Unit 11); reading model “confidence” as a probability, not a guarantee. |
| Provide computer literacy | Students assemble and test working systems (a retrieval pipeline, a tool-using agent) in a browser workbench, and run controlled comparisons across models of different sizes. |
| Indicate the limits of formal and quantitative reasoning and the potential for abuse of numerical arguments | The Midterm System Audit and the failure ledger document where statistical systems fail; Unit 01 and objective 6 take apart benchmark and marketing claims; Dialogue 4 asks what trust means for a system with no notion of certainty. |
General Education objectives
This course is proposed for the R2 (Analytic Reasoning) designation. Its primary Gen Ed objectives are applying methods of analysis to real-world problems; analytical, quantitative and critical thinking through inquiry and problem solving; communicating persuasively in writing; and technological literacy. Secondary objectives are methods of analysis, the consequences of one’s choices, oral communication, working in groups across perspectives, information literacy, and pluralistic perspective-taking: Unit 10 shows how unequal representation in training data produces biased output for under-represented groups and what that does in the real world, and Lab 9 has you compare a model’s defaults with your own community.
The course assumes basic arithmetic, percentages and reading a simple table; it teaches every other quantitative idea it uses.
How the course runs
Every week has two interactive class meetings and one lab, all done in permanent teams. No talk runs past 15 minutes before teams do something with it.
- Class meetings (2 x 75 minutes). Hook, short talk, team task, debrief. Team tasks run on the UMass AI portal and the course workbench. Dialogue seminars take part of one meeting in Weeks 5, 10, 12 and 14.
- Lab (60 minutes). Hands-on work on that week’s Workbench Lab, with the instructor in the room. In Week 7 the slot is midterm studio, in Week 12 the retraction workshop, and in Weeks 13 and 14 capstone studio and demo day.
- Teams. Permanent teams of 5 or 6, mixed by major, assigned in Week 1.
- Tools. A web browser and a laptop. Students use the UMass AI portal for frontier-quality prompting and a course workbench (prompt bench, tokenizer, sampling control, embedding plot, attention heatmap, pipeline builder) that runs small open models hosted for the course. No installation and no code.
- Materials. No textbook. Short papers and case studies on Canvas. No cost beyond standard course technology.
Assessment and grading
Grades come from six categories that sum to 100%. Every grade is given by a person against a published rubric. No AI tool assigns, suggests or changes a grade. Labs and the Midterm System Audit use published yes/no checklists, each item worth 1 or 2 points, so you can see exactly what earns credit; Dialogue papers use a three-level rubric.
| Category | Weight | How it works |
|---|---|---|
| Workbench Labs | 28% | 10 labs; saved session checked against a rubric, plus a reflection; lowest 2 dropped |
| Final Capstone | 20% | Cross-major pairs; design, audit, reflection, practitioner feedback |
| Philosophy Dialogues | 17% | 4 position papers, seminar participation, 1 formal retraction |
| Prompt Engineering Portfolio | 15% | 4 checkpoints; Weeks 2 and 12 include baseline comparisons |
| Midterm System Audit | 15% | Red-team a deployed product |
| Team Participation | 5% | In-class and lab collaboration |
Scale. A 93 to 100, A- 90 to 92, B+ 87 to 89, B 83 to 86, B- 80 to 82, C+ 77 to 79, C 73 to 76, C- 70 to 72, D+ 67 to 69, D 60 to 66, F below 60.
Late tokens. 4 per semester, applied automatically, at most 2 on one assignment, usable only on Labs and Portfolio checkpoints. Illness or emergency: contact the instructor.
Make-up work. Late tokens cover Labs and Portfolio checkpoints. Dialogue papers, the Midterm System Audit and the Capstone have no late tokens; an extension needs an excused absence under the University attendance regulation (illness, religious observance, a University-sanctioned activity, or an emergency) and a message to the instructor before the deadline when possible. Missed seminars and capstone demo day are made up with a written substitute set by the instructor.
Regrades. Through Canvas within 1 week of the grade posting, citing the rubric line.
How labs are graded. Each lab goes through four steps.
- Automatic checks. Software, not AI, checks that the saved session is complete: the required runs are there, predictions were written before the runs, and reported counts match the session log.
- AI review. An AI reviewer reads the submission with instructions the instructor writes for that lab. It points the grader to parts that deserve a close look, such as a claim the counts do not support or an explanation that does not match the prediction. It is told to stay neutral, to watch for bias (for example, against writing style, English as a second language or a student’s major), and never to produce, suggest or imply a grade.
- Human grading. A person scores the lab against the rubric and writes feedback.
- AI check-back. The AI reviewer flags anything the grader may have missed and any score that looks inconsistent with how similar work was scored, so it can be checked again. The grader decides; the AI never changes a grade.
Regrade requests always go to a person.
Assignments
Workbench Labs (10, solo, team help encouraged). Each lab produces a saved workbench session plus a 150 to 250 word reflection connecting what you did to how it works underneath. Every lab starts with a written prediction. There is a lab in the weekly lab slot in Weeks 1 to 6 and 8 to 11. Lab 1 is a self-audit of the course’s own tools, and the capstone revisits it.
Prompt Engineering Portfolio (solo, 4 checkpoints). A cumulative, revised collection showing few-shot design, decomposition and self-critique, each with test cases showing where the prompt succeeds and fails. The Week 2 and Week 12 checkpoints run the same design challenge through an unprompted model and compare.
Midterm System Audit (individual or pairs). Pick a deployed AI product relevant to your major or daily life and design a structured red-team test that exposes a specific, reproducible failure, reported as a rate with its trial counts in a 1200 to 1500 word report. Well-documented findings may join the course failure ledger, with your consent.
Philosophy Dialogues (4 papers plus a retraction). Position papers of 750 to 1000 words, each followed by a seminar in the next class meeting (Weeks 5, 10, 12 and 14), run as a fishbowl in two halves:
- Does producing correct output require understanding anything?
- Whose knowledge trains the model?
- What is authorship for statistically generated work?
- What does trust mean for a system with no notion of certainty?
In Week 12, a formal retraction: identify a claim in one of your own earlier papers that later evidence undermines, and explain why (400 to 600 words).
Final Capstone (cross-major pairs). Design and assemble one working AI system in the workbench that applies a course technique to a real problem in either partner’s field, then write a joint audit of its limits (1500 to 2000 words) and a 500-word individual reflection. Presented at demo day as a gallery walk, with feedback from practitioners.
Weekly schedule
Thirteen units in three parts, one per week, with the capstone over the last two weeks. Weeks are instructional weeks; spring break is not counted.
| Week | Unit | Topics | Due |
|---|---|---|---|
| 1 | 01 Seeing past the marketing | Hype and reality; what does “understanding” mean? | Lab 1 (self-audit of the course tools) |
| 2 | 02 Prompting as a discipline | Roles and framing; few-shot, decomposition, self-critique | Lab 2; Portfolio 1 (baseline) |
| 3 | 03 Tokens and chance | Tokenization; sampling and temperature | Lab 3 |
| 4 | 04 Meaning as position | Embeddings; similarity search | Lab 4 |
| 5 | 05 Attention | Dialogue 1 seminar; attention conceptually; a transformer block by hand | Lab 5; Dialogue 1 |
| 6 | 06 Grounding answers in sources | Why retrieval exists; building a retrieval pipeline | Lab 6; Portfolio 2 |
| 7 | 07 Red-teaming | Red-teaming method; midterm studio | Midterm System Audit |
| 8 | 08 Agents and tools | Tools and structured output; agent failure modes | Lab 7 |
| 9 | 09 Choosing the approach | Prompting, retrieval and fine-tuning compared; case studies | Lab 8; Portfolio 3 |
| 10 | 10 Whose data, whose defaults | Bias and data provenance: how unequal representation becomes biased output, with real-world cases; Dialogue 2 seminar; failure case studies | Lab 9; Dialogue 2 |
| 11 | 11 Confidence and what we can see inside | Hallucination and grounding revisited; interpretability limits | Lab 10 |
| 12 | 12 Authorship and labor | Authorship and the Dialogue 3 seminar; labor and economics; retraction workshop (lab slot) | Dialogue 3; Portfolio 4 (baseline); retraction |
| 13 | 13 Capstone | Capstone studio | |
| 14 | 13 Capstone | Demo day; synthesis and Dialogue 4 seminar | Capstone (with its self-audit section); Dialogue 4 |
Parts: How it works (Weeks 1 to 5), Building and breaking (Weeks 6 to 9), Judgment (Weeks 10 to 14).
Policies
Advance notice of AI use. This course requires substantial, direct interaction with generative AI in nearly every assignment. Students who prefer not to interact with AI should know this before enrolling. (Joint Task Force on Generative AI recommendation.)
AI use. You may use AI tools, and you must disclose every use, with transcripts of the relevant prompts and responses. For Labs and the Portfolio, the model interactions are the work and are graded as such. Using AI undisclosed on work you are meant to show yourself (a Dialogue paper, the retraction or a Capstone reflection) is a violation, however central AI is to the rest of the course. Unattributed AI-generated text in those pieces is treated as plagiarism.
Academic integrity in this course. Violations include undisclosed AI use as described above, copying another student’s Lab or Portfolio work, and contract cheating. The University statement below governs.
University policies regarding Accommodations, Academic Integrity, and Title IX apply to all courses. The required statements follow (Faculty Senate, Responsible Employee Required Syllabus Statements).
Academic Integrity Statement. UMass Amherst is strongly committed to academic integrity, which is defined as completing all academic work without cheating, lying, stealing, or receiving unauthorized assistance from any other person, or using any source of information not appropriately authorized or attributed. As a community, we hold each other accountable and support each other’s knowledge and understanding of academic integrity. Academic dishonesty is prohibited in all programs of the University and includes but is not limited to: Cheating, fabrication, plagiarism, lying, and facilitating dishonesty, via analogue and digital means. Sanctions may be imposed on any student who has committed or participated in an academic integrity infraction. Any person who has reason to believe that a student has committed an academic integrity infraction should bring such information to the attention of the appropriate course instructor as soon as possible. All students at the University of Massachusetts Amherst have read and acknowledged the Commitment to Academic Integrity and are knowingly responsible for completing all work with integrity and in accordance with the policy: (https://www.umass.edu/senate/book/academic-regulations-academic-integrity-policy)
Attendance. Class meetings are interactive and team-based, so attendance matters to the work. More than 2 unexcused lab absences affect Team Participation.
Accommodation Statement. The University of Massachusetts Amherst is committed to making reasonable, effective, and appropriate accommodations to meet the needs of students with disabilities and help create a barrier-free campus. If you have a disability and require accommodations, please register with Disability Services, meet with an Access Coordinator in Disability Services, and send your accommodation letter to your faculty. Information on services and registration is available on the Disability Services website (https://www.umass.edu/disability/)
Accessibility in this course. We adjust format, not content. The course runs in a web browser on any laptop. If a workbench tool will not run on your laptop, you will pair with a teammate or work from the instructor’s screen, with no effect on your grade.
Title IX Statement (Responsible Employee). In accordance with Title IX of the Education Amendments of 1972 that prohibits gender-based discrimination in educational settings that receive federal funds, the University of Massachusetts Amherst is committed to providing a safe learning environment for all students, free from all forms of discrimination, including sexual assault, sexual harassment, domestic violence, dating violence, stalking, and retaliation. This includes interactions in person or online through digital platforms and social media. Title IX also protects against discrimination on the basis of pregnancy, childbirth, false pregnancy, miscarriage, abortion, or related conditions, including recovery. There are resources here on campus to support you. A summary of the available Title IX resources (confidential and non-confidential) can be found at the following link: https://www.umass.edu/titleix/resources. You do not need to make a formal report to access them. If you need immediate support, you are not alone. Free and confidential support is available 24 hours a day / 7 days a week / 365 days a year at the SASA Hotline 413-545-0800.
For purposes of Title IX reporting, I am considered a “responsible employee” at UMass (https://www.umass.edu/titleix/about). That means that if you tell me about a situation involving sexual assault, sexual harassment, domestic violence, dating violence, stalking, and retaliation, I must share that information with the Title IX Coordinator. Making a report to the Title IX Coordinator is my legal obligation, meets the University’s goal of providing members of our community with supportive resources they might need, and enables the University to obtain a more accurate picture of the extent of sexual violence in our community. It will be completely up to you to determine if and how you want to work with the Title IX Coordinator’s office. You will not be in trouble for reporting to me that you have experienced any of these situations, and the law prohibits retaliation against anyone who participates in a Title IX process.
Inclusivity and conduct. Students come from every major and every level of technical background. Respectful, patient collaboration in teams, cross-major pairs and seminars is expected; condescension toward less technical peers is a participation issue.
Failure ledger and data use. Your findings join the shared failure ledger only with your consent, and with anything identifying a person’s private data removed. You may opt out with no grading penalty; your audit is graded on its own merits either way.