CS 222: AI Agents and Simulations

Paper Commentary Guidelines (h/t to CS347 Guidelines)

The goal of paper commentaries is to get you to think critically about the research that a paper presents and why that research is important. You will write brief commentary reflections—around three to four paragraphs—for each reading in the course.

Writing a strong commentary means grappling with the core ideas being presented in the reading. The goal is not to summarize the paper — everyone reading your commentary will have already read the paper!

Suggested Strategies for Writing a Commentary

Don’t:

• Nitpick low-level details
• Harp on already-acknowledged limitations or future work
• Spend too much time summarizing
• Levy judgment (“I like this!”) without digging into the why or the implications of that agreement or disagreement

Do: engage with the core contributions. To achieve this, we suggest the following three-step process:

1. Ask yourself: What is the point that this paper is trying to make? (You don’t need to write out the answer to this question in your commentary, but you do need to know the answer in order to write a good commentary.)
2. How effectively does it convince you of that argument? How could the argument be even more persuasive, on its own terms?
3. What are the implications of the argument? What future frontier projects might be suggested by this, to push the idea farther?

Some appropriate topics to address in a commentary are:

• Future research directions that this paper inspires for you
• Why the paper does/doesn’t seem important
• Observations of novel methodology or methodology that seems suspect
• Why the paper is/isn’t effective at getting its message across
• How the paper has changed your opinion or outlook on a topic

Commentaries are due at 10:00 PM the day before the lecture on Canvas. Late submissions will not be accepted. We will drop the two lowest commentary grades at the end of class: meaning, you may drop two days' worth of commentaries.

Commentary Grading

Commentaries will be graded on a check-minus/check/check-plus scale. The rubric will be:

Check-minus:

Surface-level engagement with the readings, or a repeat of a style of critique that the staff told the class to avoid. Examples of surface-level engagement include: comments about whether the commenter likes or would use the technology, a summary of the paper rather than reflections on the ideas, or critiques that engage only obliquely with the paper or indicate that the commenter didn’t fully read it. Partially complete submissions may also earn a check-minus if appropriate.

Check:

Effective engagement with the readings. Example commentaries involving check grades often indicate that they understand the main ideas of the papers, and the reflections are reasonably nontrivial observations worth discussing.

Check-plus:

Excellent engagement with the readings. Check-plus grades are reserved for rare instances where a commentary really hits on an interesting, unique, and insightful point of view worth sharing. Generally only a few submissions in each session earn a check-plus.