Always thank the reviewers anyway.
“We thank the reviewers for their time and efforts in providing thoughtful feedback on our paper. We look forward to using this feedback to improve the paper and look forward to a future submission. Thank you.” (Phrasing suggested by Eric Paulos)
“A sincere thank you to all the reviewers of this paper; your reviews were very thorough and raised excellent points that will help us improve this paper and communicate more effectively for future submissions.” (Phrasing suggested by Payod Panda)
“We thank the reviewers for their valuable and constructive critique and will use it to develop this work further for a future venue. Thank you!” (Phrasing suggested by Wendy Mackay)
(The following is from Wendy Mackay, Scott Klemmer, Gene Golovchinsky and Bjoern Hartmann)
Wendy: Rebuttals are usually the most emotionally charged part of writing a paper. Remember that, no matter what the reviews say, you need to respond in a polite, clear and factual manner. Your goal is not to argue with the reviewers, but rather to convince the program committee that your paper is worth accepting. This is a tricky writing exercise, no matter how much experience you have.
Focus on the changes you will make. i.e., have the rebuttal be forward-looking. (Rather than offering your opinion of the reviews.) Summarizing revisions that will appear is an easy way to generate positive energy. And demonstrate that you understand the concerns. Disagreeing with reviewers is sometimes essential, but most often should be a minor part of the review, not the centerpiece. And don’t recapitulate your favorite part of the review. The reviewers can read.
Show, don’t tell. Don’t say, “In revision, we will compare our work to Cosby ‘83’s Jello Paper.” Say, “The primary distinction between our work and Cosby ‘83 is the use of carrots. We will explain this and add the reference in revision.” Don’t say, “In revision, we will present statistic Y instead of / in addition to statistic X.” Say, “Statistic Y is (BLANK). We will present this instead of / in addition to statistic X.” In other words, don’t say what you will do. Do it, and summarize the upshot in the rebuttal.
Remember:
We thank the externals and the 1AC for their thorough and detailed expert reviews. All reviewers find novelty, thoughtful writing, and good coverage of implementation and generalizability. We first discuss issues identified by the 1AC, then additional individual concerns.
VALIDITY/EVALUATION:
HOW PAGES ARE ASSOCIATED TO CODE: DOES N=2 WORK?
RELATED WORK
INDIVIDUAL CONCERNS
I originally wanted to post this during the SIGCHI rebuttal period. However, I was afraid that it may hurt the acceptance rate of my paper somehow. (Maybe by revealing my identity to the external reviewers? I don’t know. Anything is possible. No?) Hence, I decided to write this immediately after the notification, while everything is still hot in my head. Maybe students that have to write a rebuttal for CHI2012 may find my post useful.
First, I will start by my opinion on why it is difficult to write a convincing rebuttal.
If 90% of writing a paper requires knowing how to explain a research idea and 10% knowing how to convince another researcher, writing a rebuttal requires 10% of the former and 90% of the latter. As a junior researcher (like myself, a graduate student), the only senior researcher you talk to in a daily basis is your advisor. Yet, your advisor is only one sample point among the pool of senior researchers. How on earth would you know how to convince another senior researcher when you do not even know how to initiate a serious conversation with them?
It is always difficult to take criticism. This is especially the case for people like us Ph.D. students who always strive to produce results that are flawless. We seldom hear that our work is a crap in our face. Additionally, the criticism that we have to take in the review seems unjust especially when you believe that the person who’s making the criticism spent maybe a day or two reading a document that you spent endless nights editing over and over again for several months. Some reviewers are actually nasty too. Although it is written in fancy and erudite terms, sometimes the reason that they are rejecting your paper is simply because, “you did not do a good job in impressing me” or “I do not buy your research story”. How would you possibly stay sane when you read these comments?
Despite, many reasons to yell at your reviewers and say, “you are full of s**t~!!”. We should not even express this in any indirect way while writing our rebuttal. In the past, I always had to discard the very first draft that I produced after reading the harsh reviews. My rebuttal was bitter, smirking, cynical and mean. This was not obvious until I slept at least two days crying over it and returned to my calm and reasonable self.
This year, due to my post (in my Korean blog) about my experience in writing a CHI rebuttal in the past years, I’ve been asked by several junior students (outside my institute) to help out in writing their rebuttal. While doing so, there are several tips that I repeated. Here are some of them.
Read your reviews with another coauthor and have an in-depth discussion. It is important to address the most important issues first and address only the problems that reviewers raise. Sometimes, I realize that I misunderstood what the reviewer meant and was addressing something that was totally unnecessary. Sometimes, I structured my arguments in the wrong order: order of least importance to most importance. Many authors actually make these mistakes during the writing process, not because they are careless, but because the reviews are somewhat encrypted. Not all of them are kind enough to tell us “A is unconvincing and A’ is my opinion. B and C are what I do not understand but authors should only address B because C is not as important”. It comes more like this, “A is weird, B is weird, C is weird”. Usually, meta-reviewer tries help us by decrypting the dialect of the external reviewers so that the authors are not at a loss. However, not all meta-reviewer are nice either. For this reason, I always talk with my advisor or one of my coauthors for 2~3 hours about the reviews before writing anything. This usually helps a lot.
Agree with your reviewers. Last year, one of the rebuttal of the paper that I reviewed basically stated that “R1 (myself) is wrong because so and so, and our paper is awesome”. This rebuttal didn’t acknowledge some of the important problems that I pointed out and tried to challenge my judgement. I was offended and became more strict in finding faults of the paper during the discussion period. This is the last thing that the authors want. Making an enemy among the reviewers. To make an ally, you have to tell them how useful their feedbacks are and you have to sincerely mean it too.
Specify how the camera ready version will be reflected based on the reviewer’s request. Often, there are rebuttals that just say, “I understand R2’s point”. This is only half-baked response. The goal of the rebuttal is to demonstrate how the camera ready version will be changed according to the issues raised. Hence, the response should be more specific and go even one step further. Like this: “the question A is raised by R2 because we only explained B in section C when it is also needed in section D. In the camera ready version we will clarify B in section D”
Do not say that the draft will be improved with a major change. I have seen several authors that say in their rebuttal “After the submission we did A,B, and C which addresses all of R1,R2, R3 questions which will be updated in the final draft”. This approach is very bad. First, you are admitting that current draft has many issues. Second, during the PC meeting, papers are discussed “as is”. If it is concluded in the PC meeting that the paper requires major revision, PC members advise that it should be submitted to a future venue. Better approach is to figure out what reviewers misunderstood. Explain why there was a miscommunication and offer ways for reviewers to solve those misunderstandings. Point to a paragraph or a figure in the paper. If needed, direct them to a reference that is not cited in your paper. This is what a rebuttal is for; to clarify.
Although 5000 characters limit may seem insufficient to explain everything, do not hesitate to allocate some of those characters in white spaces and phrases such as (in response to R1, as mentioned by R2, in our RELATED WORK section, in p8~9). At a glance, they want to see which major raised points have been addressed and which part of the paper they should read again. Sometimes, I use the web browser search tool (namely Ctrl+F) to locate my reviewer id (RX) in the rebuttal and read the accompanying paragraph more carefully to make sure that I didn’t miss anything.
Last but not least, write short and direct sentences. Any sentence that you write to explain in your rebuttal have 50/50 chance of helping your paper and hurting your paper. The longer and indirect a sentence, the higher chance of mis-interpretation. On top of that, reviewers have very short attention span. If sentence become convoluted, they will read what they think the sentence is saying as opposed to what the sentence is actually saying.
The biggest question behind all this is, “Does reviewers actually change their score after reading a rebuttal?”.
And the answer is “YES”~!!. Among the 7 papers that I reviewed this year, I increased the score by 0.5 in one paper because I was happy to learn something that I didn’t know from the rebuttal. Among my 4 papers (2 in previous years and 2 in current year) that have been accepted, 3 paper scores actually increased (by +0.4, +0.1, +0.4) after the rebuttal period.
Although it is painful and tedious process to write a good rebuttal, it is very rewarding once you DO write a good one.
© 2023 / Molly Jane Nicholas / email