Guide to a Successful HCI Archive Submission
Introduction
Considerable effort can go into preparing an HCI Archive contribution for the CHI conference. No one wants to see this effort go to waste, especially if it involves a potentially valuable contribution that could bring credit to authors and fresh insights to readers. However, CHI is very popular as a place to present work, and this inevitably means that a large proportion of submissions - typically 80 percent - get rejected. For authors such as you, this means taking special care to submit your work in a manner that guarantees the best chance of gaining acceptance. This Guide has been prepared to help you.
Like most high-profile conferences, CHI relies on a panel of reviewers to assist the selection of contributions. Your submission will be subjected to thorough reviewing by three or more experts in the relevant field. The process by which these reviewers are selected, and by which their assessments then influence the final decision, has been reworked and refined over the years in order to ensure that submissions are treated as fairly as possible. Reviewers receive guidance, too, on how to prepare their reviews. Particular care is made to ensure that the instructions issued to them are consistent with the instructions to authors in the Call for Papers and in this Guide.
Since CHI 1999, reviewers have been asked to assess submissions in terms of the contribution made to HCI, and the benefit thus gained by the target audience. Over the years, minor changes have been introduced to this review process to ensure that submissions offering strong contributions are indeed more likely to be accepted than those that do not. The review process has been fine-tuned again for 2009.
First, the papers and notes review committees have been divided into subcommittees that each focus on a subset of topics in human-computer interaction (topics are identified by keywords, a short description, and a subcommittee name). The reason for this is to better match reviewers with your paper. Each sub-committee consists of a chair and associate chairs who, as a group, are knowledgeable about the topic and who should be able to match good reviewers for your paper. As an author, you decide which subcommittee reviews your paper. Consequently, it is important that you choose the subcommittee that best matches your research. See the selecting a subcommittee page for additional details.
Second, your submission form will have asked you to select from a list of contribution types that guides how your submission should be evaluated by reviewers and by the program committee. By officially recognizing different contribution types, CHI encourages a variety of submissions, and guides referees to review your submission using criteria appropriate to the type of contribution. The CHI program committee strongly encourages reviewers to review your paper according to its contribution type. Each contribution type will have its own review criteria, but the primary criteria for significance of all submissions remains the overall contribution to HCI. See the selecting a contribution type page for additional details.
The Review Form
We start this guide with a brief overview of the on-line form used by reviewers to submit their reviews of your contribution. You may have been asked to review submissions for CHI, in which case you will need to become very familiar with this form! That's not why we describe it here, however. Rather, we suggest that familiarity with the questions on the review form will help you to decide what to include or emphasize in your submission.
Because of the inclusion of contribution types, the review form has been changed from previous years. It now includes a summary of each contribution type, along with possible review criteria specific to that contribution type (see the selecting a contribution type page for additional details). Reviewers will be asked to provide a contribution type specific rating for your paper:
- Based on the specific contribution criteria, I believe this paper provides a strong contribution to the field of HCI
The other questions on the review form, identical to previous years, ask the reviewer for the following:
- A rating of the reviewer's own expertise in the topic area of the submission, from 4 (expert) to 1 (no knowledge); this helps members of the Technical Programs Committee to resolve conflicting views on submissions.
- Statement of the submission's contribution to the field of HCI; this provides the Technical Programs Committee with a basis for assessing the significance of the contribution, and for judging whether all the reviewers agree on what the contribution is. We suggest reviewers use the contribution type as a starting point for appraising the contribution, but also state that the paper does not need to exactly match the contribution type criteria.
- A review of the submission in terms of the criteria laid out in the Call for Papers; more on this below.
- Any aspects of the submission's written presentation that need improvement.
- A rating of the submission's acceptability, from 5 (high) down to 1 (low). Submissions whose contribution is judged significant are rated from 4 to 5, depending on whether or not they contain any non-serious flaws.
Reviewers can also add further comments that they want you or the Technical Programs Committee to see. But for you, the main concerns to keep in mind are those numbered 2, 3, and 4: you should offer the reviewers a strong, well-presented contribution to HCI that meets the criteria in the Call. If you do this, your submission should get a high rating.
For further details, we recommend you look at the Guide to Reviewing CHI Papers and Notes.
Making a Significant Contribution
Your submission's reviewers will be asked to focus on the significance of the submission's contribution, the benefit others can gain from its results, the validity of the work, and its originality. Of these, the contribution and its significance are the most important. CHI Papers and Notes are expected to offer contributions that clearly and significantly advance the field of HCI. Your contribution should offer CHI attendees, and readers of the proceedings, something that adds to what they could have learned from existing publications.
HCI is a very broad field, and it opens up a correspondingly broad range of possible contributions.
The contribution you present may be one of the contribution types listed in the Papers and Notes Call for Participation. This contribution should be made clear in the abstract as well as the submission itself, outlining its direct significance to the field of HCI. Your Paper or Note's contribution may be a design briefing, a new design for an interactive technology or system, a new method or tool, an analysis of an HCI issue, results from fieldwork and ethnography, results from laboratory studies, or a theory or model. This list is not exhaustive, but it encompasses the vast majority of past CHI research contributions.
CHI asks for just one such contribution. You may be tempted to offer more than one, for example, a new design and a new method by which you evaluated it. You are advised to resist this temptation, and focus on presenting one strong contribution really well. If you offer several contributions of different types, reviewers may be confused as to which is the submission's primary focus. Furthermore, you will probably find it difficult to do justice to more than one contribution in ten pages. Should you find you have several strong contributions to offer - and you have time to spare - consider submitting them as separate contributions.
Offering Benefit to the Reader
If you are in a position to make a contribution to HCI, there will be people who stand to benefit from it. For example, if you are presenting a new interaction technique for small screens, it will be of potential use to people involved in developing handheld computers. A case study describing how you developed a new interactive product will probably catch the eye of HCI educators looking for teaching materials. The benefit thus to be gained from your contribution will be one consideration that reviewers take into account.
As you write your submission, therefore, keep in mind the kinds of people you think might benefit from reading it. Think also about how this might happen - what kinds of problems might readers be facing to which your contribution could provide the solution. Try to make sure that the submission explains the contribution in sufficient detail for the full benefit to be extracted.
If your paper is accepted, you will need to provide with your submission: a 30-word statement of contribution and benefit. This statement is not part of your submission, but is included in the final program to help the audience decide what papers to attend. We suggest it may be useful to draft this statement before you begin writing your submission, to help keep the contribution and benefit in sharp focus.
Ensuring Results are Valid
The validity of your submission's contribution must be adequately supported by appropriate arguments, analyses, evaluations, or data as best fit the contribution type. Otherwise readers will find it hard to judge whether they can confidently take up your ideas, and thus gain the benefits you are claiming to offer. Reviewers are therefore asked to assess the validity of the results you are presenting.
Demonstrating validity is one of the most troublesome aspects of writing CHI Papers and Notes. Reviewers often cite problems with validity, rather than with the contribution per se, as the reason to reject a submission. For this reason it is risky to leave validity issues (for example, evaluating a design) until the last minute. Instead you should consider, when planning the work and certainly before embarking on the submission, how you will demonstrate your contribution's validity.
Your choice of how to demonstrate validity will depend on what kind of contribution you're offering, and suggestions for how to do this are listed in the contribution type page. No matter what method you chose to show validity, you should let the reader/reviewer know why that method is appropriate for your situation.
A guiding principle here is to consider the benefits that lie in your contribution, and confirm to your own satisfaction and your colleagues', that these benefits are really there. Again, the steps you take depend on the contribution type. For example, if you are providing an incremental improvement to a well known technique, such as a new or refined type of menu capable of reducing errors in selecting menu items, you will probably run a careful and convincing experiment to measure error rates with this and other types of menu. If you have come up with a system to support a new way of collaborative writing, you may try to evaluate it in real-world conditions, offering it to a group of co-authors for use in a joint writing task, and conducting studies to determine how the system helps them - and what problems it introduces. If you have developed a new design or evaluation methodology, you may compare with other methodologies, or you may want to report on your experiences in using it. If you are developing a theory that contributes principles, concepts, or models on which work in HCI might be based, you will probably what to argue or demonstrate the value of that theory, e.g., by using it to describe, analyze, predict and/or influence HCI applications or settings. If you are contributing systems, tools, architectures or infrastructure, you will want to demonstrate how it makes systems authoring, development and/or use better, more efficient, and/or more expressive. Any evaluations like these, conducted to convince yourself that you've got something of benefit to the HCI community, can appropriately be adopted and extended to convince your reviewers and readers.
Bear in mind that reviewers of HCI Archive submissions often mention issues of an obvious or important nature that have not been addressed by the authors. They often criticize authors for conducting studies without adequate theoretical basis, or for not providing enough evidence or sound reasoning for claims. A further concern is lack of justification for design choices and not explaining why certain design features have been included. In summary, you should explain not only what you did, but also why you did it, so that readers (including reviewers) can be convinced that you made appropriate choices. Explaining your choices can also stimulate more research by helping others see alternative approaches.
Finally, remember that where you have collected data for analysis, appropriate methods should have been selected and correctly applied to support the work. You should provide sufficient data or well-supported arguments, explain what analyses were made and why, cite relevant work, and cover the important issues at the appropriate level of detail.
Gaining Credit for Originality
Originality in your submission will help it get accepted in two ways. First, it is not just helpful but essential that the submission's contribution be original, going beyond any work already reported in other journals or conference proceedings. Second, reviewers will often give credit for original approaches adopted in conducting the work, particularly if these contributed strongly to the work's success.
To demonstrate the originality of your contribution you should make sure to cite prior work (including your own) in the relevant area. If possible, explain the limitations in this work that your contribution has overcome. Make sure also to cite publications that have had a major influence on your own work. Lack of references to prior work is a frequent cause for complaint - and low rating - by reviewers. Note that reviewers are being asked to set the context for their review by identifying relevant past work; you can help them do this. You can also make it easier for them to check your references by concentrating on papers in easy-to-find publications. Allow adequate time for this part of your submission's preparation.
As regards originality in conducting your work, remember that acceptance of your submission doesn't depend on this. If the submission's contribution is a strong one, it should gain acceptance however you arrived at it. However, reviewers do appreciate novelty and elegance in conducting the work, particularly if they can see how it simplified the work, or could help others conduct work of a similar kind, or both. Thus they will probably give credit for an original way of collecting data during a study, or of choosing a means of evaluating a design, or of overcoming a weakness found in a new design. A few examples of such originality in your work will probably strengthen your submission; however, a plethora of them could drown out the central contribution - so go easy!
Describing the Work Clearly and Concisely
You might be surprised at the number of reviewer complaints about written presentation. Describing your work involves not only writing good prose, but also providing a good structure that helps the reader follow the explanation. The text should be supported with figures, tables and even videos where appropriate; these should be clear and easy to understand. Papers and Notes may use color figures, but they should be usable when printed in black and white in the paper proceedings. In summary, try to write clearly and concisely, avoid jargon, organize the submission to flow logically and smoothly, provide the right level of detail, and make good use of figures to support the text.
Although all presentations at CHI are made in English, CHI is a conference with an international audience - and an international panel of reviewers. Submissions must be written in a language that effectively communicates across national and cultural boundaries. When authors are not native speakers of English, reviewers try to assess the quality of the work independent of language issues, but good English always helps. If you are not a native English speaker but have access to those who are, it is a good idea to ask them to proof-read your submission before you submit it.
Even if your first language is English, keep in mind that non-native English speakers will be reading and reviewing the submission. Avoid long, complex sentences as well as regional colloquialisms, jokes, or puns that could be difficult for someone outside your culture to understand.
A couple of final and important points: first, CHI Papers and Notes are reviewed mostly on an 'as-is' basis. While your paper may be accepted conditionally upon making changes, these changes tend to be minor vs. major suggestions. This is unavoidable given the tight schedule of the reviewing process: there is no time for a second review after the author has made changes, so reviewers must make a decision whether the submission in its current form is acceptable for CHI. Note, however, that reviewers often do make suggestions for further improvement, and authors of accepted contributions are encouraged to make minor revisions to their work before a final draft is required. In some cases, the person handling your paper will reread it to make sure the changes have been incorporated.
Also, when writing your submission you should resist the temptation to describe future work, or work expected to be completed before the final submission or conference. Although these planned activities are often interesting, you cannot rely on them to get your submission accepted. On the contrary, they may be seen by reviewers as evidence that the submission is premature, and you may be advised to resubmit when more of the work has been completed.
Summary
With its large number of submissions, CHI's review process is bound to be highly competitive. The intent of the review process is to provide the conference with a program of submissions offering significant contributions of high potential benefit to attendees and readers. Writing such a submission for CHI is a lot of work, but it is rewarded with the visibility and influence that only high-profile publications like the CHI Proceedings can offer. We hope this document has helped give you some clear and concrete guidance on how to write a successful CHI submission. If not, feel free to send feedback and questions to papers@chi2010.org
Best wishes, and we look forward to seeing a good collection of successful submissions this year.
Written by William Newman, with contributions from Kate Ehrlich, Saul Greenberg, Scott Hudson, Robin Jeffries, Jock Mackinlay, Fred Sampson and John Tang. Originally from the CHI 2006 website.
