Home   About   Preliminary Program   Rules   Deadlines   Register   Download/Submit    Results   Organizers/Contacts   FAQ

Contest Rules:

The organization of this competition, the collection of the data, the evaluation of the results, and the maintenance of this website all require a large effort. We are committed to maintaining this site as a repository of benchmark results on our test data set in the spirit of cooperative scientific progress. In return, everyone who uses this site needs to respect the rules below.

Website Utilization:

  • This site is copyrighted. That means that users of this site may not copy or redistribute content from it without explicit permission from the maintainers of this site, listed at the bottom of this page. This also applies to all images contained on the site.
  • We welcome links to this site. You are free to establish a hypertext link to any web page.
Team Registration:

  • Anonymous registration is not allowed; you must use your real name, affiliation and e-mail address to register.
  • Once you have successfully registered a team, a Data Transfer Agreement will be forwarded to you. Please e-mail the signed document to vladimir.pekar@philips.com
  • The information you provide (except the email address) will only be listed on the site if you choose to publish the results (see below).

Data:

  • The Data Transfer Agreement outlining pledges as regards data confidentiality, ownership, distribution limitations, and exclusive and timely utilization, must be read carefully, adhered to and agreed with by the team, dated and signed by the team contact person, and faxed to the organizers. It will be cosigned by one of the organizers. This agreement applies to all data, including CT images and expert segmentations, and all other derived entities without restriction. As outlined, the data provided shall be used exclusively for the purposes of the Head and Neck Auto-Segmentation Challenge 2010.
  • In addition, when publishing the obtained results (see below), you have to acknowledge the origin of the data, namely the Head & Neck Site Group of the Radiation Medicine Program at the Princess Margaret Hospital, Toronto ON Canada.
  • The formats used for the provided data and to be used for submitted results are detailed in the Download/Submit section.

Meeting Participation:

  • At least one of the members of each team should be present at the workshop and give an oral presentation at the workshop if the team is one of the best.
  • All teams are expected to prepare a poster about their algorithm and present it at the workshop (see Preliminary program in About section).

Submitting Results:

  • Segmentation results have to be submitted in the mask format detailed in the Download/Submit section (binary .raw + ASCII .mhd header).
  • You can submit the results of your auto-segmentation on the training and/or testing data anytime before the result submission deadline. Each submission generates a request for evaluation by the organizers according to the metrics defined in the About section. In order for the organizers to be able to handle all submitted results by all teams in a timely manner, the evaluation of submitted segmentation results can only be requested once a week by each team. Segmentation results will be evaluated in the order the teams have submitted them.
  • The submission is a 2-step process (see below) with 2 separate deadlines: 1) submission of segmentation results; 2) submission of workshop paper.

Publishing Results:

  • The results of your submission will only be published on the website provided they have been evaluated by the organizers and you have also submitted a document (e.g. journal paper) describing your method before the paper submission deadline. If a commercial system is evaluated no method description is necessary in the document, but the system has to be publicly available and the exact name and version number has to be detailed instead. The organizers will check the method description / commercial system availability before publishing your results on the website.
  • The submission is a 2-step process with 2 distinct deadlines separated by about two weeks: 1) submission of segmentation results; 2) submission of workshop paper. From the results submitted, the organizers will (auto-)generate and supply to each team standard-formatted tables and text containing test statistics, and figures, all to be included in the result section of the teams' workshop paper (we encourage the authors to prepare the remaining of the paper in advance). This standardized presentation of results evaluated with common validation software is to facilitate comparisons between methods by readers.
  • When submitting results and providing the supporting document, the team allows the organizers to make the evaluation of their segmentation results publicly available on this site, in the workshop proceedings, and possibly in an international journal paper (with all the participants as co-authors).
  • If further publishing the obtained results on your own, you have to acknowledge the origin of the data, namely the Head & Neck Site Group, Radiation Medicine Program, Princess Margaret Hospital, Toronto ON Canada, and to cite the original workshop paper submitted to this Head and Neck Auto-segmentation Challenge 2010. You must notify the organizers about any publication that is (partly) based on the data, in order for us to maintain a list of publications associated with the data related to this site.
  • We do not claim any ownership or right to the algorithms or submitted documents, or create obstacles for publishing methods that use our data, as long as it respects the signed Data Transfer Agreement. On the contrary, we welcome publications that use the data on this site.
  • Teams must not report results of their segmentation algorithms on only the training data of this challenge. Instead teams must always include results on the testing data as well, i.e. the data without provided expert segmentation.


The content of this website is copyrighted © 2010
Vladimir Pekar, John Kim, Stéphane Allaire, Arish Qazi, David A. Jaffray