|
The organization of this competition, the collection of the data, the evaluation
of the results, and the maintenance of this website all require a large effort.
We are committed to maintaining this site as a repository of benchmark results
on our test data set in the spirit of cooperative scientific progress. In return,
everyone who uses this site needs to respect the rules below.
Website Utilization:
- This site is copyrighted. That means that users of this site may not copy
or redistribute content from it without explicit permission from the
maintainers of this site, listed at the bottom of this page. This also
applies to all images contained on the site.
- We welcome links to this site. You are free to establish a hypertext link
to any web page.
Team Registration:
- Anonymous registration is not allowed; you must use your real name,
affiliation and e-mail address to register.
- Once you have successfully registered
a team, a Data Transfer Agreement will be forwarded to you. Please e-mail
the signed document to
vladimir.pekar@philips.com
- The information you provide (except the email address) will only be listed
on the site if you choose to publish the results (see below).
Data:
- The Data Transfer Agreement outlining pledges as regards data
confidentiality, ownership, distribution limitations,
and exclusive and timely utilization, must be read carefully,
adhered to and agreed with by the team, dated and signed by the team
contact person, and faxed to the
organizers. It will be cosigned by one of the
organizers. This agreement
applies to all data, including CT images and expert segmentations, and all
other derived entities without restriction. As outlined, the data provided
shall be used exclusively for the purposes of the Head and Neck
Auto-Segmentation Challenge 2010.
- In addition, when publishing the obtained results (see below), you have to
acknowledge the origin of the data, namely the Head & Neck Site Group
of the Radiation Medicine Program at the Princess Margaret Hospital,
Toronto ON Canada.
- The formats used for the provided data and to be used for submitted results
are detailed in the Download/Submit
section.
Meeting Participation:
- At least one of the members of each team should be present at the workshop
and give an oral presentation at the workshop if the team is one of the best.
- All teams are expected to prepare a poster about their algorithm and present
it at the workshop (see Preliminary program in
About section).
Submitting Results:
- Segmentation results have to be submitted in the mask format detailed
in the Download/Submit section
(binary .raw + ASCII .mhd header).
- You can submit the results of your auto-segmentation on the training and/or
testing data anytime before the result submission
deadline. Each submission generates a
request for evaluation by the organizers according to the metrics
defined in the About section. In order for
the organizers to be able to handle all submitted results by all teams in a
timely manner, the evaluation of submitted segmentation results can only be
requested once a week by each team. Segmentation results will be evaluated
in the order the teams have submitted them.
- The submission is a 2-step process (see below) with 2 separate
deadlines: 1) submission of segmentation
results; 2) submission of workshop paper.
Publishing Results:
- The results of your submission will only be published on the website
provided they have been evaluated by the organizers and you have also
submitted a document (e.g. journal paper) describing your method before
the paper submission deadline. If a
commercial system is evaluated no method description is necessary in the
document, but the system has to be publicly available and the exact name
and version number has to be detailed instead. The organizers will check
the method description / commercial system availability before publishing
your results on the website.
- The submission is a 2-step process with 2 distinct
deadlines separated by about two weeks: 1) submission of
segmentation results; 2) submission of workshop paper. From the results
submitted, the organizers will (auto-)generate and supply to each team
standard-formatted tables and text containing test statistics, and figures,
all to be included in the result section of the teams' workshop paper (we
encourage the authors to prepare the remaining of the paper in advance).
This standardized presentation of results evaluated with common validation
software is to facilitate comparisons between methods by readers.
- When submitting results and providing the supporting document, the team
allows the organizers to make the evaluation of their segmentation results
publicly available on this site, in the workshop proceedings, and possibly
in an international journal paper (with all the participants as co-authors).
- If further publishing the obtained results on your own, you have to
acknowledge the origin of the data, namely the Head & Neck Site Group,
Radiation Medicine Program, Princess Margaret Hospital, Toronto ON Canada,
and to cite the original workshop paper submitted to this Head and Neck
Auto-segmentation Challenge 2010. You must notify the organizers about any
publication that is (partly) based on the data, in order for us to maintain
a list of publications associated with the data related to this site.
- We do not claim any ownership or right to the algorithms or submitted
documents, or create obstacles for publishing methods that use our data,
as long as it respects the signed Data Transfer Agreement. On the contrary,
we welcome publications that use the data on this site.
- Teams must not report results of their segmentation algorithms on only
the training data of this challenge. Instead teams must always include
results on the testing data as well, i.e. the data without provided expert
segmentation.
The content of this website is copyrighted © 2010
Vladimir Pekar,
John Kim,
Stéphane Allaire,
Arish Qazi,
David A. Jaffray
|