Frequently Asked Questions

On Tasks

Who can participate in SemEval tasks?

Anyone who wants to can participate, either independently or by forming a team.

Is there a fee to participate in a task?

No, participation is free.

Please see instructions on the website for the task (e.g. on CodaLab).

Can I participate in multiple SemEval tasks (separate competitions)?

Yes.

Can I participate in multiple teams under the same SemEval task (same competition)?

This depends on the task. Check with the task organizers to see whether they allow an individual to be part of multiple teams within the task.

What does "evaluation period" mean?

The SemEval-wide evaluation period is the time window in which the official part of each task competition takes place. Task organizers will release evaluation data to participants at the beginning of the evaluation period, and system outputs will be due before the end of the evaluation period.

Individual tasks may set their own deadlines within this period (e.g., if a task has multiple stages, there may be a deadline partway through for first-stage submissions and then a later deadline for second-stage submissions).

How many runs can be submitted for a task?

This depends on the task. Task organizers often specify how many submissions by each team will be evaluated. Contact your task organizers if these details are not already stated on the task webpage.

Where can I find datasets from past SemEval tasks?

Task datasets are distributed by task organizers, not the SemEval organization. Starting with SemEval-2020, task organizers have been asked to archive datasets on Zenodo after the evaluation period concludes: https://zenodo.org/communities/semeval/. For other tasks, check the task website and if that fails, contact the task organizers.

On Papers, Proceedings, and the Workshop

Do I need to write a paper?

All teams competing in a task are encouraged, but not required, to write a paper describing their system(s) and what was learned in the task. Even if a system did not achieve a high score in the competition, the description may be able to offer insights to the scientific community.

For my paper to appear in the proceedings, do I have to present it at the workshop?

No. Unlike many other venues, SemEval invites task participants to submit papers for the proceedings whether or not the paper is presented at the workshop. All papers do go through a peer review process, and if your paper is found to have scientific value, it will be accepted to appear in the proceedings. You may be required to make certain changes before your paper can be accepted.

After the contents of the proceedings are finalized, you will be asked whether you prefer to present the paper at the workshop, or have it appear in the proceedings only. If you change your mind later, be sure to notify SemEval organizers.

Is there a fee to submit a paper/attend the workshop?

There is no fee to submit a paper or have it published in the proceedings.

However, there is a registration fee if you choose to attend the workshop. For a paper to be presented at the workshop, at least one of its authors must be registered. The registration process is handled by the main conference (you can indicate whether you plan to attend just SemEval or other parts of the conference as well). Note that SemEval organizers do not have direct access to the registration data, and should be notified if you change your mind about registering.

How should my paper be structured?

SemEval has special paper requirements for the title, length, and other details. There are also general recommendations for writing system papers.

Where can I find published papers?

SemEval proceedings are published in the ACL Anthology: https://www.aclweb.org/anthology/venues/semeval/

Typically papers are published about a week prior to the start of the conference with which the SemEval workshop is colocated.

What is the policy regarding SemEval papers on arXiv?

Authors are free to submit SemEval papers to arXiv if they wish, at any time. Since SemEval task and system description papers are not anonymous during review, posting an initial draft to arXiv will not interfere with the review process (but if you do this please also upload the camera-ready version to arXiv when it is ready).

My question is not answered here. Who should I contact?

First, if your question is about a specific task, consult the task website. The task may also have a mailing list for discussion.

Questions not answered there should probably be emailed to task organizers. They oversee the competition as well as the review of system papers for the task.

Questions about the workshop event schedule/logistics should be sent to the SemEval organizers at semevalorganizers@gmail.com.