SIGIR 2009 Workshop on The Future of IR Evaluation
23 July 2009
|
SIGIR 2009 Workshop on The Future of IR Evaluation23 July 2009 |
| Workshop Homepage | Program | Call for Participation | Organizers |
Evaluation is at the core of information retrieval: virtually all progress owes directly or indirectly to test collections built within the so-called Cranfield paradigm. However, in recent years, IR researchers are routinely pursuing tasks outside the traditional paradigm, by taking a broader view on tasks, users, and context. There is a fast moving evolution in content from traditional static text to diverse forms of dynamic, collaborative, and multilingual information sources. Also industry is embracing "operational" evaluation based on the analysis of endless streams of queries and clicks.
The questions we expect to address can be succinctly summarized as to make IR evaluation more "realistic." There is however no consensus on what then "real" IR is, or should be, and various directions have been proposed:
The workshop brings together all stake-holders, ranging from those with novel evaluation needs, such as a PhD candidate pursuing a new IR-related problem, to senior IR evaluation experts. Desired outcomes are insight into how to make IR evaluation more "realistic," and at least one concrete idea for a retrieval track or task (at CLEF, INEX, NTCIR, TREC) that would not have happened otherwise.
| • | Workshop at SIGIR (23 July 2009) |
| • | Program and proceedings online (14 July 2009) |
| • | Notification, 20 short papers accepted (3 July 2009) |
| • | 33 submissions received (16 June 2009) |
| • | Deadline extended to June 15 (16 May 2009) |
| • | Call for papers published (20 March 2009) |
| • | Website online (10 March 2009) |
| • | Workshop proposal accepted by ACM SIGIR (6 March 2009) |
The workshop is part of SIGIR 2009 in Boston.
| June 15, 2009 | Deadline for Paper Submissions |
|---|---|
| Prepare your 2 page PDF using the ACM format Submit online using EasyChair | |
| July 2, 2009 | Notification of Acceptance |
| Details of accepted papers published online | |
| July 8, 2009 | Deadline for Camera Ready Copies |
| July 23, 2009 | SIGIR 2009 Workshop on the Future of IR Evaluation |
This workshop will be held as part of the 32st Annual International ACM SIGIR Conference on Research & Development on Information Retrieval, Boston, 2009. Information on Boston can be found in the Wikipedia.