Welcome to the CAGI experiment!

CAGI 2015 Plans

We have initiated CAGI 2015. We are preparing challenges for a prediction season in late spring & summer, assessment in late summer and fall, and a winter CAGI meeting. CAGI 2015 is enabled by funding from NHGRI.

The CAGI 2013 conference program with presenters' slides is now available.

The Critical Assessment of Genome Interpretation (CAGI, \'kā-jē\) is a community experiment to objectively assess computational methods for predicting the phenotypic impacts of genomic variation. In this experiment, modeled on the Critical Assessment of Structure Prediction (CASP), participants are provided genetic variants and make predictions of resulting molecular, cellular, or organismal phenotype. These predictions are evaluated against experimental characterizations, and independent assessors perform the evaluations. Community workshops are held to disseminate results, assess our collective ability to make accurate and meaningful phenotypic predictions, and better understand progress in the field. From this experiment, we identify bottlenecks in genome interpretation, inform critical areas of future research, and connect researchers from diverse disciplines whose expertise is essential to methods for genome interpretation. We want to emphasize that CAGI is a community experiment to understand and improve the interpretation of genome variation. It is not a contest and all predictors are awarded recognition for their participation in the meeting.

CAGI presentations

A list of past and future presentations about CAGI is available here with downloadable posters and slides.

Latest News

The CAGI 2013 predictors were: Yesim Aydin Son, Benjamin Bachman, Brady Bernard, Marcus Breese, Yana Bromberg, Chen Cao, Emidio Capriotti, Rita Casadio, Chien-Yuan Chen, Shann-Ching Chen, Yun-Ching Chen, Carla Davis, Christopher Douville, Roland Dunbrack, Carlo Ferrari, Adam Frankish, Manuel Giollo, Nina Gonzaludo, Julian Gough, Jennifer Harrow, Tadashi Imanishi, Chan-Seok Jeong, Yuxiang Jiang, Rachel Karchin, Panagiotis Katsonis, Dongsup Kim, Michael Kleyman, Pietro Di Lena, Emanuela Leonardi, Biao Li, Jun Li, Olivier Lichtarge, Chiao-Feng Lin, Rhonald Lua, Angel Mak, Pier Luigi Martelli, Sean Mooney, Zev Medoff, Matthew Mort, John Moult, Steve Mount, Eliseos Mucaki, Jonathan Mudge, Katsuhiko Murakami, Yoko Nagai, Abhishek Niroula, Yanay Ofran, Kymberleigh Pagel, Nathaniel Pearson, Vikas Pejaver, Alexandra Piryatinska, Catherine Plotts, Predrag Radivojac, Aliz Rao, Lipika Ray, Graham Ritchie, Aharon Brodie, Peter Rogan, Jana Marie Schwarz, George Shackelford, Nuttinee Teerakulkittipong, Janita Thusberg, Silvio Tosatto, Ron Unger, Gurkan Ustunkar, Jouni Valiaho, Mauno Vihinen, Mary Wahl, Qiong Wei, Yuedong Yang, Christopher Yates, Yizhou Yin, Chen-Hsin Yu, Dejian Yuan, Maya Zuhl

The CAGI 2013 Conference took place 17 - 18 July 2013 at the Max Planck Institute for Molecular Genetics in Berlin, Germany. This year's The CAGI 2013 results are now posted. The conference program page contains the full set of slides that were presented at the meeting. The abstract book contains abstracts describing the prediction methods. Each challenge page has the slides for talks given for that challenge as well as the challenge's answer key, assessor summary, and predictions.

The CAGI prediction season concluded on 25 April 2013, and the preliminary assessment phase concluded 1 June 2013. There were a total of 188 predictions submitted from 33 groups, a >50% increase in both submissions and groups over the previous CAGI experiment. Predictors hailed from 14 countries representing academics, research institutes, and industry.

The challenges and assessors:
  • Crohn’s disease: 56 submissions. Assessor: Alexander Morgan.
  • PGP: 16 submissions. Assessor: Sean Mooney
  • MRN variants: 22 submissions. Assessor: Sean Tavtigian.
  • P16 variants: 22 submissions. Assessor: Silvio Tosatto.
  • BRCA variants: 14 submissions. Assessor: Robert Nussbaum
  • Splicing: 5 submissions. Assessor: Jeremy Sanford.
  • FCH and HA: 39 submissions. Assessor: Shamil Sunyaev
  • MR-1 fitness: 0 submissions.
  • RiskSNPs: 12 submissions. Assessor: John Moult.

CAGI 2013

The 2013 prediction season ended on 25 April 2013. The meeting to discuss the results will be held in Berlin, 17 - 18 July 2013 (the two days preceding the ISMB SIGs).

CAGI 2011

The second round of CAGI was completed in 2011, with 11 challenges provided to predictors. The experiment yielded a total of 117 predictions from 21 groups from 18 countries, with 55 people attending the December 2011 meeting. Experimental results for the following CAGI 2011 challenges have been published: Mouse exomes challenge: Fairfield et al., Mutation discovery in mice by whole exome sequencing. Genome Biology 2011, 12:R86. doi:10.1186/gb-2011-12-9-r86 Cancer cell lines challenge: Heiser LM et al., Subtype and pathway specific responses to anticancer compounds in breast cancer. doi: 10.1073/pnas.1018854108. CBS challenge: Mayfield JA, et al. Surrogate genetics and metabolic profiling for characterization of human disease alleles. Genetics. 2012, 190(4):1309-23. doi:10.1534/genetics.111.137471. CAGI was presented at ISMB/EECB 2011 in a special session in July, covered in GenomeWeb and in a special session at the Pacific Symposium on Biocomputing in January 2012. The flagship manuscript describing the meeting as well as a collection of papers describing the predictions and challenges is currently underway.

CAGI 2010

The first round of CAGI was organized in the fall 2010. We want to express our appreciation to everyone who participated in CAGI and RiskSNPs 2010. This initial experiment was a great success with over 100 prediction submissions from 8 countries. The workshop on 10 December 2010 gathered an enthusiastic international group of 40 people at Berkeley. A summary of the CAGI 2010 Conference proceedings has been published in Nature News.