CAGI Newsletter

warning: Creating default object from empty value in /data/cagi-live/drupal/modules/taxonomy/taxonomy.pages.inc on line 33.

CAGI Gathering at ASHG

We will be holding an informal CAGI gathering at ASHG in Vancouver on either Wednesday, 19 October from 6pm-7pm or Thursday, 20 October from 3:30pm-4:30pm.

Please fill in the Doodle pool:

http://doodle.com/poll/7ygfgve6bnqsreba

to help us select a suitable time. The Wednesday time partially overlaps with an Award presentation and the Thursday time with the poster session. We will confirm the location at a closer time.

Please write us at cagi@genomeinterpretation.org to meet with us if you cannot make these times.

There will also be a CAGI poster presented at the ASHG conference in Vancouver. Come and meet Gaia on Wednesday, 3:00-4:00 pm, at poster 1822 which has material in progress for the flagship paper.

We are currently looking for new data sets and new data providers for CAGI 5, and people we could meet in Vancouver. If you have ideas for contacts or recommendations for CAGI challenge datasets, we would much appreciate hearing about them. We have a few leads on challenges, including those that provide continuity from the previous experiments.

* CAGI special issue

We would like to thank all assessors and predictors who submitted their manuscripts. Of these,10 papers are currently under review. We expect to have a total of 23 papers for this special issue.

* CAGI flaghship

We continue to develop the flagship manuscript and welcome further input expanding the 15 themes that will be described in the manuscript, list below.

CAGI themes:

- Top missense prediction methods are highly statistically significant, but individual variant accuracy is limited

- Missense methods tend to correlate better with each other than with experiments

- Structure-based missense methods excel in a few cases, while evolution-based methods have more consistent performance

- Phenotype assignment based on rare variant analysis sometimes succeeds for a range of challenges

- Predicting complex traits from exomes is fraught

- Predictors were able to identify causal variants that were overlooked by a clinical laboratory

- Bespoke approaches often enhance performance

- Creating a good challenge that provides a reliable gold standard is difficult

- PGP ‘improvements’ (overall years)

- The clinical use of multiple uncalibrated missense methods is not advised

- Some challenges are just too hard for the current state of the art

- Interpretation of non-coding variants shows promise but is not at the level of missense

- The CAGI community has potential for addressing unanswered questions in genome interpretation

- CAGI challenges have contributing to interpretation of cancer variants

- There might be potential for missense interpretation at the extreme of the distribution

We hope to see you in Vancouver!

Best wishes,
Gaia, John, Steven

CAGI manuscripts update; welcome Gaia Andreoletti

Dear CAGI participant,

In this newsletter we have updates on the assessor and predictor manuscripts for the CAGI collection and special issues of Human Mutation and the Annals of Human Genetics. We particularly encourage predictors to submit manuscripts--see details below.

We are pleased to welcome and introduce Gaia Andreoletti, who has joined us to lead future CAGI experiments. We also have updates on the flagship manuscript.

* CAGI paper collection
We strongly encourage all assessors and predictors to submit manuscripts to the CAGI collection in Human Mutation and Annals of Human Genetics. We are extending the deadline for manuscript submissions to 25 August 2016. If there are special circumstances that preclude submitting your manuscript by the deadline, please let us know and we’ll try to accommodate. Presently we have 17 manuscripts confirmed, listed below along with submission details below, as well as plans for several more.

* Welcome Gaia Andreoletti
We are delighted to introduce Gaia Andreoletti, who has joined us to lead the CAGI experiment as a postdoctoral fellow. Gaia has recently completed a Ph.D. in genomic medicine at the University of Southampton. As you may know, Roger is unable to continue as lead scientist and will leave us soon. Many of you met Gaia at the CAGI 4 conference. She and Roger are overlapping this week, so she will be able to hit the ground running. We look forward to initiating the CAGI 5 experiment with her. She can be reached as gandreoletti@berkeley.edu.

* Flagship manuscript and authorship
We continue to develop the flagship manuscript and welcome further input expanding the themes previously circulated. Based on feedback, we have revised the authorship plan: We will submit the manuscript with all interested CAGI participants as named authors; we estimate this will be ~270 authors. If the journal refuses, then we will propose a single banner author, such as “CAGI Participants,” with all authors (including organizers) named at the back of the paper and searchable in PubMed. (This plan could still change pending feedback from more authors and Journal policy.)

* CAGI papers collection logistics and details
There will be no publication charges for manuscripts accepted for the special issues of the two journals. There will be no page charges and no charges for color figures. All papers in the special issues will be free online. If you would like full open access to your paper (called OnlineOpen at Wiley), the fee for Human Mutation is $3,500. Wiley has also confirmed that preprints may be submitted to bioRxiv.

The editors have requested that submitting authors include “CAGI special issue” near the beginning of their cover letter. They also request that submitting authors recommend 5 or 6 potential referees, as these are difficult to recruit in the summer months.

Previously we did not correctly describe the procedure for resubmission of returned manuscripts. If a manuscript is deemed not to meet the criteria for acceptance at Human Mutation, the corresponding author will receive a rejection letter and the reviews. If the authors would like to submit the manuscript to the Annals of Human Genetics, then the reviews from Human Mutation may be included with the cover letter in the submission, and this may expedite a decision by the editors at the Annals.

Please let us know if you are planning to submit a manuscript and to which journal. We are tracking the manuscripts in preparation, so that the journal editors can prepare. Our current confirmed list includes 8 manuscripts on assessment of challenge results and 9 manuscripts from predictors:

Assessor manuscripts:
p16 challenge. Silvio C.E. Tosatto, et al.
NAGLU challenge. Wyatt T. Clark, et al.
SUMO ligase challenge. Nick Grishin, et al.
CBS challenges, Iddo Fridberg et al (tentative)
Predicting gene expression from DNA sequence in a massively parallel reporter assay: a comparative study, Anat Kreimier, Nir Yosef, et al.
Predicting disease and drug phenotypes from exome data, Roxana Daneshjou, Alexander A. Morgan, et al.
PGP challenges. Sean D. Mooney, et al.
Hopkins clinical panel challenge. John-Marc Chandonia, Shamil Sunyaev, et al.
SickKids clinical genomes challenge, M. Stephen Meyn, et al.

Predictor manuscripts:
Improving prediction performance: lessons learned from CAGI challenges. Abhishek Niroula and Mauno Vihinen.
eQTL-causal SNPs. Michael Beer
MutPred performance in nonsynonymous variant challenges. Predrag Radivoja, et al.
Evolutionary Action performance in nonsynonymous variant challenges. Olivier Lichtarge, et al.
Cautions in bipolar phenotype prediction using exome sequencing SNPs. Maggie Haitian Wang, Billy Chang, Rui Sun, Inchi Hu, Benny Zee.
Crohn’s exome challenges, Lipika R. Pal, John Moult, et al.
SickKids clinical genomes challenge, Lipika R. Pal, John Moult, et al.
Hopkins clinical panel challenge. Kunal Kundu, John Moult, et al.
NAGLU and SUMO ligase challenges, Yizhou Yin, John Moult, et al.
Bipolar exomes challenge. Laksshman Sundaram, Rajendra Rana Bhat, and Xiaolin Li.

With all best regards,

Gaia, Roger, John, and Steven

CAGI manuscripts update; welcome Gaia Andreolett

Dear CAGI participant,

In this newsletter we have updates on the assessor and predictor manuscripts for the CAGI collection and special issues of Human Mutation and the Annals of Human Genetics. We particularly encourage predictors to submit manuscripts--see details below.

We are pleased to welcome and introduce Gaia Andreoletti, who has joined us to lead future CAGI experiments. We also have updates on the flagship manuscript.

* CAGI paper collection
We strongly encourage all assessors and predictors to submit manuscripts to the CAGI collection in Human Mutation and Annals of Human Genetics. We are extending the deadline for manuscript submissions to 25 August 2016. If there are special circumstances that preclude submitting your manuscript by the deadline, please let us know and we’ll try to accommodate. Presently we have 17 manuscripts confirmed, listed below along with submission details below, as well as plans for several more.

* Welcome Gaia Andreoletti
We are delighted to introduce Gaia Andreoletti, who has joined us to lead the CAGI experiment as a postdoctoral fellow. Gaia has recently completed a Ph.D. in genomic medicine at the University of Southampton. As you may know, Roger is unable to continue as lead scientist and will leave us soon. Many of you met Gaia at the CAGI 4 conference. She and Roger are overlapping this week, so she will be able to hit the ground running. We look forward to initiating the CAGI 5 experiment with her. She can be reached as .

* Flagship manuscript and authorship
We continue to develop the flagship manuscript and welcome further input expanding the themes previously circulated. Based on feedback, we have revised the authorship plan: We will submit the manuscript with all interested CAGI participants as named authors; we estimate this will be ~270 authors. If the journal refuses, then we will propose a single banner author, such as “CAGI Participants,” with all authors (including organizers) named at the back of the paper and searchable in PubMed. (This plan could still change pending feedback from more authors and Journal policy.)

* CAGI papers collection logistics and details
There will be no publication charges for manuscripts accepted for the special issues of the two journals. There will be no page charges and no charges for color figures. All papers in the special issues will be free online. If you would like full open access to your paper (called OnlineOpen at Wiley), the fee for Human Mutation is $3,500. Wiley has also confirmed that preprints may be submitted to bioRxiv.

The editors have requested that submitting authors include “CAGI special issue” near the beginning of their cover letter. They also request that submitting authors recommend 5 or 6 potential referees, as these are difficult to recruit in the summer months.

Previously we did not correctly describe the procedure for resubmission of returned manuscripts. If a manuscript is deemed not to meet the criteria for acceptance at Human Mutation, the corresponding author will receive a rejection letter and the reviews. If the authors would like to submit the manuscript to the Annals of Human Genetics, then the reviews from Human Mutation may be included with the cover letter in the submission, and this may expedite a decision by the editors at the Annals.

Please let us know if you are planning to submit a manuscript and to which journal. We are tracking the manuscripts in preparation, so that the journal editors can prepare. Our current confirmed list includes 8 manuscripts on assessment of challenge results and 9 manuscripts from predictors:

Assessor manuscripts:
p16 challenge. Silvio C.E. Tosatto, et al.
NAGLU challenge. Wyatt T. Clark, et al.
SUMO ligase challenge. Nick Grishin, et al.
CBS challenges, Iddo Fridberg et al (tentative)
Predicting gene expression from DNA sequence in a massively parallel reporter assay: a comparative study, Anat Kreimier, Nir Yosef, et al.
Predicting disease and drug phenotypes from exome data, Roxana Daneshjou, Alexander A. Morgan, et al.
PGP challenges. Sean D. Mooney, et al.
Hopkins clinical panel challenge. John-Marc Chandonia, Shamil Sunyaev, et al.
SickKids clinical genomes challenge, M. Stephen Meyn, et al.

Predictor manuscripts:
Improving prediction performance: lessons learned from CAGI challenges. Abhishek Niroula and Mauno Vihinen.
eQTL-causal SNPs. Michael Beer
MutPred performance in nonsynonymous variant challenges. Predrag Radivoja, et al.
Evolutionary Action performance in nonsynonymous variant challenges. Olivier Lichtarge, et al.
Cautions in bipolar phenotype prediction using exome sequencing SNPs. Maggie Haitian Wang, Billy Chang, Rui Sun, Inchi Hu, Benny Zee.
Crohn’s exome challenges, Lipika R. Pal, John Moult, et al.
SickKids clinical genomes challenge, Lipika R. Pal, John Moult, et al.
Hopkins clinical panel challenge. Kunal Kundu, John Moult, et al.
NAGLU and SUMO ligase challenges, Yizhou Yin, John Moult, et al.
Bipolar exomes challenge. Laksshman Sundaram, Rajendra Rana Bhat, and Xiaolin Li.

With all best regards,

Gaia, Roger, John, and Steven

CAGI publication plans; conference slides and video now available

Dear CAGI community,

The CAGI 4 conference was held 25-27 March 2016 in San Francisco. Presentations by data providers, assessors, and predictors, and discussion sessions, are now available as slides and video to registered users:
https://genomeinterpretation.org/content/4-Conference-program
You are encouraged to use the remixable slides in your own presentation per the items provided.

Flagship manuscript

We are now preparing a CAGI "flagship" manuscript. We propose that the >250 participants in the four CAGI experiments be recognized by banner authorship as "CAGI Participants" and searchable in PubMed as "investigator name". The manuscript will address all of CAGI for 2010-2016. Rather than presenting details of every challenge, we are organizing the manuscript around key theme lessons learned. Some themes include:
* Top missense prediction methods are highly statistically significant, but individual variant accuracy is limited.
* Missense methods tend to correlate better with each other than with experiments.
* Structure-based missense methods excel in a few cases.
* Predictors were able to identify causal variants that were overlooked by the clinical laboratory.
* Predicting complex traits from exomes is fraught.
* Bespoke approaches often enhance performance.
* Clinical implications.
* Creating a good challenge that provides a reliable gold standard is difficult.
Feedback and suggestions for additional themes are welcome.

Assessor and predictor manuscripts

We are also organizing a series of manuscripts for special issues of Human Mutation (HuMu) and the Annals of Human Genetics (Annals). All CAGI assessors and predictors are encouraged to submit manuscripts on their approaches and results in the four CAGI experiments held to date. HuMu has a longer format and may be appropriate for most assessment manuscripts and prediction reports that encompass multiple challenges or provide substantial insights; Annals may be appropriate for shorter reports such as predictions with an established method.

Manuscripts are to be submitted directly to the journals through the normal process. In your cover letter, please reference the "CAGI special issue." The submission deadline is 30 July 2016. Manuscripts that are not accepted for publication in HuMu can be considered for publication in Annals without resubmission or re-review. Papers accepted for the special issues will appear online without delay in advance of the publication in print, which is planned for Spring 2017.

If you are interested in submitting a manuscript for one of the special issues, please contact us now for further information.

With all best regards,

Roger, John, and Steven

Register now for the CAGI 4 conference, deadline extended; Lead Scientist job open

Dear CAGI Participant,

The CAGI 4 Conference will held 25-27 March 2016 at UCSF Mission Bay in San Francisco, California. All CAGI 4 participants are encouraged to attend and should register now. The standard conference fee is available through Wednesday, 9 March 2016. This is a 5-day extension of the original deadline. After that deadline, a late registration fee will apply.

Fellowships are available, and in previous years we were able to provide support to most eligible applicants. To be eligible for fellowship support you must be a student or postdoc and a US citizen or permanent resident. We may also be able to support some members of under-represented groups, including those with disabilities, regardless of citizenship. All eligible participants are strongly encouraged to apply for a fellowship and are likely to receive support.

Observers who did not participate in CAGI 4 are also welcome, but their attendance is limited by the capacity of the meeting space.

The conference will include presentations on all 11 challenges, with talks from the dataset providers, assessors, and predictors with particularly interesting methods and results.

Register now for the conference:
https://genomeinterpretation.org/content/registration-cagi-4-conference
Submit your abstract:
https://genomeinterpretation.org/content/cagi-4-conference-abstract
Apply for a fellowship:
https://genomeinterpretation.org/content/cagi-4-conference-fellowship-ap...

We will not be having a room block, and in addition to traditional hotels, we advise considering AirBNB for accommodation. Also consider accommodation options in Oakland and the East Bay, which are connected to the conference site by public transportation (look for hotels near BART).

If you know of an organization that may wish to support the CAGI conference, please let us know!

* * *

We are currently seeking a successor Lead Scientist or Postdoctoral researcher for CAGI. If you know of any outstanding candidates, please refer them to us and encourage them to apply. The initial review of applications is now underway, and we hope to move quickly to allow substantial overlap of this appointment with Roger Hoskins.
http://compbio.berkeley.edu/jobs and https://genomeinterpretation.org
Apply at
https://aprecruit.berkeley.edu/apply/JPF00954 (Scientist)
or by email to "jobs@compbio.berkeley.edu" (Postdoc)

We remind you to check the website and follow us @CAGInews for more updates.

We look forward to your contribution to an exciting conference.

Roger, John, and Steven

Register now for the CAGI 4 conference; Lead Scientist job open

Dear CAGI Participant,

Registration is now open for the CAGI 4 Conference to be held 25-27 March 2016 at UCSF Mission Bay in San Francisco, California. All CAGI 4 participants are encouraged to attend.

Fellowships are available, and in previous years we were able to provide support to most eligible applicants. To be eligible for fellowship support you must be a student or postdoc and a US citizen or permanent resident. We may also be able to support some members of under-represented groups, including those with disabilities, regardless of citizenship. All eligible participants are strongly encouraged to apply for a fellowship and are likely to receive support.

Observers who did not participate in CAGI 4 are also welcome, but their attendance is limited by the capacity of the meeting space. A late fee will apply to registrations received after 4 March 2016.

Register now for the conference:
https://genomeinterpretation.org
Submit your abstract:
https://genomeinterpretation.org/node/add/cagi-4-abstract
Apply for a fellowship:
https://genomeinterpretation.org/node/add/cagi-4-fellowship
See the Conference page for further details and updates:
https://genomeinterpretation.org/content/4-conference

We will post logistics and other information soon. We will not be having a room block, and we advise exploring AirBNB for accommodation, in addition to traditional hotels. Also consider accommodation options in Oakland and the East Bay, which are connected to the conference site by public transportation.

If you belong to an organization that would like to support the CAGI conference, please let us know!

* * *

We are currently seeking a successor Lead Scientist or Postdoctoral researcher for CAGI. If you know of any outstanding candidates, please refer them to us and encourage them to apply. The initial review of applications is now underway, and we hope to move quickly to allow substantial overlap of this appointment with Roger Hoskins.
http://compbio.berkeley.edu/jobs and https://genomeinterpretation.org
Apply at
https://aprecruit.berkeley.edu/apply/JPF00954 (Scientist)
or by email to "jobs@compbio.berkeley.edu" (Postdoc)

We remind you to check the website and follow us @CAGInews for more updates.

We look forward to your contribution to an exciting conference.

Roger, John, and Steven

CAGI conference confirmed: 25-27 March 2016; Lead Scientist job open

Dear CAGI Participant,

* The CAGI 4 conference will be held on 25-27 March 2016 at Genentech Hall on the UCSF Mission Bay campus in San Francisco, California. Although this overlaps Good Friday and Easter Sunday, and also affects Purim, our survey indicates that most likely attendees prefer to hold the conference on these dates. The week leading up to the conference is Spring Break at many universities in the US, which is convenient for those with teaching responsibilities. We sincerely apologize to those of you who are unable to attend due to this conflict.

We will post registration and logistics information soon. We will not be having a room block, and we advise exploring AirBNB for accommodation in additional traditional hotels.

* We remind you that three CAGI 4 challenges are still open:
Bipolar exomes (closes 25 January 2016):
Distinguish between exomes of bipolar disorder patients and unaffected individuals
https://genomeinterpretation.org/content/4-bipolar-exomes

Hopkins clinical panel (closes 18 January 2016):
Predict patients’ clinical descriptions and pathogenic variants from gene panel sequences
https://genomeinterpretation.org/content/4-Hopkins_clinical_panel

SickKids clinical genomes (closes 1 February 2016):
Predict patients’ clinical descriptions and pathogenic variants from their genome sequences
https://genomeinterpretation.org/content/4-SickKids_clinical_genomes

* We are seeking a successor Lead Scientist or Postdoctoral researcher for CAGI. If you know of any outstanding candidates, please refer them to us and encourage them to apply. The initial review of applications is now underway, and we hope to move quickly to allow substantial overlap of this appointment with Roger Hoskins.
http://compbio.berkeley.edu/jobs and https://genomeinterpretation.org
Apply at
https://aprecruit.berkeley.edu/apply/JPF00954 (Scientist)
or by email to "jobs@compbio.berkeley.edu" (Postdoc)

We remind you to check the website and follow us @CAGInews for more updates.

Finally, our colleagues at CAMDA have just released three challenges. Please see details, below.

Best wishes for 2016.

Roger, John, and Steven

Call for Contributions: Critical Assessment of Massive Data Analysis
| www.camda.info | 8-9 July Orlando, Flordia ---

CAMDA was founded in 2000 (Nature 411, 885. Nature 424, 610) to provide a forum for the critical assessment of different techniques used in large-scale data analysis in the life-sciences, including but not limited to high-dimensional -omics data (see www.camda.info for general background). It aims to establish the state-of-the-art in analysis methods, as well as identify progress and highlight promising directions for future efforts. To this end, CAMDA adopted the approach of a community-wide contest, with the scientific community analysing the same data sets. Besides traditional classification problems, there is a regular focus on

measurement accuracy / data processing
complex integrative open ended biological inference

Researchers worldwide are invited to take the CAMDA challenge, which has become a prominent fixture (cf. Nature Methods 5, 569). Accepted contributions are presented though talks and posters, and the results and methods of the different analyses are discussed and compared at the conference, with selected presentations published in a special Open Access proceedings issue in collaboration with the F1000 Research Journal, which are fully pubmed indexed.

This year's contest offers three challenges:

The Oxford Nanopore ‘wiggle space’ challenge (Mason lab, New York, original unpublished data). Several microbiota samples had their DNA sequenced by Nanopore long read next-next-generation sequencing as well as more established sequencing technology. Additional ‘mystery’ samples provide an independent blind test.
Sequencing Quality Control neuroblastoma study (SEQC, Fischer lab, Köln). A comparison of RNA-seq and Agilent microarray gene expression profiles for clinical endpoint prediction (Zhang et al, Genome Biology 2015) assessed 498 children patients. The published summary data are complemented by raw signal level data sets for sequencing and arrays, and extended clinical meta-data (event-free & overall survival times, multiple prognostic markers, therapy data). In addition, we newly provide: whole genome shotgun (WGS) data of 56 patients for both cancer tissue and normal cells (~30x coverage), and array CGH data of 200 patients (for CNV and SNP analysis).
Sequencing Quality Control toxicogenomics project (SEQC consortium) – an extension building on a contest of CAMDA 2015 for continuity. A comparison of RNA-seq and Affymetrix microarray gene expression profiles (Wang et al, Nature Biotech. 2014) assessed 27 drugs representing 7 biochemical modes, testing over 100 rat livers. Summary & raw data available. New: Rat gene models can now be extended through the RatBodyMap project, which provides RNA-seq profiles of over 300 biologically distinct samples (both sexes, 10 tissues, 4 developmental stages; Yu et al, Nature Comm. 2014).

Running as an official Satellite meeting of ISMB provides further opportunities for interaction with colleagues.

Keynotes by leading researchers in the field provide further focus points for discussion at the meeting, with both experimental and theoretical issues covered, and recent keynote speakers including Mark Gerstein, John Quackenbush, Chris Sander, Eran Segal, Temple Smith and Terry Speed.

We look forward to your contribution to an exciting meeting!

CAGI 4 conference date query

Dear CAGI Participant,

The CAGI 4 conference is scheduled for 25-27 March 2016. It was just brought to our attention that this overlaps Good Friday and Easter Sunday, which we appreciate will be a concern for some participants. Unfortunately, it will be very difficult for us to re-schedule for a different date at this point, and all other dates are also likely to have conflicts.

Please let us know whether or not the conflict with the holiday is a significant concern for you by completing the short survey at:

https://docs.google.com/forms/d/1WzE9AYB8N0aevS4BU8EGy71JGKNq-owoyqqEwLv...

Taking into account your feedback, we will announce a final decision regarding the meeting dates in early January. The most likely alternative would be the weekend before.

Also, as announced in the last newsletter, we are seeking a successor Lead Scientist or Postdoctoral researcher for CAGI. If you know of any outstanding candidates, please refer them to us and encourage them to apply. The first review of applications will be on 7 January 2016, to allow substantial overlap of this appointment with Roger Hoskins.
http://compbio.berkeley.edu/jobs and https://genomeinterpretation.org
Apply at
https://aprecruit.berkeley.edu/apply/JPF00954 (Scientist)
or by email to jobs@compbio.berkeley.edu (Postdoc)

We remind you to check the website and follow us @CAGInews for more updates.

We again wish you the best of the season.

Roger, John, and Steven

CAGI lead job, SickKids genomes, Bipolar exomes, CAGI conference

Dear CAGI Participant,

We are writing to announce the release of a new challenge, extension of an existing one, as well as details of the CAGI 4 conference to be held in March. In addition, we are opening the search for a successor CAGI Lead Scientist or Postdoctoral Researcher.

*New SickKids clinical genomes challenge
The SickKids clinical genomes challenge provides clinical whole genome sequences from a diagnostic laboratory used in medical evaluation of children with suspected genetic disorders who were referred for clinical genome sequencing. Predictors will aim to determine patients’ referring indications and causal variants. Challenge closes 1 February 2016.
https://genomeinterpretation.org/content/4-SickKids_clinical_genomes

*Bipolar exomes challenge deadline extended
Due to the time required to execute the Data Transfer Agreement contracts required for participation in this challenge, the close date of the Bipolar exomes challenge has been extended to 25 January 2016 (previously it was 7 January).
https://genomeinterpretation.org/content/4-bipolar-exomes

*CAGI 4 Conference
The CAGI 4 conference will be 25-27 March 2016 on the UCSF Mission Bay campus in San Francisco, California, starting Friday morning. The CAGI 2011 conference was held at this location with a similar schedule, which worked very well. All CAGI predictors assessors, and data providers are invited to attend. Registration fees and other logistics will be shared later.

* CAGI Lead Scientist or Postdoctoral Researcher position open
Take the lead of the CAGI experiment! We are searching for a CAGI Lead Scientist or Postdoctoral Researcher to join us in early 2016, as Roger Hoskins is unable to continue beyond next summer. Job descriptions posted at:
http://compbio.berkeley.edu/jobs
Applications for the Lead Scientist position should be made at:
https://aprecruit.berkeley.edu/apply/JPF00954
Applications for the Postdoctoral Researcher and other inquiries should be sent to:
jobs@compbio.berkeley.edu
The first review is 7 January 2016. We will also be seeking a developer for CAGI.

We remind you to check the website and follow us @CAGInews for more updates.

We warmly wish you and your families happiness and health in this holiday season and for the upcoming year.

Roger, Steven, and John

New challenges released: Hopkins clinical panel and Pyruvate kinase

Dear CAGI Participant,

* New challenges
This newsletter announces the release of two new CAGI 4 challenges that expand the scope of CAGI.

The Hopkins clinical panel challenge provides real clinical gene panel sequences from a diagnostic laboratory used in medical evaluation of patients. Predictors will aim to determine patientsÕ disease and causal variants.

Hopkins clinical panel (released 18 November 2015):
Predict patientsÕ clinical descriptions and pathogenic variants from gene panel sequences. Challenge closes 18 January 2016.
https://genomeinterpretation.org/content/4-Hopkins_clinical_panel

In addition to enzyme activity, the Pyruvate kinase challenge asks for prediction of the effect of non-synonymous variants on allosteric regulation.

Pyruvate kinase (released 11 November 2015):
Liver pyruvate kinase (L-PYK): predict the effects of missense mutations on kinase activity and allosteric regulation. Challenge closes 11 January 2016.
https://genomeinterpretation.org/content/4-Pyruvate_kinase

* Challenges to be released
Two additional challenges are being developed and will be released soon. Because challenges are currently under development, we cannot guarantee the release of each challenge.

SickKids clinical genomes: Genome sequences from sick children (provided by M. Stephen Meyn, Hospital for Sick Children). Challenge: match patientsÕ genomes to their clinical descriptions and predict causal variants.

Splice acceptors: High-throughput splicing assay (data provided by William Fairbrother, Brown University). Challenge: predict the effect on splicing of naturally occurring intronic variants near 3Õ splice sites.

* Deadlines
Prediction submission deadlines for five challenges are approaching.

NAGLU: Predict the effect of naturally occurring missense mutations on cellular enzymatic activity. Challenge closes 8 December 2015.
https://genomeinterpretation.org/content/4-NAGLU

eQTL causal SNPs: Identify regulatory sequences and eQTL-causal variants, and estimate their effects on activation of transcription in a massively parallel reporter assay. Challenge closes 10 December 2015.
https://genomeinterpretation.org/content/4-eQTL-causal_SNPs

Crohn's exomes: Distinguish between exomes of Crohn's disease patients and healthy individuals. Challenge closes 9 December 2015.
https://genomeinterpretation.org/content/4-crohns-exomes

Warfarin exomes: Estimate patientsÕ therapeutic warfarin doses from their exome sequence. Challenge closes 8 December 2015.
https://genomeinterpretation.org/content/4-Warfarin_exomes

PGP: The Personal genome project: predict individuals' phenotypes. Challenge closes 7 December 2015.
https://genomeinterpretation.org/content/4-PGP

We remind you to check the website and follow us @CAGInews for more updates.

Bipolar exomes; deadline extensions; publication plans

Dear CAGI Participant,

This newsletter announces the release of a new challenge, Bipolar exomes; the extension of the prediction submission deadlines for the NAGLU, Crohn's exomes, and Warfarin exomes challenges; deadlines updates; and publication plans for CAGI 4.

* New challenge released 4 November 2015:
Bipolar exomes: Distinguish between exomes of bipolar disorder patients and unaffected individuals. Challenge closes 7 January 2016.
https://genomeinterpretation.org/content/4-bipolar-exomes

* Prediction submission deadlines have been extended for three challenges:

NAGLU: N-acetyl-glucosaminidase (NAGLU): predict the effect of naturally occurring missense mutations on cellular enzymatic activity. Challenge now closes 8 December 2015.
https://genomeinterpretation.org/content/4-NAGLU

Warfarin exomes: Estimate patientsÕ therapeutic warfarin doses from their exome sequences. Challenge now closes 8 December 2015.
https://genomeinterpretation.org/content/4-Warfarin_exomes

Crohn's exomes: Distinguish between exomes of Crohn's disease patients and healthy individuals. Challenge now closes 9 December 2015.
https://genomeinterpretation.org/content/4-crohns-exomes

* Challenge submission deadline close time
These are the final changes to prediction submission deadlines for released challenges. Most challenge close at 9:00 PM PST (Pacific Standard Time) on the specified date; see individual pages for precise times.

* Prediction submissions for two challenges are approaching soon:

NPM-ALK: The Nucleophosmin - Anaplastic Lymphoma Kinase fusion protein: predict the effect of mutations in the kinase domain on kinase activity and Hsp90 binding affinity. Challenge closes 9 November 2015.
https://genomeinterpretation.org/content/4-NPM-ALK

SUMO ligase: Human SUMO ligase (UBE2l): predict the effects of missense mutations on competitive growth in a high-throughput yeast complementation assay. Challenge closes 18 November 2015. The experimental distribution of growth scores has been provided.
https://genomeinterpretation.org/content/4-SUMO_ligase

* Data for one challenge has been updated:

eQTL-causal SNPs: Identify regulatory sequences and eQTL-causal variants, and estimate their effects on activation of transcription in a massively parallel reporter assay. The sample dataset has been updated with edits to two variants.
https://genomeinterpretation.org/content/4-eQTL-causal_SNPs

* Publication plans for CAGI 4
We are in communication with editors at three journals about a virtual collection of publications on the CAGI 4 experiment. We plan a flagship paper reporting an overview of the challenges and general results, for which all CAGI participants will be listed as authors or in banner authorship. This is to be written within 6 weeks after the CAGI conference in late March 2016.

We will also have a series of papers from assessors and predictors on the detailed results of individual challenges or sets of related challenges. The timeline for submission of papers will be 3 to 6 months following the CAGI conference. The papers would be available free online, with the option to pay for full open access. Papers on individual challenges are to be organized primarily by the assessors and predictors, with support from the organizers as needed.

We are also working on publications from previous CAGI experiments, a matter of great concern to us.

* Rapid updates
There will be more news as challenges come to a close, not all of which will be in a newsletter, to avoid filling your inbox. All changes are highlighted on the challenge webpages. Please watch our twitter feed @CAGInews for udpates.

Challenge updates: PGP, Crohn's, eQTL; server security

Dear CAGI Participant,

This newsletter provides updates on the PGP, Crohn's exomes, and eQTL-causal SNPs challenges, and information on an intrusion of the CAGI server.

PGP challenge updates:
https://genomeinterpretation.org/content/4-PGP
On 12 October, we released 145 matched genomes & profiles as training data. On 26 October, we increased the number of challenge genomes from 13 to 23, and we decreased the number of challenge profiles from 228 to 101 to match the ratio of profiles to genomes in the PGP challenge in CAGI 2013.

Crohn's exomes challenge update:
https://genomeinterpretation.org/content/4-crohns-exomes
On 23 October, we released age of disease onset data for the Crohn's disease challenges in CAGIs 2011 and 2013 for use in training.

eQTL-causal SNPs challenge update:
https://genomeinterpretation.org/content/4-eQTL-causal_SNPs
The challenge submission deadline has been extended to 10 December 2015. On 30 October, we corrected the numbers of variants and eQTLs in the datasets.

Additional challenges will be released soon. Because challenges are under development, we cannot guarantee the release of each challenge. The current likelihood of release is indicated: + low, ++ moderate, and +++ high likelihood. Estimated closing dates for these challenges are in mid-December.
https://genomeinterpretation.org/content/developing-challenges-4

Protein-Coding Variants
+++ Pyruvate kinase mutations cause hereditary non-spherocytic hemolytic anemia (data provided by Aron Fenton, The University of Kansas Medical School). Challenge: predict the effects of single-amino-acid mutations on enzyme activity, and allosteric activation and inhibition in cell extracts.

Splicing Variants
+++ Splice acceptors (data provided by William Fairbrother, Brown University). Challenge: predict the effect of naturally occuring intronic variants near 3Õ splice sites of medically actionable genes in a high-throughput splicing assay.

Clinical Sequences
+ SickKids clinical genomes: genome sequences and microarrays from sick children (provided by M. Stephen Meyn, Hospital for Sick Children). Challenge: match patientsÕ genomes to their clincal descriptions and predict causal pathogenic variants.

++ Hopkins clinical panel (provided by Garry Cutting, Johns Hopkins). Challenge: match the patientsÕ gene panel sequences to their clinical descriptions and predict the causal pathogenic variants.

Research Exomes: predict phenotypes and mutations
++ Exomes, RNA-seq profiles, and microbomes from pre-diabetes patients and healthy individuals (provided by Mike Snyder, Stanford University). Challenge: predict each individualÕs fasting glucose level, glucose metabolic rate, and insulin response.

+++ Bipolar exomes (data provided by Peter Zandi, James B. Potash, and Richard McCombie). Challenge: distinguish between 1,000 exomes of bipolar disorder patients and healthy individuals.

Also, we wanted to make you aware of an intrusion of the server running the CAGI website on Saturday, 15 August, for approximately 30 minutes. Our investigation showed that the attacker installed software in an attempt to gain access to other systems. Logs show this server had no large data transfers to any unknown systems during the affected period. Therefore the CAGI challenges and registration data appear secure.

In an abundance of caution, we have deployed a rebuilt server. As a further precaution, you might wish to change your password using the "Request new password" button in the upper right corner of the home page. Although we deem it extremely unlikely that data were compromised, we wanted to report this incident to you.

New CAGI challenges released; close dates

Dear CAGI participant,

Three new challenges were released today:
https://genomeinterpretation.org/content/challenges-4

More challenges will be released over the next few weeks:
https://genomeinterpretation.org/content/developing-challenges-4

All released challenges now have posted close dates, with the first close on 9 November 2015.

Challenges released today:

* PGP: predict individuals' phenotypes from their genomes

* SUMO ligase: predict the effects of missense mutations on competitive growth in a high-throughput yeast complementation assay

* eQTL-causal SNPs: identify regulatory sequences and eQTL-causal variants, and estimate their effects on activation of transcription in a massively parallel reporter assay

Additional challenges being developed with the likelihood of release is indicated: + low, ++ moderate, and +++ high likelihood:

Protein-Coding Variants
-----------------------
+++ Pyruvate kinase mutations cause hereditary non-spherocytic hemolytic anemia (data provided by Aron Fenton, The University of Kansas Medical School). Challenge: predict the effects of single-amino-acid mutations on enzyme activity, and allosteric activation and inhibition in cell extracts.

Splicing Variants
-----------------
+++ Pre-mRNA splicing (data provided by William Fairbrother, Brown University). Challenge: predict the effect of naturally occuring intronic variants near 3’ splice sites of medically actionable genes in a high-throughput splicing assay.

Clinical Sequences
------------------
+ Genome sequences and microarrays from sick children (provided by M. Stephen Meyn, Hospital for Sick Children). Challenge: match patients’ genomes to their clinical descriptions and predict causal pathogenic variants.

++ Clinical gene panel sequences (provided by Garry Cutting Johns Hopkins). Challenge: match the patients’ gene panel sequences to their clinical descriptions and predict the causal pathogenic variants.

Research Exomes
---------------
+ Exomes, RNA-seq profiles, and microbomes from pre-diabetes patients and healthy individuals (provided by Mike Snyder, Stanford University). Challenge: predict each individual’s fasting glucose level, glucose metabolic rate, and insulin response.
+ Exomes of bipolar disorder patients and healthy individuals (provided
by Peter Zandi, James B. Potash, and Richard McCombie). Challenge: predict which individuals have bipolar disorder.

These challenges are anticipated to have close dates in early December.

We are currently exploring CAGI meeting dates in late March. Most likely the meeting will be 18-19 March in the San Francisco Bay Area.

Please distribute widely and follow our Twitter feed @CAGInews and the web site for updates.

CAGI 4 challenges and timeline

Dear CAGI participant,

Based on CAGI participants' feedback, the fourth CAGI experiment will run from August 2015 to March 2016 as follows:

Prediction season: August to November 2015, with about 3 months per challenge. We are rolling out further challenges now. A specific close date will be posted for each challenge.

Assessment season: December 2015 to February 2016.

Conference: tentatively scheduled for two days among 18-20 or 25-27 March 2016 in the San Francisco Bay Area.

CAGI 4 launched on 3 August 2015, with four challenges released thus far, below. More challenges will be released over the next 2 to 3 weeks. See Developing challenges, https://genomeinterpretation.org/content/developing-challenges-4.

Challenges released so far:
* Crohn's exomes: Distinguish between exomes of Crohn's disease patients and healthy individuals
* NAGLU: N-acetyl-glucosaminidase: predict the effect of naturally occurring missense mutations on cellular enzyme activity
* NPM-ALK: Anaplastic Lymphoma Kinase: predict the effect of mutations in the kinase domain of the NPM-ALK fusion gene on kinase activity and Hsp90 binding affinity
* Warfarin exomes: Estimate patients' therapeutic warfarin doses from their exome sequences

Additional challenges cover:
* Protein-Coding Variants
* Splicing Variants
* Regulatory Variants
* Clinical Sequences
* Research Exomes: predict phenotypes and mutations

Please distribute widely and follow our Twitter feed @CAGInews and the web site for updates.

CAGI 2015 about to launch!

Dear CAGI Newsletter Subscriber,

We expect to begin releasing CAGI 2015 challenges on 3 August 2015. The list of challenges under consideration follows. Please distribute widely and follow our Twitter feed @CAGInews and the web site for updates. Because challenges are currently under development, we cannot guarantee the release of each challenge. The likelihood of release is indicated: + low, ++ moderate, and +++ high likelihood. Please see below regarding CAGI timing.

Protein-Coding Variants
+++ N-Acetyl-Glucosaminidase (NAGLU) mutations cause mucopolysaccharidosis IIIB (data provided by Jon LeBowtiz, Biomarin Pharmaceutical). Challenge: predict the effect of natually occuring single-amino-acid mutations on enzyme activity in cell extracts.

+++ Pyruvate kinase mutations cause hereditary non-spherocytic hemolytic anemia (data provided by Aron Fenton, The University of Kansas Medical School). Challenge: predict the effects of single-amino-acid mutations on enzyme activity, and allosteric activation and inhibition in cell extracts.

+++ Anaplastic Lymphoma Kinase (ALK) (data provided by Paolo Bonvini and Federica Lovisa, Padua Children's Hospital). Challenge: predict the effects of kinase domain mutations in the oncogenic NPM-ALK fusion gene on kinase activity and Hsp90 binding affinity in transfected cell extracts.

+++ Human SUMO ligase (Ube2I) (data provided by Fritz Roth, University of Toronto). Challenge: predict the effects of a library of amino-acid mutations on competitive growth in a high-throughput yeast complementaion assay.

Splicing Variants
+++ Pre-mRNA splicing (data provided by William Fairbrother, Brown University). Challenge: predict the effect of naturally occuring intronic variants near 3’ splice sites of medically actionable genes in a high-throughput splicing assay.

Regulatory Variants
++ Regulatory sequences and variants associated with eQTLs (data provided by
Ryan Tewhey and Pardis Sabeti, Broad Institute). Challenge: predict the effects of eQTL-associated variants on activation of transcription in a high-throughput reporter assay.

Clinical Sequences
++ Genome sequences and microarrays from sick children (provided by M. Stephen Meyn, Hospital for Sick Children). Challenge: match patients’ genomes to their clincal descriptions and predict causal pathogenic variants.

++ Clinical gene panel sequences (provided by Garry Cutting Johns Hopkins). Challenge: match the patients’ gene panel sequences to their clinical descriptions and predict the causal pathogenic variants.

Research Exomes: predict phenotypes and mutations
+++ Exomes of Crohn's disease patients and healthy individuals (provided
by Andre Franke). Challenge: predict which individuals have Crohn's disease.

++ Exomes, RNA-seq profiles, and microbomes from pre-diabetes patients and healthy individuals (provided by Mike Snyder, Stanford University). Challenge: predict each individual’s fasting glucose level, glucose metabolic rate, and insulin response.

++ Exomes of patients on warfarin (provided by Roxana Daneshjou and Russ Altman, Stanford University School of Medicine). Challenge: predict each patient’s stable warfarin dose.

+ Harvard Personal Genomes Project (provided by George Church, Harvard Medical School). Challenge: predict clinical phenotypes from genome sequences and match individual’s genomes to their health records.

+ Exomes of bipolar disorder patients and healthy individuals (provided
by Peter Zandi, James B. Potash, and Richard McCombie). Challenge: predict which individuals have bipolar disorder.

*** Timeline for CAGI 2015 ***
The prediction season start date of 3 August 2015 is later than planned. We would now like your feedback on two options for the full experiment timeline:

Option 1: Hold the CAGI conference around 12 December 2015 as originally planned, with the prediction season ending early October (i.e. two months in total with less time for some challenges). That is short, potentially reducing the number of participants and the number of challenges each participant can attempt. It also leaves a short assessment season.

Option 2: Hold the CAGI conference later, perhaps March 2016, allowing time for a full-length prediction season (ending some time in November) and an adequate assessment season, providing the assessors time to do a deeper analysis of the results. In addition, a full-length prediction season will allow the CAGI organizers to provide updates on the details of how challenges will be assessed.

Please go to the following URL and let us know which you think is better.
https://docs.google.com/forms/d/1ZaIXIGdwd1PC-DFCHVI1-m7Qr9e4-ZPhoo-LU_j...

Further information:
This is the second newsletter for CAGI 2015. Please see the CAGI website (https://genomeinterpretation.org/) if you missed the first one. Briefly, CAGI has now received funding from NHGRI and NCI, and so we expect to conduct one experiment a year for the next three years.

The Critical Assessment of Genome Interpretation (CAGI) is a community experiment to assess computational methods for predicting the phenotypic impacts of genomic variation. CAGI 2015 will have approximately 10 challenges which will be made available on our web site:
https://genomeinterpretation.org/

In the CAGI experiment, participants are provided genetic variants and make predictions of resulting phenotypes. Independent assessors then evaluate these predictions against experimental characterizations. The primary goals of the experiment are to establish the current state of the art, identify bottlenecks in genome interpretation, inform critical areas of future research, and connect researchers from diverse disciplines whose expertise is essential for advancing methods for interpreting genomic variation. Anonymous submissions, with limitations, are possible:
https://genomeinterpretation.org/content/anonymity-policy
In order to access the challenges and submit predictions for CAGI 2015,
please register as soon as possible at our web site.

We encourage use of both established methods and experimental approaches, and we welcome predictors of all backgrounds. An NHGRI R13 grant will help support travel and participation in the meeting for junior investigators.
Previous CAGI experiments have highlighted striking breakthroughs as well as disappointing failures. Registered users have access to slides and posters presentations describing the results of previous CAGI experiments at:
https://genomeinterpretation.org/content/cagi-presentations
The results from the CAGI 2015 challenges will be posted as well, and publication of results from the experiment are planned and encouraged.

We look forward to your participation and predictions in CAGI 2015!

With all best regards,

Roger Hoskins, CAGI Organizer
John Moult, CAGI Co-Chair
Steven Brenner, CAGI Chair
email: "cagi@genomeinterpretation.org"

CAGI 2015

Dear CAGI Participant / Newsletter Subscriber,

We are writing to you as a registered CAGI participant or newsletter subscriber to update you regarding preparations for the 2015 CAGI experiment.

The general format for CAGI 2015 will be similar to that of previous CAGIs. We are aiming for about 10 challenges, with a three month prediction season in the summer. We tentatively plan to have the CAGI conference on 11 - 12 December 2015 in the San Francisco Bay Area. This is possible because NHGRI and NCI have recently co-funded our grant proposal to support CAGI.

We have been fortunate to hire an outstanding CAGI lead scientist, Roger Hoskins. Roger comes to us from the Drosophila Genome Center at Lawrence Berkeley National Laboratory. He will be in charge of all the organizational aspects of the CAGI experiment. He can be contacted at "hoskins@berkeley.edu".

We are now ramping up collection of datasets for this year's CAGI. In addition to new data sets similar to previous years, we are interested in identifying data sets produced using new and emerging methods. If you or someone you know has a dataset that may be appropriate for CAGI 2015, please let us know. We are also looking to recruit a CAGI developer for the technical and informatics aspects of the experiment.

We are looking forward to an exciting and productive CAGI, and we hope that you will participate in making it a great success!

With all best regards,

Steven Brenner and John Moult

CAGI 2015 plans - CAGI scientist position & Challenge datasets

We are happy to announce plans for CAGI 2015. Our intention is to have a prediction season in the spring & summer, assessment in late summer and fall, and a winter CAGI meeting.

CAGI 2015 is enabled by funding from NHGRI that will support hiring a scientist to lead this experiment. We would greatly appreciate referrals of individuals who may be suitable for this interesting and essential job. There are many types of people who could fill this position, and accordingly we have two job postings, one for a postdoctoral researcher

http://compbio.berkeley.edu/jobs/CAGI-pd-e908.pdf

and for staff scientist:

http://compbio.berkeley.edu/jobs/CAGI-ps-ea01.pdf

Please share these links and re-post as appropriate, and advise us of good places to advertise. It is an unusual post, and we would be happy to speak with prospective candidates.

If you have ideas for contacts or recommendations for CAGI challenge datasets, we would much appreciate hearing about them. We have a few leads on challenges, including those that provide continuity from the previous experiments. We will not be developing challenges in detail until the CAGI scientist is hired, but are starting preliminary work so as to move rapidly when the time comes.

We look forward to your participation in the next CAGI experiment!

Steven Brenner
John Moult

P.S. There will be a presentation about CAGI at the ASHG meeting: 6:15pm on Monday 20 October 2014 (Session 37, "From Bytes to Phenotypes"; Hall B1, Ground Level). CAGI will also be presented at the HGVS satellite meeting at 2:30pm on 18 October 2014. Come say hello!

Early registration for CAGI conference extended until 10 June 2013

Dear CAGI Community:

The early registration rate for the CAGI conference in Berlin has been extended until just after the ISMB early deadline, to allow more coordinated planning. CAGI early registration rates are now valid until 10 June 2013.

In order to register for the conference, you must first create a CAGI user account.
- https://genomeinterpretation.org/user/register
This necessary for us to verify your identity and online signature on the data use agreement, given the unpublished human data used in the CAGI meeting. We aim to approve all new CAGI users within 24 hours.

Once approved, you will be notified of your new CAGI account, whereupon you can immediately register for the conference.
- https://genomeinterpretation.org/content/register

Meanwhile, the assessors are completing the first stage of their evaluations (blinded to predictor identities and methods), and the preliminary results are striking. We have learned that several groups have been able to map some PGP genomes to trait profiles. On the Crohn’s challenge last year, one group used exomes to identify 80% of affected individuals before the first false positive healthy person. This year the challenge uses an improved dataset and the results are more tantalizing: several groups performed remarkably well -- and one achieved a ROC AUC of 0.94. Assessment of the challenge to understand Familial Combined Hyperlipidemia revealed a twist wherein real-world data differed sharply from theoretical models. Predictions on MRN proteins Rad50 (from last year), Mre11, and Nbs1 show how methods differ sharply in their effectiveness even amongst proteins in the same complex. These and the HA, p16, BRCA, splicing, and riskSNPs challenges will be discussed in full at the conference.

See you in Berlin!

Sincerely,

CAGI Organizers
cagi@genomeinterpretation.org

Early registration for CAGI conference extended until June 10th

Dear CAGI Community:

The early registration rate for the CAGI conference in Berlin has been extended until just after the ISMB early deadline, to allow more coordinated planning. CAGI early registration rates are now valid until 10 June 2013.

The regular fellowship application has passed, but late applications will be considered if funds are available. Abstracts are due on 15 Jun 2013.

See you in Berlin!

Sincerely,

CAGI Organizers
cagi@genomeinterpretation.org

Registration for the CAGI 2013 conference is now open! Call for abstracts and fellowship applications

Dear CAGI Participant,

Registration for the CAGI 2013 conference is now open! The conference
will be held 17-18 July 2013 at the Max Planck Institute for Molecular
Biology in Berlin, Germany. To register, please visit:
- https://genomeinterpretation.org/content/register

The early registration deadline is 31 May 2013, with a fee of $490 for
ISCB members; see the website for all other registration options:
- https://genomeinterpretation.org/content/cagi-2013-conference
The meeting is open to all participants (predictors, assessors, data
providers, board & council) as well as sponsors. Observers may also
be accommodated, depending upon space availability and anticipated
contributions to the meeting.

We welcome participants to submit abstracts for oral and poster
presentations. We especially encourage abstracts that present methods
which were used for predictions on the CAGI 2012/2013 challenges. To
submit an abstract, please visit:
- https://genomeinterpretation.org/content/cagi-2013-conference-abstract

Fellowships will be available to a limited number of US trainees from
an NHGRI R13 grant; we may also be able offer fellowships or discounts
to others depending upon funds. We particularly encourage trainees
from underrepresented groups to apply for fellowships to attend the
meeting. Please apply at:
- https://genomeinterpretation.org/content/cagi-2013-conference-fellowship

As with the previous conferences, the aim of this CAGI Conference is
to disseminate the results of the most recent CAGI experiment and
assess our collective ability to make accurate and meaningful
predictions of phenotypes from genomic information. CAGI assessors and
selected predictors will give the majority of the talks at the
conference, with significant time allocated for discussions.

Some important dates to keep in mind:
31 May 2013: Early registration closes
31 May 2013: Deadline for fellowship applications (including abstracts)
15 June 2013: Deadline for abstract submission (participants not
requesting fellowships)
21 June 2013: Regular registration closes

Much more information is available on the CAGI website:
- https://genomeinterpretation.org/content/cagi-2013-conference

If you are interested in sponsoring the CAGI conference--or have ideas
of potential sponsors--please contact us at
cagi@genomeinterpretation.org. Sponsorship will help support
additional predictor fellowships, assessors, and meeting amenities.

We look forward to seeing you in Berlin in July!

Sincerely,

Steven Brenner, CAGI Chair
John Moult, CAGI Chair
Daniel Barsky, CAGI 2012 Organizer
Stephen Yee, CAGI Conference 2013 Organizer

CAGI assessment underway; Conference registration to open soon

Dear CAGI Community,

The CAGI prediction season concluded on 25 April 2013, and we have entered the assessment phase. There were a total of 188 predictions submitted from 33 groups, a >50% increase in both submissions and groups over the previous CAGI experiment. Predictors hailed from 14 countries representing academics, research institutes, and industry.

Registration for the CAGI 2013 conference will open imminently. The conference will be held 17-18 July 2013 at the Max Planck Institute for Molecular Biology in Berlin, Germany. The early registration deadline is 31 May 2013, with a fee of $490 for ISCB members; see the website for all other registration options. The meeting is open to all participants (predictors, assessors, data providers, board & council) as well as sponsors. Observers may also be accommodated, depending upon space availability and anticipated contributions to the meeting.

Fellowships will be available to a limited number of US trainees from an NHGRI R13 grant; we may also be able offer fellowships or discounts to others depending upon funds. If you are interested in sponsoring the CAGI conference--or have ideas of potential sponsors--please contact us at cagi@genomeinterpretation.org.

We will post registration and fellowships links soon.

The assessment phase is now underway, with initial blind evaluations due 1 June. This year we intend to facilitate dialog between predictors and assessors in advance of the meeting, but predictors should not contact assessors at this point. The challenges and assessors:

Crohn’s disease: 56 submissions. Assessor: Alexander Morgan.
PGP: 16 submissions. Assessor: Sean Mooney
MRN variants: 22 submissions. Assessor: Sean Tavtigian.
P16 variants: 22 submissions. Assessor: Silvio Tosatto.
BRCA variants: 14 submissions. Assessor: Robert Nussbaum
Splicing: 5 submissions. Assessor: Jeremy Sanford.
FCH and HA: 39 submissions. Assessor: to be confirmed
MR-1 fitness: 0 submissions.
RiskSNPs: 12 submissions. Assessor: John Moult.

We look forward to seeing you in Berlin in July!

Sincerely,

Steven Brenner, CAGI Chair
John Moult, CAGI Chair
Daniel Barsky, CAGI 2012 Organizer
Stephen Yee, CAGI Conference 2013 Organizer

CAGI2012-13 challenges close tonight--send your predictions before 11:59 PM PDT!

Dear CAGI Participants,

This is a reminder that today is the last day to send your predictions for CAGI2012-13. The submission site will close at 11:59 PM PDT!
https://genomeinterpretation.org/all-submission-forms

(Note that the PGP challenge already closed yesterday.)

Hope to see many of you in Berlin. Details will be posted in the next few weeks.
https://genomeinterpretation.org/content/cagi-2012-conference

Sincerely,

CAGI Organizers

PGP Challenge deadline is 24 April--ONE DAY EARLIER than others; PGP templates have been modified

Dear CAGI participant,

Please note that we have had to bring the deadline for the PGP challenge forward 24 hours to 11:59PM PST on 24 April 2013. Please accept our apologies for this additional change and notification, but this is necessary to allow free exchange of the genomic information the next morning at the GET meeting (25 Apr).

Please also note that the two submission template files have been modified slightly as follows: 1) A training genome (474576e34ad0f39488d9c9b75946f7a7a4248427) that should not have been in the templates has been replaced by the missing challenge genome (fb74440463ffaaedecdac54b38ba2db7915965a0). 2) The templates have now been alpha-numerically sorted. Please download and use the corrected submission templates.
https://genomeinterpretation.org/content/PGP2012#templates

Sincerely,

CAGI organizers

CAGI prediction season extended to April 25; submission forms ready; various errata

There is additional time to tackle CAGI challenges! After consulting with predictors, we are making a final extension of the CAGI prediction deadline for all open challenges until 11:59 pm PDT on 25 April 2013.

Prediction submission forms are now available. You may make up to 6 predictions for each challenge, and any of these may be anonymous.
https://genomeinterpretation.org/all-submission-forms
(You must be registered and logged in to access the submission page.)

Following are additions, clarifications, and errata for several challenges:

1. MRN challenge: Previous versions of the datasets did not designate the anomalous variants as described in the challenge description (There is one such variant for MRE11, and there are five variants for NBS1). The dataset has been updated appropriately. In addition, there was an error in a previous version of the NBS1 Variant Dataset where the mutation p.V314E was incorrectly denoted c.941T>A.

2. HA challenge: We have been asked to explain why members of the family have variants reported on the Y chromosome. We note that this occasionally occurs in females due to mismapping to pseudo-autosomal regions of sex chromosomes.

3. FCH challenge: When identifying a specific gene, please include the gene name in front of the variant with a colon (e.g., NM_004006.1(DMD):c.3G>T).

4. The p16 challenge now includes training data.
https://genomeinterpretation.org/sites/default/files/protected_files/CAG...
(Only available to registered and signed-in users)

Critical Assessment of Genome Interpretation (CAGI)--Open Challenges and Conference

Following is an omnibus update on CAGI, including new information about
the CAGI conference in July and the recently added anonymity policy.
Please distribute widely and follow our Twitter feed @CAGInews.

The Critical Assessment of Genome Interpretation (CAGI) is a community
experiment to assess computational methods for predicting the
phenotypic impacts of genomic variation. The current CAGI experiment
has eight open challenges, available on the CAGI website:
https://genomeinterpretation.org/

In the CAGI experiment, participants are provided genetic variants and
make predictions of resulting phenotypes. Independent assessors then
evaluate these predictions against experimental characterizations.
The primary goals of the experiment are to establish the current
state of the art, identify bottlenecks in genome interpretation,
inform critical areas of future research, and connect researchers
from diverse disciplines whose expertise is essential for advancing
methods for interpreting genomic variation.

The deadline for current CAGI predictions is
***EDIT: extended to 25 April 2013.***
Anonymous submissions, with limitations, are allowed this year.
https://genomeinterpretation.org/content/anonymity-policy
We encourage use of both established methods and experimental
approaches, and we welcome predictors of all backgrounds.

The current CAGI experiment will culminate in a conference in Berlin,
on 17-18 July 2013, immediately before the ISMB SIGs. An NHGRI R13
grant will help support travel and participation in the meeting.
https://genomeinterpretation.org/content/cagi-2012-conference

Previous CAGI experiments have highlighted striking breakthroughs
as well as disappointing failures. Publications from the previous
CAGI are underway; slides and posters presentations about CAGI may
be found at:
https://genomeinterpretation.org/content/cagi-presentations
The results from the current CAGI challenge will be published as well.

The currently open CAGI challenges are:

+ Seventy-seven PGP genomes (provided by George Church).
Challenge: Predict clinical phenotypes from genome data, and match
individuals to their health records.
https://genomeinterpretation.org/content/PGP2012

+ Exomes of Crohn's disease patients and healthy individuals (provided
by Andre Franke). Challenge: predict which individuals have Crohn's.
https://genomeinterpretation.org/content/new-crohns-dataset

+ Exomes from two families with lipid metabolism disorders (provided
by John Kane and Pui-Yan Kwok). Challenge: predict lipid profiles
and a causative variant.
https://genomeinterpretation.org/content/FCH
https://genomeinterpretation.org/content/HA

+ Variants in DNA double-strand break repair genes (provided by Sean
Tavtigan). Challenge: predict probability of each variant occurring
in a breast cancer case versus healthy control.
https://genomeinterpretation.org/content/MRN

+ Mutations in p53 gene exons affecting mRNA splicing (provided by
Jeremy Sanford). Challenge: predict how variants impact splicing.
https://genomeinterpretation.org/content/Splicing-2012

+ Variants of a p16 tumor suppressor protein (provided by Silvio
Tosatto). Challenge: predict how well variants inhibit cell
proliferation.
https://genomeinterpretation.org/content/p16_2012

+ Shewanella oneidensis MR-1 gene disruptions (provided by Adam Arkin).
Challenge: Predict impact of microbial gene disruptions on cell
growth under stress conditions
https://genomeinterpretation.org/content/MR-1_2012

+ riskSNPs disease-associated loci (provided by John Moult). Challenge:
identify potential causative SNPs.
https://genomeinterpretation.org/content/risksnps2012

We are also soliciting challenges for the next CAGI. Please contact us
at cagi@genomeinterpretation.org with proposals for suitable datasets.

In order to access the current challenges and submit predictions for CAGI,
please register at https://genomeinterpretation.org/.

Registered users also have access to presentations from the previous
CAGI conferences, as well as posters and talk slides that summarize
the results.

Sincerely,

Daniel Barsky, CAGI 2012 Organizer
Steven E. Brenner, CAGI Chair
John Moult, CAGI Chair

cagi@genomeinterpretation.org

CAGI News: deadlines, meeting, PGP, p16, and Crohn's challenges

Dear CAGI Community,

We have several updates for the CAGI 2012 experiment, including the prediction submission deadline, two challenge releases, one major challenge update, and tentative CAGI meeting plans.

Deadline for prediction submissions:
The deadline for prediction submissions for the CAGI 2012 challenges is 28 March 2013 at 11:59 pm PDT. All CAGI 2012 challenges (except BRCA, which already closed) will be open until that time.

PGP challenge released:
The PGP challenge has just been released. This year the challenge is to submit predictions that match 54 genomes to phenotypic profiles (trait lists). Please visit the challenge page for details and to download the data: https://genomeinterpretation.org/content/PGP2012

p16 challenge released:
The p16 challenge has also just been released. The challenge is to evaluate how different variants of the p16 tumor suppressor protein impact its ability to block cell proliferation.
https://genomeinterpretation.org/content/p16_2012

Reworked Crohn's challenge data posted:
The reworked Crohn's disease dataset is now available on the Crohn's challenge page. The reworked dataset consists of variant calls made for all 66 exomes together, providing better quality variant calls and critical information for interpreting relatives. Please visit the challenge page for more information and to download the data: https://genomeinterpretation.org/content/new-crohns-dataset

News on the meeting for CAGI 2012 challenges:
We are looking into having the postponed CAGI 2012 meeting in Berlin, on 18-19 July 2013, immediately before the ISMB SIGs. More information will be available on the CAGI website as the plans develop. If you would be interested in helping with the meeting in any capacity, please let us know.

CAGI 2013 plans:
Pending funding from NIH, we hope to have a CAGI 2013 on a usual cycle (predictions in summer, meeting in December). To that end, we are beginning to solicit data for 2013 challenges. Please let us know if you know of any datasets appropriate for next year's CAGI experiment.

The CAGI organizers wish you and your families a joyous holiday season and happy 2013!

New CAGI challenges and updates: Mechanisms of risk SNPs, bacterial fitness, Crohn's, and splicing

Crohn's challenge update: The data providers are in the process of re-basecalling all samples together so as to provide one VCF for all the data. The revised dataset will provide all variants in the capture region (exome target) as well as the particular capture technology. It is expected that this updated dataset will be available in mid-December.

Splicing challenge update: The TP53 minigene experiments were done in HEK293T cells. Previously, the particular cell line was not specified.

We have also just released two new CAGI 2012 challenges:

1. Assign possible mechanisms for SNPs associated with risk of seven complex trait diseases. https://genomeinterpretation.org/content/risksnps2012

2. Predict impact of Shewanella oneidensis MR-1 gene disruptions on cell growth under stress conditions. https://genomeinterpretation.org/content/MR-1_2012

Coming soon: We will shortly be releasing a "PGP challenge" wherein predictors are asked to match individual genomes to individual trait profiles.

Feedback requested for the CAGI 2012 meeting plans

We would like to solicit your input regarding the postponed CAGI 2012 meeting. Please take this brief survey so that we can decide when, where, and if to hold a separate meeting to discuss the CAGI 2012 experiment.
http://www.surveymonkey.com/s/NB3CBW5

ASHG attendees: Informal CAGI gathering Thursday at 1:15pm

An informal CAGI gathering will be held Thursday, Nov. 8, 1:15 to 2:15 pm in Moscone South. Please write cagi@genomeinterpretation.org for location details. Also please write if you want to meet with Steven Brenner but cannot make that time.

Two new CAGI 2012 challenges: lipid metabolism diseases FCH and HA

We have recently released two new CAGI 2012 challenges:

1. Using exome sequencing data from a family, identify which individual has hypoalphalipoproteinemia (HA) and other disease phenotypes.
https://genomeinterpretation.org/content/HA-2012

2. In a family affected by familial combined hyperlipidemia (FCH), identify mutation(s) conferring low-density lipoprotein cholesterol (LDL-C) disease phenotype and identify individuals with abnormal triglycerides (TG) and high-density lipoprotein cholesterol (HDL-C) levels.
https://genomeinterpretation.org/content/FCH

Important! since this challenge was first posted, there have been important updates, namely
(a) a correction regarding who has the elevated LDL-C phenotype (Patient-17 is the unaffected daughter) and
(b) a clarification about how the variant calling was performed.