Have questions? Mail aitxchallenge@gmail.com !

:thinking: :speech_balloon: Frequently Asked Questions

Genetic Diagnoses

Are we interested in targeted treatments for cancer?

Tumor somatic sequencing results and precision oncology targeted treatments are out of scope. However, germline variants that indicate a predisposition to cancers, including rare cancers, are within scope. Other somatic variant testing (e.g., to detect mosaicism or guide treatment decisions for congenital vascular anomalies) mentioned in the patient.clinical_context input field and/or as part of the question stem are also in scope.

What about structural variants?

We are limiting question submissions to simple (single-nucleotide and insertion/deletion) variants that can be represented in HGVS cDNA and protein formats.

Why would there be multiple diagnostic variants?

Our input patient.genotype field is a list of variants, allowing for multiple diagnostic variants.

In recessive disorders, compound heterozygous variants (i.e., two different variants inherited in trans from two parents) might be necessary for disease manifestation. It is also possible that a patient may be affected by multiple monogenic conditions simultaneously.

Can we assume that the input genotype (list of variants) is confirmed diagnostic?

Yes.

Why are we focusing on just rare genetic diseases?

  1. Rare genetic diseases represent a uniquely underserved area in therapeutic development. Fewer than 5% of known monogenic disorders have targeted treatments. Unlike relatively common genetic disorders (e.g., cancers with recurrent “hotspot” mutations across patients), rare diseases often involve private or ultra-rare variants unique to individuals or families. This means that therapeutic strategies must account for variant-specific impacts.

  2. Classes of personalized treatments (e.g., antisense oligonucleotides [ASOs], gene therapies) may be especially suited for treating rare genetic diseases! However, determining the feasibility of such treatments requires answering complex molecular eligibility questions based on diverse, dynamic (up-to-date) data sources.

Question Scope

Will we be asked to recommend non-intervention medical tests?

We will not ask you to recommend surveillance steps (e.g., imaging studies) following a diagnosis. Results of such studies provided in the input patient.clinical_context field and/or as part of the question stem are considered in scope. For example, MRI evidence of gadolinium enhancement in X-linked Adrenoleukodystrophy patients indicates eligibility for haematopoietic stem cell transplantation, whereas an MRI showing no contrast enhancement would not indicate treatment.

What if there are no established targeted therapies or clinical trials?

Established therapies and active clinical trials may not exist for every case, but when they are available, identifying them accurately is especially important.

Are only established (e.g., not under development) drug repurposing methods valid?

For questions in the Drug Development and Repurposing category, retrieval of pertinent information for all scientific strategies (e.g., target/pathway reasoning, connectivity mapping, structural modeling) are welcome, provided the reasoning is clearly explained and evidence-based. Synthesis of insights across multiple data sources for specific candidates is highly encouraged, as is the analysis of candidate groups to identify shared mechanisms, patterns, or therapeutic opportunities.

What will be included in the patient.clinical_context field?

You can expect a brief description of the patient’s symptoms/condition and, if relevant, the patient’s age, family history, care setting, comorbidities, and/or other medications as they pertain to the question. This description is free text and may just be a comma-separated list rather than complete sentences.

Model Development Specifications

Can we use closed models?

Yes. However, we will evaluate and award prizes to submitted models across different tiers; use of large vs. small models as well as use of open-source vs. proprietary models will be distinguished during evaluation.

Can we use agents? What tier do they fall into?

Agents use LLMs to decide what to do next (e.g., breaking a task into steps, calling tools, retrieving data, running code), and we do not consider them models themselves, but rather an “orchestration layer”. Therefore, you can use an agent in any tier, but the underlying LLM must follow the tier rules (i.e., open weights for Tiers 1 & 2 and ≤8B parameters in Tier 1 or ≤70B parameters in Tier 2).

Is RAG no longer relevant?

Retrieval-Augmented Generation is relevant! Grounding claims in retrieved sources is the essential component of the final Actionability Report Generation task. What has aged is the classic pipeline: embed a corpus, retrieve the top-k chunks, and paste them into the prompt. That fits this challenge poorly, because an actionability report needs current records (is that trial still recruiting?) and citations that resolve to a specific accession rather than a paraphrase of a nearby chunk.

The framework we would point you to instead is agentic retrieval, where the model plans a query, calls a live API, reads what comes back, and queries again until it can actually support the claim. Although longer context windows newly enable keeping whole records in context instead of fragments, sometimes this is not desirable. A prebuilt index is still useful for finding candidate claims quickly, just verify each claim against the live record before you cite it.

Can I wrap or call medically-trained models?

Yes! Some recent, open-access examples include MedGemma and TxGemma.

Will there be a dev dataset?

For the current, final challenge phase: No. However, a dev dataset for the Question-Answering Task had been made available through our AI-Tx Challenge Hugging Face organization, and may be useful for the Actionability Report Generation Task as well.

Which resource tier would a mixture-of-experts model fall into?

A mixture-of-experts (MoE) model can be run with far less compute, but still requires high VRAM to run since all experts are loaded into memory. Resource tiers are determined by the total (not active) number of parameters, and so, for example, GPT-OSS-120B would fall into the “Tier 2: Large” tier, as we would count it as having ~117B parameters, not ~5.1B active parameters.

How soon can I submit my model?

All deadlines are listed on our Participate Landing Page.

Can teams submit models, or just individuals?

Teams can participate!

Who can help me debug technical issues I run into?

Google Health AI Developer Foundations (HAI-DEF) has offered to help participants and developers with Google-related technical queries related to this challenge, including broad Google Cloud Platform questions as well as questions about their models (e.g., MedGemma). You can post your questions directly on their forum: https://discuss.ai.google.dev/c/hai-def/.

Am I eligible to receive cloud credits for participation or as a prize?

Unfortunately, our intention to award cloud credits to participants and as prizes did not pan out due to payment reporting complications. We are actively working with our internal Finance and Legal teams as well as Google to prevent similar hiccups for any future agreements that involve Google Cloud Platform (GCP) credits. In the meantime, there are other alternatives to explore:

  1. The Cloud for Education program offers research credits of up to $5,000 for faculty and students. This program is managed by Cloud is open to accepting credit requests. You can find more information on the program here: Cloud for Education - Research Credits. To apply, the researcher can use this link: Cloud for Education Research Application.

  2. Gemini for Research: Qualified academic researchers (such as faculty, staff, and PhD students) in supported countries can apply to receive up to $20k in Gemini API credits and higher rate limits for research projects.

What else is happening in this space?

There are other complementary challenges relating to therapeutics and rare disease:

Still have questions? Mail aitxchallenge@gmail.com !

Google logo

SAIL logo           Hugging Face logo           NEJM-AI logo