Submit Your Model

Individuals or teams participating in the AI-Tx Challenge should register in advance. Each submission must be a Hugging Face Space exposing a required /answer_question API endpoint.


Step 1: Understand Submission Tiers

When submitting a model, you must choose one of three resource tiers (defined by the total, rather than active, number of model parameters). These tiers help us ensure a fair evaluation across models of different sizes and computational complexity. Each tier corresponds to a collection within our Hugging Face organization and has its own evaluation track.

Tier Description Intended Use
Tier 1: Small Models that can run efficiently on low-resource environments, typically with ≤8 billion parameters.* All model weights must be open-source. Ideal for compact, open models such as Qwen3-8B, Mistral-7B, or medically oriented models like BioMistral-7B, MedGemma-4B-multimodal, or TxGemma-2B.
Tier 2: Large Larger models with ≤70 billion parameters.* All model weights must be open-source. Suitable for advanced open models such as Llama-3.3-70B, Mixtral 8×7B, or domain-adapted larger models like MedGemma-27B-text-only, OpenBioLLM-70B and Meditron-70B.
Tier 3: Unrestricted Any model, including those with >70 billion parameters.*, closed-source, or relying on private/commercial APIs. Includes systems such as GPT-5.1, Claude Opus 4.1, Gemini 2.5 Pro, etc.

*Resource tiers are specified by the total number of parameters, as opposed to the active number of parameters.


Step 2: Build Your Space

  • Implement /answer_question
    • Accept the expected input format and return responses in the required output format
  • You may host/call any model, including private and API-based (see Step 1, Tier 3: Unrestricted).
Development Dataset

A small development dataset with questions and answers is available on Hugging Face: AI-Tx Challenge Phase 1 Model Validator.


Step 3: Set Access

  • Public Space: no action needed
  • Private Space: provide a read-only Hugging Face token:
    • Go to SettingsAccess TokensCreate new token
    • Select a fine-grained token
    • Grant read access only to the submitted Space repository
    • Do NOT use write tokens or tokens with access to unrelated repositories.

Step 4: Submit & Validate

Additional Requirements
  • Individuals or teams with the top performing models for Tier 1: Small and Tier 2: Large will be required to submit their training data and code (although you can and should still maintain licensing).
    • Exceptions can be made for the data sharing requirement with reasonable justification (e.g., patient consent, third-party licensing). At minimum, you will need to share a dummy dataset to include with your codebase.

Step 5: Evaluation

  • Agree to no-logging rules: no logging, storing, redistributing, or training on evaluation queries or responses
  • We will run your Space on a private test set and share results.

Google logo

SAIL logo           Hugging Face logo           NEJM-AI logo