Paper

One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Models

Clinical events captured in Electronic Health Records (EHR) are irregularly sampled and may consist of a mixture of discrete events and numerical measurements, such as laboratory values or treatment dosages. The sequential nature of EHR, analogous to natural language, has motivated the use of next-token prediction to train prior EHR Foundation Models (FMs) over events. However, this training fails to capture the full structure of EHR. We propose ORA, a marked time-to-event pretraining objective that jointly models event timing and associated measurements. Across multiple datasets, downstream tasks, and model architectures, this objective consistently yields more generalizable representations than next-token prediction and pretraining losses that ignore continuous measurements. Importantly, the proposed objective yields improvements beyond traditional classification evaluation, including better regression and time-to-event prediction. Beyond introducing a new family of FMs, our results suggest a broader takeaway: pretraining objectives that account for EHR structure are critical for expanding downstream capabilities and generalizability.

Download:

• Paper on arXiv

Format

• Paper shared 1 week ahead for pre‑reading
• 10–15 min summary by a volunteer (no slides — we’ll scroll through the PDF together)
• Open discussion on methodology, strengths/weaknesses, clinical relevance, and future directions
• Papers chosen collaboratively to balance foundational ML research and cutting‑edge medical AI
• Who should attend? Researchers, clinicians, students — anyone curious about ML/AI in medicine.
• How to join? Simply show up (no registration required). Looking forward to seeing you there!

Organizers

Questions or paper suggestions? Contact us via Email.