Software Engineer, Machine Learning Infrastructure

About Us:

Hippocratic AI has developed a safety-focused Large Language Model (LLM) for healthcare. The company believes that a safe LLM can dramatically improve healthcare accessibility and health outcomes in the world by bringing deep healthcare expertise to every human. No other technology has the potential to have this level of global impact on health. 

Why Join Our Team:

• Innovative Mission: We are developing a safe, healthcare-focused large language model (LLM) designed to revolutionize health outcomes on a global scale.
• Visionary Leadership: Hippocratic AI was co-founded by CEO Munjal Shah, alongside a group of physicians, hospital administrators, healthcare professionals, and artificial intelligence researchers from leading institutions, including El Camino Health, Johns Hopkins, Stanford, Microsoft, Google, and NVIDIA.
• Strategic Investors: We have raised a total of $278 million in funding, backed by top investors such as Andreessen Horowitz, General Catalyst, Kleiner Perkins, NVIDIA’s NVentures, Premji Invest, SV Angel, and six health systems.
• World-Class Team: Our team is composed of leading experts in healthcare and artificial intelligence, ensuring our technology is safe, effective, and capable of delivering meaningful improvements to healthcare delivery and outcomes.

For more information: Visit www.HippocraticAI.com.

We value in-person teamwork and believe the best ideas happen together. Our team is expected to be in the office five days a week in Palo Alto, CA, unless explicitly noted otherwise in the job description.

About the Role:

This role at Hippocratic AI is focused on building and optimizing the data infrastructure that powers our machine learning (ML) operations, including our generative AI models and large language models (LLMs) for healthcare. You will design and scale reliable, data-driven services for ML model training, data processing, and deployment, ensuring our Research Scientists can seamlessly transition from experimentation to production. This role involves working with massive datasets, managing ETL pipelines, and building scalable solutions to handle data ingestion, transformation, and storage across Hippocratic AI’s systems.

Responsibilities:

• Build powerful, flexible, and user-friendly data and ML infrastructure that supports all ML Speech and LLM operations across Hippocratic AI.
• Design and develop fast, reliable data services for ML model training, ETL pipelines, and deployment, scaling infrastructure across multiple regions.
• Create services and libraries that enable ML engineers to move efficiently from data experimentation to production, especially for generative AI models.
• Collaborate closely with product teams, data engineers, and research scientists to develop data-focused infrastructure that supports production-ready generative AI and LLM/ Multimodal models for healthcare applications.

Must-Have:

• 5-8 years of experience in building software applications for large-scale distributed data systems.
• Strong engineering background with experience in infrastructure and/or distributed systems, with proficiency in Python, Java, or similar languages.
• Solid experience with ETL processes and data pipeline design, ensuring high-quality data for ML model development and deployment.
• Familiarity with the complete software development life cycle, from design and implementation to testing and deployment.
• Proven track record in building and maintaining high-availability, low-latency systems with a focus on reliability, testing, and observability.
• Pragmatic approach to problem-solving, knowing when to aim for ideal solutions and when to adjust course.
• Experience with big data technologies such as Apache Spark for data processing and large-scale data analytics.
• Strong sense of curiosity and a collaborative mindset, eager to learn new technologies and share knowledge within the team.

Preferred:

• 5+ years of experience supporting machine learning and generative AI infrastructure.
• Hands-on experience optimizing the end-to-end performance of distributed data systems, particularly for Multimodal LLMs and other generative AI applications.
• Experience with Audio/ Speech Training Infrastructure is a plus.

Why You’ll Love Working Here:

At Hippocratic AI, we are revolutionizing the healthcare landscape through cutting-edge technology. We want talented individuals who thrive at the intersection of innovation and impact. You’ll work alongside some of the brightest minds in healthcare and AI to shape the future of healthcare accessibility.