ML Pipeline Architect
We’re on a mission to democratize AI by building the definitive AI data development platform. The AI landscape has gone through incredible change between 2016, when Snorkel started as a research project in the Stanford AI Lab, to the generative AI breakthroughs of today. But one thing has remained constant: the data you use to build AI is the key to achieving differentiation, high performance, and production-ready systems. We work with some of the world’s largest organizations to empower scientists, engineers, financial experts, product creators, journalists, and more to build custom AI with their data faster than ever before. Excited to help us redefine how AI is built? Apply to be the newest Snorkeler!
As a ML Pipeline Architect you’ll lead the successful design and implementation of Snorkel Flow and associated model artifacts for our enterprise customers on premises or hosted cloud environments. This is a new role to Snorkel and will be its founding member responsible for shaping the approach we apply to future customer engagements. You’ll work closely with our internal engineering teams improving the design, architecture and implementation of Snorkel Flow, data integrations and model deployments, as well as represent our customer facing teams as Snorkel’s ML pipeline expert. This is a phenomenal opportunity to join our customer success team and drive its strategy for the future.
Main Responsibilities
- Design, prototype, and refine scalable infrastructure for operating Snorkel's machine learning pipeline at scale within customer on-premises and cloud environments
- Bridge the gap between data science and production, ensuring Snorkel Flow ML pipelines are deployed efficiently, reliably, and securely
- Work closely with our Product Management, Product Engineering, and Support teams to resolve issues that occur during the implementation
- Guide the team in adopting the best MLOps practices, ensuring a consistent and production-ready approach across all ML developments
- Assist in troubleshooting and resolving ML operations related issues
- Submit product feature requests to drive the platform forward
- Contribute to the Snorkel user documentation, including but not limited to design, testing, delivery and training
- Work closely with Snorkel’s field engineering teams and customers
Minimum Qualifications
- B.S. degree in Computer Science, Engineering, or comparable degree/experience
- 5+ years experience in a Platform Engineering, ML Operations or similar role
- Previous experience with cloud infrastructure providers such as Amazon Web Services, Microsoft Azure, or Google Cloud Platform
- Hands-on experience with common container and container orchestration solutions, e.g., Docker and Kubernetes
- Solid experience in developing data pipelines, deploying machine learning models as well as building model tracking, monitoring and online learning tools
- You are experienced with infrastructure-as-code tools such as terraform
- Experience with ML operation frameworks like Kubeflow, MLflow, and Airflow
- You’re deeply technical with a scrappy mindset
Preferred Qualifications
- Experience operating Kubernetes orchestration tools in a production setting
- Proficiency in Python, especially when setting up automation and system monitoring
- Exposure to authentication protocols like LDAP, SAML and OAuth2
- Experience implementing software products or solutions to large enterprise companies
- Comfortable working in a fast-paced, small-team environment where you’ll wear many hats
#LI-JD1