Senior Python Engineer

About Us

At Union, we are solving one of the hardest challenges in AI infrastructure today: enabling high-velocity iteration while maintaining seamless production-readiness for AI workloads at scale. Flyte, the open-source project we steward, is the emerging standard for modern data and AI orchestration, with numerous leading technology organizations - like LinkedIn, Spotify, and Gojek - running millions of mission-critical workloads on the platform. We have a deep bench of infrastructure veterans from companies in the Big Three and beyond and a technical founding team who originally created Flyte while at Lyft.

The Opportunity

Flytekit is the main entrypoint for users into the Flyte world. It's a Python SDK that serves as the gateway for the 'authoring' (writing tasks and workflows) part of Flyte. It's also the basis of the core tenet, the core mission of Union AI, and underscores why the Flyte project has found the success that it has. 

We're looking for someone to join us and become the core maintainer of flytekit, with the goal of continuing to push on the mission of bringing as much ease-of-use and user-delight as possible while modernizing and elevating the existing code base. You will not only lead Python development at Union, but you will be a voice in the industry, leading other companies by example. 

Prior work should have motivated you to develop a mastery of advanced Python topics - you should be able to seamlessly navigate things like asyncio and python threading, metaclasses, mros, module loading, and be able to use all this idiomatically, enumerating and explaining tradeoffs among the various ways of doing things, while integrating with and improving the flyte DSL. 

In this role, you will:

• Lead the development of Python products at Union (e.g. flytekit, UnionML, pandera)
• Advocate for best practices across the Python language across our internal engineering members and external community contributors 
• Go deep. Implement a DSL that mirrors the Python core language
• Get satisfaction out of understanding and solving our users’ ML and data challenges
• Since Union AI is focused heavily on machine learning and data applications, you should either bring with you or be willing to develop experience with ML libraries, learning and appreciating how AI practitioners use Python and the various ML toolchains that already exist
• Learn about new tools and libraries in the ML landscape, understanding their impact and use-cases, and implementing elegant integrations within Flyte

About You:

• 6+ years experience as a python backend engineer
• In-depth understanding of Python internals (CPython), lower-level programming, and infrastructure-related code development
• You have an interest in contributing to open source projects
• Bachelor’s Degree in Computer Science or related field, or equivalent experience
• Experience with ML Frameworks (Pytorch, Tensorflow, etc) and data processing frameworks (Spark, Flink, Beam etc) a plus
• Knowledge of Rust and PyO3 is a plus
• Open-source contributions in Python is a plus
• Knowledge of Rust and PyO3 is a plus

Benefits & Belonging

At Union.ai we know that employees who feel their best can build amazing things and we are proud to offer best in class benefits that will continually evolve and grow as the needs of our employees do. Benefits may vary based on country

• Excellent medical - We pay 100% of your premiums and 90% for your dependents
• Generous dental and vision plans- We pay 80% of the premiums for you and your dependents
• Meaningful equity in the form of options – all employees are owners here
• Unlimited PTO + 12 company holidays 
• 401K match - Union.ai matches 100% of contributions up to the first 3%, and 50% up to 5%
• 12 weeks paid parental leave for primary and secondary caregivers
• Flexible work schedule at a remote friendly company

We believe that our differences are what bring us together to achieve truly special outcomes. We strive to be inclusive and focus on building teams that embody that quality too. Union.ai is an equal-opportunity employer and we encourage you to apply, even if your experience doesn’t align exactly with our job description.