Senior Software Engineer- Data Acquisition
We are looking for people with experience gathering data from the web at scale. You’ll be part of Luma’s infrastructure team, working cross-functionally with Applied AI and Research on mission critical work-streams.
Responsibilities
Own / lead engineering projects for data acquisition.Work with Research and Applied AI to ensure their data and technical needs are met.Develop and deploy highly scalable distributed systems capable of handling petabytes of data.Build and gather high quality datasets by collaborating closely with the research team.Build and maintain backend services for data storage.Architect and implement algorithms for data management including indexing & search.Conduct and analyze system performance experiments. Experience
Proficiency in Python.Experience of 5+ years in web-scraping at-scale.Experience building and working efficiently in very large distributed systems.Experience working with Kubernetes, proxies and other relevant tools.Experience working with research datasets.Please note this role is not meant for recent grads. The pay range for this position in California is $180,000 - $250,000yr; however, base pay offered may vary depending on job-related knowledge, skills, candidate location, and experience. We also offer competitive equity packages in the form of stock options and a comprehensive benefits plan.
All your applications are reviewed by real people.
Apply for this job →