You'll lead the technical direction of the Machine Learning Training Platform team, which is developing a unified solution for all the machine learning teams at Pinterest. The system will encompass not just the model learning itself, but also many practical challenges in ML systems: feature extraction/feature definition, data validation, monitoring, and management of features/models. Comparable systems include Facebook's FBLearner Flow, Google's TFX, and Uber's Michelangelo.
What you'll do:
- Work with existing ML users across the company to understand their requirements.
- Design and develop a framework for easily assembling data pipelines for machine learning.
- Design and develop tools that allow large teams of modeling engineers to iterate on model quality and that help them keep their ML systems running smoothly.
What we're looking for:
- A solid understanding of the auxiliary practical concerns in production ML systems
- Good software design skills.
- Expertise with big data computation frameworks (e.g. Hadoop, Cascading/Scalding, or Spark)
- Expertise in Java and/or Python