Summary:
Plume develops and deploys cloud based control planes with scale to manage tens of millions of customer homes through some of the world’s largest Internet Service Providers. Our smart home services and experiences include WiFi network management and optimization, online protection, IoT security, motion detection, and end customer user interaction through mobile apps. We serve the consumer home market via our B2B and B2C product offerings.
To manage the explosive growth in data and applications, and power our Business Intelligence and Data products, we are building a world class team for our Enterprise Data Warehouse initiative.
The Opportunity:
As a Principal Data Engineer at Plume, you will focus on providing actionable insights and build highly available resilient systems that will impact and help make business decisions. You will ensure that the platform we are delivering will meet both ours and our customers’ reliability and scalability expectations.
What you will do:
- Interact with business stakeholders to understand and analyze BI requirements and design and document updates to dimensional model and schemas to support desired queries
- Understand available sources of data and design, implement, validate, deploy, manage, tune and monitor data pipelines to deliver the target schemas to the performance and quality spec.
- Adhere to data protection requirements including data access, retention, residency and de-identification.
- Create data validation checks and scripts to ensure high data quality and availability.
- Be responsive in triaging and fixing any issues related to production data quality and availability.
- Mentor and assist junior team members and new hires to become successful and productive.
- Build infrastructure and abstractions that can enable anyone (engineer or data scientist) to craft scalable pipelines for whatever the purpose is: metrics, analysis, machine learning, dashboard visualizations
Who you are:
- BA/BS in Computer Science, Information Systems or equivalent.
- Experienced: 6+ years building, scaling, tuning, managing and supporting production ETL pipelines for multiple terabytes of data.
- Prolific Programmer - 3+ years of Scala or Java.
- Technology Framework Expert - in a variety of data infrastructures, such as:
- Batch processing: Apache Spark
- Messaging: Kafka, Zookeeper, Pulsar
- Storage: Hive, Mongo DB, Athena, Cassandra, PostgreSQL
- A technical leader: you make intuitive decisions about what services, frameworks, and capabilities need to be in place before they are needed.
- Self Driven - able to own a project from inception to completion
- Mentor - open and active in sharing knowledge as well as excellent communication skills
- SQL Savvy - able to query and discover data with SQL
