Associate Architect - Data - PySpark + Streaming

Quantiphi

Education
Benefits
Qualifications
Skills

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.

If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!

Job Role : Associate Architect - Data

Experience : 6+ Years

Location : Mumbai/Bangalore/Trivandrum (Hybrid)

Must have skills:

  • 6+ years of hands-on experience in building large-scale data pipelines.
  • Strong experience with Spark Streaming, AWS Glue, and EMR for processing both real-time and batch workloads
  • Proficiency in Pyspark/Python, including building Kafka producers for data ingestion
  • Experience working with Confluent Kafka for streaming ingestion from on-premise sources.
  • Solid understanding of AWS services including S3, Redshift, Glue, CloudWatch, and Secrets Manager
  • Experience working with Medallion architecture and hybrid data destinations (e.g., Redshift and on-premise Oracle)
  • Ability to implement monitoring dashboards and observability for pipelines using tools like CloudWatch or Datadog
  • Strong SQL skills for validating data accuracy and developing job-level metrics
  • Experience with building alerting mechanisms for pipeline failures and performance issues
  • Strong collaboration and communication skills
  • Proven ability to take ownership of technical deliverables, driving execution from design to deployment
  • Experience in guiding junior developers or data engineers, providing technical direction and code reviews to uphold best practices

Good to have skills :

  • Experience with orchestration tools like Apache Airflow or AWS Step Functions
  • Exposure to Terraform for infrastructure automation
  • Familiarity with CI/CD pipelines for data workflows
  • Prior experience in on-prem to cloud data migration
  • AWS Solution Architect Certification

If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!

Read Full Description
Confirmed 13 hours ago. Posted 3 days ago.

Discover Similar Jobs

Suggested Articles