Python Pyspark - (CREQ219377)
Senior Data Engineer
Position Summary
The Senior Data Engineer leads complex data engineering projects working on designing data architectures that align with business requirements This role focuses on optimizing data workflows managing data pipelines and ensuring the smooth operation of data systems
Minimum Qualifications
8 Years overall IT experience with minimum 5 years of work experience in below tech skills
Tech Skill
Strong experience in Python Scripting and PySpark for data processing
Proficiency in SQL dealing with big data over Informatica ETL
Proven experience in Data quality and data optimization of data lake in Iceberg format with strong understanding of architecture
Experience in AWS Glue jobs
Experience in AWS cloud platform and its data services S3 Redshift Lambda EMR Airflow Postgres SNS Event bridge
Expertise in BASH Shell scripting
Strong understanding of healthcare data systems and experience leading data engineering teams
Experience in Agile environments
Excellent problem solving skills and attention to detail
Effective communication and collaboration skills
Responsibilities
Leads development of data pipelines and architectures that handle large scale data sets
Designs constructs and tests data architecture aligned with business requirements
Provides technical leadership for data projects ensuring best practices and high quality data solutions
Collaborates with product finance and other business units to ensure data pipelines meet business requirements
Work with DBT Data Build Tool for transforming raw data into actionable insights
Oversees development of data solutions that enable predictive and prescriptive analytics
Ensures the technical quality of solutions managing data as it moves across environments
Aligns data architecture to Healthfirst solution architecture
: IN-TN-Chennai
: Full Time
: Individual Contributor
: Experienced
: No
: 14/05/2025, 5:00:53 AM
Read Full Description