Data Engineer (Pyspark) - (CREQ225424)
o Minimum 4+ years of development and design experience in experience as Data Engineer
o Hands-on experience in building PySpark
o Should be able to perform data cleansing and transformations in PySpark.
o Unit testing using Python/PySpark code
o Experience on Big Data platforms and distributed computing (e.g. Hadoop, Map/Reduce, Spark, HBase, Hive)
o Experience in data pipeline software engineering and best practice in python (linting, unit tests, integration tests, git flow/pull request process, object-oriented development, data validation, algorithms and data structures, technical troubleshooting and debugging, bash scripting )
o Experience in Data Quality Assessment (profiling, anomaly detection) and data documentation(schema,dictionaries)
Experience in data architecture, data warehousing and modelling techniques (Relational, ETL, OLTP) and consider performance alternatives
: IN-TN-Chennai
: Full Time
: Individual Contributor
: Experienced
: No
: 10/07/2025, 12:31:13 PM
Read Full Description