Sr. Hadoop Developer
Work Location: Remote
Project Scope: Looking for a solid developer to design and code build ETL using Spark/Scala. This is for both batch jobs and real-time streaming jobs.
Description:
Senior Hadoop Developer is responsible for development, programming, coding of Information Technology solutions in Hadoop/ Cloudera Platform. Responsible for documenting detailed system specifications, participation in unit testing and maintenance of planned and unplanned internally developed applications, evaluation and performance testing of purchased products. Responsible for including IT Controls to protect the confidentiality, integrity, as well as availability of the application and data processed or output by the application.
Essential Functions:
- Write code for moderately complex system designs. Write programs that span platforms.
- Write code for enhancing existing projects or developing new project.
- Review code developed by other IT Developers.
- Provide input to and drive programming standards.
- Write detailed technical specifications for subsystems. Identify integration points.
- Report missing elements found in system and functional requirements and explain impacts on subsystem to team members.
- Consult with other IT Developers, Business Analysts, Systems Analysts, Project Managers and vendors.
- Seek review from other IT Developers, Business Analysts, Systems Analysts or Project Managers on estimates.
- Perform unit testing and debugging. Set test conditions based upon code specifications. May need assistance from other IT Developers and team members to debug more complex errors.
- Supports transition of application throughout the Product Development life cycle. Document what has to be migrated. May require more coordination points for subsystems.
- Accountable for including IT Controls and following standard corporate practices to protect the confidentiality, integrity, as well as availability of the application and data processed or output by the application.
- The essential functions listed represent the major duties of this role, additional duties may be assigned.
What you will need:
- 6+ years software development coding experience
- Extensive hands-on ETL coding experience in Spark and Scala
- Experience writing complex SQL queries and extensive data warehousing knowledge
- Experience working with different file formats including XML, Json.
- Experience developing code to process large volume data in batch and real time using Spark Streaming.
- Experience leading technical resources on day-to-day basis and/or in large scale project
- Experience in Hadoop stack
- Experience building data flow and process flow diagrams
- Proficiency using versioning & debugging tools
Job Requirements:
- Experience and understanding with unit testing, release procedures, coding design and documentation protocol as well as change management procedures
- Extensive hands-on coding experience in Spark and Scala
- Experience writing complex SQL queries and extensive data warehousing knowledge
- Experience working with different file formats including XML, Json.
- Experience developing code to process large volume data in batch and real time using Spark Streaming.
- Extensive experience working with SQL & NoSQL databases (Mongo, Postgres)
- Proficiency using versioning tools
- Demonstrated organizational, analytical and interpersonal skills
- Flexible team player
- Ability to manage tasks independently and take ownership of responsibilities
- Must demonstrate initiative and effective independent decision-making skills
- Ability to communicate technical information clearly and articulately
- Ability to adapt to a rapidly changing environment
REQUIRED EDUCATION/EXPERIENCE:
- Related Bachelor's degree in an IT related field or relevant work experience
- 5-8 years related work experience of IT development/programming/coding professional within a Hadoop environment
- Hadoop EcoSystems with Spark/ Scala
- Kafka, Spark streaming/Batch/ real-time processing
- Hive, HBase
- MongoDB, Postgres DB
- Shell Scripts
- SQL
- Java/Node/JavaScript PREFERRED
#DICE
efinn@c4techservices.com
Read Full Description