Strategic Staffing Solutions is currently looking for a Data engineer, for a W-2 contract opportunity.
Data Engineer
Location: Charlotte, NC,28206
Type: W-2 Contract (NO C2C)
Contract: 12 months
Schedule: Hybrid
This is a W-2 contract opportunity, and candidates must be able to work on our W-2 only. NO C2C or 1099.
Position Overview
Required Technical Skills
- Python: Advanced proficiency in Python 3.8+ for data processing, scripting, and automation
- PySpark: Extensive experience with PySpark for large-scale data processing and transformations on EMR clusters
- AWS Lambda: Hands-on experience building serverless functions for data processing and orchestration
- AWS Glue: Expertise in AWS Glue for ETL jobs, Data Catalog management.
- Amazon Athena: Experience with Athena for SQL-based data analysis and querying data lakes
- Amazon EMR: Strong experience with EMR for running Spark jobs, EMR Notebooks, and cluster management
- Amazon Redshift: Knowledge of Redshift for data warehousing, query optimization, and integration with data lakes
- Kafka: Experience with Confluent Kafka for real-time data streaming, including Schema Registry, source/sink connectors, and producers/consumers
- Amazon S3: Deep understanding of S3 for data lake architecture, bucket policies, and data organization (raw, curated layers)
- CloudWatch: Proficiency in CloudWatch for logging, monitoring, and alerting on data pipelines
- Terraform: Experience with Infrastructure as Code using Terraform for provisioning AWS resources
Key Responsibilities
- Design, develop, and maintain scalable ETL pipelines using AWS Glue, EMR (PySpark), and Lambda functions
- Implement real-time data streaming solutions using Confluent Kafka with source and sink connectors
- Develop and optimize PySpark jobs on EMR clusters for large-scale data transformations
- Build serverless data processing workflows using AWS Lambda
- Design and optimize queries in Amazon Athena for analytical workloads
- Manage data warehouse solutions in Amazon Redshift including data modeling and performance tuning
- Implement monitoring and alerting using CloudWatch for data pipeline health and performance
- Provision and manage AWS infrastructure using Terraform
- Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements
- Implement data governance and security best practices using Lake Formation, IAM, and KMS
Additional Technical Experience
- Knowledge of AWS EventBridge and Step Functions for workflow orchestration
- Familiarity with EMR runtime roles and execution roles for secure data access
- Experience with DynamoDB for NoSQL data storage
- Knowledge of API Gateway for building data APIs
Preferred Qualifications
- Bachelor’s degree in Computer Science, Data Engineering, Information Technology, or related field
- Strong understanding of distributed computing and big data processing frameworks
- Experience with Agile development methodologies
- Excellent problem-solving and analytical skills
- Strong communication and collaboration abilities
Development and Data Tools
- EMR Notebooks or Jupyter for interactive development
- VS Code or similar IDE for Python development
- Git for version control
- AWS CLI and boto3 for AWS service interaction
- Terraform for infrastructure provisioning
- DBeaver, Data Studio for database management
“Beware of scams. S3 never asks for money during its onboarding process.”
Read Full Description