About this role
Data Engineer
Description
-
Remote
-
Our client seeks a Data Engineer to design and implement high-performance data pipelines at scale. The role will focus on building reliable ingestion and transformation processes, enforcing data quality and lineage, and delivering resilient solutions across AWS services.
-
Rate: $50.00 to $57.00/hr. w2
Responsibilities
- Design and implement high-performance data pipelines that handle large volumes of data.
- Ensure data integrity through advanced quality checks and lineage tracking.
Experience Requirements:
- Expert-level proficiency in Python and PySpark.
- Deep hands-on experience with AWS Glue and Amazon EMR.
- Proven track record building scalable pipelines for large-scale data, preferably at terabyte scale.
- Experience implementing frameworks for data lineage and data quality.
- Strong experience with S3-based data lakes and relational databases such as RDS, PostgreSQL, or MySQL.
- Experience in Financial Services is a plus.
- Experience with real-time processing using Kinesis or Kafka.
- Familiarity with CI/CD for data pipelines and Infrastructure as Code.