About this role
Job Description:
Years of Experience : 6-10 Years
Key Responsibilities
-
Design, build, and optimize scalable ETL/ELT data pipelines across GCP.
-
Develop modular, object‑oriented Python applications for data ingestion, orchestration, and transformation.
-
Architect, deploy, and maintain data solutions leveraging GCP services: BigQuery, GCS, Cloud Functions, Cloud Run, Cloud Composer, Pub/Sub, Eventarc, and DataForm.
-
Integrate with external APIs to ingest datasets into BigQuery and other storage layers.
-
Implement event-driven and real-time data processing architectures.
-
Build and automate CI/CD workflows for deploying pipelines and services using Cloud Build.
-
Ensure optimal performance, scalability, and cost efficiency of pipelines and data infrastructure.
-
Establish and enforce data quality, governance, lineage, and security standards.
-
Work closely with cross-functional teams to convert business needs into scalable data solutions.
-
Maintain detailed technical documentation and contribute to engineering best practices.
-
Must have skills
-
Proven experience building modular, scalable ETL/ELT pipelines.
-
Hands-on expertise with GCP services such as BigQuery, Cloud Functions, Cloud Run, Cloud Composer, GCS, and Pub/Sub.
-
Hands-on Experience in Airflow
-
Ability to design and implement real-time event-driven architectures using Pub/Sub and Eventarc.
-
Experience integrating and consuming REST APIs for data ingestion.
-
Strong understanding of data governance, quality, privacy, and security practices.
-
Experience automating builds and deployments using CI/CD pipelines (Cloud Build preferred).
-
Ability to collaborate effectively with data science, product, engineering, and compliance teams.
-
Strong proficiency in Python with solid OOP principles, including encapsulation, inheritance, polymorphism, abstraction, SOLID, and design patterns.
-
Strong problem-solving, debugging, and system design skills.
-
Education Qualification
-
Bachelor’s or Master Degree or equivalent Degree
-
Certification If Any
-
- GCP Professional Data Engineer Certification.
-
- Snowflake Associate / Core
-
Shift timing
-
12 PM to 9 PM and / or 2 PM to 11 PM - IST time zone
Location
DGS India - Pune - Indiqube Orchid
Brand:
Merkle
Time Type:
Full time
Contract Type:
Permanent