About this role
Job Summary
The Software Engineer Team Member is part of the Marketing Technology Team; under general direction, they will design, develop, and implement systems to ensure customer requirements are met in a timely and cost-effective manner. The role is responsible for enterprise Marketing Technology. Key development responsibilities include the development of internal systems, management & integration of third-party software systems, and systems support.
Job Description
Experience Required: 5 to 10 years
Experience Desired: Experience working in Data Engineering + retail supply chain environment Education Desired: Bachelors Degree
Job Responsibilities
- Design, build, and maintain scalable batch and streaming data pipelines on Databricks using PySpark, Spark SQL, and Delta Lake.·
- Develop ingestion frameworks to bring data from databases, APIs, files, and third-party marketing platforms into the Lakehouse.·
- Build and maintain curated data models using the medallion (bronze/silver/gold) architecture for analytics, reporting, and marketing use cases.·
- Implement data governance, security, and access control using Unity Catalog.·
- Build data quality checks, validation, and monitoring to ensure the accuracy, completeness, and reliability of data.·
- Orchestrate and schedule pipelines using Databricks Workflows/Jobs and Azure Data Factory where applicable.·
- Optimize Spark jobs, cluster configurations, and Delta tables (partitioning, Z-ordering, file compaction) for performance and cost.· Support the migration and modernization of legacy Oracle-based ETL and PL/SQL processes to Databricks.·
- Collaborate with data analysts, data scientists, business stakeholders, and application teams to deliver data solutions.·
- Troubleshoot and resolve pipeline failures, data issues, and performance bottlenecks.