Senior Data Engineer Data and Analytics | Data Engineering

Job title: Senior Data Engineer Data and Analytics | Data Engineering

About the Role An experienced Senior Data Engineer who will design, develop, and optimize ETL/ELT pipelines and end-to-end data solutions. You will lead design discussions, mentor engineers, and collaborate with client architects within Deloitte's Project Delivery Model to deliver on-site, long-term data projects.

What You'll Do

• Communicate regularly with Engagement Managers (Directors), project team members, and representatives from various functional and/or technical teams, escalating matters when needed.
• Design, develop and optimize ETL/ELT pipelines using Azure Data Factory and Databricks
• Write and tune PySpark/Spark SQL notebooks for large-scale data transformation
• Architect end-to-end data solutions across dev, UAT, and prod environments using Unity Catalog
• Lead and drive design discussions with client architects and other counterparts
• Collaborate on data contracts and schema agreements
• Lead design and optimization of high-volume data pipelines
• Define and enforce data engineering standards — naming conventions, partitioning strategies, cluster configurations, Spark tuning
• Drive performance optimization — AQE tuning, liquid clustering, broadcast joins, shuffle partition management
• Design Databricks cluster policies, autoscaling configurations, and cost optimization strategies
• Conduct root cause analysis on production incidents and implement permanent fixes
• Mentor junior and mid-level engineers through code reviews and pair programming
• Evaluate new technologies and recommend adoption

What We're Looking For

• Experience designing, developing, and optimizing ETL/ELT pipelines using Azure Data Factory and Databricks
• Proficiency with PySpark and Spark SQL notebooks for large-scale data transformation
• Ability to architect end-to-end data solutions across dev, UAT, and production using Unity Catalog
• Strong collaboration and communication skills to lead design discussions with client teams
• Experience with data contracts, schema governance, and high-volume pipeline design
• Expertise in Spark performance tuning
• Experience with Databricks cluster policies, autoscaling, and cost optimization
• Root cause analysis skills for production incidents and permanent fixes
• Mentorship of junior/mid-level engineers
• Willingness to evaluate and adopt new technologies

Nice to Have

• Familiarity with Delta Lake concepts and newer Delta features is a plus
• Prior experience in on-site client delivery under a dedicated talent model

Compensation & Benefits

• Not disclosed