About this role
About the Role We are the Lakeflow Engineering team, responsible for Databricks ETL product line: Jobs, Spark Declarative Pipelines, and Genie Code for Data Engineering. We run one of the world's biggest data engineering platforms - processing exabytes of data daily for tens of thousands of customers. We're seeking a dedicated Engineering Leader to spearhead the Pipelines Engine engineering team. The team builds next-generation Runtime ETL features and ensures Lakeflow Pipelines has state-of-the-art performance for ETL workloads. You will also lead the Agentic Data Engineering infrastructure, building the next generation engine to power agentic pipeline authoring, execution and maintenance.
What You'll Do
- Lead an engineering team building the next-generation ETL features for the Databricks Lakeflow platform.
- Oversee sustained recruitment of top-tier talent and upskilling of the team.
- Build processes to implement product vision and strategy according to organizational goals and priorities.
- Build software that is high quality and easy to operate.
- Manage technical debt, including long-term architecture decisions and balancing the product roadmap.
What We're Looking For
- Minimum 3 years of experience managing top-tier engineering teams.
- 5+ years experience building data infrastructure systems such as Apache Spark or database internals.
- Passion for database systems, storage systems, distributed systems, or performance optimization.
- Experience working with product management and directly with customers; ability to understand customer needs.
- Ability to ensure the team builds high quality and reliable infrastructure services.
- Experience being responsible for testing, quality, and SLAs of a product.
- Experience building and managing teams in a complex technical domain, such as distributed data systems or database internals.
- Expertise in attracting, hiring and coaching engineers who meet Databricks hiring standards.
- Experience up-leveling teams via hiring top-notch talent and growing existing team members.
Compensation & Benefits
- Local Pay Range: $190,000—$261,250 USD
- The total compensation may include annual performance bonus, equity, and benefits as listed.
- Compliance with Databricks' commitment to diversity and inclusion is part of the culture.