Lead Data Engineer

About the Role The Lead Data Engineer will drive the design, development, and optimization of scalable data pipelines and ETL/ELT workflows using Azure Databricks, PySpark, SQL, and Delta Lake. You will lead a team of data engineers, mentor best practices, and collaborate with stakeholders to deliver secure, high-quality data solutions for donors and patients. What You'll Do

• Lead design, development, and optimization of scalable data pipelines and ETL/ELT workflows using Azure Databricks, PySpark, SQL, and Delta Lake
• Enhancement of CI/CD pipelines and Azure DevOps, Git, and automated deployment frameworks to streamline development and release processes
• Ensure data quality and consistency through data integration, transformation, and cleansing
• Familiarity with security and governance best practices using Unity Catalog, Azure RBAC, managed identities, and data access controls to ensure compliance and secure data management is preferred
• Responsibility includes leading a team of data engineers and mentoring in best practices
• Utilize Python for data manipulation, automation, and custom data solutions
• Collaborate with stakeholders to understand data needs and translate them into technical solutions
• Participate in on-call rotation and assist with after-hours issues, as required
• Perform other duties as assigned What We're Looking For
• Bachelor’s degree in computer science or information systems science, or equivalent experience
• Three (3) years experience with Azure Databricks platform
• Five (5) years of comprehensive data warehouse analysis and design experience, with full knowledge of data warehouse methodologies, data modeling and familiarity with operational data stores
• Five (5) years of experience with ETL/ELT Development, Python, PySpark, SQL, and Terraform experience
• Databricks Data Engineer Professional or Databricks Data Engineer Associate Certification preferred
• Ability to adapt to changing priorities and respond to problems creatively
• Ability to assist clients with migrating from legacy to modern data platform
• Proficiency with technical, conceptual, interpersonal, written, and verbal communication skills
• Knowledge of general business, quality and regulatory requirements preferred
• Familiarity with Power BI and MicroStrategy architecture is a plus Nice to Have
• Familiarity with Power BI and MicroStrategy architecture is a plus Compensation & Benefits
• Formal training
• Outstanding plans for medical, dental, and vision insurance
• Health savings account (HSA)
• Flexible spending account (FSA)
• Tuition Reimbursement
• Employee assistance program (EAP)
• Wellness program
• 401k retirement plan
• Paid time off
• Company paid holidays
• Personal time