Senior Azure Databricks Engineer(ID: 4002)

As a Senior Azure Databricks Engineer, you will:

• Design, develop, and maintain scalable and reliable data processing solutions using Azure Databricks.

• Build and manage robust batch and streaming data pipelines within Databricks environments.

• Develop and optimize data transformation and processing solutions using Python, PySpark, and SQL.

• Design and optimize data models to support scalable processing, performance, and reliability.

• Manage multiple parallel data processing workflows and shared data sources efficiently.

• Implement and maintain CI/CD pipelines using Azure DevOps and YAML-based configurations.

• Apply Infrastructure as Code (IaC) using ARM/Bicep for deployment and infrastructure automation.

• Monitor, troubleshoot, and optimize data processing workloads and Databricks environments.

• Collaborate with cross-functional engineering and business teams to deliver reliable and maintainable data solutions.

• Contribute to Agile development practices and continuously improve engineering standards, system stability, and performance.

What You Bring to the Table:

• Strong hands-on experience with Azure Databricks as a core data engineering platform.

• Strong proficiency in Python, PySpark, and SQL.

• Hands-on experience developing batch and streaming data pipelines.

• Experience with data modeling, transformation, and optimization within Databricks environments.

• Good understanding of Azure cloud services relevant to data engineering.

• Experience with Azure DevOps, CI/CD, and YAML-based pipeline configurations.

• Hands-on experience with Infrastructure as Code, particularly ARM/Bicep.

• Experience working in Agile engineering and delivery environments.

• Understanding of modern cloud-based data architectures and end-to-end data engineering solutions.

• Strong communication, collaboration, troubleshooting, and problem-solving skills.

You Should Possess the Ability to:

• Build scalable, high-performance, and reliable data processing solutions using Azure Databricks.

• Develop efficient PySpark and SQL-based data transformations.

• Design and manage complex batch and streaming workloads.

• Optimize data pipelines, processing performance, and resource utilization.

• Troubleshoot complex data engineering issues and improve system reliability.

• Implement automated deployment and infrastructure management practices.

• Make pragmatic technical decisions while maintaining scalability and maintainability.

• Work effectively with engineering, architecture, and business stakeholders.

• Drive continuous improvement and maintain high standards of code and solution quality.

What We Bring to the Table:

• Opportunity to work on enterprise-scale Azure and Databricks data engineering initiatives.

• Exposure to modern cloud-based data platforms and engineering practices.

• A collaborative Agile environment focused on technical excellence and innovation.

• Opportunities to work with advanced data processing, pipeline engineering, and cloud technologies.

• Continuous learning and opportunities for technical and professional growth.

• A culture focused on quality, ownership, scalability, and sustainable engineering solutions.