About this role
Data Architect + AI
Overall Stack: Databricks Lakehouse Platform, Apache Spark, Delta Lake, Unity Catalog, and modern cloud data architecture
Must Have
- Lakehouse & Medallion Architecture: Expertise in designing end-to-end data architectures (Bronze, Silver, Gold layers) for reliable, production-ready pipelines
- Databricks & Spark Internals: Deep understanding of distributed computing. They must know how to troubleshoot and tune large-scale Spark jobs using caching, partitioning, and broadcast joins
- Delta Lake: Must understand ACID transactions, schema enforcement, time travel, and optimization operations like Z-ordering
- Unity Catalog & Data Governance: Proven ability to design unified governance models for data and AI assets, including role-based access control (RBAC), row/column-level security, and data lineage
- Cloud Infrastructure (AWS, Azure, or GCP): Strong grasp of the native cloud ecosystem they work in (e.g., ADLS/Entra for Azure, S3/IAM for AWS), including Virtual Network (VNet) setups and IAM roles
- Coding Proficiency: Advanced SQL skills and fluency in Python or Scala
- Cost Optimization & Performance Tuning: Ability to monitor DBUs (Databricks Units), right-size serverless and multi-node clusters, and implement best practices for avoiding cloud bill shock
Nice to Have
- Databricks Certifications: Candidates holding valid Databricks Certified Data Architect or Databricks Certified Data Engineer Professional badges generally have a proven, up-to-date baseline of the platform's features
- Generative AI & MLflow Integration: Experience building, deploying, and monitoring GenAI applications and ML models using Databricks Model Serving, Vector Search, and the Mosaic AI suite
- CI/CD & DevOps Practices: Experience automating Databricks workflows using Git (Databricks Repos) and orchestration tools like dbt, Azure Data Factory, or Apache Airflow
- Streaming Data: Familiarity with Databricks Structured Streaming and Auto Loader for real-time data ingestion and processing
- Data Warehousing & BI: Understanding of Databricks SQL, Serverless Warehouses, and integration with downstream BI tools like Power BI
Remote
Adavenced english
Client
Engineering
Estado remoto
Completamente remoto
Cliente
Techmahindra
We help our clients enhance their talent attraction capacities, especially in technological profiles.
We constantly innovate and actively seek to find the best solutions for clients and professionals. We understand the needs of our customers and aim to be the industry specialists.
We offer consulting services to technology companies in various areas, including IT, software development, cybersecurity, and project management. Our employees are the reason for the company's existence, and their satisfaction translates into that of our customers.
Fundada en
2016
Compañeros
24
Engineering
Guadalajara
Completamente remoto
Cargando formulario de solicitud