Databricks SME (Remote)

About the Role GovCIO is hiring a Databricks SME to support the VA Data Modernization initiative. In this role, the Databricks SME will guide data scientists, analysts, infrastructure specialists and business SMEs to deliver production-ready data solutions. The role focuses on workspace automation, onboarding support, data platform usability, infrastructure provisioning, and white-glove customer success to ensure VA users can rapidly adopt and leverage modern data tools. This position is fully remote within the United States. What You'll Do

• Implement and manage linked services, mount points, and catalog configuration within Databricks and VA’s VHA Data Lake.
• Develop and optimize PySpark (Python/Scala) jobs, notebooks, and workflows in Databricks.
• Build ETL/ELT pipelines using Azure Databricks Lakehouse features.
• Design and implement Spark-based data processing for analytics workloads.
• Optimize Databricks Spark jobs for performance, cost, and scalability (partitioning, caching, tuning, etc.).
• Collaborate with data scientists, analysts, infrastructure specialists and business SMEs to deliver production-ready data solutions.
• Experience with Delta Lake and Synapse.
• Ensure data quality, governance, and security best practices.
• Act as subject matter expert to workgroups from start to finish in migration to the cloud.
• Help workgroups set up their resources and migrate their workloads.
• Support automated data warehouse access provisioning for eligible users, ensuring seamless integration with VA Cloud Data Warehouse (CDW) systems.
• Provide white‑glove customer success services, including onboarding sessions, office hours, rapid async support, and creation of reusable templates, guides, and best‑practice documentation.
• Collaborate with Customer Success, Business Analysts, and Platform Engineering teams to refine intake processes, improve user satisfaction, and standardize workspace deployment.
• Develop migration guidance and self‑service resources to accelerate platform adoption and reduce user friction when transitioning from legacy data systems.
• Troubleshoot user issues related to data access, workspace configuration, pipelines, cataloging, or permissions, escalating to engineering teams when needed. What We're Looking For
• Bachelor’s degree in Information Technology or a related field. (or commensurate experience)
• 12+ years of experience in business analysis, project management, or a similar role.
• Strong experience with Databricks and Azure data services, including workspace administration, catalog creation, linked services, storage mounts, and access configuration.
• Proficiency in data engineering fundamentals such as ETL/ELT pipeline development, data modeling, and managing structured and unstructured datasets.
• Ability to automate provisioning workflows and infrastructure using scripting or infrastructure‑as‑code tools (Python, PowerShell, Bash, Terraform, ARM, or Bicep).
• Strong troubleshooting and problem‑solving abilities for diagnosing platform, data access, and pipeline issues and collaborating across teams for resolution.
• Effective communication skills with the ability to document technical processes, create reusable templates, and support users through onboarding, office hours, and white‑glove assistance.
• Clearance: Public Trust required or ability to obtain. Nice to Have
• Experience with Data Analysis, Data Curation, Data Visualization (PowerBi), Python, Azure Synapse/Azure Data Factory, Azure Data Lake Storage, SQL Server.
• Certifications: Azure Databricks and Spark for Data Engineers, PySpark and SQL, Microsoft Azure Cert, SQL, SQLlite, Python Pandas, Sybase/SAP PowerDesigner, Databricks DAB, Databricks Unity Catalog, and Databricks lakflow pipelines.
• Exposure to Starburst, PowerBI, SSRS, SSIS and Unity Catalog.
• Familiarity with modern data lakehouse architectures, distributed data systems, and cloud data engineering practices.
• Strong understanding of Azure data services, including storage, compute, identity and access management, and DevOps workflows.
• Experience with workflow automation, CI/CD pipelines, or infrastructure‑as‑code (Terraform, ARM, Bicep).
• Background in user‑facing or customer‑success‑adjacent work such as onboarding, training, communications, or technical support.
• Ability to translate business needs into technical requirements and develop scalable solutions that improve platform efficiency.
• Knowledge of enterprise data systems common to healthcare or federal environments, especially data governance and data‑security considerations.
• Experience with Jira, ServiceNow, or SharePoint for intake, change management, and support processes. Compensation & Benefits
• Salary: USD 175,000 - 195,000 per year.
• This role is fully remote within the United States.