Sr Analyst, Data Integration & Workflows

Job title: Sr Analyst, Data Integration & Workflows

About the Role The Senior Analyst, Data Integration & Workflows plays a critical role within the Data AI & Enablement organization, serving as a senior technical leader responsible for designing, implementing, and operationalizing production-grade data pipelines and workflow automation that power SPDJI's index and analytical solutions. This role combines hands-on technical expertise with leadership capabilities to drive delivery excellence, mentor technical talent, and ensure all data integration solutions meet enterprise standards for quality, reliability, and maintainability.

What You'll Do

• Lead complex data integration initiatives from design through production deployment, ensuring solutions are scalable, observable, and aligned with enterprise architecture standards
• Design and implement production-grade data pipelines (batch and streaming) that transform raw inputs into trusted curated outputs, incorporating robust error handling, validation, and reconciliation controls
• Establish and evangelize engineering best practices for ETL/ELT patterns, workflow orchestration, data quality controls, and operational observability across the team and value streams
• Drive technical decision-making for pipeline architecture, technology selection, and design patterns, balancing business requirements with technical feasibility and long-term maintainability
• Partner with PPD on technical planning and feasibility, providing realistic estimates, identifying technical dependencies, and shaping scope to ensure achievable delivery commitments
• Enablement & Co-Development: Lead hands-on enablement with value stream SMEs through pair programming, structured guidance, and co-development sessions—adapting approach based on SME technical capability
• Assess SME technical readiness and recommend appropriate engagement models (SME-led with review, co-development, or led build with validation)
• Build reusable automation components and templates (frameworks for ingestion, validation, transformation, publishing, backfills) that accelerate consistent delivery across domains
• Develop SME technical capabilities through targeted coaching, code reviews, and knowledge transfer, fostering a culture of engineering excellence and continuous learning
• Create and maintain technical documentation, including reference architectures, design patterns, coding standards, and implementation guides
• Quality Assurance & Production Readiness: Conduct comprehensive code reviews for SME-built and team-developed pipelines, ensuring adherence to standards for maintainability, testing, logging, data validation, and documentation
• Implement data reliability controls including validation rules, reconciliation checks, anomaly detection, and completeness/timeliness monitoring that protect downstream index processes
• Engineer observability and monitoring solutions by implementing logging standards, metrics, alerts, and runbooks that enable effective production support
• Prepare IT-ready handover artifacts including technical documentation, test evidence, operational procedures, and clear support boundaries
• Partner with IT during QA and deployment, resolving issues quickly and ensuring solutions meet enterprise standards for security, supportability, and operational excellence
• Operational Excellence & Continuous Improvement: Monitor operational metrics related to pipeline reliability, data quality, performance, and cost efficiency; drive continuous improvement initiatives
• Collaboration & Stakeholder Management: Collaborate with Data Integration Lead to shape team strategy, prioritize initiatives, and align technical approaches with organizational goals; Engage with Data Value Streams, Data Governance teams, Data Services & Strategy and IT for cross-functional alignment

What We're Looking For

• Demonstrated experience designing, implementing, and operating production-grade data pipelines (batch and streaming) with robust error handling, validation, and reconciliation
• Deep knowledge of ETL/ELT patterns, workflow orchestration, data quality controls, observability, and data reliability controls
• Experience enabling and coaching SME capabilities, pair programming, code reviews, and development of reusable automation components
• Proficiency in creating reference architectures, coding standards, and implementation guides; strong documentation skills
• Strong collaboration across Data Integration, AI Solutions, Data Governance, Data Services & Strategy, and IT; excellent communication and mentoring abilities
• Familiarity with metadata and lineage tracking and the ability to ensure proper visibility into data processing pipelines