Principal Data Engineer

Principal Data Engineer

Remote-US

What You’ll Do

• Lead Data Architecture and System Design

• Own the technical vision and architecture for scalable, resilient, and cost-effective data platforms and pipelines.

• Design end-to-end data systems spanning batch and streaming ingestion, transformation, data modeling, storage, orchestration, serving layers, and consumption patterns.

• Make and document high-quality architecture decisions, including tradeoffs involving performance, reliability, scalability, latency, security, maintainability, and cost.

• Define reusable architectural patterns for common data engineering challenges, including CDC, event-driven data flows, schema evolution, slowly changing dimensions, data quality, and multi-tenant or domain-oriented data models.

• Establish clear boundaries and interfaces between operational systems, data platforms, analytics layers, and downstream applications.

• Lead technical design reviews for high-impact initiatives and provide clear, constructive direction on implementation approaches.

• Identify architectural risks early and drive plans to mitigate technical debt, single points of failure, data inconsistency, and operational fragility.

• Set Engineering Standards and Best Practices

• Establish and continuously improve standards for data engineering development, testing, deployment, documentation, monitoring, incident response, and lifecycle management.

• Define best practices for version-controlled data transformations, infrastructure as code, CI/CD, code review, automated testing, and environment promotion.

• Build a culture of data reliability by implementing standards for data contracts, schema management, lineage, observability, freshness, completeness, accuracy, and reconciliation.

• Champion security, privacy, access control, governance, retention, and compliance requirements throughout the data lifecycle.

• Develop practical engineering guardrails that enable teams to move quickly without sacrificing quality, trust, or operational excellence.

• Drive adoption of well-defined SLIs, SLOs, alerting, runbooks, and post-incident learning practices for critical data products.

• Drive Technical Decision-Making

• Serve as a trusted technical authority for complex, cross-functional decisions involving data architecture, tooling, platform capabilities, and engineering investments.

• Facilitate technical discussions that turn ambiguous requirements into durable, well-reasoned solutions.

• Evaluate build-versus-buy decisions, platform technologies, and vendor capabilities using clear criteria and documented recommendations.

• Align architecture decisions with organizational priorities, business outcomes, data strategy, and long-term platform sustainability.

• Create and maintain architecture decision records, reference architectures, roadmaps, and technical standards that provide clarity across teams.

• Influence leaders and stakeholders through concise communication of technical options, risks, tradeoffs, and recommended paths forward.

• Build and Improve Data Products

• Lead the design and development of high-value, production-grade data pipelines, datasets, and platform capabilities.

• Partner with data analysts, data scientists, software engineers, and product teams to deliver well-modeled, discoverable, and trustworthy data products.

• Optimize data systems for reliability, usability, performance, scalability, and cost efficiency.

• Drive modernization initiatives, including migration from legacy platforms, consolidation of fragmented pipelines, and reduction of manual operational processes.

• Ensure data products are designed with clear ownership, documentation, lineage, quality expectations, and support models.

• Mentor and Multiply Engineering Excellence

• Mentor data engineers at all levels, with a particular focus on growing senior engineers into strong technical leaders.

• Provide practical coaching on system design, data modeling, debugging, performance tuning, code quality, and production operations.

• Lead architecture reviews, technical workshops, design sessions, and knowledge-sharing forums.

• Establish reusable templates, reference implementations, and playbooks that improve consistency and accelerate delivery.

• Foster an inclusive, high-accountability engineering culture centered on thoughtful collaboration, continuous learning, and ownership.

• Act as a technical multiplier: helping teams make better decisions independently while maintaining alignment with shared architectural principles.

• What you bring

• Required

• 10+ years of experience in software engineering, data engineering, platform engineering, or a related technical discipline.

• 5+ years designing and operating large-scale data platforms, distributed data systems, or enterprise-grade data products.

• Demonstrated expertise in end-to-end system design, including defining requirements, evaluating tradeoffs, designing resilient architectures, and delivering production systems.

• Deep experience with modern data architecture concepts, such as lakehouse or warehouse platforms, batch and streaming processing, data modeling, orchestration, metadata management, and data governance.

• Strong proficiency in SQL and at least one general-purpose programming language, such as Python, Java, Scala, or similar.

• Experience with distributed data processing frameworks and technologies such as Spark, Kafka, Flink, Snowflake, Databricks, BigQuery, Redshift, or equivalent platforms.

• Strong knowledge of data modeling practices across analytical, operational, and domain-oriented use cases.

• Experience establishing engineering standards for testing, CI/CD, observability, infrastructure as code, reliability, and security.

• Proven ability to lead technical decisions across multiple teams without relying on direct authority.

• Exceptional written and verbal communication skills, including the ability to explain complex technical concepts and tradeoffs to technical and non-technical stakeholders.

• Demonstrated experience mentoring engineers and raising technical standards across an organization.

• Preferred

• Experience with cloud-native data platforms on AWS, Azure, or GCP.

• Experience implementing data mesh, domain-oriented data product, or self-service platform patterns.

• Familiarity with data governance, cataloging, lineage, privacy, and access-control tooling.

• Experience supporting financial, revenue, operational, customer, or enterprise analytics domains.

• Experience with ML data pipelines, feature stores, or AI/LLM data foundations.

• Experience managing major data-platform migrations, modernization programs, or large-scale architectural transformations.

• Leadership Competencies

• Systems thinker: Connects individual technical decisions to broader platform, organizational, and business outcomes.

• Pragmatic architect: Balances ideal-state architecture with incremental delivery and operational realities.

• Technical judgment: Makes clear recommendations amid ambiguity and communicates tradeoffs transparently.

• Builder and operator: Designs for production from the beginning, including reliability, security, observability, and supportability.

• Mentor and multiplier: Develops others and creates scalable mechanisms for technical excellence.

• Collaborative influencer: Builds alignment across engineering, analytics, security, product, and business teams.

• Our Technology Stack

• Languages & Databases: Python, SQL, MySQL, NoSQL databases

• Orchestration & Transformation: Airflow, dbt

• Data Platforms: Snowflake, AWS, GCP, BigQuery, Redshift

• Data Integration: Fivetran, c