Senior Data Engineer
Location:
Remote, USA (candidates in PST, MT, or CST preferred)
Duration:
4-month contract
Pay Range:$86–91/hr
Schedule:
9:00 AM – 6:00 PM with flexibility
Summary
Our client's Core Data team builds and scales the data platform and analytics capabilities that power decision-making across the organization. Within Core Data, the Vehicle Catalog team owns the vehicle domain — ingesting, curating, and organizing vehicle data for analytics, ML, and BI use cases relied on by hundreds of stakeholders across dozens of domain teams. The team is migrating the Vehicle Catalog to a new Databricks account while simultaneously building out a new vehicle catalog on that platform. As a Senior Data Engineer, you will help refactor and move existing ETL/ELT and analytics pipelines into the new environment, migrate the underlying data, and validate that everything lands correctly. This is hands-on migration work alongside the engineering team on well-defined pipelines with clear acceptance criteria, with related migration requests — including dashboard and reporting migrations — picked up as they arise.
Job Responsibilities
In this role, you will manage and support data migration projects within the Core Data department. Key responsibilities include:
- Working with a Staff Data Engineer to understand the migration plan, sequence work against it, and provide regular progress updates.
- Refactoring existing ETL/ELT code written in PySpark, DABs, and dbt from the current Databricks environment to meet the patterns and standards of the new account.
- Migrating data pipelines and their underlying datasets, including backfills, and validating parity between source and target.
- Building and running reconciliation checks — row counts, schema conformance, freshness, completeness — to confirm migrated data matches the legacy environment.
- Migrating dashboards and downstream reporting assets to point at the new datasets, coordinating with the analysts who own them.
- Updating orchestration, scheduling, alerting, and runbooks so migrated pipelines are supportable after cutover.
- Troubleshooting pipeline failures and data discrepancies during and after cutover.
- Documenting what was migrated, what changed, and anything the team needs to know to own it going forward.
- Taking on adjacent migration and data engineering requests as the program evolves.
Essential Job Duties and Job Functions
- 6+ years of experience in Data Engineering, Analytics Engineering, or Software Engineering working with production data systems.
- Strong expertise in SQL and Python for building scalable data pipelines and transformations.
- Hands-on experience building ELT pipelines using modern cloud data platforms (Databricks experience required).
- Experience building robust testing frameworks for data migration (completeness, quality, etc.).
- Deep experience with dbt, including model development, testing, documentation, and CI/CD integration.
- Experience designing and operating production-grade data pipelines with monitoring and observability.
- Prior data or platform migration experience.
Knowledge and Skills
- Strong familiarity with AI development tools such as Claude Code or Devin for the software development process.
- Strong collaboration skills and ability to partner with engineering teams, analysts, and operational stakeholders.
- Familiarity with Fivetran connector management and ingestion architecture (nice-to-have).
- Experience with data contracts and schema governance (nice-to-have).
- Knowledge of streaming or near-real-time data pipelines (nice-to-have).
Education and Experience
Bachelor's or Master's degree in Computer Science, Engineering, Mathematics, or a related field, or equivalent practical experience.
...