Own end-to-end lakehouse implementation and collaborate with the Data team on platform architecture.
Build ingestion patterns for APIs, public-source text, and external databases.
Resolve product identities across retailers and dates while preserving source-level traceability.
Consolidate existing food-enrichment pipelines into a unified pipeline.
Build connectors that deliver processed food data to WISE Intel and the consumer application.
Maintain fail-fast configuration practices and ensure missing settings produce clear errors.
Establish ingestion capabilities and data-quality reporting for Data Assembly readiness.
Write concise, dated architectural decision records before implementation.
Requirements:
3+ years of experience in a similar software, data, or platform engineering role.
Experience designing, shipping, and operating a lakehouse or data warehouse in production for at least one year.
Native proficiency in Python and SQL.
Experience with DuckDB, DuckLake, S3 Parquet, Postgres, and DynamoDB.
Experience with Prefect and Kubernetes or Fargate job runners; familiarity with dbt, Dagster, or Airflow is preferred.
Experience treating data identity and entity resolution as core architectural concerns.
Experience operating batch pipelines that use paid cloud or third-party APIs with cost and budget controls.
Experience collaborating daily with engineers in a fast-paced, ambiguous environment.
Experience with AWS services including S3, ECS, Lambda, EventBridge, SNS, Terraform, and IAM; GitHub Actions experience.
Preferred: experience comparing embedded query engines such as DuckDB with distributed systems such as Spark, Databricks, or Snowflake, plus data acquisition methods such as web scraping or OCR APIs.
Benefits:
Competitive base salary, bonus, and meaningful equity.
Fully remote work.
Unlimited PTO and 10 paid holidays.
401(k) with an immediately vested 4% company match.
Medical, dental, vision, life, and ancillary coverage, with employees covered at 100% and dependents at 80%.
Employer-paid short- and long-term disability, identity theft, and life insurance.
FSA and HSA plans, including employer HSA contributions of $200/month for employee plans and $400/month for employee-plus-dependent plans.
Tags
Carta de presentación
Inicia sesión para generar una carta de presentación para esta vacante.