Tasks:
- * Build AI-powered data quality diagnosis and data discovery capabilities
- * Build and maintain batch and streaming data pipelines
- * Develop layered data models and standardized metrics
- * Evolve Iceberg lakehouse architecture
- * Improve CI workflows, documentation, data quality, and alerting
- * Optimize query performance and compute costs
Perks/Benefits:
- + AI-first working methodology
- + Collaborative engineering culture
- + High impact ownership
- + Modern data stack experience
- + Rapid feedback
- + Shared best practices and SOPs
Skills/Tech stack required:
[Apache Iceberg] [Apache Spark] [AWS Glue] [Batch Processing] [CI/CD] [ClickHouse] [Dagster] [Data Backfilling] [Data Lake] [Data Lake Architecture] [Data Modeling] [Data Quality] [Data quality monitoring] [DBT] [Distributed Computing] [Go] [Idempotency] [Incremental loading] [Java] [Metric Standardization] [Python] [Quality monitoring] [S3-compatible] [S3-compatible storage] [SQL] [Stream processing]
Educational requirements:
N/A
Role(s):
[Data Engineer] [Engineer]