Tasks:
- * Analyze datasets and generate insights
- * Analyze requirements and implement application enhancements
- * Build and maintain distributed data processing applications
- * Build microservices and data ingestion services
- * Collaborate with product owners, analysts, business teams, and QA
- * Create high-level and low-level system designs
- * Design and develop scalable batch and streaming data pipelines
- * Develop data flow diagrams and interface specifications
- * Implement automated build, test, and deployment workflows
- * Perform application security analysis, testing, and debugging
- * Source, cleanse, transform, enrich, reconcile, and validate data
- * Support and troubleshoot application systems
- * Tune Spark performance, partitioning, and execution plans
- * Use AI developer tools for prototyping, code review, testing, and documentation
- * Write and optimize Spark SQL queries and transformations
Perks/Benefits:
Skills/Tech stack required:
[Apache Spark] [Automated testing] [Bitbucket] [CI/CD] [Data Analysis] [Dataframes] [Data Modeling] [Data pipeline] [Data Pipeline Development] [Datasets] [Devin AI] [Distributed Computing] [Git] [GitHub] [GitHub Copilot] [Harness] [Java] [Jenkins] [Performance Tuning] [Pipeline Development] [PySpark] [Python] [RDDs] [Scala] [Schema evolution] [Shell Scripting] [SonarQube] [Spark performance tuning] [Spark SQL] [Spring] [Spring Boot] [SQL] [Tableau] [Test-Driven] [Test-Driven Development] [Unix Shell] [Unix/Shell Scripting]
Educational requirements:
N/A
Role(s):
[Analyst] [Applications Development] [Applications Development Programmer] [Applications Development Programmer Analyst] [Big Data Engineer] [Data Engineer] [Developer] [Engineer] [Java] [Java Developer] [Programmer Analyst] [Scala Developer] [Senior Big Data Engineer]