Tasks:
- * Accelerate data loading and preprocessing
- * Build training performance monitoring systems
- * Design and optimize large-model training frameworks
- * Optimize memory and GPU utilization
- * Optimize parallelism strategies and communication-computation overlap
- * Profile and tune training performance
- * Resolve distributed training communication bottlenecks
Perks/Benefits:
Skills/Tech stack required:
[C++] [CI/CD] [Data parallelism] [DeepSpeed] [Distributed Training] [Expert parallelism] [GPU memory] [GPU Memory Optimization] [High Performance] [High-Performance Computing] [Infiniband] [Megatron-LM] [Memory Optimization] [NCCL profiling] [Nsight] [Performance Computing] [Performance Profiling] [Pipeline parallelism] [Python] [PyTorch] [RoCEv2] [Tensor Parallelism]
Educational requirements:
N/A
Role(s):
[AI Infrastructure Engineer] [Deep Learning Systems Engineer] [Distributed Training Engineer] [Engineer] [Framework Engineer] [Infrastructure Engineer] [Learning Systems Engineer] [Machine Learning Framework Engineer] [Systems Engineer] [Training Engineer]