Tasks:
- * Build cache architectures across GPU host memory, NVMe SSDs, and remote parallel file systems
- * Design and develop distributed multi-tier cache acceleration systems
- * Design intelligent data placement and lifecycle management policies
- * Design metadata management systems
- * Develop large-scale distributed file cache systems for AI training clusters
- * Evaluate and deploy high-performance storage hardware and software technologies
- * Implement access-hotness-based tiered scheduling
- * Implement efficient metadata path lookup and caching
- * Implement TTL/LRU eviction and data prefetching
- * Optimize end to end data paths
- * Reduce GPU stalls caused by storage I/O
Perks/Benefits:
Skills/Tech stack required:
[AI coding] [AI coding tools] [Burst Buffer] [C#] [C++] [Coding Tools] [Consistency] [Data placement] [Data prefetching] [Direct Storage] [Distributed Caching] [Distributed file systems] [Distributed Storage] [Distributed Storage Systems] [Distributed Systems] [Fault Tolerance] [Filesystem Development] [File systems] [Golang] [GPU Direct] [GPU direct storage] [Lifecycle Management] [Linux storage] [Linux Storage Stack] [LRU] [NVMe-oF] [NVME SSD] [Parallel File Systems] [PyTorch Dataloader] [PyTorch Dataset] [RDMA] [Replication] [Rust] [SGLang] [Sharding] [Storage stack] [Storage Systems] [TTL] [User space] [User-space filesystems] [VLLM]
Educational requirements:
[Bachelor's Degree]
Role(s):
[Distributed Storage Engineer] [Engineer] [Storage Engineer] [Storage Systems Engineer] [Systems Engineer]