Member of Technical Staff
true-talent-solutions-partners
- $200,000–$300,000 per year
- Onsite
- Full Time
About the role
About the Role
Join a small AI infrastructure team building efficient inference systems for open-source language models. You will optimize models and serving infrastructure across diverse hardware, helping improve performance, reliability, and cost per token.
What You'll Do
Ship support for new open-source models, tuning them for latency and throughput.
Optimize inference serving with techniques such as batching, KV caching, speculative decoding, and quantization.
Write and tune kernels using CUDA, HIP, and Triton across a range of accelerator platforms.
Design, deploy, and operate heterogeneous clusters across infrastructure vendors.
Run production inference across a mixed hardware fleet, improving reliability, observability, and cost efficiency.
Work with customers to understand technical needs and solve infrastructure problems.
What We're Looking For
Typically 1 to 6 years of experience in backend, infrastructure, systems, or low-level engineering, including experience at a technology company or early-stage startup.
Strong computer science fundamentals, including computer architecture, memory hierarchy, and operating system internals.
Experience owning a challenging system end to end, notable technical achievements, or evidence of rapid growth.
Systems engineering experience beyond application-level or full-stack development, with a non-research-focused background.
Familiarity with CUDA, Triton, or GPU programming; experience with HIP, TPU, or AWS Trainium is also relevant.
Knowledge of inference optimization techniques, such as quantization, batching, KV caching, and speculative decoding.
Experience deploying or operating infrastructure at scale, such as clusters or GPU fleets.
Experience using AI agents and coding tools to improve engineering workflows, and comfort communicating with customers.
Strong internships or equivalent hands-on experience are welcome, including for recent graduates.
Compensation & Benefits
Base salary: $200,000 to $300,000 annually. Visa sponsorship is available.
Location
On-site in San Francisco, California, five days per week.