Machine Learning, Platform Engineer

Position Overview

Our team focuses on enabling custom models and dedicated inference on Together. We are responsible for building a container platform, optimizing autoscaling, minimizing cold starts, achieving the best end-to-end model performance, and providing a best-in-class developer experience with great tooling. We work on video or audio generation across the stack – from CUDA kernels and PyTorch optimization to inference engines, container orchestration, and queueing theory. An ideal candidate will excel in profiling and optimization, possess solid knowledge of Kubernetes and multi-cluster scheduling, and understand ML bottlenecks.

Key Responsibilities

• Work on multi-cluster orchestration, portfolio optimization, predictive autoscaling, control panes, model bring-up, model optimization, APIs for managing deployments, inference worker SDKs, and CLI tools.
• Analyze and improve the robustness and scalability of existing distributed systems, APIs, databases, and infrastructure.
• Partner with product teams to understand functional requirements and deliver solutions that meet business needs.
• Write clear, well-tested, and maintainable software and Infrastructure as Code (IaC) for both new and existing systems.
• Conduct design and code reviews, create developer documentation, and develop testing strategies for fault tolerance and robustness.

Required Qualifications

• 5+ years of demonstrated experience in building large scale, fault tolerant, distributed systems.
• Experience running serverless inference platforms, handling model bring-up on short notice, being on call, or operating a cloud provider is a very big plus.
• Ability to thoughtfully discuss past experiences and learnings from system failures.
• Experience in designing, analyzing, and improving efficiency, scalability, and stability of system resources.
• Excellent understanding of low-level operating system concepts including concurrency, networking, storage, performance, and scale.
• Expert-level programming in one or more of Python, Golang, Rust, C++, or Haskell.
• Proficiency in writing and maintaining Infrastructure as Code (IaC) using tools like Terraform.
• Experience with Kubernetes internals or other container orchestration systems.
• Sound judgment on when to use—and refrain from using—LLMs for code.
• Bachelor's or Master's degree in Computer Science, Computer Engineering, or a related field, or equivalent practical experience.
• Experience in writing-heavy roles or companies is a plus.

Benefits & Perks

• Compensation: Competitive base salary ranging from $160,000 to $250,000, plus equity and benefits.
• Equity: Startup equity is offered.
• Health Insurance: Comprehensive health insurance plan provided.
• Other Benefits: Additional competitive benefits tailored for our team.

About Together AI

Together AI is a research-driven artificial intelligence company committed to open and transparent AI systems. Our mission is to significantly lower the cost of modern AI systems by co-designing software, hardware, algorithms, and models. We have contributed to leading open-source research, models, and datasets and have been behind innovations such as FlashAttention, Hyena, FlexGen, and RedPajama. Join our passionate group of researchers and engineers as we build next-generation AI infrastructure.

Equal Opportunity

Together AI is an Equal Opportunity Employer and proudly offers equal employment opportunity to all candidates regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more. For more details, please review our privacy policy at https://www.together.ai/privacy.