Software Engineer, Model Performance Tooling

Position Overview

Baseten powers mission-critical inference for the world's most dynamic AI companies. By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable companies to bring cutting-edge models into production. We are looking for early-career Software Engineers to join our team in Vancouver, BC in a specialized role at the intersection of high-performance computing (HPC) and Large Language Model (LLM) engineering. In this role, you will build the automated "speedometer and diagnostic" suite for our next-generation AI infrastructure – tearing models apart to understand their performance on the metal, measuring GPU FLOPS, stress-testing InfiniBand clusters, and creating benchmarks to ensure production-readiness.

Location: Vancouver, BC
Compensation: CA$130K - CA$200K
Experience Required: Early-career / Entry-level

This role offers a direct impact on defining performance standards, world-class expertise in GPU orchestration, and high ownership in a small team led by industry experts.

Key Responsibilities

• Performance Benchmarking: Run and automate standard LLM quality benchmarks (GSM8K, MMLU) alongside custom performance suites for specific workloads (e.g., long-context window, KV cache reuse).
• Infrastructure Validation: Create automated acceptance tests for new GPU clusters across x86 and ARM systems, measuring GPU memory bandwidth, networking throughput, and multi-node networking performance.
• Model Dev Experience: Develop and maintain internal GPU-enabled development environments (similar to GitHub Codespaces) to ensure seamless, high-performance setups optimized for model experimentation.
• Tool Development: Build and contribute to tools such as InferenceMAX and genai-bench to automate model evaluation and optimization.
• Deep Hardware Profiling: Use PyTorch Profiler and NVIDIA Nsight Systems to collect performance profiles, identify bottlenecks, and debug the NVIDIA compute/networking stack.
• Monitoring & Observability: Develop real-time dashboards and alerts to monitor system health, model startup times, and runtime performance.
• Continuous Integration: Automate performance testing via CI/CD pipelines to catch regressions before they hit production.
• Optimization Automation: Build tools to identify the optimal configuration (latency vs. cost vs. quality) for various models and workloads.

Required Qualifications

• A Love for Systems & Hardware: A keen interest in understanding GPU memory subsystems, InfiniBand, and data movement across clusters.
• An Automation Mindset: A belief that repetitive tasks should be scripted, with a passion for stress-testing and fuzzy testing to expose system limits.
• Mathematical Curiosity: A desire to grasp the underlying math of Transformers and its translation into FLOPs and memory requirements.
• Interest in Optimization: Excitement to explore quantization, speculative decoding, disaggregated serving, and kernel-level optimizations.
• Technical Toolkit: Familiarity with Python and an eagerness to master the NVIDIA software stack; C++ experience is a plus.

Benefits & Perks

• Competitive compensation, including meaningful equity.
• 100% coverage of medical, dental, and vision insurance for employee and dependents.
• Generous PTO policy including a company-wide Winter Break (offices closed from Christmas Eve to New Year’s Day!).
• Paid parental leave.
• Company-facilitated 401(k).
• Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities.