Machine Learning Engineer II – San Francisco, CA

Position Overview

Zendesk is seeking a Machine Learning Engineer II specializing in GenAI & LLM Infrastructure to join the AI/ML Platform team in San Francisco (Hybrid). In this hands-on engineering role, you will design, implement, and improve critical components of our GenAI infrastructure—from our LLM Proxy to evaluation frameworks and orchestration tools—ensuring that our AI-driven products are safe, scalable, and developer-friendly.

Key Responsibilities

• Develop Zendesk’s LLM Proxy, ensuring secure and cost-optimized access to multiple foundation models.
• Build and maintain benchmarking and A/B testing frameworks to measure LLM performance, latency, and cost.
• Assist in creating orchestration systems that power multi-step, tool-using AI agents.
• Collaborate closely with applied ML, product, and platform teams to align infrastructure with product needs.
• Deliver well-tested, maintainable, and performant production-ready code.

Required Qualifications

• 2–4 years of hands-on experience developing and deploying ML systems or infrastructure.
• Familiarity with LLM applications, vector databases, or ML/AI infrastructure components.
• Exposure to AWS, GCP, or Azure and understanding of Kubernetes, Docker, or similar containerized environments.
• Proficiency in Python and experience with at least one other backend language (Java, Scala, Golang, or Ruby).
• Knowledge of CI/CD workflows, testing frameworks, and software design principles.
• Ability to work effectively in cross-functional teams and to take feedback constructively.

Preferred Qualifications

• Experience developing small-scale ML services or APIs.
• Exposure to ML observability tools and evaluation frameworks.
• Understanding of prompt engineering or fine-tuning workflows tailored for LLMs.

Benefits & Perks

• Full ownership of your projects with a huge impact on product evolution.
• Work alongside a team of passionate professionals in an inclusive environment.
• Opportunity to implement your own ideas, drive improvements, and grow your career.
• Compensation: The US annual base salary range is $110,000.00–$166,000.00, with potential bonus, benefits, and other incentives.
• Enjoy a Hybrid work model that combines onsite collaboration with remote work flexibility.