AI/ML Prompt Scientist/Engineer

Position Overview

STAND 8 provides end-to-end IT solutions to enterprise partners across the United States and globally. With offices in Los Angeles, New York, New Jersey, Atlanta, Mexico, India, and more, we are at the forefront of technology innovation. We are looking for an AI/ML Prompt Scientist/Engineer to design, evaluate, and optimize prompt-based systems that drive real-world impact in cutting-edge AI applications. This role is ideal for a Data Scientist or Machine Learning Engineer experienced in prompt optimization, orchestration frameworks, retrieval pipelines, and evaluation tooling. You will collaborate with AI Engineers and Data Scientists to build scalable, reliable, and safe AI systems.

Key Responsibilities

• Design, optimize, and evaluate prompts and orchestration pipelines to enhance LLM performance and scalability, supporting both data science and engineering workflows.
• Implement programmatic prompting using frameworks such as LangChain, LlamaIndex, DSPy, Guidance, and LangGraph.
• Define and manage evaluation frameworks, including A/B testing, human and automated scoring, and model quality assessments using tools like Ragas, UpTrain, DeepEval, TruLens, Promptfoo, and OpenAI Evals.
• Engineer and maintain context management systems, including conversation memory, retrieval logic, and context compression strategies (chunking, summarization, prioritization).
• Design and optimize retrieval and RAG pipelines by integrating vector databases (e.g., Weaviate) and rerankers (e.g., Cohere) to enhance relevance and accuracy.
• Establish observability and traceability mechanisms (e.g., Langfuse) to monitor performance metrics, latency, and operational costs.
• Implement guardrails, safety mechanisms, and bias mitigation policies using frameworks such as Guardrails AI, NeMo Guardrails, Outlines, and Rebuff.
• Collaborate with cross-functional teams—including Data Scientists, ML experts, and AI Engineers—to ensure high-quality, compliant, and well-documented system delivery.

Required Qualifications

• Proven experience in LLM and NLP model operations, including tokenization, context window management, and safety considerations.
• Strong technical expertise in prompt engineering, orchestration frameworks, and context management.
• Hands-on experience with retrieval-augmented generation, evaluation frameworks, and observability tooling.
• Sound understanding of evaluation metrics such as faithfulness, groundedness, coherence, toxicity, latency, and cost.
• Experience with compliance and ethical AI practices, including bias and hallucination mitigation.
• Excellent analytical, documentation, and communication skills with a proven ability to collaborate effectively across teams.

Preferred Qualifications

• Experience in model fine-tuning and alignment (e.g., LoRA, DPO/RLHF, preference tuning).
• Background in developing AI agents and integrating tool-use capabilities.

Benefits & Perks

• Compensation: Base range is $100,000 - $200,000 per year, depending on experience.
• Medical coverage and Health Savings Account (HSA) through Anthem.
• Dental, Vision, and various Ancillary coverages through Unum.
• 401(k) retirement savings plan.
• Paid-time-off options.
• Company-paid Employee Assistance Program (EAP) and discount programs through ADP WorkforceNow.