Senior Classified HPC Engineer

Position Overview

We are hiring a Senior Linux HPC Systems Engineer to design, operate, and maintain clusters, servers, and workstations supporting the critical services where science happens at ORNL. This role is part of the Emerging Technologies & Computing team within the Research Computing group in the Information Technology Services Directorate. You will help design, deploy, optimize, monitor, and provide tooling support for various clustered infrastructures that drive ORNL’s scientific breakthroughs.

Key Responsibilities

• Advocate and promote HPC and clustered computing services to researchers processing large datasets or developing code.
• Ensure the availability, performance, scalability, and security of production systems.
• Leverage automation and monitoring solutions to minimize day-to-day maintenance and optimize system performance.
• Collaborate with technical points-of-contact to install and fine-tune various scientific toolsets.
• Optimize workflows and monitoring solutions to support a 24/7 operations staff, reducing off-hours support.
• Deliver ORNL’s mission by aligning actions with core values: Impact, Integrity, Teamwork, Safety, and Service.

Required Qualifications

• BS degree in computer science, computer engineering, information technology, or a related discipline with 8-12 years of aligned professional experience (or equivalent combination of education and experience).
• Masters degree candidates require 7-10 years and PhD candidates require 4-6 years of relevant experience.
• At least 5 years managing UNIX/Linux systems.
• Minimum of 3 years using configuration management and automation tools such as Git, Jenkins, Ansible, or Puppet.
• Moderate proficiency in a scripting language (e.g., Bash or Python).
• Experience in advanced troubleshooting and system administration with Linux servers.
• Experience supporting large data systems.
• Strong desire to innovate and communicate the benefits of new technologies.
• A collaborative and upbeat approach to build trust and credibility with research teams.

Preferred Qualifications

• Active DOE Q, active DOD Top Secret, or active DOD TS/SCI clearance is heavily preferred.
• Solid understanding of multiple operating systems and cluster technologies.
• Experience with Rocky/CentOS/RHEL, Ubuntu, and VMware.
• Familiarity with HPC platforms supporting SLURM job submissions and troubleshooting.
• Experience building and running containerized applications in an HPC environment.
• Experience with deployment mechanisms such as Diskless, Warewulf, and traditional methods (e.g., Cobbler, PXEboot, Bright).
• Experience managing GPU (NVIDIA and AMD) clusters for AI/ML or image processing.
• Knowledge of networking fundamentals including TCP/IP, traffic analysis, and network diagnostics.
• Experience with Infiniband networks and diagnostics.
• Extensive experience with high-performance parallel file systems (e.g., Lustre, WEKA, GPFS).
• Familiarity with performance and diagnostic tools for systems, networking, and storage benchmarking.
• Experience with monitoring systems such as Grafana, CheckMK, Nagios, Zabbix, SolarWinds, or Ganglia.
• Previous experience in a government, scientific, or highly technical environment.
• Strong documentation skills, including the ability to prepare simple documentation web pages.

Benefits & Perks

• Competitive Pay and comprehensive benefits including medical and retirement plans.
• Flexible work hours and employee amenities such as on-site fitness, banking, and cafeteria facilities.
• Additional benefits include Prescription Drug, Dental, and Vision Plans, 401(k) Retirement Plan, Contributory Pension, Life Insurance, Disability Benefits, Generous Vacation and Holidays, Parental Leave, Legal Insurance with Identity Theft Protection, Employee Assistance, Flexible Spending Accounts, Health Savings Accounts, Wellness Programs, Educational Assistance, Relocation Assistance, and Employee Discounts.