Distinguished HPC Engineer
Datainscience, United States
What you will do
- Design and implement high-performance computing clusters to ensure they meet the required
performance, scalability, reliability and availability standards.
- Provide subject matter expertise in HPC storage, file systems and Linux to DST clients and
internal teams.
- Deploy and maintain cluster management software such as Bright Cluster Manager to
streamline administration and operations.
- Set up, configure, and troubleshoot HPC job schedulers like SLURM, LSF, or UGE to optimize
job scheduling and resource allocation.
- Install, configure and maintain containers, container applications and orchestration tools.
- Deploy and manage advanced health monitoring systems to ensure the HPC infrastructure is
operating efficiently and to preemptively identify issues.
- Automate software installations and system configurations using Ansible to ensure consistency
and efficiency across deployments.
- Stay up to date with the latest developments in HPC technologies and research new solutions to
improve system performance and capabilities.
- Provide Jira support to maintain and improve DST’s project and ticket management.
- Maintain clear and effective communication with clients, management, and internal teams to
ensure project goals and expectations are met. Requirements:
- Extensive experience with Linux operating systems, including system configuration,
troubleshooting, and performance tuning.
- Advanced expertise with parallel storage systems, InfiniBand, and Ethernet networking to
ensure high-performance data transfer and storage solutions.
- Proven expertise in designing and building HPC clusters from the ground up, including
hardware and software integration.
- Advanced knowledge and hands-on experience with Bright Cluster Manager, Slurm for job
scheduling, and Ansible for automation.
- Intermediate knowledge of GPUs and associated hardware/software, including configuration,
optimization, and debugging.
- Intermediate Jira skills
- Strong organizational skills to manage multiple projects efficiently and excellent
communication skills to interact effectively with various stakeholders