Software Engineer, Inference at Thinking Machines Lab – San Francisco or New York

Looking for software engineer inference jobs in the USA with visa sponsorship? Thinking Machines Lab is hiring a Software Engineer, Inference in San Francisco or New York.

Photo: Programming on laptop via Wikimedia Commons (Flickr), licensed CC BY-SA.

Software Engineer Inference Jobs in the USA: Overview

About the Employer

Thinking Machines Lab is an AI research and product company building systems that extend human will and judgment. It trains frontier models (Inkling), develops Tinker for custom model fine-tuning, and focuses on infrastructure quality for research and production.

Job Responsibilities

• Operate and scale production inference systems that serve live traffic, including Tinker’s multi-tenant serving platform
• Own the rollout process for new models, model versions and inference optimisations
• Build and improve observability, alerting and capacity planning
• Partner with inference and research teams to productionise new serving techniques
• Lead incident response for production inference issues and drive root cause analysis
• Design for graceful degradation, failover and redundancy
• Manage capacity and cost tradeoffs as traffic and model sizes scale

Requirements

• Experience operating large-scale, latency-sensitive production systems
• Proficiency in Python and Go or another systems language
• Experience with observability, monitoring and incident response
• Strong understanding of distributed systems and how they fail at scale
• Preferred: experience running production inference for large language models, deployment systems, capacity planning for GPU/TPU, familiarity with batching/caching/quantization

Visa Sponsorship and International Applicants

Thinking Machines Lab states: We sponsor visas. While we cannot guarantee success for every candidate or role, if you are the right fit we are committed to working through the visa process together. Relocation support is available as needed. This is an on-site role in San Francisco or New York.

Salary and Benefits

• Generous health, dental and vision benefits
• Unlimited PTO
• Paid parental leave
• Relocation support as needed

How to Apply

Apply directly via Ashby. A CV is required.

Apply to this Job

Application Tips

• Highlight experience with large-scale latency-sensitive systems and incident response
• Mention any production LLM inference or GPU infrastructure work

Browse more Technology Jobs and Visa Sponsorship Jobs on CMAJobs.