New 
Principal Software Engineer
 Microsoft | |
 United States, Washington, Redmond | |
 Aug 19, 2025 | |
|
OverviewWe are the AI Frameworks team at Microsoft, part of the CoreAI organization, and we are at the forefront of driving innovation in large-scale AI. Our mission is to enable state-of-the-art large language model (LLM) training and inference through deep optimization across the full software and hardware stack. We work across PyTorch, ONNX, and other open AI frameworks, pushing the boundaries of performance, scalability, and efficiency on a wide range of hardware accelerators. This position is part of a highly specialized sub-team focused on building the end-to-end software stack for Microsoft's first-party (1P) AI accelerators. Our work is foundational to Microsoft's long-term AI infrastructure strategy, powering next-generation models and services across Azure and Microsoft products. We collaborate closely with hardware architects, compiler teams, and model experts to co-design software solutions that unlock the full potential of custom silicon. The work is deeply technical and impactful-advancing AI systems used at global scale. If you are passionate about AI systems, low-level performance optimization, and solving hard cross-discipline engineering problems, we invite you to join us and help shape the future of AI at Microsoft.
ResponsibilitiesDesign, implement, and optimize core components of the AI software stack targeting Microsoft's first-party AI accelerators, including runtime, kernel libraries, and framework integration layers. Collaborate with hardware, compiler, and model teams to co-design solutions that maximize performance, efficiency, and reliability across the full AI stack.Develop performance-critical infrastructure to support inference of large language models (LLMs) at scale. Identify and address software bottlenecks, and drive end-to-end performance tuning and debugging across framework, runtime, and hardware layers.Work closely with partner teams across Azure, research, and product groups to align technical direction and deliver high-impact capabilities for real-world AI workloads.Participate in design reviews, code reviews, and architectural discussions to ensure high-quality and maintainable software.Stay current with advancements in AI frameworks, compiler technologies, and hardware acceleration, and bring relevant innovations into our software stack. | |