Developer Guides

Use the NxD Inference (neuronx-distributed-inference) Developer Guides to learn how to use NxD Inference.

• NxD Inference Features Configuration Guide

• NxD Inference - Production Ready Models

• Onboarding models to run on NxD Inference

• vLLM User Guide for NxD Inference

• Testing modeling code with NxD Inference

• Migrating from NxD Core inference examples to NxD Inference

• Migrating from Transformers NeuronX to NeuronX Distributed(NxD) Inference

• LLM Inference Benchmarking guide

• Accuracy Evaluation of Models on Neuron Using Open Source Datasets

• Custom Quantization