Tutorials

This section includes tutorials that you can follow to get started with the NeuronX Distributed (NxD) Inference library.

• Tutorial: Deploying Llama3.1 405B (Trn2)

• Tutorial: Deploying Llama3.2 Multimodal Models

• Tutorial: Using Speculative Decoding to improve Llama-3.3-70B inference performance on Trn2 instances

• Tutorial: Multi-LoRA serving for Llama-3.1-8B on Trn2 instances

• Tutorial: Using Speculative Decoding and Quantization to improve Llama-3.1-405B inference performance on Trn2 instances

• Tutorial: Evaluating Accuracy of Llama-3.1-70B on Neuron using open source datasets