Executive Summary
Amazon Web Services has announced the general availability of Amazon SageMaker Inference for custom Amazon Nova models. This new capability allows customers to deploy their fine-tuned Nova Micro, Nova Lite, and Nova 2 Lite models on a production-grade, managed service. The offering aims to provide customers with greater control over cost and performance by enabling configuration of instance types, auto-scaling, and other advanced inference parameters for production workloads.
Key Takeaways
* New Service: Custom-trained Amazon Nova models can now be deployed using the managed Amazon SageMaker Inference service.
* Supported Models: The service supports Nova Micro, Nova Lite, and Nova 2 Lite models that have been customized through continued pre-training, supervised fine-tuning, or reinforcement fine-tuning.
* Cost & Performance Optimization: Users can reduce inference costs through optimized GPU utilization on Amazon EC2 G5 and G6 instances, auto-scaling policies, and configurable parameters like context length, concurrency, and batch size.
* Deployment Flexibility: Models can be deployed through the SageMaker Studio UI or programmatically using the SageMaker AI SDK.
* Availability: The service is now generally available in the US East (N. Virginia) and US West (Oregon) AWS Regions.
* Pricing: Customers pay per hour for the compute instances used, with no minimum fees or commitments.
Strategic Importance
This launch creates a seamless, end-to-end AWS-native workflow for customizing and serving Amazon's proprietary foundation models, strengthening the SageMaker ecosystem against competitors and third-party model providers.