Amazon Web Services (AWS) has expanded its SageMaker AI Serverless Model Customization service to support Full Fine-Tuning (FFT) of large language models (LLMs). FFT enables the comprehensive updating of all model parameters during fine-tuning, allowing models to better capture complex, domain-specific knowledge compared to parameter-efficient approaches like Low-Rank Adaptation (LoRA).
Fine-tuning adjusts a pretrained LLM on specific organizational data, improving accuracy on targeted tasks and reducing inference costs. Traditionally, parameter-efficient fine-tuning methods like LoRA modify a small subset of model parameters, lowering computational overhead but limiting the model’s representational capacity. This low-rank restriction can limit performance on complex problems and large datasets.
FFT updates the entire weight matrix of the model, utilizing the full representational power of the LLM's parameters. According to AWS experiments with code generation tasks on a 110,000-pair instruction dataset, FFT achieved significantly higher scores on semantic correctness and text similarity metrics compared to LoRA. The quality advantage increases with dataset size, demonstrating FFT’s strength for large-scale, complex fine-tuning.
The SageMaker AI Serverless Model Customization platform abstractions allow users to run FFT jobs without manual GPU provisioning, cluster setup, or distributed training configuration. Users define datasets, hyperparameters, and base models, while SageMaker manages compute resources and checkpointing automatically, enabling rapid experimentation and deployment.
However, FFT comes with trade-offs. Updating all parameters risks catastrophic forgetting of general knowledge—a problem where domain-specific fine-tuning degrades broader model capabilities. AWS’s evaluation using the MMLU benchmark showed that careful hyperparameter tuning, especially low learning rates and limited epochs, is critical to preserving general knowledge during FFT. Aggressive training degraded performance drastically.
In contrast, parameter-efficient fine-tuning like LoRA generally retains pretraining knowledge better and can outperform FFT on some subjective quality metrics, suggesting no one-size-fits-all method. Organizations should consider dataset size, task complexity, and acceptable trade-offs when choosing between FFT and PEFT approaches.
Overall, AWS’s addition of Full Fine-Tuning to SageMaker AI Serverless offers enterprises a powerful option for deep domain model customization without infrastructure complexity. It facilitates improved task-specific accuracy, particularly for tasks like advanced code generation or specialized reasoning, provided the datasets are sufficiently large and training parameters are carefully managed.
Source & further reading
Full Fine-Tuning (FFT) on SageMaker AI Serverless Model Customization ↗Source published 2026-11-19. Claims and limitations should be checked against the original announcement or research.