Is fine-tuning the same as RAG?

No. Fine-tuning alters the model's internal weights to change how it behaves. RAG leaves the model alone and simply provides factual text in the prompt.

When should I use fine-tuning instead of prompt engineering?

Use fine-tuning when you need the model to adopt a highly specific tone, format, or specialized domain language that is too complex to fit in a system prompt.

Is fine-tuning expensive?

It can be. It requires curated datasets, significant computational resources to run the training, and often costs more to host the custom model afterward.

Techniques

What is Fine-tuning?

The process of training a pre-trained model on a smaller, specific dataset to adapt it for a particular task.

Short answer

Fine-tuning is the process of taking a pre-trained AI model and training it further on a smaller, specific dataset to adapt its behavior or knowledge to a particular task or domain.

Fine-tuning adjusts the weights of an LLM to better perform in a specific domain (like legal or medical) or to adopt a specific style. It differs from RAG, which provides knowledge at runtime; fine-tuning bakes knowledge or behavior into the model itself.

Build your own AI Agent

Ready to put this concept into action? Create your own custom AI agent with Agent One in minutes. No coding required.

Build Your AI Agent

Frequently Asked Questions

Related Terms

Retrieval-Augmented Generation (RAG)

A technique that enhances the accuracy and reliability of generative AI models with facts fetched from external sources.

Prompt Engineering

The practice of designing inputs for AI models to produce optimal outputs.

Zero-shot Learning

The ability of a model to perform a task without having seen any specific examples of that task during training.