What is Transfer Learning?

What is the common confusion about Transfer Learning?

Transfer is not limited to later layers, and successful transfer is not guaranteed when source and target tasks differ sharply.

Learn Transfer Learning in the course

Lessons that name Transfer Learning in a title or section

• Transfer Learning & Fine-Tuning

  Somebody else spent a million GPU hours teaching a network what edges, textures, and object parts look like. You should borrow those features before training your own.

  Phase 04: Computer Vision

Covered in Phase 04: Computer Vision.

Related terms

• Fine-tuningContinuing training from pretrained parameters on a narrower dataset or objective. Depending on the method, you may update all parameters,…
• FeatureAn individual measurable property of the data. In classical ML, you engineer features by hand.
• SFT (Supervised Fine-Tuning)Fine-tuning a pretrained model on paired inputs and desired responses so it learns the demonstrated behavior under the training…
• Zero-ShotPerforming a task from instructions or task framing without including task-specific demonstrations in the immediate input.

More terms in Math & training

• Activation Checkpointing
• Activation Function
• Adam (Optimizer)
• AdamW
• Autograd
• Backpropagation
• Batch Size
• Contrastive Learning
• Cross-Entropy
• Data Augmentation
• DPO (Direct Preference Optimization)
• Dropout
• Eigenvalue
• Epoch
• Fine-tuning
• Gradient
• Gradient Accumulation
• Gradient Clipping
• Gradient Descent
• Hyperparameter
• JAX
• Knowledge Distillation
• Learning Rate
• Learning Rate Schedule
• LoRA (Low-Rank Adaptation)
• Loss Function
• Mixed Precision
• NaN (Not a Number)
• Normalization
• Optimizer
• Overfitting
• QLoRA
• ReLU
• RLHF (Reinforcement Learning from Human Feedback)
• SFT (Supervised Fine-Tuning)
• Softmax
• Stochastic Gradient Descent (SGD)
• Underfitting
• Warmup
• Weight
• Weight Decay

Open the Math & training list in the glossary