Investigated parameter-efficient fine-tuning with Low-Rank Adaptation for RoBERTa-base on the AG News text classification dataset.
Overview
The project focuses on adapting a transformer language model without updating all model weights. LoRA enables efficient experimentation while reducing the amount of trainable parameter overhead compared with full fine-tuning.
Technical Focus
- RoBERTa-base text classification
- LoRA-based parameter-efficient fine-tuning
- AG News dataset preprocessing and evaluation
- Notebook-based experimentation for reproducible model training
Repository
The fine-tuning workflow is available on GitHub.