NLP with Transformers

BERT Fine-Tuning for Sentiment Analysis

Fine-tuned BERT-base-uncased on 1.6 million tweets achieving 87% accuracy and F1 score 0.85 on an NVIDIA RTX A6000 GPU.

BERTPyTorchNLPTransformersHugging FaceSentiment Analysis
GitHub Repository

Overview

A fine-tuning pipeline for BERT-base-uncased targeting sentiment analysis. The model was trained on the Sentiment140 dataset (1.6 million tweets) - a large-scale, real-world NLP benchmark with noisy labels from Twitter.

Methods

  • Added a classification head on top of BERT-base-uncased for binary sentiment classification
  • Trained for 10 epochs on the Sentiment140 dataset on an NVIDIA RTX A6000 GPU
  • Applied rule-based filtering to handle noisy labels inherent in Twitter data
  • Addressed class imbalance using class weighting in the loss function
  • Optimized batch size and learning rate for convergence stability

Results

  • Achieved 87% accuracy on the held-out test set
  • F1 score of 0.85 indicating balanced precision and recall
  • Class weighting successfully mitigated class imbalance effects
  • Model generalized well to unseen tweets despite noisy training labels