Fine-Tuning DistilBERT for Emotion Classification

Project Snapshot

Executive Summary

This project demonstrates an end-to-end pipeline for fine-tuning a transformer-based language model (DistilBERT) to perform multi-class emotion classification. The system adapts a general-purpose pre-trained model to understand nuanced emotional signals in text and deploys it as a production-ready web application with real-time inference.

Problem Statement

Generic sentiment analysis models often fail to capture fine-grained emotional context, as they are typically limited to binary or polarity-based outputs. This results in:

  • Inability to distinguish between closely related emotions
  • Reduced accuracy in conversational and user-generated text
  • Poor performance when applied without task-specific adaptation
  • Limited usefulness in real-world NLP applications

Solution Overview

To address these limitations, I fine-tuned DistilBERT on an emotion-labeled dataset, enabling the model to learn emotion-specific linguistic patterns while retaining the efficiency of a lightweight transformer.

  • Task-specific fine-tuning using supervised learning
  • Multi-class emotion classification (joy, sadness, anger, fear, etc.)
  • Optimized for low-latency, real-time inference
  • Deployed as a scalable Flask web application

Architecture & Approach

  • Data preprocessing and tokenization using Hugging Face Transformers
  • Fine-tuning DistilBERT with PyTorch on labeled emotion data
  • Evaluation using accuracy, precision, recall, and F1-score
  • Model hosting via Hugging Face Hub for lightweight deployment
  • Flask-based inference service with HTML/CSS frontend

Key Capabilities

• Transformer Fine-Tuning

Adapted a pre-trained DistilBERT model to a task-specific emotion classification problem using supervised fine-tuning.

• High-Accuracy Emotion Detection

Achieved 82.4% classification accuracy by learning nuanced emotional representations from labeled text data.

• Production-Ready Inference

Deployed the fine-tuned model via Flask with real-time inference, optimized for low latency and scalability.

• Lightweight & Scalable Deployment

Hosted the trained model on Hugging Face Hub, avoiding large model files in the repository while enabling cloud deployment.

Impact & Outcomes

  • Improved emotion detection accuracy compared to zero-shot approaches
  • Enabled real-time emotion analysis for user-generated text
  • Demonstrated full ML lifecycle ownership from training to deployment
  • Created a reusable blueprint for NLP model fine-tuning projects

Tech Stack

Languages: Python
Frameworks: Hugging Face Transformers, PyTorch, Flask
Model: DistilBERT (fine-tuned)
Training: Google Colab
Deployment: Render, Hugging Face Hub
Frontend: HTML, CSS, Bootstrap