Transformer-Based Ensemble Model for Classification of Documents Based on English Vocabulary Words

Nisar Ahmad Kangoo; Nisar Iqbal Wani

doi:10.36948/ijfmr.2026.v08i01.66106

Transformer-Based Ensemble Model for Classification of Documents Based on English Vocabulary Words

Author(s)	Mr. Nisar Ahmad Kangoo, Dr. Nisar Iqbal Wani
Country	India
Abstract	Document classification remains a key challenge in natural language processing, particularly when dealing with complex vocabulary structures and imbalanced datasets. While traditional classifiers and individual deep learning models achieve moderate success, they often struggle to capture both local lexical patterns and long-range contextual dependencies. To overcome these limitations, we propose a Transformer-based Ensemble Model that integrates BERT with Convolutional Neural Networks (CNNs) and Bidirectional Long Short-Term Memory (BiLSTMs) networks. BERT embedding serves as the foundation, offering contextualized semantic representations. CNN extracts local n-gram features, while BiLSTM models sequential dependencies across the text. The outputs from these complementary models are combined using a stacking ensemble strategy with a meta-classifier to generate the final prediction. An experimental evaluation of an English vocabulary-based document dataset demonstrates that the proposed ensemble achieves higher accuracy, F1-score, and robustness compared to baseline deep learning and traditional machine learning methods. These results underscore the effectiveness of hybrid Transformer-driven ensembles in advancing document classification tasks.
Keywords	: Transformer, BERT, Convolutional Neural Network (CNN), BiLSTM, Document Classification, Vocabulary Complexity
Field	Computer > Artificial Intelligence / Simulation / Virtual Reality
Published In	Volume 8, Issue 1, January-February 2026
Published On	2026-01-11
DOI	https://doi.org/10.36948/ijfmr.2026.v08i01.66106

View / Download PDF File

E-ISSN 2582-2160

doi

CrossRef DOI is assigned to each research paper published in our journal.

IJFMR DOI prefix is
10.36948/ijfmr

Downloads

Research Paper Format Copyright Permission Form and Undertaking Form Cover Page Vol 8 Isu 3 Cover Page Vol 8 Isu 2 Cover Page Vol 8 Isu 2

All research papers published on this website are licensed under Creative Commons Attribution-ShareAlike 4.0 International License, and all rights belong to their respective authors/researchers.

CC-BY-SA

About IJFMR Fees & Payment Current Issue Publication Archive	Submit Research Paper Track Submission Status Publication Guidelines Publication Ethics Peer Review & Plagiarism	Join as a Reviewer Editors & Reviewers Reviewer Referral Program Get Reviewer Membership Certi.	Website/Journal Policies Usage Policy Content Policies Privacy Policy

Contact Us		+91-9687-828-838	editor@ijfmr.com

International Journal For Multidisciplinary Research

A Widely Indexed Open Access Peer Reviewed Multidisciplinary Bi-monthly Scholarly International Journal

Transformer-Based Ensemble Model for Classification of Documents Based on English Vocabulary Words

Share this