International Journal For Multidisciplinary Research
E-ISSN: 2582-2160
•
Impact Factor: 9.24
A Widely Indexed Open Access Peer Reviewed Multidisciplinary Bi-monthly Scholarly International Journal
Home
Research Paper
Submit Research Paper
Publication Guidelines
Publication Charges
Upload Documents
Track Status / Pay Fees / Download Publication Certi.
Editors & Reviewers
View All
Join as a Reviewer
Get Membership Certificate
Current Issue
Publication Archive
Conference
Publishing Conf. with IJFMR
Upcoming Conference(s) ↓
Conferences Published ↓
ISC-AISGF-TBRF
DePaul-2026
IC-AIRCM-T3-2026
NSSFIGTMA-2025
SPHERE-2025
AIMAR-2025
SVGASCA-2025
ICRTET-4
ICCE-2025
Chinai-2023
PIPRDA-2023
ICMRS'23
Contact Us
Plagiarism is checked by the leading plagiarism checker
Call for Paper
Volume 8 Issue 6
November-December 2026
Indexing Partners
MoSLM: Sparse Orchestration of Specialized Small Language Models
| Author(s) | Mr. Damodhar Meshram, Mr. Manas Tarare, Ms. Samruddhi Tonghale |
|---|---|
| Country | India |
| Abstract | This proposal presents MoSLM (Mixture of Small Language Models), a task-level orchestration system inspired by Mixture-of-Experts routing. A small orchestrator (1–3B parameters) decomposes requests into typed subtasks and routes each to suitable small expert models for math, code, retrieval QA, tool calling, and general chat; low-confidence subtasks can be escalated to a larger model. The proposed orchestrator training has two stages: supervised routing from oracle labels, followed by reinforcement learning (GRPO) with a reward that combines correctness, serving cost, latency, and a regularizer intended to discourage expert collapse. Capability embeddings will be used to test whether new experts can be added without retraining the orchestrator. The study will test the hypothesis that MoSLM can match or beat a monolithic 8–14B model on mixed reasoning, coding, and tool-use benchmarks while reducing measured serving cost per successful query by 40–60%. This is a target, not a result; cost will include orchestration, retrieval, model loading, and fallback. If completed, planned artifacts include the code, router weights, and MixBench. |
| Keywords | Keywords: Small language models; LLM orchestration; task-level routing; expert models; cost-aware inference; reinforcement learning; tool use; retrieval-augmented question answering. |
| Field | Computer > Artificial Intelligence / Simulation / Virtual Reality |
| Published In | Volume 8, Issue 6, November-December 2026 |
| Published On | 2026-11-05 |
Share this

E-ISSN 2582-2160
CrossRef DOI prefix of IJFMR is 10.36948/ijfmr
All research papers published on this website are licensed under Creative Commons Attribution-ShareAlike 4.0 International License, and all rights belong to their respective authors/researchers.
Powered by Sky Research Publication and Journals