Federated Learning Implementation Framework using Databricks: Privacy-Preserving Model Training at Scale

Chandra Sekhar Oleti

doi:10.36948/ijfmr.2024.v06i06.55515

Federated Learning Implementation Framework using Databricks: Privacy-Preserving Model Training at Scale

Author(s)	Chandra Sekhar Oleti
Country	United States
Abstract	The exponential growth of data-driven applications across organizations has created unprecedented opportunities for collaborative machine learning while simultaneously intensifying privacy and regulatory compliance challenges. This research presents a comprehensive federated learning framework leveraging Databricks infrastructure to enable privacy-preserving distributed model training at enterprise scale. Our approach addresses fundamental challenges in cross-organizational machine learning collaboration, including data sovereignty requirements, regulatory compliance obligations, and secure model aggregation protocols. The proposed framework integrates advanced differential privacy mechanisms with innovative secure aggregation techniques to ensure participant data remains confidential throughout the federated learning process. Through extensive experimentation across healthcare and financial services domains, we demonstrate the framework's ability to achieve 89% model accuracy while maintaining strict privacy guarantees equivalent to centralized approaches. The system successfully orchestrates federated learning workflows across geographically distributed Databricks clusters, enabling organizations to benefit from collaborative intelligence without compromising sensitive data assets. Key contributions include a novel secure aggregation protocol optimized for cloud environments, automated compliance verification mechanisms for HIPAA and GDPR regulations, and scalable orchestration algorithms that adapt to heterogeneous participant capabilities and network conditions.
Keywords	Federated learning, privacy-preserving machine learning, differential privacy, secure aggregation, distributed computing, regulatory compliance
Published In	Volume 6, Issue 6, November-December 2024
Published On	2024-12-11
DOI	https://doi.org/10.36948/ijfmr.2024.v06i06.55515

View / Download PDF File

E-ISSN 2582-2160

doi

CrossRef DOI prefix of IJFMR is 10.36948/ijfmr

Downloads

Research Paper Format Copyright Permission Form and Undertaking Form Cover Page Vol 8 Isu 3 Cover Page Vol 8 Isu 2 Cover Page Vol 8 Isu 2

All research papers published on this website are licensed under Creative Commons Attribution-ShareAlike 4.0 International License, and all rights belong to their respective authors/researchers.

CC-BY-SA

About IJFMR Fees & Payment Current Issue Publication Archive	Submit Research Paper Track Submission Status Publication Guidelines Publication Ethics Peer Review & Plagiarism	Join as a Reviewer Editors & Reviewers Reviewer Referral Program Get Reviewer Membership Certi.	Website/Journal Policies Usage Policy Content Policies Privacy Policy

Contact Us		+91-9687-828-838	editor@ijfmr.com

International Journal For Multidisciplinary Research

A Widely Indexed Open Access Peer Reviewed Multidisciplinary Bi-monthly Scholarly International Journal

Federated Learning Implementation Framework using Databricks: Privacy-Preserving Model Training at Scale

Share this