How Apache Iceberg Outperforms Traditional and Hybrid Table Formats for Large-Scale Data Engineering

Author(s)	Urvangkumar Kothari
Country	United States
Abstract	The amount of data generated every day is exploding, which demands efficient, scalable and flexible data management systems. Apache Iceberg: Open Table Format for Big Data In recent years, Apache Iceberg, a new open table format, has become a compelling successor to traditional or hybrid table formats, and significantly boosting up the performance higher for large scale data sets. In this paper, we delve into the architectural benefits of Apache Iceberg over legacy table formats and highlighting its deployment in big tables. It also discusses the infusion of AI and ML in data engineering pipelines and how Iceberg’s capabilities support advanced analytics and predictive applications.
Keywords	Apache Iceberg, Data Engineering, Large Tables, Table Formats, AI Integration, Machine Learning, Data Lakes, Metadata Management.
Field	Engineering
Published In	Volume 6, Issue 2, March-April 2024
Published On	2024-03-08
DOI	https://doi.org/10.36948/ijfmr.2024.v06i02.50256
Short DOI	https://doi.org/g9tzbc

About IJFMR Fees & Payment Current Issue Publication Archive	Submit Research Paper Track Submission Status Publication Guidelines Publication Ethics Peer Review & Plagiarism	Join as a Reviewer Editors & Reviewers Reviewer Referral Program Get Reviewer Membership Certi.	Website/Journal Policies Usage Policy Content Policies Privacy Policy

Contact Us		+91-9687-828-838	editor@ijfmr.com

International Journal For Multidisciplinary Research