International Journal For Multidisciplinary Research

E-ISSN: 2582-2160     Impact Factor: 9.24

A Widely Indexed Open Access Peer Reviewed Multidisciplinary Bi-monthly Scholarly International Journal

Call for Paper Volume 7, Issue 2 (March-April 2025) Submit your research before last 3 days of April to publish your research paper in the issue of March-April.

Automating Data Pipelines in Azure Data Factory to Improve Data Management in Large Enterprises

Author(s) Upesh Kumar Rapolu
Country United States
Abstract To handle the advanced phase of the data-driven basement, there is various prominent data management which plays a major role among the large enterprises. By the use of Azure Data Factory (ADF), regular data pipelines provide cost prominent solutions to pass the data integration, scalability and processes among various features. The implementation of large enterprises to handle the data workflows, development of operational efficiency and lowering the manual intervention were discussed in this paper. The connection with Azure services like Synapse Analytics, Power BI, Data Lake, low-code interface and dynamic parameterization get leverage, where the quick targets of organizations can fix the data consistency and decision-making. To improve the reliability, this paper improves the better handling pipelines under handling and logging. Along with the enterprise compliance standards, this study even explains various performances such as scalability, governance and security, as it provides the features to structure the ADF deployment. The advantages of ADF include the improvement of data-driven decision-making, which lowers the operational prices through recognizing the highlight.
Keywords Data Management, Data Automation, Data Pipelines, Scalability, Large Enterprises, Azure Data Factory, Cloud Integration, Operational Efficiency, Real-Time Analytics, Data Governance.
Field Engineering
Published In Volume 5, Issue 3, May-June 2023
Published On 2023-05-16
DOI https://doi.org/10.36948/ijfmr.2023.v05i03.36367
Short DOI https://doi.org/g83mc4

Share this