A Secure and Automated Framework for GCP-Based D0/D1 Data Processing, Ingestion, and SQL Generation

Author(s)	Ms. Tanaya Kate, Mr. Abhishek Singh
Country	India
Abstract	This paper presents a comprehensive analysis of a Streamlit-based D0→D1 automation system built on Google Cloud Platform (GCP) components, including Cloud Storage (GCS), BigQuery, and Vertex AI. The system supports ingesting Excel files from local or GCS sources, transforming them into structured BigQuery tables, exploring schemas, and generating stored procedures using a generative model . We analyze the implementation of the application , identify operational and security risks—such as hard-coded credentials, full-table in-memory downloads, PII exposure to LLM prompts, and unsafe SQL execution—and provide an improved, production-ready D0→D1 architecture. We conclude with a prioritized implementation plan, suggested code-level changes, and governance recommendations to make the pipeline secure, scalable, and auditable.
Keywords	vertexAI, SQL, Pipeline, Stored Procedure, GCP, Bigquery, Excel
Field	Engineering
Published In	Volume 8, Issue 2, March-April 2026
Published On	2026-03-01

About IJFMR Fees & Payment Current Issue Publication Archive	Submit Research Paper Track Submission Status Publication Guidelines Publication Ethics Peer Review & Plagiarism	Join as a Reviewer Editors & Reviewers Reviewer Referral Program Get Reviewer Membership Certi.	Website/Journal Policies Usage Policy Content Policies Privacy Policy

Contact Us		+91-9687-828-838	editor@ijfmr.com

International Journal For Multidisciplinary Research