Vishal Awate

Lead Data Engineer with 9+ years of experience designing petabyte-scale data platforms — and a hands-on builder of production GenAI/LLM systems.

I build GenAI for a global banking client — a production service that translates complex SQL and multi-language code (Python, Scala, Java, and more) into plain-English business explanations: hybrid RAG combining deterministic structured retrieval with vector search, database-driven prompt management, LLM-as-judge quality evaluation, and fully deterministic outputs built for regulated environments, running on Llama 3.3 through an enterprise LLM gateway.

Independently, I designed and operate a second production GenAI system end to end — multi-source data ingestion (25+ sources), an LLM pipeline with structured extraction using tool schemas, embedding-based semantic search (RAG) with vector storage in PostgreSQL, multi-stage match scoring with model cascading that keeps inference costs under control, LLM-as-judge evaluation gating notifications, and LLM-generated documents delivered over WhatsApp and email.

I architect end-to-end data solutions: petabyte-scale data lake pipelines in the payments domain (PayPal/Braintree) using Spark/PySpark, lakehouse platforms on Azure Databricks, and the orchestration, quality, and cost controls that keep them reliable in production — plus platform governance frameworks automating storage, compliance, and cost monitoring across 10+ Hadoop databases serving 15+ teams.

Tech: Spark · PySpark · Python · Azure Data Factory · Azure Databricks · AWS (S3, Redshift, Lambda) · GCP core services · Hive · Impala · HDFS · SQL (MySQL, Oracle, PostgreSQL) · Shell · CI/CD (Jenkins, uDeploy)

AI: LLMs (Claude, OpenAI, Llama 3.3) · Hybrid RAG (structured + vector) · Embeddings & Vector Search (pgvector) · AI Agents · Prompt Engineering · LLM-as-Judge · Model Cost Optimization · Deterministic LLM Outputs