🇮🇳 Engineering India Pulse:

A Semantic Data Intelligence Layer for India’s Next Billion Users

Data for everyone. Insights for billions.

🔗 Live App: here!
💻 GitHub: here!
🏆 Built at: Build for Billion Hackathon (Round 2)

1. Why India Pulse Exists

India generates massive amounts of public data — pollution levels, crime statistics, UPI metrics, census data, education indices.

But here’s the uncomfortable truth:

Most of this data is practically unusable for ordinary citizens.

Why?

  • Scattered across CSVs, PDFs, and legacy portals

  • English-only interfaces

  • Requires data science skills to interpret

  • Dashboards are rigid, non-interactive, and outdated

India Pulse was built to solve one core problem:

👉 How do we make India’s public data accessible, visual, multilingual, and queryable by anyone?

2. What Is India Pulse?

India Pulse is an AI-powered data intelligence web app that lets users explore India’s public datasets using natural language.

Instead of digging through government portals, users can simply ask:

  • “Compare air pollution in Delhi vs Karnataka over 5 years”

  • “Which states have the fastest MSME growth?”

  • “Show crime trends in Delhi from 2015–2024”

And get:

  • Interactive charts

  • Maps

  • Clean summaries

  • Visual stories

All without writing a single line of code.

3. Core Capabilities

🔍 Natural Language Querying

Users interact using plain language — text or voice — instead of filters and dropdowns.

🌐 Multilingual Access

Supports regional languages (starting with English, Hindi, Kannada, Tamil) using Indic NLP.

📊 Visual-First Insights

Results are rendered as:

  • Interactive charts

  • Geo-maps

  • Data stories

📱 Mobile-First Design

Optimized for low-bandwidth devices and first-time internet users.

4. High-Level System Architecture

India Pulse is built around an Intent → Insight pipeline.

User Query (Text / Voice)
        ↓
Semantic Parsing (Gemini + LangChain)
        ↓
Entity Resolution & Data Orchestration
        ↓
India Open Data + India Stack APIs
        ↓
Visualization Engine (Charts, Maps, Stories)

This separation allows the system to scale across new datasets, new languages, and new use cases without rewriting the frontend.

5. Architecture Deep Dive

🧠 Semantic Middleware

At the heart of India Pulse is a semantic layer that converts natural language into structured data queries.

Responsibilities:

  • Extract entities (state, city, metric, year)

  • Understand intent (compare, rank, trend)

  • Handle ambiguity in real-world queries

Example:

“Bangalore pollution vs Delhi last 10 years”
→ Normalized to Bengaluru, matched to CPCB datasets, time-filtered, and visualized.

🗺️ Entity Resolution (The Hard Part)

Government datasets are messy.

  • “Bangalore” vs “Bengaluru”

  • “Odisha” vs “Orissa”

  • Inconsistent district naming

Solution:
A fuzzy-matching + normalization layer maps all inputs to a unique internal geo-ID before querying data.

🔄 Data Normalization Layer

Public datasets rarely follow a standard schema.

So every dataset passes through a custom transformer that:

  • Cleans keys

  • Standardizes units

  • Aligns time-series formats

This ensures the visualization layer remains clean and predictable.

6. APIs & Integrations

SourcePurpose
IndiaStack.OnlineAadhaar auth, UPI trends, DigiLocker
Open Government Data (data.gov.in)Crime, health, education, census
LatLong APIGeo-mapping
eSamudaayMSME & local economy data
Knobly MediaVisual storytelling

7. Tech Stack (Why These Choices)

Frontend

  • React (Vite) — fast HMR, modular UI

  • Tailwind CSS — rapid iteration

  • Recharts + Mapbox + D3.js — rich visualizations

Backend

  • Node.js + Express

  • LangChain for orchestration

AI Layer

  • Gemini 1.5 Pro / GPT APIs

  • Used only for intent parsing, not raw data generation

Infrastructure

  • Vercel — edge caching + CI/CD

  • GitHub — version control

8. Engineering Challenges & Solutions

❌ Inconsistent Data Schemas

Problem: No two datasets look alike.
Solution: Built a Data Intelligence Layer that standardizes everything into a single schema.

❌ UI Performance Bottlenecks

Problem: Heavy maps blocked the main thread.
Solution:

  • React.memo

  • Async rendering

  • Deferred visualization loading

❌ Language & Naming Ambiguity

Problem: Users don’t speak like databases.
Solution: Semantic parsing + fuzzy geo-resolution.

9. Performance & Reliability

  • ⚡ Edge caching via Vercel CDN

  • 📱 Mobile-first rendering

  • 🧠 Minimal AI calls → predictable latency

Even on low-end devices, the core experience remains usable.

10. Future Scope

  • 🔐 DigiLocker integration for document-based insights

  • 🔊 Voice-based regional language queries

  • 📲 Mobile app for rural entrepreneurs

  • 🧠 Vector-based entity matching (replace fuzzy logic)

11. Who Built This?

👩‍💻 Project Lead: Aarti Panchal
🏫 Institution: PES University, Bangalore
🏆 Hackathon: Build for Billion (Round 2)
🤝 Mentors & Sponsors: NITK, IndiaStack.Online, eSamudaay, LatLong, Knobly Media

12. Final Thought

India Pulse is not just a dashboard.

It’s an attempt to answer a bigger question:

What happens when data finally speaks the language of the people it belongs to?

👩‍💻 About Me

Aarti Panchal
Engineering student & builder focused on scalable systems, AI-driven platforms, and digital infrastructure.