RAGA - Batch 6 level 3 project

1. Problem Statement

Indian classical singers require live instrumental accompaniment during practice sessions, typically a tanpura drone and a tabla or mrudangam player. This dependency creates real barriers to consistent practice:

• Expensive — professional accompanists charge per session
• Logistically difficult — scheduling around another person's availability is time-consuming
• Unavailable at odd hours — singers cannot practice spontaneously, early morning or late at night
• A barrier for intermediate and advanced students, who need frequent self-directed practice to progress

No existing application provides intelligent, raga-aware accompaniment for both Hindustani and Carnatic traditions in a single platform. Raga addresses this gap.

---

2. Know Your Audience

Raga is built for:

• Intermediate and advanced Indian classical singers, across both the Hindustani and Carnatic traditions
• Students in gurukul or institutional settings without regular access to accompanists
• Singers in geographies where accompanists are scarce or unavailable

These users need a practice tool that is musically correct — not a generic metronome or drone app, but one that understands raga grammar and traditional tuning systems.

---

3. Highlight the Impact

Raga removes the single biggest logistical barrier to self-directed classical music practice: dependency on another person. By providing an always-available, raga-aware accompanist, it enables:

• Practice on demand, at any hour, without cost per session
• Immediate feedback on pitch accuracy and raga adherence, which most singers never get outside a guru's presence
• A path for students in under-resourced settings to practice as often as more privileged peers

---

4. Dive into the Details: The Work

Overview

Raga is a web-based AI-powered practice companion for Indian classical singers, supporting both Hindustani and Carnatic traditions. It provides on-demand instrumental accompaniment, real-time pitch detection, raga validation, and practice session tracking, enabling singers to practice anytime without needing a live accompanist.

Scope

Raga's current scope covers four pillars:

1. Tanpura Drone — real recorded tanpura samples across all 12 shrutis, with pitch-shifting
2. Raga Selection — an interactive Katapayadi circle for all 72 Melakarta ragas
3. ML Pitch Detection — the YIN algorithm, via TensorFlow.js, identifying swaras in real time
4. Raga Validation — checks sung notes against the selected raga's arohana and avarohana

Tabla/mrudangam accompaniment, melodic AI (violin/harmonium), gamaka detection, and janya raga support are explicitly out of scope for the current build and are addressed under Future Plans below.

Technical Features

Frontend

• React.js — component-based UI
• Tone.js — audio engine for tanpura playback and pitch-shifting
• Web Audio API — real-time microphone input
• TensorFlow.js — ML library for numerical pitch computation

Backend

• Firebase Authentication — Google Sign-In
• Cloud Firestore — NoSQL database for sessions, favourites, and user settings

AI/ML

• YIN algorithm — pitch detection from microphone input
• DSP onset detection — RMS energy tracking for tempo drift detection
• Katapayadi algorithm — mathematical generation of all 72 Melakarta scales

Features built to date (all complete):

Core

• Tanpura Drone — real recordings in F#, G, G#, pitch-shifted to all 12 shrutis
• Seamless Looping — trimmed recordings with fade in/out
• Google Sign-In — Firebase Authentication
• Practice History — sessions saved to Firestore per user
• Favourites — save and load shruti + taal combinations
• Settings — toggle drift detection, metronome, audio quality
• Drift Detection — DSP-based tempo comparison via onset detection

AI/ML

• Pitch Detection — YIN algorithm, real-time swara identification
• Raga Validation — validates sung swara against the selected Melakarta
• Katapayadi Circle — interactive SVG wheel for all 72 Melakarta ragas

What makes Raga unique is the Katapayadi system: the 72 Melakarta ragas are mathematically generated using the ancient Katapayadi encoding system, rather than stored as a manually curated database.

• 72 ragas = 2 (Ma variants) × 6 (Ri+Ga combinations) × 6 (Dha+Ni combinations)
• 12 chakras of 6 ragas each
• The first 36 ragas use Shuddha Madhyama (M1); the next 36 use Prati Madhyama (M2)
• Each raga's number is encoded in the first two syllables of its name (e.g., "Harikambhoji" → Ha=8, ri=2, reversed → the 28th Melakarta)

The app computes any raga's scale algorithmically from its number, with no manual database entry required.

ML pitch detection, in more detail:

The YIN algorithm (de Cheveigné & Kawahara, 2002) detects pitch through four steps: a difference function that computes the squared difference between time-shifted copies of the audio buffer; a cumulative mean normalized difference that removes bias toward zero; threshold detection that identifies the first tau below a 0.1 threshold; and parabolic interpolation that refines the period estimate.

The detected frequency is converted to a MIDI note number, then to a semitone offset relative to Sa, then mapped to a Carnatic swara name using raga-context-aware logic that resolves shared semitones (for example, R2=G1).

For raga validation, the sung semitone is checked against a set of allowed semitones derived from the selected Melakarta's Ri, Ga, Ma, Dha, and Ni positions, returning a simple valid/invalid result.

For drift detection, RMS energy tracking detects note onsets; inter-onset intervals are averaged to compute the singer's BPM, which is compared to the app's BPM with a tolerance of plus or minus 5 BPM.

Competitor Analysis

Existing tanpura and metronome apps provide static drone playback or tempo tracking, but none combine raga-aware pitch validation for both Hindustani and Carnatic traditions with an algorithmically generated Melakarta database. Raga's differentiation is structural: because the 72 Melakarta scales are computed rather than hand-entered, the system can validate against any of them from day one, without the manual curation bottleneck competing apps face when trying to cover the full raga system.

Design and Implementation

Database architecture (Firebase Firestore):

Collection	Fields	Purpose
Sessions	userId, shruti, taal, laya, tradition, duration, date	Practice history
Favourites	userId, name, shruti, taal, laya, tradition	Saved combinations
Users	uid, metronome, drift, hq, streak, lastPracticeDate	Settings and gamification
Ragas (Phase 2)	name, melakartaNumber, arohana, avarohana, chakra, vadi, samvadi, pakad	Raga grammar database

Sample raga record (Phase 2 design):

{
  "name": "Mayamalavagowla",
  "melakartaNumber": 15,
  "chakra": "Agni",
  "tradition": "carnatic",
  "arohana": ["S", "R1", "G3", "M1", "P", "D1", "N3", "S'"],
  "avarohana": ["S'", "N3", "D1", "P", "M1", "G3", "R1", "S"],
  "ri": "R1", "ga": "G3", "ma": "M1", "dha": "D1", "ni": "N3",
  "equivalentHindustani": "Bhairav",
  "pakad": ["S", "R1", "G3", "M1", "P"],
  "time": "Early morning",
  "mood": "Devotional, serious"
}

Tech stack summary:

Layer	Technology
Frontend Framework	React.js
Audio Playback	Tone.js
Audio Capture	Web Audio API
Pitch Detection	YIN Algorithm + TensorFlow.js
Authentication	Firebase Auth (Google Sign-In)
Database	Cloud Firestore
Deployment	Vercel (planned)
Version Control	GitHub

Challenges and Plan B: Raga validation depends on clean microphone input; in noisy environments, YIN's threshold detection can misfire. The current mitigation is the RMS-based onset detection layer, which filters low-energy input before it reaches the pitch estimator. If ambient noise remains a persistent issue for users, a fallback of user-adjustable sensitivity thresholds is planned.

Timelines and Costs

The current build (tanpura drone, shruti support, Google authentication, practice history, favourites, streaks, drift detection, pitch detection, raga validation, and the Katapayadi circle) is complete. Phase 2 and Phase 3 timelines are outlined below under Future Plans; costs are limited to Firebase/Firestore usage and Vercel hosting, both on free or low-tier plans at current scale.

---

5. What's Next

Phase 2 (pre-launch):

• Tabla stroke samples with a taal sequencer engine
• Mrudangam stroke samples with a talam sequencer
• Raga-aware violin/harmonium melodic accompaniment

Phase 3 (post-launch):

• Gamification
• Gamaka detection via a CNN trained on the Saraga dataset
• Pitch this project to music teachers and work on their feedbacks
• A full raga practice coach with session-level feedback
• Janya raga support (300+ derived ragas)

---

6. References and Attributions

• YIN pitch detection algorithm: de Cheveigné, A., & Kawahara, H. (2002)
• Katapayadi Melakarta encoding system: traditional Carnatic music theory
• Tone.js, Web Audio API, TensorFlow.js, Firebase, and Cloud Firestore: respective open-source and platform documentation

---

7. Conclusion

Raga demonstrates that a combination of traditional music theory (the Katapayadi system, swara sthanas), modern web audio technology (Web Audio API, Tone.js), and machine learning (YIN pitch detection, TensorFlow.js) can solve a real, practical problem for Indian classical musicians. The mathematical elegance of the 72 Melakarta system, where every raga's scale is computable from its number, makes algorithmic raga validation possible without a manually curated dataset. This is Raga's core technical innovation.