Source-linked AI summary
StocksTalk: A Voice-Enabled Conversational Agent for Structured Query Generation over Web Data
Akshat Parmar, Vikranth Udandarao, Abhay Shakya, Tanmay Hire, Avinash Anand, Rajiv Ratn Shah, Daniel Wang Zhengkui
TL;DR
Spoken financial requests are difficult to translate into reliable structured queries because finance-specific constraints, transcription errors, and ambiguity complicate natural-language understanding. StocksTalk combines retrieval grounding, constrained SQL generation, validation, and human verification, outperforming plain GPT-4o by 27.8 pp in logical consistency and 37.4 pp in multi-turn stability.
Problem
Finance-focused spoken query generation must handle domain terminology, temporal and multi-attribute constraints, transcription errors, and ambiguity without producing inconsistent executable queries.
Method
StocksTalk uses a modular voice-to-query pipeline combining speech interaction, retrieval-augmented constraint extraction, schema-grounded SQL generation, rule-based validation, and human verification.
Results
27.8 pp and 37.4 pp: StocksTalk outperforms plain GPT-4o on logical consistency and multi-turn stability, respectively, across 150 spoken prompts.
Takeaways & Limitations
The evaluation supports reliable voice-driven financial query induction when retrieval grounding, constrained generation, validation, and human verification address distinct failure modes.
Takeaways & Limitations
Performance is bounded by the fixed Screener.in schema and remains sensitive to ASR errors, especially when transcribing numeric thresholds and domain-specific metric names.
Abstract
from arXiv · showhide
StocksTalk is a voice-enabled conversational system for transforming spoken financial screening requests into executable and validated structured queries over real-world market data. The system combines streaming speech recognition, retrieval-augmented constraint extraction, schema-grounded LLM-based SQL generation, rule-based validation, and human-in-the-loop verification within an interactive dashboard. Unlike traditional template-driven financial assistants, StocksTalk exposes intermediate reasoning artifacts, including extracted constraints, normalized financial metrics, operator grounding, and generated queries, allowing users to inspect and refine each stage before execution. To evaluate the system, we curate a benchmark of 150 spoken financial prompts spanning multiple investment strategies and input noise conditions. Experimental results show that retrieval grounding, constrained query generation, and interactive verification substantially improve constraint extraction accuracy, SQL executability, logical consistency, and multi-turn stability compared to baseline LLM-based approaches. StocksTalk demonstrates how transparent, voice-driven interfaces can bridge natural language interaction and structured financial analysis, providing an effective framework for conversational stock screening and decision support.
1 Introduction
StocksTalk frames spoken financial screening as interactive structured prediction under uncertainty, requiring executable, semantically aligned, and logically consistent SQL over a predefined financial schema. It addresses this challenge through a transparent voice-driven pipeline and evaluates it on 150 spoken prompts across clean and noisy conditions.
- Spoken financial screening is difficult because high-stakes finance combines noisy input, domain terminology, temporal qualifiers, and multi-attribute constraints with transcription errors and ambiguity.These challenges complicate mapping natural language into executable structured queries.
- The system must infer valid, executable SQL aligned with a predefined financial schema while preserving semantic intent and logical consistency.StocksTalk frames this task as interactive structured prediction under uncertainty.
- Real-world financial screening additionally requires grounding constraints to financial metrics, normalizing units and thresholds, aligning temporal qualifiers with data fields, and preventing inconsistent queries.These requirements extend beyond typical text-to-SQL settings using clean text and fixed schemas.
- StocksTalk integrates streaming speech recognition, retrieval-augmented constraint extraction, schema-grounded constrained SQL generation, rule-based validation, and human-in-the-loop verification.The interface exposes extracted constraints, normalized metrics, and generated SQL before execution.
- The study contributes a transparent modular pipeline, a publicly released benchmark of 150 spoken financial screening prompts, and an evaluation of distinct failure modes addressed by constrained decoding, RAG grounding, and interactive verification.The benchmark spans clean and noisy conditions.
2 Positioning and Related Work
StocksTalk is positioned at the intersection of text-to-SQL, retrieval-augmented generation, conversational voice agents, and financial LLM systems. It addresses the underexplored problem of structured query induction over live market data while exposing intermediate reasoning for verification.
- Text-to-SQL and NLIDBs: Text-to-SQL and NLIDB research maps unstructured language into executable query structures, including financial-domain benchmarking and visual query systems.Song et al. target financial text-to-SQL and propose tree-based edit distance as an evaluation metric.
- Retrieval-Augmented Generation: Retrieval-augmented generation improves factuality and domain grounding, while knowledge-oriented retrieval, hallucination mitigation, and agentic RAG extend reliability and tool use.StocksTalk complements these directions in financial screening.
- Conversational and Voice Agents: Conversational agents and agentic workflow interfaces improve structured data-collection accuracy and user experience but typically do not expose intermediate reasoning.Commercial voice assistants remain limited to intent classification and text generation.
- Financial LLM Systems: Despite extensive work on context engineering and information retrieval with LLMs, finance-specific structured query induction over live market data remains underexplored.StocksTalk directly targets this gap.
3 System Architecture and Workflow
StocksTalk is a modular Web-native pipeline that converts spoken financial requests into validated executable queries through speech processing, retrieval-augmented understanding, constrained query induction, and live market-data integration. A synchronized dashboard exposes intermediate constraints, validation status, generated queries, results, and conversation history for inspection and control.
- Pipeline overview: The architecture comprises four stages: low-latency speech interaction, retrieval-augmented intent understanding, structured query induction, and real-time Web data integration.These stages form the system’s modular Web-native pipeline.
- Speech interaction: Streaming speech recognition converts unconstrained utterances into structured text while preserving discourse cues, tracking multi-turn state, and supporting region-specific conversational agents.The interface uses ElevenLabs STT and produces an intermediate representation for downstream modules.
- Retrieval-augmented understanding: GPT-4o-based retrieval-augmented generation uses schema-grounded prompting and a financial knowledge base to ground constraint extraction and reduce hallucination.The knowledge base covers metric definitions, operator conventions, and Screener.in sector taxonomies.
- Retrieval-augmented understanding: The RAG module retrieves domain rules, extracts financial screening constraints, and accumulates or updates constraint slots across multi-turn conversations.Extracted constraints include valuation thresholds, sector preferences, growth filters, and temporal qualifiers.
- Structured query induction: A constrained generation module maps extracted constraints into executable SQL-like queries, validating coherence, grounding operators, verifying patterns, and synthesizing explanatory SQL.These checks prevent malformed or semantically inconsistent structures while exposing queries for auditability and user control.
- Integration and interface: A synchronized Flask/SSE dashboard exposes voice controls, query visualizations with validation status, sortable results, and conversation history with highlighted constraints.Validated queries execute against Screener.in’s live market-data API, with fallback strategies under rate-limit conditions and structured dashboard formatting.
4 Evaluation
StocksTalk was evaluated on a 150-prompt spoken financial-screening benchmark spanning investment strategies and clean/noisy recording conditions. Results and ablations show that retrieval grounding, constrained decoding, validation, and human verification contribute distinct improvements, while ASR noise remains a major bottleneck.
- Benchmark: FinScreenBench contains 150 spoken prompts across growth-focused, dividend-oriented, and value-based strategies, with 2–5 constraints drawn from 28 financial metrics.Prompts were authored by three finance-background annotators and cover diverse phrasing styles.
- Benchmark: 300 total audio samples were recorded by six speakers under clean and noisy conditions, with cafeteria noise at 55–65 dB SNR in the noisy condition.The benchmark is intended for reproducible evaluation of voice-driven text-to-SQL systems.
- Evaluation setup: The evaluation compares StocksTalk with GPT-4o (plain), GPT-4o + RAG, and GPT-4o + RAG + Validation baselines that progressively ablate pipeline components.Metrics include CEA, EX, LCR, QED, multi-turn stability, and end-to-end latency.
- Main results: The validation layer provides the largest single gain in logical consistency (+18.3 pp), while human-in-the-loop verification particularly improves multi-turn stability.Adding RAG substantially improves constraint extraction, and plain GPT-4o has reasonable executability but low logical consistency.
- Robustness and ablations: The clean–noisy gap is largest for CEA (–12.8 pp) and MTS (–14.3 pp), reflecting ASR sensitivity to numeric thresholds and domain-specific metric names.Ablations identify RAG as critical for extraction, constrained decoding for executability, validation for consistency, and HITL for preventing multi-turn constraint drift.
- Human verification: HITL provides the largest category-specific benefit for value-based prompts (+8.5 pp), while turn-1 constraint errors persist into turn 3 in 34% of cases without HITL versus 9% with verification.Intermediate user correction absorbs grounding errors before they propagate into query generation.
5 Demonstration and Use Cases
StocksTalk demonstrates an end-to-end, speech-only workflow that converts spoken financial requests into validated queries and live market-data results with synchronized dashboard updates. Its use cases include growth screening, dividend retrieval, and value exploration through conversational multi-constraint filters.
- End-to-End Demonstration: Each spoken query runs through speech recognition, RAG-based reasoning, SQL induction, and Web data execution with live dashboard updates.The demonstration is conducted entirely through speech.
- Interactive Dashboard: The system presents synchronized Speech, Query, Results, and Reasoning views, enabling users to correct extracted constraints before execution.The Query View shows generated SQL and validation status, while the Results View provides sortable market data.
- Use Cases: Growth Screening filters revenue growth, market capitalization, and sector thresholds through voice.This use case applies conversational voice interaction to growth-oriented stock screening.
- Use Cases: Dividend Retrieval uses conversational filters for yield, payout ratio, and dividend history, while Value Exploration screens P/E, P/B, ROE, and debt-to-equity ratios.These use cases support dividend-focused retrieval and multi-constraint value exploration.
6 Conclusion and Future Extensions
StocksTalk makes voice-driven financial query induction reliable by combining grounding, constrained generation, validation, and human verification, outperforming plain GPT-4o on logical consistency and multi-turn stability. Future extensions add news-aware, portfolio-aware, and agentic screening capabilities toward conversational financial analysis.
- Conclusion: 27.8 pp on logical consistency and 37.4 pp on multi-turn stability: the full pipeline outperforms a plain GPT-4o baseline on 150 spoken prompts.The pipeline uses GPT-4o as its backbone and combines RAG grounding, constrained SQL generation, rule-based validation, and human-in-the-loop verification.
- Future Extensions: Real-time earnings call transcripts and financial news feeds will be added as RAG sources for event-driven screening queries.The proposed examples include finding pharmaceutical companies that beat EPS estimates last quarter.
- Future Extensions: The query schema will support relative constraints against users’ existing holdings, enabling portfolio-aware screening such as lower-volatility stock searches.This extension compares candidate stocks with a user’s current portfolio.
- Future Extensions: Agentic screening workflows will chain multiple screening steps autonomously, such as applying a valuation filter after a growth screen without per-step voice input.This direction is intended to move StocksTalk toward a general-purpose platform for conversational financial analysis.