Building StoxFlow: Hybrid Local/Cloud AI Agents Architecture Complete Guide (2026)

Building production-grade AI agents for financial research presents a unique engineering challenge: processing vast amounts of unstructured text without incurring exorbitant API costs. Piping full news articles and earnings transcripts directly into cloud LLMs is prohibitively expensive at scale.

To solve this, we engineered StoxFlow, an AI stock research agent for Indian equities that implements a hybrid local/cloud multi-agent architecture.

This reference guide breaks down the complete architecture, cost-optimization strategies, and production implementation details for building hybrid LLM workflows using LangGraph and Ollama.

---

1. Executive Summary

StoxFlow is a decoupled three-tier application that automates equity research by querying fundamentals, parsing technical OHLC (Open, High, Low, Close) candles, and synthesizing market news.

The Core Innovation: StoxFlow uses a hybrid LLM pipeline. It routes high-volume text extraction tasks to a local, small language model (SLM) via Ollama, and routes the final reasoning and synthesis tasks to a cloud frontier model (Google Gemini).

This approach reduces cloud token consumption by over 80% while preserving the reasoning capabilities required for a high-quality investment thesis.

---

2. Why This Matters

As AI agents move from prototypes to production, Token Economics (LLMOps) becomes the primary constraint.

When scraping 10 news articles for a single stock ticker, the input payload can easily exceed 20,000 tokens. Doing this sequentially for an entire portfolio using GPT-4o or Claude 3.5 Sonnet is financially unsustainable. By introducing a local “compression layer,” platform engineers can decouple raw data ingestion from cognitive synthesis.

---

3. Core Architecture

StoxFlow is divided into three distinct service layers to ensure scalability and separation of concerns.

Tier 1: Frontend Dashboard (NiceGUI)

A Python-based reactive UI that accepts stock tickers, streams real-time execution node states, and renders the final structured JSON as interactive analytical tabs.

Tier 2: Backend API (FastAPI)

A stateless REST API that exposes endpoints for triggering the research pipeline. It acts as the boundary between the client and the heavy execution engine.

Tier 3: Execution Engine (LangGraph & LiteLLM)

A stateful workflow engine that coordinates the agent nodes, manages API rate limits via ThreadPoolExecutor, and standardizes model communication via LiteLLM.

graph TD
    A["NiceGUI Frontend Dashboard :8080"] -->|HTTP GET Request| B["FastAPI Backend Server :8000"]
    B -->|Trigger Agent Graph| C["LangGraph Research Agent"]
    
    subgraph LangGraph Pipeline
        C --> D["Node: Resolve Company"]
        D -->|Parse Symbols| E["Node: Fetch & Preprocess Data"]
        E -->|ThreadPoolExecutor Concurrency| F["Upstox API Fundamentals & OHLC Candles"]
        E --> G["Node: Digest News (Local)"]
        G -->|LiteLLM + Qwen 2.5 3B| H["Node: Synthesize Report (Cloud)"]
        H -->|LiteLLM + Gemini 2.5 Flash| I["JSON Report Output"]
    end
    
    I -->|Save File| J[("reports/research_TICKER.json")]
    I -->|Return JSON| B
    B -->|Payload Response| A
    
    subgraph Telemetry
        K["Arize Phoenix Server :6006"]
        C -.->|Auto-Instrumentation OpenInference| K
    end

---

4. The LangGraph Pipeline Workflow

The research workflow is modeled as a cyclic state-graph using LangGraph. This ensures predictable execution sequences and strict error boundaries.

Node 1: resolve_company

Resolves a user’s raw semantic query (e.g., “TCS” or “Adani Port”) into a standardized ISIN and Upstox instrument key.

Node 2: fetch_company_data

Retrieves the company’s financial profile, technical OHLC candle series, and recent news URLs. To minimize latency, this node utilizes a ThreadPoolExecutor to execute network-bound API calls concurrently.

Node 3: digest_news (The Compression Layer)

Crawls the full DOM text of recent news articles. It feeds this raw text into a local instance of qwen2.5:3b via Ollama. The local model is prompted strictly for extraction: identifying financial facts, filtering out ad noise, and scoring market sentiment. It outputs a highly compressed JSON summary.

Node 4: synthesize_report (The Reasoning Layer)

Compiles the preprocessed financial metrics, technical baselines (like the 40-Week SMA), and the compressed news JSON into a final prompt. This prompt is sent to a cloud model (Gemini 2.5 Flash) to synthesize the comprehensive investment thesis.

---

5. Cost Analysis: The Hybrid LLM Approach

The financial impact of a hybrid architecture is substantial when analyzing high-frequency data.

Metric	Pure Cloud Architecture	Hybrid Architecture (StoxFlow)
Raw Input Size (Per Ticker)	~25,000 tokens	~25,000 tokens
Local SLM Processing	0 tokens	25,000 tokens (Free)
Cloud LLM Input	25,000 tokens	~3,500 tokens
Cloud Cost Savings	0%	86% Reduction
Latency	~8 seconds	~14 seconds

Tradeoff Note: While the hybrid approach significantly reduces cloud API costs, it introduces higher latency due to local SLM inference times. This is acceptable for asynchronous research tasks but may not be suitable for real-time trading execution.

---

6. Real-Time Observability with Arize Phoenix

Debugging non-deterministic LLM agents requires granular trace telemetry. StoxFlow implements OpenInference to stream execution traces to a local Arize Phoenix server.

This provides:

1. Node Latencies: Exact millisecond execution times for each LangGraph state transition.
2. Trace Replays: The exact system instructions, user prompts, and raw completions for both local and cloud LLMs.
3. Token Accounting: Automatic aggregation of token consumption to monitor ongoing cost metrics.

---

7. Security and Privacy Considerations

By implementing local LLMs for the initial ingestion phase, StoxFlow naturally enhances data privacy.

When scraping external data sources or processing proprietary internal financial documents, passing the raw documents through a local Ollama instance ensures that sensitive PII or corporate secrets never leave your internal network. Only the sanitized, aggregated summaries are transmitted to external APIs for final synthesis.

---

8. Step-by-Step Implementation Guide

To run the complete hybrid architecture locally:

1. Provision the Local Model

Ensure you have Ollama installed, then pull the target extraction model:

ollama pull qwen2.5:3b

2. Configure Environment Variables

You will require an Upstox Analytics Token for structural data, and a Google Gemini API Key for synthesis.

UPSTOX_API_KEY="your_upstox_analytics_token"
GOOGLE_API_KEY="your_gemini_api_key"

3. Execute the Service Stack

Install the required dependencies and start the FastAPI/NiceGUI server stack:

pip install -r requirements.txt
python run.py

Check out the full implementation in the official StoxFlow GitHub Repository.

---

9. Key Takeaways

1. Decouple Extraction from Reasoning: Do not use expensive frontier models for basic data extraction and summarization.
2. Utilize Local SLMs: Models like Qwen 2.5 3B or Llama 3.2 1B excel at compressing large text payloads locally.
3. Graph State Management: LangGraph provides the necessary deterministic framework to manage data handoffs between local and cloud models reliably.
4. Mandatory Telemetry: Never deploy multi-agent systems without trace observability (like Arize Phoenix) to monitor latency and token expenditure.

---

Disclaimer: This software and architectural guide is for educational and engineering research purposes only. It does not constitute financial advice.