Multi-agent systems shine when the work can be decomposed — one agent extracts, another analyzes, a third synthesizes. Financial document processing is a perfect fit: structured extraction is a distinct step from credit reasoning, and both are distinct from report generation. In this guide you'll wire Microsoft AutoGen to Lekha to build a three-agent pipeline that turns raw Indian financial documents into an actionable credit summary.

What you'll build

A coordinated AutoGen group chat where:

DocumentAgent — calls Lekha to extract structured JSON from bank statements, salary slips, and CAS documents

AnalystAgent — runs financial ratios (EMI/income, savings rate, net worth) on the extracted data

ReporterAgent — synthesizes a concise credit/underwriting summary

The pipeline handles the full document stack an Indian lending team needs: 3-month bank statements + latest salary slip + optional CAS.

Prerequisites

pip install pyautogen>=0.2 requests
Get your Lekha API key at lekhadev.com
export LEKHA_API_KEY="lk_live_..."
export OPENAI_API_KEY="sk-..."   # or configure any AutoGen-compatible LLM

Step 1: Lekha extraction helper

Lekha's REST API accepts a PDF and returns structured JSON. Wrap it in a callable function your agents can invoke as a tool.

import os
import requests
import base64
from pathlib import Path
LEKHA_API_KEY = os.environ["LEKHA_API_KEY"]
LEKHA_BASE = "https://api.lekhadev.com/v1"
def extract_document(file_path: str, doc_type: str | None = None) -> dict:
    """
    Send a financial document to Lekha and return structured JSON.
    doc_type: 'bank_statement' | 'salary_slip' | 'cas' | None (auto-detect)
    """
    pdf_bytes = Path(file_path).read_bytes()
    b64 = base64.b64encode(pdf_bytes).decode()
payload: dict = {"document": b64, "format": "base64"}
    if doc_type:
        payload["document_type"] = doc_type
resp = requests.post(
        f"{LEKHA_BASE}/extract",
        json=payload,
        headers={"Authorization": f"Bearer {LEKHA_API_KEY}"},
        timeout=60,
    )
    resp.raise_for_status()
    return resp.json()["data"]

Test this against a sample PDF before wiring it into agents:

data = extract_document("samples/hdfc_statement.pdf", "bank_statement")
print(data["account"]["holder_name"])   # "Priya Sharma"
print(data["summary"]["closing_balance"])  # 84250.0

Lekha auto-classifies documents when doc_type is omitted — handy when users upload an unknown file. See the classification docs for details.

Step 2: Define the AutoGen agents

AutoGen's ConversableAgent lets you attach Python tools that agents call mid-conversation.

import autogen
llm_config = {
    "model": "gpt-4o",
    "temperature": 0.1,
    "api_key": os.environ["OPENAI_API_KEY"],
}
--- DocumentAgent: calls Lekha, passes raw extraction downstream ---
document_agent = autogen.ConversableAgent(
    name="DocumentAgent",
    system_message="""You extract financial data from Indian documents using the
extract_document tool. Always call the tool for each file provided, then output
the raw JSON. Do not interpret the numbers — just extract and pass them on.""",
    llm_config=llm_config,
)
--- AnalystAgent: computes ratios, no tool calls needed ---
analyst_agent = autogen.ConversableAgent(
    name="AnalystAgent",
    system_message="""You are a senior credit analyst specialising in Indian retail lending.
Given extracted document JSON from DocumentAgent, compute:
Average monthly income (last 3 months net salary or bank credits)
Average monthly obligations (loan EMIs, recurring debits)
EMI-to-income ratio (< 40% is healthy)
Average monthly savings
Savings rate as % of income
Net worth from CAS if available
Return a structured analysis dict with these keys: income, obligations, emi_ratio,
savings, savings_rate, net_worth. Flag any anomalies (bounced ECS, salary delays).""",
    llm_config=llm_config,
)
--- ReporterAgent: writes the final summary ---
reporter_agent = autogen.ConversableAgent(
    name="ReporterAgent",
    system_message="""You write concise credit summaries for Indian lending teams.
Given the analysis from AnalystAgent, produce a 150-200 word summary covering:
eligibility verdict (Approve / Refer / Decline), key risk factors, and
recommended loan amount (if applicable). Write in plain English — no jargon.""",
    llm_config=llm_config,
)
user_proxy = autogen.UserProxyAgent(
    name="UserProxy",
    human_input_mode="NEVER",
    max_consecutive_auto_reply=1,
    code_execution_config=False,
)

Register extract_document as a callable tool on the DocumentAgent:

autogen.register_function(
    extract_document,
    caller=document_agent,
    executor=user_proxy,
    name="extract_document",
    description="Extract structured JSON from an Indian financial PDF using Lekha.",
)

Step 3: Run the group chat

AutoGen's GroupChat routes messages between agents. Set speaker_selection_method="round_robin" so the conversation flows Document → Analyst → Reporter without extra back-and-forth.

group_chat = autogen.GroupChat(
    agents=[user_proxy, document_agent, analyst_agent, reporter_agent],
    messages=[],
    max_round=8,
    speaker_selection_method="round_robin",
)
manager = autogen.GroupChatManager(
    groupchat=group_chat,
    llm_config=llm_config,
)
files = {
    "bank_statements": [
        "samples/hdfc_april.pdf",
        "samples/hdfc_may.pdf",
        "samples/hdfc_june.pdf",
    ],
    "salary_slip": "samples/salary_june.pdf",
    "cas": "samples/cas_june.pdf",       # optional
}
initial_message = f"""
Assess this loan applicant's creditworthiness.
Files to process:
Bank statements (3 months): {files['bank_statements']}
Salary slip: {files['salary_slip']}
CAS (mutual fund portfolio): {files['cas']}
DocumentAgent: extract all documents. AnalystAgent: compute ratios.
ReporterAgent: write the final credit summary.
"""
user_proxy.initiate_chat(manager, message=initial_message)

Step 4: Interpreting the output

A typical run produces an exchange like this (abbreviated):

DocumentAgent → extracted 3 bank statements + salary slip + CAS
AnalystAgent  → {
  "income": 95000,
  "obligations": 22000,
  "emi_ratio": 0.23,
  "savings": 18500,
  "savings_rate": 0.19,
  "net_worth": 412000
}
ReporterAgent → VERDICT: Approve (refer for final credit committee sign-off)
  Monthly net income ₹95,000 with stable salary credits on the 1st.
  Existing EMI burden 23% — well within the 40% ceiling.
  Savings rate 19% over 3 months shows disciplined cash management.
  MF portfolio ₹4.1L adds additional collateral comfort.
  Recommended sanction: up to ₹4,50,000 personal loan at standard rate.
  No bounced ECS or return transactions observed in the review period.

All figures come directly from Lekha's structured extraction — no OCR guesswork, no hallucinated amounts. Try it live at lekhadev.com/playground.

Handling errors gracefully

Lekha returns success: false with a structured error when a document is unreadable or password-protected. Catch this in the tool before AutoGen sees it:

def extract_document(file_path: str, doc_type: str | None = None) -> dict:
    ...
    result = resp.json()
    if not result.get("success"):
        error = result.get("error", {})
        return {
            "error": True,
            "code": error.get("code"),
            "message": error.get("message"),
            "file": file_path,
        }
    return result["data"]

When DocumentAgent returns an error dict, AnalystAgent skips that file and notes the gap in its output — the pipeline continues rather than crashing.

Scaling to production

The pattern above runs synchronously. For a production lending API:

Parallel extraction: call extract_document concurrently for each statement using asyncio.gather or a thread pool — Lekha is stateless and handles concurrent requests.

Persistent context: pass cache_seed to llm_config so AutoGen caches LLM responses for identical inputs — useful when re-running the same document set.

Webhook flow: trigger the AutoGen run from a webhook when a user uploads documents via your UI; stream the ReporterAgent output back to the client using AutoGen's streaming callbacks.

Full docs at lekhadev.com/docs.

FAQ

Can AutoGen handle documents in regional languages? Lekha's extraction layer handles Hindi, Tamil, Telugu, and other Indic scripts in headers and remarks — the structured output it returns to AutoGen is always English JSON, so your agents work regardless of the document's language. What if a bank statement spans multiple PDFs? Pass each PDF file separately to extract_document. Lekha returns per-statement JSON; AnalystAgent merges the monthly figures in its analysis prompt. For single multi-page PDFs, Lekha handles them natively — no splitting needed. Does this work with open-source LLMs? Yes. Replace the llm_config dict with an Ollama or vLLM endpoint following AutoGen's local LLM guide. Lekha's extraction is model-agnostic — only the analysis and reporting agents need a capable LLM. How accurate is Lekha's bank statement extraction? Lekha uses vision AI trained on Indian bank formats (HDFC, ICICI, SBI, Axis, Kotak, and 30+ others) and achieves >99% field accuracy on clean PDFs. See the supported banks list and the playground to test your own documents.

---

Building a lending product or credit assessment tool? Sign up for a Lekha API key and start parsing documents in minutes — no training data or ML infrastructure required.