Parse Form 16 and ITR with AI: Tax Verification Agent

Income verification is the most common friction point in Indian lending. A borrower submits a Form 16 or ITR, your team spends 20 minutes cross-checking numbers, and the loan still gets delayed by a day waiting for a human review. AI-powered document extraction collapses that to under two seconds.

This guide shows you how to extract structured JSON from Form 16 and ITR documents using Lekha, then wire it into a TypeScript tax verification agent that lending, KYC, and rental applications can use directly.

What Are Form 16 and ITR?

Form 16 is a TDS (Tax Deducted at Source) certificate that employers issue to employees every year by June 15. It has two parts:

Part A — employer and employee PAN/TAN details, total TDS deposited quarter by quarter

Part B — full salary breakdown: basic pay, HRA, allowances, perquisites, deductions under 80C/80D/80CCD, and final taxable income

ITR (Income Tax Return) is the self-declaration filed with the Income Tax Department. The form type varies by income source — ITR-1 (salary), ITR-2 (capital gains), ITR-3 (business), and so on. A verified ITR acknowledgement (ITR-V) with a digital stamp from the CPC is the gold standard for income proof.

Both documents are dense PDFs with structured tables, multi-page layouts, and varying formats across assessment years and filing portals. Traditional OCR struggles with table alignment and misreads amounts. Vision AI reads the semantic structure, not just the pixels.

What Lekha Extracts

When you send a Form 16 or ITR to Lekha, the API returns a normalized JSON object with:

From Form 16:

{
  "document_type": "form_16",
  "employer": {
    "name": "Infosys Limited",
    "tan": "BLRN12345A",
    "pan": "AAACI1681G"
  },
  "employee": {
    "name": "Priya Sharma",
    "pan": "ABCPS1234D",
    "assessment_year": "2025-26"
  },
  "income": {
    "gross_salary": 1800000,
    "hra_exemption": 240000,
    "standard_deduction": 50000,
    "net_taxable_salary": 1510000
  },
  "deductions": {
    "section_80c": 150000,
    "section_80d": 25000,
    "total_deductions": 175000
  },
  "tax": {
    "total_taxable_income": 1335000,
    "tax_payable": 201500,
    "tds_deducted": 201500,
    "balance_tax": 0
  }
}

From ITR (ITR-1/2):

{
  "document_type": "itr",
  "itr_form": "ITR-1",
  "assessment_year": "2025-26",
  "filing_date": "2025-07-28",
  "acknowledgement_number": "123456789012345",
  "taxpayer": {
    "name": "Priya Sharma",
    "pan": "ABCPS1234D"
  },
  "income_summary": {
    "salary_income": 1800000,
    "house_property": -200000,
    "other_sources": 45000,
    "gross_total_income": 1645000
  },
  "deductions": {
    "section_80c": 150000,
    "section_80d": 25000,
    "total": 175000
  },
  "tax_summary": {
    "total_taxable_income": 1470000,
    "tax_liability": 221500,
    "advance_tax_paid": 50000,
    "tds_credit": 171500,
    "refund_due": 0
  },
  "verified": true
}

All amounts are numbers, all dates are ISO 8601 — no post-processing needed.

Uploading Documents to Lekha

The API accepts PDFs via Base64 or multipart form. Here's the basic extract call:

import fs from "fs";
const apiKey = process.env.LEKHA_API_KEY!;
async function extractTaxDocument(filePath: string) {
  const fileBuffer = fs.readFileSync(filePath);
  const base64 = fileBuffer.toString("base64");
const response = await fetch("https://lekhadev.com/api/extract", {
    method: "POST",
    headers: {
      Authorization: Bearer ${apiKey},
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      document: base64,
      filename: "form16.pdf",
    }),
  });
const result = await response.json();
if (!result.success) {
    throw new Error(Extraction failed: ${result.error.message});
  }
return result.data;
}

Lekha's classifier automatically detects whether the document is Form 16 Part A, Part B, or ITR and routes it to the correct extraction prompt. You don't need to specify the document type.

Building the Tax Verification Agent

Real-world income verification needs more than raw extraction — it needs decisions. Is the income consistent? Is the ITR verified? Does the declared income match the TDS? Let's build an agent that checks all three.

import Anthropic from "@anthropic-ai/sdk";
import fs from "fs";
const client = new Anthropic();
const lekhaKey = process.env.LEKHA_API_KEY!;
// Tool: extract any tax document via Lekha
async function extractDocument(base64: string, filename: string) {
  const res = await fetch("https://lekhadev.com/api/extract", {
    method: "POST",
    headers: {
      Authorization: Bearer ${lekhaKey},
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ document: base64, filename }),
  });
  return res.json();
}
// Tool: cross-check income between Form 16 and ITR
function crossCheckIncome(form16: any, itr: any) {
  const form16Income = form16.income?.gross_salary ?? 0;
  const itrIncome = itr.income_summary?.salary_income ?? 0;
  const variance = Math.abs(form16Income - itrIncome);
  const variancePct = form16Income > 0 ? (variance / form16Income) * 100 : 100;
return {
    form16_income: form16Income,
    itr_income: itrIncome,
    variance_amount: variance,
    variance_pct: variancePct.toFixed(2),
    consistent: variancePct < 5, // allow 5% tolerance for rounding
  };
}
const tools: Anthropic.Tool[] = [
  {
    name: "extract_tax_document",
    description:
      "Extract structured data from a Form 16 or ITR PDF using Lekha API. Returns income, deductions, and tax summary.",
    input_schema: {
      type: "object" as const,
      properties: {
        base64: { type: "string", description: "Base64-encoded PDF content" },
        filename: {
          type: "string",
          description: "Filename, e.g. form16.pdf or itr.pdf",
        },
      },
      required: ["base64", "filename"],
    },
  },
  {
    name: "cross_check_income",
    description:
      "Compare income declared in Form 16 vs ITR. Returns variance percentage and consistency flag.",
    input_schema: {
      type: "object" as const,
      properties: {
        form16: { type: "object", description: "Extracted Form 16 JSON" },
        itr: { type: "object", description: "Extracted ITR JSON" },
      },
      required: ["form16", "itr"],
    },
  },
];
async function runTaxAgent(form16Path: string, itrPath: string) {
  const form16B64 = fs.readFileSync(form16Path).toString("base64");
  const itrB64 = fs.readFileSync(itrPath).toString("base64");
const messages: Anthropic.MessageParam[] = [
    {
      role: "user",
      content: Verify income for a loan application.
Form 16 (base64): ${form16B64.slice(0, 20)}... [truncated]
ITR (base64): ${itrB64.slice(0, 20)}... [truncated]

Steps:
Extract both documents using extract_tax_document
Cross-check income consistency using cross_check_income
Verify the ITR has an acknowledgement number (CPC verified)
Return a verification summary with: approved/rejected, declared income, consistency status, and any flags,
    },
  ];
// Agentic loop
  while (true) {
    const response = await client.messages.create({
      model: "claude-opus-4-8",
      max_tokens: 4096,
      tools,
      messages,
    });
messages.push({ role: "assistant", content: response.content });
if (response.stop_reason === "end_turn") {
      const textBlock = response.content.find((b) => b.type === "text");
      return textBlock?.type === "text" ? textBlock.text : null;
    }
// Process tool calls
    const toolResults: Anthropic.ToolResultBlockParam[] = [];
    for (const block of response.content) {
      if (block.type !== "tool_use") continue;
let result: unknown;
      if (block.name === "extract_tax_document") {
        const input = block.input as { base64: string; filename: string };
        result = await extractDocument(
          input.base64 === form16B64.slice(0, 20) + "..." ? form16B64 : itrB64,
          input.filename,
        );
      } else if (block.name === "cross_check_income") {
        const input = block.input as { form16: any; itr: any };
        result = crossCheckIncome(input.form16, input.itr);
      }
toolResults.push({
        type: "tool_result",
        tool_use_id: block.id,
        content: JSON.stringify(result),
      });
    }
messages.push({ role: "user", content: toolResults });
  }
}
// Example usage
const report = await runTaxAgent("./form16_2025.pdf", "./itr_2025.pdf");
console.log(report);

Practical Use Cases

Lending and NBFC underwriting — Parse Form 16 to get the exact taxable income figure lenders care about, not the gross CTC. Many borrowers inflate salary claims; the TDS certificate can't lie because it's employer-issued. Rental verification — Landlords increasingly ask for ITR as proof of income stability. Extract the three-year income trend from consecutive ITRs to flag applicants with declining income. Insurance premium assessment — Group health and life insurers verify employer details from Part A of Form 16. Automating this eliminates manual data entry in onboarding queues. GST registration and compliance — GST authorities cross-reference declared turnover in ITR-3 against GST filings. Agents parsing both documents can flag discrepancies before they become audit triggers.

Handling Edge Cases

Password-protected PDFs — Send the password in the request body:

body: JSON.stringify({
  document: base64,
  filename: "form16.pdf",
  password: "ABCPS1234D",
});

Most Form 16 PDFs from payroll software use the employee's PAN as the password.

Multi-year ITRs — When borrowers upload multiple years, Lekha returns an array of documents. Process them in order and compare assessment_year fields to build an income trend. Scanned or photographed documents — Lekha handles images (JPEG, PNG, WEBP) as well as PDFs. For physical Form 16 copies, pass the image directly — the vision model reads the table layout regardless of scan quality.

Try extracting a sample document in the Lekha playground — no code required.

FAQ

Does Lekha verify authenticity of Form 16 or ITR? Lekha extracts the data exactly as it appears in the document. For authenticity, pair it with ITR-V verification via the Income Tax e-filing portal or use CBDT's TDS verification API to confirm TAN-level TDS deposits. What ITR form types does Lekha support? ITR-1 (Sahaj), ITR-2, ITR-3, and ITR-4 (Sugam). These cover salaried individuals, capital gains, business income, and presumptive income — the four most common types in retail lending workflows. How does Lekha handle scanned Form 16 vs digitally generated? Both work. Digitally generated PDFs from TRACES or payroll platforms like Razorpay Payroll are text-native and extract with higher accuracy. Scanned copies go through the vision model. Either way the output schema is identical. Can I use Lekha to extract Form 16 from multiple employees in bulk? Yes. Use the batch endpoint or run parallel requests — Lekha handles concurrency at the API level. See the batch processing guide for production-grade queue patterns.

---

Tax document parsing is one of the highest-leverage automations you can add to an Indian lending or KYC stack. Two API calls replace 20 minutes of manual review per application.

Get your API key and start extracting in minutes at lekhadev.com. The API docs have full schema references for every supported document type.