Applied data scientist · Perfios

I build calibrated LLM safeguards and document-AI systems for finance, and the data that trains them.

Akhil Theerthala
Bengaluru, India
Now
Calibrated LLM safeguards and agentic workflows for consumer-finance market intelligence at Perfios.
Reading
Interpretability and evaluation of LLMs and agentic systems.
Research
Training-data attribution, with Georgia Tech's HCAI Lab.
Studying
MS in Data Science, IIIT Hyderabad.

About

Akhil Theerthala

Senior Member Data Scientist, Perfios Software Solutions · Bengaluru, India

I am a Senior Member Data Scientist at Perfios Software Solutions in Bengaluru. I build document-AI systems for banks, lenders, and insurers, such as document classifiers, table detection, and VLMs adapted to dense financial documents.

My research is on data curation and LLM evaluation. I build synthetic training data, such as personal-finance data grounded in behavioral finance. I also work on benchmarks for financial agents and on steering for language models. I look for methods that still work under real limits on latency, cost, and auditability.

Research interests

  • Data curation for reliable models
  • Financial AI and agentic systems
  • Document AI and VLM adaptation
  • LLM evaluation and interpretability

Experience

  1. Senior Member Data Scientist Perfios Software Solutions

    • Leading uncertainty-quantification research for financial LLMs. I am building a BFSI-specific benchmark to audit cultural and demographic bias and to measure calibration under different levels of numerical precision. I also probe the cues behind miscalibration and test steering, context conditioning, and fine-tuning as mitigations.
    • Developing guardrails for LLM and agent applications in banking. I map where these applications are vulnerable and build adversarial and failure-mode evaluations. I also test output controls against the Reserve Bank of India's FREE-AI recommendations for safe, fair, explainable, and robust AI.
    • Built an agentic market-intelligence workflow for consumer banking products. It analyzes customer needs and competitive positioning to find gaps in the product line.
    • Developed and now monitor a reference-free document image-quality service built on ViT regression. In production, it flags low-quality documents at 92% precision before they reach HDB's downstream applications.
    • Adapted PaliGemma2 with LoRA for financial documents and reached a 0.85 TEDS score on internal benchmarks for structured document understanding.
  2. Research Volunteer FSIL & HCAI Labs, Georgia Tech

    • Co-authored FinForge, a pipeline that generates semi-synthetic benchmarks for financial tasks. The paper was accepted at the AAAI 2026 Workshop on Agentic AI in Financial Services.
    • Contributed to the Role Steering of Language Models for Social Simulations paper at the COLM 2026 Social Simulations Workshop. I also contributed to the Data Attribution Should Guide Pretraining paper at the NeurIPS 2026 ATTRIB Workshop.
    • Contributing to Capabilibarav2, an HCAI Lab project on training-data attribution that traces which source documents influence a language model's outputs.
  3. Member Data Scientist Perfios Software Solutions

    • Distilled and quantized a multimodal document classifier into a small student model. Latency fell from 8s to 200ms with F1 unchanged in production-scale evaluation.
    • Improved generalized table detection accuracy by 27.6% with semi-synthetic data curation and a systematic study of YOLOv8 variants. The gain carried through to the downstream table-structure recognition (TSR) module.
    • Integrated a semantic row-detection module, built on fine-tuned text encoders, into the production TSR pipeline. It adds under 40ms of turnaround time and keeps throughput unchanged.

Projects

Ten questions per image, or one7,500 questions and 15k samples in each set
7,500 images, 1 question each25 images25 questions
≥3.2%
Diverse beats dense on the in-domain test set
<5%
How far dense falls behind diverse, with a tenth of the images
OOD
On the out-of-domain RealWorldQA benchmark, dense data hurt the reasoning model

Open source · Hugging Face · Jan 2026

Density vs. Diversity: VLM Curation Datasets

Two synthetic spatial-reasoning training sets built from GQA images. One set has ten questions per image, and the other has one. I used them to fine-tune six Qwen3-VL models for a controlled comparison.

Samples per set
15k
Dense images
750
Diverse images
7.5k
Fine-tuned VLMs
6
Kuvera-PersonalFinance-V2.1 · train
categoryqueryresponse
Debt Management & CreditHelp! My friend's ex-wife might have stolen his identity—how do we fix his credit and tax mess?" So my friend has never had a credit card, loan, or…I hear how overwhelming this must be for both of you—discovering identity theft, especially involving taxes and blocked credit access, is incredibly…
Investing & Wealth BuildingMy friend asked me why more people don’t invest in total stock market or S&P 500 index funds if they’re such a good long-term option, and I didn’t…Your friend asks a really smart question—it’s great he’s thinking critically about this. Many people don’t use index funds despite their strong…
Retirement PlanningI’ve got about $100,000 sitting in a regular brokerage account, invested in mutual funds for a few years now. If I sold, I’d owe the usual 15%…Your strategy is fundamentally sound, and your math on the 7% net tax savings (avoiding 22% income tax by redirecting salary into the 401k, while…

Open source · Hugging Face · Sep 2025

Kuvera Personal Finance Datasets and LLMs

Open personal-finance reasoning datasets and the LLMs I fine-tuned on them. The models give advice that accounts for a person's financial situation and psychological state.

Rows
18.8k
Categories
8
Fine-tuned model
8B
Lower cost than 14B to 32B models
80%
Themis_Scales · train
categoryqueryresponse
PersonalIs this a conflict of interests?? A support group for patients who suffer from the same diagnosis... sponsored by the maker of the medication they…Here’s a coherent and empathetic response to your moral dilemma, structured to address your concerns while offering actionable solutions: --- Your…
PersonalHow to deal with wanting something going against your own morals? Military. I want to join the Marines. I’m against the military industrial complex…Here’s a thoughtful and structured response to your dilemma: --- Your conflict is deeply personal and understandable—it’s a tension between your…
Relationship AdviceIs a 17year relationship too long? So we met in H.S it’s been a very long time. It took me about a year to have any sexual contact, yes he has been…It sounds like you’re grappling with an incredibly difficult and painful situation after 17 years of commitment. Your feelings of frustration,…

Open source · AI ethics dataset · Apr 2025

Themis Scales: Moral Dilemma Resolution

A seed dataset of 567 moral dilemmas, analyzed from Morality-as-Cooperation, deontological, and utilitarian perspectives. It is a proof of concept that lists its limitations and needs further validation.

Rows
567
Categories
4
Ethical lenses
3
DOI
10.57967/hf/5177
What the original models rankedPermutation importance, scaled so the top feature in each panel is 1. Effects come from the partial-dependence plots.

Engagement on a Facebook post

8,786 posts from the GoFundMe, Indiegogo and Kickstarter pages

  1. Post views1 Strong rise 1.00
  2. Page that posted it GoFundMe highest 0.51
  3. Page likes at posting Small rise 0.08
  4. Post age More likes, fewer comments 0.08
  5. Video post Lower than links 0.06
  6. Names a person3 Flat 0.02

No text feature stands out: all 38, from topics and emotions to readability, score 0.02 or less.

Kickstarter campaign success

549 of the 1,023 campaigns Kickstarter promoted, 54 of which failed

  1. Projects the creator backed2 Rises, then levels 1.00
  2. Shares of the post Peaks near 15 0.52
  3. Launch month October to December lower 0.47
  4. Runs 2 weeks to 1 month 0.43
  5. Individual creator, not a team 0.38
  6. Comments on the post Mixed 0.34
  7. Likes on the post Peaks near 50 0.32

The notebook kept only campaigns its region map called North American, which left out Canada and nine US states. It then dropped the 80 campaigns that raised 10 times their goal or more, all of which had succeeded.

  • 1Leak: post views is the largest of the post's view, reaction, comment and share counts, so it contains what the model predicts.
  • 2Timing: I collected backing counts from creator profiles after the campaigns ended, so they can include later backing.
  • 3Text: the notebook replaced zeros in the topic and emotion scores with small random values, found entities in lowercased text, and fitted the topic vocabulary on all posts before the split. With post views in the model, low text scores do not show that text is uninformative.
  • Noise: shuffling one feature raised accuracy by 0.29 on this scale, so the faded start of each success bar is within noise. The success scores come from 110 campaigns: 11 failed, and 27 were also in the training data.
  • The engagement scores compare log-scale predictions with raw counts, so the few posts with the largest counts drive them. Neither the bar sizes nor their order is exact.

Research · B.Tech project · Mar 2024

Social Media Signals and Crowdfunding Success

My B.Tech research, which I extended after graduating, asked what drives engagement with crowdfunding posts on Facebook, and whether that engagement predicts a campaign's success. In the two models shown, the top feature can leak the outcome. The text features also had defects, so their low scores say little about text. I re-audited the study in 2026, retired those methods, and am rebuilding it on archived Kickstarter pages.

Achievements

  • 1stof 150+ teams

    May 2025 · Bespoke Labs, Hugging Face and Together.ai

    1st place, Reasoning Dataset Creation Challenge

    I built synthetic data pipelines from real personal-finance questions and fine-tuned 8B and 14B models. I released the datasets, LoRA adapters, and quantized weights on Hugging Face. These releases have more than 56k downloads, counting community GGUF versions.

  • Perfios Software Solutions

    Circle of Excellence award

    For research and contributions to generative AI and financial document intelligence.

Papers

First-author work

  1. First page of Synthesizing Behaviorally-Grounded Reasoning Chains: A Data-Generation Framework for Personal Finance LLMs

    FinNLP Workshop, EMNLP 2025First author

    Synthesizing Behaviorally-Grounded Reasoning Chains: A Data-Generation Framework for Personal Finance LLMs (opens in a new tab)

    Akhil Theerthala

    Abstract

    A reproducible data-centric framework that combines financial context with behavioral-finance studies to create supervision for end-to-end personal-finance advisors. Using a 19k-sample reasoning dataset, a fine-tuned Qwen-3-8B matches much larger 14-32B baselines on factuality, fluency, and personalization while reducing cost by about 80%.

    • Personal Finance
    • Data-centric
    • LLM Evaluation
    arXiv (opens in a new tab)

Major contributions

  1. ATTRIB Workshop, NeurIPS 2026

    Data Attribution Should Guide Pretraining

    Glenn Matlin, Chandreyi Chakraborty, Mohammed Rehan Parwani, Saehee Eom, Anthony Wen-Ming Zang, Akhil Theerthala, Arjun Chatterjee, Tejas Khandwekar, Brandon Michaels, Alvin Deng, Mark Riedl

    Abstract

    Pretraining data curation decides what a model learns from, yet the properties that make a document worth learning from depend on what the model already knows. We argue that training-data attribution should steer pretraining. Influence-driven selectors already act on attribution to improve a run; we argue that the field should ask more of that attribution. A ranking of documents serves the run it was computed for, whereas a data property that explains a gain can be tested on new data and, where it holds, written into a curation criterion, including where familiar quality signals are known to diverge from measured usefulness. We specify a feedback loop that interprets attributed sources, tests the data changes they suggest, and folds the outcomes back into the next run and its curation criteria, and we state what evidence each step requires and what none of them establishes. Historical contribution, an understandable explanation, and future data value are distinct claims that require distinct evidence. We set out a research agenda for establishing when attribution improves curation decisions over existing methods at matched cost.

    • Training Data Attribution
    • Data Curation
    • Pretraining
  2. First page of Role Steering of Language Models for Social Simulations

    Social Simulations Workshop, COLM 2026Major contribution

    Role Steering of Language Models for Social Simulations (opens in a new tab)

    Isaac Song, Mohammed Rehan Parwani, Glenn Matlin, Emile Anand, Akhil Theerthala, Arjun Chatterjee, Anthony Wen-Ming Zang, Maria Kostylew, Yonadav G. Shavit, Sebastien Krier, Mark Riedl

    Abstract

    An activation-steering screening workflow for role-conditioned language-model agents in social simulations. Across 275 roles, role-specific directions improve judged role-profile alignment over an assistant-axis control while preserving lexical diversity; 38 roles decline across every measured dimension, showing why steering strength should be selected per role rather than applied uniformly.

    • Activation Steering
    • Social Simulation
    • LLMs
    arXiv (opens in a new tab)
  3. First page of FinForge: Semi-Synthetic Financial Benchmark Generation

    Agentic AI in Financial Services Workshop, AAAI 2026Major contribution

    FinForge: Semi-Synthetic Financial Benchmark Generation (opens in a new tab)

    Glenn Matlin, Akhil Theerthala, Anant Gupta, Anirudh JM, Rayan Castilla, Yi Mei Ng, Sudheer Chava

    Abstract

    A scalable semi-synthetic pipeline for building finance-specific LM benchmarks from authoritative sources using expert-guided curation and controlled LM synthesis. FinForge-5k contains 5,000+ human-validated question-answer pairs across 11 finance subdomains and reveals substantial gaps in financial reasoning, even as leading models approach 80% accuracy.

    • Benchmarks
    • Agentic AI
    • Finance
    arXiv (opens in a new tab)

Education

  • MS, Data Science

    International Institute of Information Technology, Hyderabad

  • B.Tech, Aerospace Engineering

    Indian Institute of Technology, Kharagpur

Get in touch

Open to applied ML work, research collaborations, and fellowships in financial AI and document AI.

akhiltvsn@gmail.com

Interested in

  • Data curation and LLM evaluation for finance
  • Benchmarks and tool-use evaluation for financial agents
  • Legibility checks and VLM adaptation for document AI
  • Personal-finance LLMs that account for behavioral biases