a dependable, low-latency, cost-efficient RAG system on a SQL desk that shops massive paperwork in long-text fields — with out altering the present schema?
This isn’t a theoretical drawback.
In most enterprises, crucial enterprise information already lives inside conventional relational databases. Proposals, experiences, contracts, articles — all saved in TEXT or LONGTEXT columns — designed for key phrase matching and aggregations, not semantic retrieval.
With the appearance of LLMs, enterprise calls for have advanced to structured computation, deep semantic understanding and contextual insights in a pure, conversational method.
For instance:
- What number of initiatives over $1M have been authorised from 2023 to 2025?
- Summarize the most important tendencies seen in know-how during the last 6 months
- What has been the differentiators of successful proposals in 2025?
They require a retrieval technique that may resolve when to compute, when to semantically search, and when to mix each. On this article, I’ll display an Agentic RAG structure that operates immediately on high of a standard SQL database — with out schema adjustments — and talk about the design ideas required to make it dependable in manufacturing.
System setup
For this illustration, I’ve used a subset of the Social Animal 10K Articles with NLP dataset, which has numerous information articles and weblog posts together with metadata. The SQL database created has the next columns — url, title, authors, published_date, article_category, word_count and the full_content.
The title may be thought of to be a singular identifier (main key) for the content material. The article classes are know-how, enterprise, sports activities, journey, well being, leisure, politics and trend. The articles are distributed roughly evenly throughout the classes. The LLM used is gemini-2.5-flash and FAISS to index and retailer the vector embeddings. The design is relevant for any alternative of LLM or vector database.
Structure
Moreover embedding the uncooked textual content, we mirrored the vector retailer metadata with the identical fields current in SQL (besides the total content material). This enables for Filtering, as we are going to see within the outcomes. For lengthy paperwork, a sliding window chunking and embedding technique may be adopted with the metadata connected to every embedding.
The metadata code snippet is connected
for idx, row in df_sql.iterrows():
content material = str(row['full_content']).strip()
if not content material:
proceed
metadata = {
"supply": row.get('url', ''),
"title": row.get('title', ''),
"authors": str(row.get('authors', '')),
"article_category": str(row.get('article_category', 'unknown')),
"published_date": str(row.get('published_date', '')),
"word_count": int(row.get('content_word_count', 0))
}
doc = Doc(page_content=content material, metadata=metadata)
paperwork.append(doc)We constructed two specialised, clever instruments that the ReAct agent can invoke utilizing the next structure. The ReAct (router) agent orchestrates the whole question pipeline by intelligently deciding which instrument to invoke based mostly on the character of the question. It makes use of the metadata and question context to find out whether or not the SQL instrument, the vector instrument, or a hybrid strategy is most acceptable. The next determine depicts the question resolution stream:
The instruments are as follows:
- search_database (SQL instrument): Handles questions that require computation, aggregation or complicated logic. It executes SQL queries
- search_articles (Vector instrument): Handles questions on content material, subject or particular entities. Accepts a pure language question, and optionally, metadata filters to execute a worldwide semantic search (eg: “articles about kids“) or search a subset of the info
As may be seen within the above determine, a question can take the next pathways:
- For computations (eg; what number of articles..), inequalities/vary (eg: Articles revealed between Jan and Apr 2023) or aggregations (eg; What’s the common phrase rely..), use the SQL instrument solely.
- Semantic search, with or with out filters, use the Vector instrument as defined above.
- Hybrid question: Hybrid queries are important once you want each structured knowledge (e.g., filtering by date) and unstructured content material (e.g., semantic search on articles). The question has a metadata filter standards (often class or a date vary), for which the SQL instrument is used to fetch articles. Then the record of titles is handed to the Vector instrument to carry out a semantic search on solely these articles. An instance could be “between march and will 2023 is there any article about mom’s day in trend“
Outcomes
Listed here are the outcomes of some queries of every sort. It could be fascinating to notice the instrument pathway adopted in every case.
1. “what are the articles by 12 months and class?” — It is a complicated computation question, so solely the SQL instrument will probably be used.
The result’s right here
QUERY: what are the articles by 12 months and class ?
TOOL SELECTION:
- search_database: {'question': "SELECT strftime('%Y', published_date) AS 12 months, article_category, COUNT(*) FROM articles GROUP BY 12 months, article_category"}
DOCUMENTS SELECTED:
- Software Output: [('1970', 'business', 1), ('2001', 'politics', 1), ('2013', 'politics', 1), ('2018', 'entertainment', 1), ('2020', 'entertainment', 1), ('2020', 'sports', 1), ('2022', 'fashion', 1), ('2023', 'business', 19), ('2023', 'entertainment', 16), ('2023', 'fashion', 16), ('2023', 'health', 18), ('2023', 'politics', 16), ('2023', 'sports', 16), ('2023', 'technology', 18), ('2023', 'travel', 18)]
RESULT:
Listed here are the variety of articles by 12 months and class:
* 1970: enterprise (1)
* 2001: politics (1)
* 2013: politics (1)
* 2018: leisure (1)
* 2020: leisure (1), sports activities (1)
* 2022: trend (1)
* 2023: enterprise (19), leisure (16), trend (16), well being (18), politics (16), sports activities (16), know-how (18), journey (18)2. “what articles about kids do you’ve gotten?” — This doesn’t match any of metadata classes that the schema consists of, so the agent decides to make use of the Vector instrument with a worldwide semantic search.
The result’s right here
QUERY: what articles about kids do you've gotten
TOOL SELECTION:
- search_articles: {'question': 'kids'}
DOCUMENTS SELECTED:
- Software Output: --- Doc 1 ---
Supply: https://www.filmcompanion.in/options/indepth-stories/streaming/best-childrens-movies-on-amazon-prime-video-netflix-hotstar-klaus-kung-fu-panda-jagga-jasoos-childrens-day-2020
Title: 10 Movies That Youngsters Ought to Watch
Class: leisure
Date: 2020-11-14 02:30:36
Content material: On this kids's day, here is an inventory of tales designed to assist younger ones perceive the world round them, achieve life classes and be dazzled by vibrant creativeness. It is a good time to be a...
- https://www.filmcompanion.in/options/indepth-stories/streaming/best-childrens-movies-on-amazon-prime-video-netflix-hotstar-klaus-kung-fu-panda-jagga-jasoos-childrens-day-2020
- https://africabusiness.com/2023/04/07/save-the-children-and-thinkmd-expand-partnership-to-improve-the-lives-of-children-globally/
- https://www.tcpalm.com/story/information/training/st-lucie-county-schools/2023/04/11/books-stay-in-st-lucie-county-schools-but-most-move-to-high-school/70098338007/
RESULT:
Listed here are some articles about kids:
1. 10 Movies That Youngsters Ought to Watch (leisure)
2. Save the Youngsters and THINKMD Develop Partnership to Enhance the Lives of Youngsters Globally (well being)
3. St. Lucie County College Board decides to maintain challenged books in class libraries (well being)3. “what are the tendencies in trend?” — The agent finds the class = trend, and runs the semantic match utilizing the Vector instrument with this filter standards.
The result’s right here
QUERY: what are the tendencies in trend
TOOL SELECTION:
- search_articles: {'question': 'tendencies', 'filter_category': 'trend'}
DOCUMENTS SELECTED:
- Software Output: --- Doc 1 ---
Supply: https://www.sightunseen.com/2023/04/the-best-thing-we-saw-in-milan-today-india-mahdavi-for-gebruder-thonet-vienna/
Title: The Greatest Factor We Noticed in Milan At the moment: India Mahdavi for Gebrüder Thonet Vienna - Sight Unseen
Class: trend
Date: 2023-04-18 12:00:00
Content material: How one can Dwell With Objects
The Greatest Factor We Noticed in Milan At the moment: India Mahdavi for Gebrüder Thonet Vienna
Sight Unseen is on the bottom on the Milan Furnishings Honest proper now and we’ll be bringing you load...
- https://www.sightunseen.com/2023/04/the-best-thing-we-saw-in-milan-today-india-mahdavi-for-gebruder-thonet-vienna/
- https://themoderndaygirlfriend.com/clean-make-up-skincare-brand-in-2023/
- https://poprazzi.com/the-80s-inspired-jewelry-trend-im-absolutely-fawning-over/
RESULT:
The search outcomes point out the next tendencies in trend: India Mahdavi for Gebrüder Thonet Vienna, clear make up & skincare, and 80's-inspired jewellery.4. “inform me know-how articles about crypto in 2023” — It is a hybrid question the place the SQL instrument will probably be used to get the titles in 2023 for class = know-how, then the Vector instrument will probably be invoked with the question = crypto and the title record. The end result will probably be discovered inside that subset.
The result’s right here
QUERY: inform me know-how articles about crypto in 2023
TOOL SELECTION:
- search_database: {'question': "SELECT title FROM articles WHERE article_category = 'know-how' AND published_date LIKE '2023%'"}
- search_articles: echojobs.io', 'Driving data to deliver net zero with life cycle assessments', 'The Future of Work: Trends and Predictions for the Next Decade', 'Risks of Outsourcing Software Development to Uruguay', 'Kochie Arrest News Addressed, Affected By Scam', 'Paying attention to employee retention
DOCUMENTS SELECTED:
- Software Output: [('NPR quits Twitter, says Musk-led platform is “undermining our credibility”',), ('Crypto.com arena considers rebrand after FTX fallout reignites investor anger',), ("World's Biggest EV Battery Maker Unveils S",), ('What Checks Should You Perform Before Buying a Used Car? - Free Car Mag',), ('Preeclampsia Diagnostics Market 2023-2030 Overall Analysis Report | Fortune Business Insights',), ('Spanish startup on ‘mission to save planet’s beer’ from climate crisis',), ('Best third-party apps for Re...
- Tool Output: --- Document 1 ---
Source: https://www.analyticsinsight.net/crypto-com-arena-considers-rebrand-after-ftx-fallout-reignites-investor-anger/
Title: Crypto.com arena considers rebrand after FTX fallout reignites investor anger
Category: technology
Date: 2023-04-19 01:15:52
Content: The Crypto.com Arena, formerly known as the Staples Center, is reportedly considering a rebrand after the recent controversy surrounding FTX, a cryptocurrency exchange. The FTX fallout reignited anger among investors, wh...
- https://www.analyticsinsight.net/crypto-com-arena-considers-rebrand-after-ftx-fallout-reignites-investor-anger/
- https://theprint.in/ani-press-releases/aave-chainlink-and-binance-are-concentrating-on-enhancing-the-shapella-while-big-eyes-coin-is-aiming-for-a-significant-final-push/1527613/
- https://echojobs.io/job/stripe-software-engineer-service-platform-wndr6
RESULT:
Here are some technology articles from 2023 that mention crypto:
1. **Crypto.com arena considers rebrand after FTX fallout reignites investor anger**: This article discusses the possibility of the Crypto.com Arena rebranding due to the controversy surrounding the FTX cryptocurrency exchange.
2. **Aave, Chainlink, and Binance are concentrating on enhancing the Shapella, while Big Eyes Coin is aiming for a significant final push**: This article mentions the Shapella upgrade and its implications for cryptocurrencies on the Ethereum network, including Binance, Aave, and Chainlink.
3. **Software Engineer, Service Platform at Stripe | echojobs.io**: This article mentions Stripe, a financial infrastructure platform that allows businesses to accept payments.Key Considerations
As with any architecture, there are design principles to be considered for a robust application. Here are a few of them:
- Tool docstrings vs system prompt: These are two types of instructions that guide the agent behavior in different ways. It is important to use them for the intended purposes without any overlap or conflict for a reliable agent performance. Tool docstring, located inside the
@tooldecorator, describes what the tool does and how to use it. Besides the tool name, it defines the parameters, types and descriptions.
Here is the example of the search_articles tool docstring.
@tool
def search_articles(query: str, filter_category: Optional[str] = None, ...):
"""Helpful for locating details about particular subjects, summaries, or particulars inside articles.
You'll be able to filter by metadata for precision:
- `filter_category`: 'well being', 'tech', and many others.
- `filter_titles`: Record of actual titles to retrieve (BATCH MODE).
- `filter_date`: Printed date (YYYY-MM-DD) for EXACT or PARTIAL match solely.
...
"""- However, the system immediate intelligently guides the routing technique for the agent, enabling it to resolve when to make use of the SQL instrument, Vector instrument or a mixture. Additionally it is probably the most complicated and fragile part of the appliance. It defines how instruments are mixed in hybrid workflows, gives examples of right instrument utilization, and specifies obligatory guidelines and constraints. To adequately design the system immediate, It’s essential to start with a check case repository of anticipated person queries, present examples within the system immediate, and proceed enriching it for deviations that come up for edge circumstances throughout operations.
Here’s a pattern of the system immediate
system_prompt = (
"1. **LISTING/BROWSING QUERIES** (e.g., 'what articles are in politics'):n"
" - **ALWAYS use [search_database] to record titlesn"
" - DO NOT use [search_articles] and not using a semantic queryn"
...
"### MANDATORY RULESn"
"1. **DATE RANGES & INEQUALITIES**: Use SQL first, then go titles to vector tooln"
...
)- Pre and Put up filtering vector databases: It is a refined level that may have unintended and hard-to-explain outcomes for particular queries. Take into account the next two queries the place the one distinction is the mis-spelt identify: “summarize articles about Doo ley in politics on seventeenth apr 2023” and “summarize articles about Dooley in politics on seventeenth apr 2023“. Each the queries comply with similar paths, whereby the SQL instrument efficiently selects the titles for this class and date (there’s just one article mentioning Decide Dooley), then the Vector instrument known as on this title record with the question. Unusually, for the primary question, the Vector instrument returns
"Software Output: No paperwork discovered matching the factors."for this minor spelling error even when the record has just one article to pick from, whereas for the second question it returns the right article.
Right here is the results of the primary question
QUERY: QUERY: summarize articles about Doo ley in politics on seventeenth apr 2023
TOOL SELECTION:
- search_database: {'question': "SELECT title FROM articles WHERE published_date LIKE '2023-04-17%' AND article_category = 'politics'"}
- search_articles: {'question': 'Doo ley', 'filter_category': 'politics', 'filter_titles': ['Judge Dooley Ends Hartford Police Consent Decree Despite Concerns']}
DOCUMENTS SELECTED:
- Software Output: [('Judge Dooley Ends Hartford Police Consent Decree Despite Concerns',)]
- Software Output: No paperwork discovered matching the factors.And the second question
QUERY: summarize articles about Dooley in politics on seventeenth apr 2023
TOOL SELECTION:
- search_database: {'question': "SELECT title FROM articles WHERE published_date LIKE '2023-04-17%' AND article_category = 'politics'"}
- search_articles: {'question': 'Dooley', 'filter_category': 'politics', 'filter_titles': ['Judge Dooley Ends Hartford Police Consent Decree Despite Concerns']}
DOCUMENTS SELECTED:
- Software Output: [('Judge Dooley Ends Hartford Police Consent Decree Despite Concerns',)]
- Software Output: --- Doc 1 ---
Supply: https://www.nbcconnecticut.com/information/native/judge-ends-hartford-police-consent-decree-despite-concerns/3015203/
Title: Decide Dooley Ends Hartford Police Consent Decree Regardless of Considerations
Class: politics
Date: 2023-04-17 05:36:24
Content material: Decide Dooley has ended the almost 50 years of federal oversight of police in Hartford, regardless of continued considerations the division nonetheless has not employed sufficient minority officers to replicate the town's massive Black and Hispanic populations.And the reason being not only a weaker embedding on account of incorrect spelling. It’s as a result of FAISS (and Chroma and many others) carry out post-filtering — first do a worldwide seek for the question, after which filter the outcomes for the metadata (= the title record). On this case, the right article doesn’t characteristic within the top_k = 3 articles after semantic search. A pre-filtering database, then again, would have carried out the semantic search solely on the articles within the title record and located the right article even with the inaccurate spelling.
- Can all metadata filters be faraway from the Vector Software?: Sure, it’s potential, however its a larger price possibility, as easy semantic queries with a metadata filter (equivalent to class or creator), will turn into a hybrid question, requiring two instrument calls, including to token utilization and latency. A realistic center floor could be to maintain dates (and presumably different numeric metadata equivalent to phrase counts on this case) within the SQL solely, and mirror all textual content and categorical metadata within the vector database.
Conclusion
Constructing RAG on high of SQL shouldn’t be about including embeddings. It’s about designing the correct retrieval technique.
When structured metadata and long-form content material reside in the identical desk, the true problem is orchestration — deciding when to compute with SQL, when to semantically search, and when to mix each. Refined particulars like metadata filtering and gear routing could make the distinction between a dependable system and one which silently fails.
With a well-designed Agentic RAG layer, legacy SQL databases can energy semantic functions with out schema adjustments, pricey migrations, or efficiency trade-offs.
Join with me and share your feedback at www.linkedin.com/in/partha-sarkar-lets-talk-AI
Reference
Social Animal 10K Articles with NLP — Dataset by Alex P (Proprietor) (CC BY-SA 4.0)
Pictures used on this article are generated utilizing Google Gemini. Dataset used beneath CC-BY-SA 4.0 license. Figures and underlying code created by me.
