Vedant Khandelwal

Vedant Khandelwal

Ph.D. candidate in Computer Science at the University of South Carolina

GEAR-Up: Generative AI and External Knowledge-based Retrieval Upgrading Scholarly Article Searches for Systematic Reviews

Published in Thirty-Eighth AAAI Conference on Artificial Intelligence, 2024

GEAR-Up supports the early stages of systematic reviews by combining query expansion, generative reformulation, and retrieval ranking in a librarian-centered pipeline.

Why it matters

  • Systematic reviews are time-intensive and require precise query formulation.
  • Researchers depend on expert librarians to define research questions and develop structured search protocols.
  • Query development and translation can affect reproducibility and search quality.
  • Librarian time and bandwidth are limited, creating a need for AI-assisted support in early systematic review stages.

What we did

  • We introduce GEAR-Up, a modular pipeline that assists librarians in systematic review query construction.
  • We expand natural language queries using knowledge graphs (for example, PubMed) and pretrained masked language models.
  • We generate related reformulated queries using ChatGPT prompted with expanded concepts and relations.
  • We retrieve and rank relevant articles using PubMed searches combined with a FAISS-powered retriever.
  • We evaluate retrieved results qualitatively with an in-house librarian, reporting favorable outcomes in reducing librarian burden and improving search quality.

How it works

  • Query Expansion Module: extracts seed concepts from a natural language query and augments them using knowledge graphs (for example, PubMed) and masked language models.
  • Related Query Generation Module: prompts ChatGPT with expanded concepts and relationships to generate reformulated queries.
  • Article Search and Retrieval Module: executes PubMed searches and applies a FAISS-powered retriever to narrow results to the most relevant titles, abstracts, and passages.
  • Librarian-in-the-loop refinement: incorporates librarian feedback for relevance, safety, ethics, and bias controls.
GEAR-Up modular pipeline showing query expansion, related query generation, and FAISS-based article retrieval with librarian feedback.
Figure 1. Overview of GEAR-Up's modular pipeline: query expansion, related query generation, and FAISS-based article retrieval with librarian feedback.

Figure 1 illustrates how seed concepts are expanded using knowledge graphs before prompting ChatGPT for structured query reformulation.

Key contributions

  • A modular architecture integrating generative AI with external knowledge graphs for systematic review query construction.
  • Automation of the first two systematic review stages: issue identification and protocol-based query formulation.
  • Integration of FAISS-based dense retrieval for narrowing PubMed search results.
  • Qualitative evaluation with an in-house librarian showing reduced burden and improved search quality.

Recommended citation: Kaushik Roy, Vedant Khandelwal, Harshul Surana, Valerie Vera, Amit Sheth, and Heather Heckman. (2024). "GEAR-Up: Generative AI and External Knowledge-based Retrieval Upgrading Scholarly Article Searches for Systematic Reviews." Proceedings of the AAAI Conference on Artificial Intelligence.
Download Paper