I clarify how you can construct an app that generates a number of selection questions (MCQs) on any user-defined topic. The app is extracting Wikipedia articles which might be associated to the person’s request and makes use of RAG to question a chat mannequin to generate the questions.
I’ll display how the app works, clarify how Wikipedia articles are retrieved, and present how these are used to invoke a chat mannequin. Subsequent, I clarify the important thing elements of this app in additional element. The code of the app is offered here.
App Demo
The gif above reveals the person coming into the training context, the generated MCQ and the suggestions after the person submitted a solution.
On the first display the person describes the context of the MCQs that ought to be generated. After urgent “Submit Context” the app searches for Wikipedia articles which content material matches the person question.
The app splits every Wikipedia web page into sections and scores them based mostly on how intently they match the person question. These scores are used to pattern the context of the following query which is displayed within the subsequent display with 4 decisions to reply. The person can choose a selection and submit it by “Submit Reply”. It’s also doable to skip this query through “Subsequent Query”. On this case it’s thought-about that the query didn’t meet the person’s expectation. It is going to be averted to make use of the context of this query for the technology of following questions. To finish the session the person can select “Finish MCQ”.
The subsequent display after the person submitted a solution reveals if the reply was appropriate and gives an extra rationalization. Following, the person can both get a brand new query through “Subsequent Query” or finish the session with “Finish MCQ”.
The tip session display reveals what number of questions had been accurately and wrongly answered. Moreover, it additionally incorporates the variety of questions the person rejected through “Subsequent Query”. If the person selects “Begin New Session” the beginning display will likely be displayed the place a brand new context for the following session could be offered.
Idea
The goal of this app is to provide top quality and up-to-date questions on any user-defined matter. Thereby person suggestions is taken into account to make sure that the generated questions are assembly the person’s expectations.
To retrieve high-quality and up-to-date context, Wikipedia articles are chosen with respect to the person’s question. Every article is break up into sections whereas each part is scored based mostly on its similarity with the person question. If the person rejects a query the respective part rating will likely be downgraded to cut back the chance of sampling this part once more.
This course of could be separated into two workflows:
- Context Retrieval
- Query Era
That are described under.
Context Retrieval
The workflow how the context of the MCQs is derived from Wikipedia based mostly on the person question is proven under.
The person inserts the question that describes the context of the MCQs at first display. An instance of the person question may very well be: “Ask me something about stars and planets”.
To effectively seek for Wikipedia articles this question is transformed into key phrases. The key phrases of the question above are: “Stars”, “Planets”, “Astronomy”, “Photo voltaic System”, and “Galaxy”.
For every key phrase a Wikipedia search is executed of which the highest three pages are chosen. Not every of those 15 pages are match to the question offered by the person. To take away irrelevant pages on the earliest doable stage the vector similarity of the embedded person question and web page excerpt is calculated. Pages which similarity is under a threshold are filtered out. In our instance 3 of 15 pages had been eliminated.
The remaining pages are learn and divided into sections. As not your complete web page content material could also be associated to the person question, splitting the pages into sections permits to pick out components of the web page that match particularly effectively to the person question. Therefore, for every part the vector similarity towards the person question is calculated and sections with low similarity are filtered out. The remaining 12 pages contained 305 sections of which 244 had been stored after filtering.
The final step of the retrieval workflow is to assign a rating to every part with respect to the vector similarity. This rating will later be used to pattern sections for the query technology.
Query Era
The workflow to generate a brand new MCQ is proven under:
Step one is to pattern one part with respect to the part scores. The textual content of this part is inserted along with the person question right into a immediate to invoke a chat mannequin. The chat mannequin returns a json formatted response that incorporates the query, reply decisions, and an evidence of the proper reply. In case the context offered just isn’t appropriate to generate a MCQ that addresses the person question the chat mannequin is instructed to return a key phrase to establish that the query technology was not profitable.
If the query technology was profitable, the questions and the reply decisions are exhibited to the person. As soon as the person submits a solution it’s evaluated if the reply was appropriate, and the reason of the proper reply is proven. To generate a brand new query the identical workflow is repeated.
In case the query technology was not profitable, or the person rejected the query by clicking on “Subsequent Query” the rating of the part that was chosen to generate the immediate is downgraded. Therefore, it’s much less probably that this part will likely be chosen once more.
Key Parts
Subsequent, I’ll clarify some key elements of the workflows in additional element.
Extracting Wiki Articles
Wikipedia articles are extracted in two steps: First a search is run to seek out appropriate pages. After filtering the search outcomes, the pages separated by sections are learn.
Search requests are despatched to this URL. Moreover, a header containing the requestor’s contact info and a parameter dictionary with the search question and the variety of pages to be returned. The output is in json format that may be transformed to a dictionary. The code under reveals how you can run the request:
headers = {'Consumer-Agent': os.getenv('WIKI_USER_AGENT')}
parameters = {'q': search_query, 'restrict': number_of_results}
response = requests.get(WIKI_SEARCH_URL, headers=headers, params=parameters)
page_info = response.json()['pages']After filtering the search outcomes based mostly on the pages’ excerpts the textual content of the remaining pages is imported utilizing wikipediaapi:
import wikipediaapi
def get_wiki_page_sections_as_dict(page_title, sections_exclude=SECTIONS_EXCLUDE):
wiki_wiki = wikipediaapi.Wikipedia(user_agent=os.getenv('WIKI_USER_AGENT'), language='en')
web page = wiki_wiki.web page(page_title)
if not web page.exists():
return None
def sections_to_dict(sections, parent_titles=[]):
consequence = {'Abstract': web page.abstract}
for part in sections:
if part.title in sections_exclude: proceed
section_title = ": ".be part of(parent_titles + [section.title])
if part.textual content:
consequence[section_title] = part.textual content
consequence.replace(sections_to_dict(part.sections, parent_titles + [section.title]))
return consequence
return sections_to_dict(web page.sections)To entry Wikipedia articles, the app makes use of wikipediaapi.Wikipedia, which requires a user-agent string for identification. It returns a WikipediaPage object which incorporates a abstract of the web page, web page sections with the title and the textual content of every part. Sections are hierarchically organized that means every part is one other WikipediaPage object with one other listing of sections which might be the subsections of the respective part. The operate above reads all sections of a web page and returns a dictionary that maps a concatenation of all part and subsection titles to the respective textual content.
Context Scoring
Sections that match higher to the person question ought to get the next chance of being chosen. That is achieved by assigning a rating to every part which is used as weight for sampling the sections. This rating is calculated as follows:
[s_{section}=w_{rejection}s_{rejection}+(1-w_{rejection})s_{sim}]
Every part receives a rating based mostly on two components: how usually it has been rejected, and the way intently its content material matches the person question. These scores are mixed right into a weighted sum. The part rejection rating consists of two elements: the variety of how usually the part’s web page has been rejected over the very best variety of web page rejections and the variety of this part’s rejections over the very best variety of part rejections:
[s_{rejection}=1-frac{1}{2}left( frac{n_{page(s)}}{max_{page}n_{page}} + frac{n_s}{max_{s}n_s} right)]
Immediate Engineering
Immediate engineering is an important facet of the Studying App’s performance. This app is utilizing two prompts to:
- Get key phrases for the wikipedia web page search
- Generate MCQs for sampled context
The template of the key phrase technology immediate is proven under:
KEYWORDS_TEMPLATE = """
You are an assistant to generate key phrases to seek for Wikipedia articles that comprise content material the person desires to be taught.
For a given person question return at most {n_keywords} key phrases. Be certain each key phrase is an effective match to the person question.
Moderately present fewer key phrases than key phrases which might be much less related.
Directions:
- Return the key phrases separated by commas
- Don't return anything
"""This technique message is concatenated with a human message containing the person question to invoke the Llm mannequin.
The parameter n_keywords set the utmost variety of key phrases to be generated. The directions make sure that the response could be simply transformed to a listing of key phrases. Regardless of these directions, the LLM usually returns the utmost variety of key phrases, together with some much less related ones.
The MCQ immediate incorporates the sampled part and invokes the chat mannequin to reply with a query, reply decisions, and an evidence of the proper reply in a machine-readable format.
MCQ_TEMPLATE = """
You're a studying app that generates multiple-choice questions based mostly on instructional content material. The person offered the
following request to outline the training content material:
"{user_query}"
Based mostly on the person request, following context was retrieved:
"{context}"
Generate a multiple-choice query instantly based mostly on the offered context. The right reply should be explicitly said
within the context and will all the time be the primary choice within the decisions listing. Moreover, present an evidence for why
the proper reply is appropriate.
Variety of reply decisions: {n_choices}
{previous_questions}{rejected_questions}
The JSON output ought to comply with this construction (for variety of decisions = 4):
{{"query": "Your generated query based mostly on the context", "decisions": ["Correct answer (this must be the first choice)","Distractor 1","Distractor 2","Distractor 3"], "rationalization": "A quick rationalization of why the proper reply is appropriate."}}
Directions:
- Generate one multiple-choice query strictly based mostly on the context.
- Present precisely {n_choices} reply decisions, making certain the primary one is the proper reply.
- Embody a concise rationalization of why the proper reply is appropriate.
- Don't return anything than the json output.
- The offered rationalization shouldn't assume the person is conscious of the context. Keep away from formulations like "As said within the textual content...".
- The response should be machine readable and never comprise line breaks.
- Verify whether it is doable to generate a query based mostly on the offered context that's aligned with the person request. If it isn't doable set the generated query to "{fail_keyword}".
"""
The inserted parameters are:
user_query: textual content of person questioncontext: textual content of sampled partn_choices: variety of reply decisionsprevious_questions: instruction to not repeat earlier questions with listing of all earlier questionsrejected_questions: instruction to keep away from questions of comparable nature or context with listing of rejected questionsfail_keyword: key phrase that signifies that query couldn’t be generated
Together with earlier questions reduces the possibility that the chat mannequin repeats questions. Moreover, by offering rejected questions, the person’s suggestions is taken into account when producing new questions. The instance ought to make sure that the generated output is within the appropriate format in order that it may be simply transformed to a dictionary. Setting the proper reply as the primary selection avoids requiring an extra output that signifies the proper reply. When exhibiting the alternatives to the person the order of decisions is shuffled. The final instruction defines what output ought to be offered in case it isn’t doable to generate a query matching the person question. Utilizing a standardized key phrase makes it simple to establish when the query technology has failed.
Streamlit App
The app is constructed utilizing Streamlit, an open-source app framework in Python. Streamlit has many capabilities that enable so as to add web page parts with just one line of code. Like for instance the aspect wherein the person can write the question is created through:
context_text = st.text_area("Enter the context for MCQ questions:")the place context_text incorporates the string, the person has written. Buttons are created with st.button or st.radio the place the returned variable incorporates the knowledge if the button has been pressed or what worth has been chosen.
The web page is generated top-down by a script that defines every aspect sequentially. Each time the person is interacting with the web page, e.g. by clicking on a button the script could be re-run with st.rerun(). When re-running the script, it is very important carry over info from the earlier run. That is performed by st.session_state which may comprise any objects. For instance, the MCQ generator occasion is assigned to session states as:
st.session_state.mcq_generator = MCQGenerator()in order that when the context retrieval workflow has been executed, the discovered context is offered to generate a MCQ on the subsequent web page.
Enhancements
There are a lot of choices to boost this app. Past Wikipedia, customers may additionally add their very own PDFs to generate questions from customized supplies—akin to lecture slides or textbooks. This may allow the person to generate questions on any context, for instance it may very well be used to organize for exams by importing course supplies.
One other facet that may very well be improved is to optimize the context choice to reduce the variety of rejected questions by the person. As a substitute of updating scores, additionally a ML mannequin may very well be skilled to foretell how probably it’s {that a} query will likely be rejected with respect to options like similarity to accepted and rejected questions. Each time one other query is rejected this mannequin may very well be retrained.
Additionally, the generated query may very well be saved in order that when a person desires to repeat the training train these questions may very well be used once more. An algorithm may very well be utilized to pick out beforehand wrongly answered questions extra often to concentrate on bettering the learner’s weaknesses.
Abstract
This text showcases how retrieval-augmented technology (RAG) can be utilized to construct an interactive studying app that generates high-quality, context-specific multiple-choice questions from Wikipedia articles. By combining keyword-based search, semantic filtering, immediate engineering, and a feedback-driven scoring system, the app dynamically adapts to person preferences and studying objectives. Leveraging instruments like Streamlit permits speedy prototyping and deployment, making this an accessible framework for educators, college students, and builders alike. With additional enhancements—akin to customized doc uploads, adaptive query sequencing, and machine learning-based rejection prediction—the app holds sturdy potential as a flexible platform for customized studying and self-assessment.
Additional Studying
To be taught extra about RAGs I can suggest these articles from Shaw Talebi and Avishek Biswas. Harrison Hoffman wrote two glorious tutorials on embeddings and vector databases and building an LLM RAG Chatbot. The way to handle states in streamlit could be present in Baertschi’s article.
If not said in any other case, all photographs had been created by the writer.
