Document Summarization

flask-cors
tiktoken
unstructured
Flask==2.0.1
llama-index
llama-index-vector-stores-qdrant
llama-index-llms-ollama
llama-index-embeddings-ollama
torch
sentence-transformers      
Werkzeug==2.2.2
qdrant-client
fastembed

• Create config.py
  • Configure all required parameters in the config file by replacing the necessary configuration values.
    • QDRANT_HOST, QDRANT_API_KEY, QDRANT_COLLECTION_NAME, OLLAMA_URL, LLM_MODEL,EMBED_MODEL, UPLOAD_FILE_PATH, INDEX_STORAGE_PATH
    • Get the cofiguration values for QDRANT_HOST, QDRANT_API_KEY, QDRANT_COLLECTION_NAME, OLLAMA_URL, LLM_MODEL,EMBED_MODEL from the Ollama and Qdrant Setup.

import os
# http api port
HTTP_PORT = os.getenv('HTTP_PORT', 7654)
#ollama
OLLAMA_URL= os.getenv('OLLAMA_URL','http://localhost:11434')
LLM_MODEL = os.getenv('LLM_MODEL','llama3.2')
EMBED_MODEL = os.getenv('EMBED_MODEL','mxbai-embed-large:latest')
# qdrant vector store config
#Replace this url with Qdrant actual url
QDRANT_HOST = os.getenv('QDRANT_URL','https://XXXXXXXX.us-west-2-0.aws.cloud.qdrant.io:6333')
#Replace this url with Qdrant API KEY
QDRANT_API_KEY = os.getenv('QDRANT_API_KEY','eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.XXXXXXXXXXXXXXXXXX')
QDRANT_COLLECTION_NAME = os.getenv('QDRANT_COLLECTION_NAME','advrag3')
UPLOAD_FILE_PATH = os.getenv('UPLOAD_FILE_PATH','XXXXX/work/file_storage')
INDEX_STORAGE_PATH = os.getenv('INDEX_STORAGE_PATH','XXXXX/work/index_storage')

• Create api.py
  • We will have three routes to support upload, embed, and chat functionalities:
  • /api/upload: Handles document uploads to the configured folder.
  • /api/embed: Internally calls the reload_index method from the app module to create embeddings and index.
  • /api/question: Internally calls the chat method to process the query and generate a response.”

from flask import Flask
from flask import jsonify
from flask import request
from flask_cors import CORS
import logging
import sys
from app import *
from config import *


app = Flask(__name__)
CORS(app)


logging.basicConfig(stream=sys.stdout, level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')


@app.route('/api/embed', methods=['GET'])
def createEmbed():
 if request.method == "GET":
 load_index()
 return jsonify({'status':'OK','message':'Created embeddings successfully!'}),200


@app.route('/api/upload', methods=['POST'])
def upload():
 if request.method == "POST":
 files = request.files.getlist("file")
 for file in files:
 file.save(os.path.join(f"{UPLOAD_FILE_PATH}", file.filename))
 return jsonify({'status':'OK','message':'File(s) uploaded successfully!'}),200


@app.route('/api/question', methods=['POST'])
def post_question():
 json = request.get_json(silent=True)
 question = json['question']
 user_id = json['user_id']
 logging.info("post question `%s` for user `%s`", question, user_id)
 resp = chat(question, user_id)
 data = jsonify({'answer':resp})

 return data, 200


if __name__ == '__main__':
 init_llm()
 init_index()
 init_query_engine()

 app.run(host='0.0.0.0', port=HTTP_PORT, debug=True)

• Initinatlizing init_llm(), init_index() and init_query_engine() methos in __main__ top leve code code environment from app module.

• Create app.py

• init_llm(): Initialize Ollama llm and embedding models

def init_llm():
llm=Ollama(base_url=f"{OLLAMA_URL}", model=f"{LLM_MODEL}", temperature=0.8, request_timeout=300,
system_prompt="You are an agent who consider the context passed "
"in, to answer any questions dont consider your prior "
"knowledge to answer and if you dont find the answer "
"please respond that you dont know.")

embed_model=OllamaEmbedding(base_url=f"{OLLAMA_URL}", model_name=f"{EMBED_MODEL}")
Settings.llm=llm
Settings.embed_model=embed_model

• init_index(): This method initializes the index using the configured persistent storage. On the first run, the persistent folder will be empty, and the load_index() method will be called to build the index, which will then be stored in the persistent store.

 

def init_index():
 global index
 global vector_store
 # create qdrant client
 qdrant_client = QdrantClient(
 url=f"{QDRANT_HOST}", 
 api_key=f"{QDRANT_API_KEY}",
 )
 
 # qdrant vector store with enabling hybrid search
 vector_store = QdrantVectorStore(
 collection_name=f"{QDRANT_COLLECTION_NAME}",
 client=qdrant_client,
 enable_hybrid=True,
 batch_size=20
 )
 
upload_files = len(os.listdir(f"{UPLOAD_FILE_PATH}"))
 if upload_files > 1:
 index_storage_files = len(os.listdir(f"{INDEX_STORAGE_PATH}"))
 if index_storage_files > 1:
 new_storage_context = StorageContext.from_defaults( vector_store=vector_store, persist_dir=f"{INDEX_STORAGE_PATH}")
 index = load_index_from_storage(new_storage_context)
 else:
 load_index()

• load_index(): This method loads files from the upload path, converts them into documents, and creates a vector store using the sentence window node parser and embed model. This method is responsible for creating embeddings, storing them in the vector store, and ultimately creating an index that is persisted to the index storage path.

• init_query_engine(): Loads the index from the local index storage path, applies the metadata replace post processorand re-rank nodes, and creates a query engine to enable user chat functionality.

• chat():

def chat(input_question, user): global query_engine
response = query_engine.query(input_question) logging.info("response from llm - %s", response)
 
# view sentece window retrieval window and origianl text logging.info("sentence window retrieval window - %s", response.source_nodes[0].node) logging.info("sentence window retrieval orginal_text - %s", response.source_nodes)
return response.response

• Install the dependencies using the following command

pip install -r requirements.txt

• Before running app.py, ensure that the LLM and Embed Model are running in the Ollama ecosystem and that the Qdrant vector store is set up

• Once the above steps are complete, run api.py using the following command

python api.py