Skip to Main Content
Lightning Talk Intermediate

OCR Multimodal Retrieval using ColiPali Model

Proposal status is Rejected
Session Description

I'll discuss ColPali, a model that extends PaliGemma-3B to generate ColBERT-style multi-vector representations for both text and images. We'll explore how ColPali enhances multimodal retrieval and search across text and image data, showcasing its recent benchmark success in vision retrieval tasks. The session will also cover the key concepts of ColBERT-style multi-vector representations and their impact on retrieval performance.


github link: https://github.com/samvardhan777/Colipali


ppt: https://github.com/samvardhan777/Colipali/blob/main/ppt/ColiPali%20Model%20OCR.pdf

References

https://medium.com/@samvardhan777/colbert-paligemma-colipali-for-document-retrieval-3bb8cc80326c

Session Categories

FOSS

Speakers

sam
sam Data Engineer | Thoughtworks

I have been working as a data engineer for two years. 🛠️ My professional experience includes significant projects in both the Pharma and BFSI sectors. 🏥💼 Known for my quick learning abilities, I thrive on solving complex data challenges. Additionally, I have a particular interest in developing high-velocity data pipelines and in data wrangling. Currently, I am deeply interested in the field of GenAI, especially the latest developments in large language models (LLMs). 🚀