Paid Project , Project ,

Improving Text Recognition capabilities using Transformer

In this project, we’ll improve CRNN model using the Beam Search technique for text generation. Then we’ll improve the text recognition capabilities using a State-of-the-art transformer model. We’ll use TrOCR model from Microsft for building the OCR model. For that we’ll use the base model and make inferences on some images using that, and then we’ll fine-tune that TrOCR base model on the IAM dataset using HuggingFace tool. On completion of the training of the model, we’ll use TrOCR_hadwritting model (which is fine-tuned on the IAM lines dataset).

View Project details

What will you Learn in the Project?

Making inference using CRNN model with beam search decoding
Understand HuggingFace model and processor module
Inferencing on TrOCR model using HuggingFace libraries
Fine-tuning the OCR model on the IAM dataset

Tools & Technologies Used

Google Colab
HuggingFace (transformers, datasets)
PyTorch (Dataset)
sklearn
Pandas

Prerequisite

Working knowledge of tools such as Tensorflow, Huggingface (transformer, datasets), library
Understanding of Dataset module of Pytorch library
Good theoretical understanding of concepts related to Transformer [Encoder-Decoder] architecture and text generation concepts such as Beam Search.
Understanding of TrOCR model architecture (which is the Transformer based OCR model).