Multi-Oriented and multi-scaled text character analysis and recognition in graphical documents and their apllications to document image retrieval

Abstract

With the advent research of Document Image Analysis and Recognition (DIAR), an important line of research is explored on indexing and retrieval of graphics rich docu- ments. It aims at nding relevant documents relying on segmentation and recognition of text and graphics components underlying in non-standard layout where commercial OCRs can not be applied due to complexity. This thesis is focused towards text infor- mation extraction approaches in graphical documents and retrieval of such documents using text information. Automatic text recognition in graphical documents (map, engineering drawing, etc.) involves many challenges because text characters are usually printed in multi- oriented and multi-scale way along with di erent graphical objects. Text characters are used to annotate the graphical curve lines and hence, many times they follow curvi-linear paths too. For OCR of such documents, individual text lines and their corresponding words/characters need to be extracted. For recognition of multi-font, multi-scale and multi-oriented characters, we have proposed a feature descriptor for character shape using angular information from con- tour pixels to take care of the invariance nature. To improve the e ciency of OCR, an approach towards the segmentation of multi-oriented touching strings into individual characters is also discussed. Convex hull based background information is used to segment a touching string into possible primitive segments and later these primitive segments are merged to get optimum segmentation using dynamic programming. To overcome the touching/overlapping problem of text with graphical lines, a character spotting approach using SIFT and skeleton information is included. Afterwards, we propose a novel method to extract individual curvi-linear text lines using the fore- ground and background information of the characters of the text and a water reservoir concept is used to utilize the background information. We have also formulated the methodologies for graphical document retrieval ap- plications using query words and seals. The retrieval approaches are performed using recognition results of individual components in the document. Given a query text, the system extracts positional knowledge from the query word and uses the same to generate hypothetical locations in the document. Indexing of documents is also per- formed based on automatic detection of seals from documents containing cluttered background. A seal is characterized by scale and rotation invariant spatial feature descriptors computed from labelled text characters and a concept based on the Generalized Hough Transform is used to locate the seal in documents. Keywords: Document Image Analysis, Graphics Recognition, Dynamic Pro- gramming, Generalized Hough Transform, Character Recognition, Touching Charac- ter Segmentation, Text/Graphics Separation, Curve-Line Separation, Word Retrieval, Seal Detection and Recognition.