Personal tools
You are here: Home Invited speakers Stephen Watt

Stephen Watt

The Mathematics of Mathematical Handwriting Recognition

Stephen M. Watt

Computer Science Department
The University of Western Ontario
London, Ontario CANADA

Accurate computer recognition of handwritten mathematics offers to provide a
natural interface for mathematical computing, document creation and
collaboration.   Mathematical handwriting, however, provides a number of
challenges beyond what is required for the recognition of handwritten natural
languages. For example, it is usual to use symbols from a range of different
alphabets and there are many similar-looking symbols.  Many writers are
unfamiliar with the symbols they must use and therefore write them incorrectly.
Mathematical notation is two-dimensional and size and placement information is
important.   Additionally, there is no fixed vocabulary of mathematical
``words'' that can be used to disambiguate symbol sequences.  On the other hand
there are some simplifications. For example, symbols do tend to be
well-segmented.   With these charactersitics, new methods of character
recognition are important for accurate handwritten mathematics input.

We present a geometric theory that we have found useful for recognizing
mathematical symbols.   Characters are represented as parametric curves
approximated by certain truncated orthogonal series.  This maps symbols to the
low-dimensional vector space of series coefficients. The Euclidean distance in
this space is closely related to the variational integral between two curves
and may be used to find similar symbols very efficiently.   Training data sets
with hundreds of classes are seen to be almost linearly separable, allowing
classification by ensembles of linear SVMs.  We have seen that the distance to
separating planes provides a reliable confidence measure for classifications.
In this setting, we find it particularly effective to classify symbols by their
distance to the convex hulls of nearest neighbors from known classes.
Additionally, by choosing the functional basis appropriately, the series
coefficients can be computed in real-time, as the symbol is being written and,
by using trancated series for integral invariant functions,
orientation-independent recognition is achieved.

The beauty of this theory is that a single, coherent view provides several
related geometric techniques that give a high recognition rate and do not
rely on peculiarities of the symbol set.
Document Actions