Named Entity Recognition

Let's say you would like to automatically identify people, places, organizations or brands within blocks of text. What you need is a system that will perform Named Entity Recognition.

NER is a challenge that has been extensively studied over the last several years. And there are many software packages available that you can use to identify various different types of entities. Here's a list that I recently compiled, mainly open source, mainly English-centric, but in no particular order.

GATE

http://gate.ac.uk/

An open source, Java framework for various different types of text processing, including NER.

ANNIE

http://www.aktors.org/technologies/annie/

Open source Java, from the University of Sheffield. Part of GATE.

Python NLTK

https://sites.google.com/site/naturallanguagetoolkit/book

Very widely used open source Natural Language Tool Kit for Python.

Lingpipe

http://alias-i.com/lingpipe/index.html

Commercially supported Java, straight out of Brooklyn. Also, Lingpipe has a nice list of competitors (some of which duplicates what I’ve listed in this post):

http://alias-i.com/lingpipe/web/competition.html

MinorThird

https://github.com/TeamCohen/MinorThird/wiki

Java open source code from CMU.

Stanford Named Entity Recognizer

http://nlp.stanford.edu/software/CRF-NER.shtml

Java open source code from Stanford.

Illinois Named Entity Tagger

http://cogcomp.cs.illinois.edu/page/software_view/4

Java open source code from Illinois.

Factorie

http://code.google.com/p/factorie/

Probabilistic programming with imperatively-defined factor graphs. I don’t know what this means, either. It is written in Scala, though, which is the latest hipster programming language.

Apache Mahout

http://mahout.apache.org//

Java, open source, classification. (I’m not actually certain it does NER, but it looks interesting).

A couple of academic papers I found, that evaluate NER systems:

http://nlp.cs.nyu.edu/sekine/papers/li07.pdf

http://www.cicling.org/2009/RCS-41/047-058.pdf