What's That Noise?! [Ian Kallen's Weblog]

« Regexp'ing simple... | Main | MSN bows to China »

Friday January 06, 2006

Open Source Language Detection

No, not a typo. OSDL is something else. I'm interested in OSLD. I've used Language::Guess to detect languages in arbitrary text with Perl, it works pretty well. But how are folks solving the problem in Java?

It looks like Oracle has language detection as part of their "Globalization Development Kit" ... but what about open source? Sadly, the Nutch Language Identifier Plugin only supports European languages, no CJK. What are the other options?

opensource open source i18n language java perl nutch oracle

( Jan 06 2006, 02:22:54 PM PST ) Permalink

Comments:

Technorati

Who?

I'm Ian Kallen. I'm an entrepreneur and technologist working on Blockboard. My technical interests are focused on mobile, location, cloud computing, big data, real time event systems, search, devops and agile web application code. I'm a long time open source enthusiast and enjoy right-sizing code architecture and infrastructure with the scope and stage a technology startup is operating at.

Previously, I was an independent technology consultant. Prior to that, I did a long stint with Technorati leading the Core Services engineering group in architecting a backend to support search and exploring the web as it changes in real time. I have an eclectic background in technology, publishing, social software and ... life. I like to think: well rounded. I have been online participating in virtual communities since 1988 and developing online resources, services and systems as well as enterprise software since 1991. Once in a while, I have time and inspiration to blog.

The views expressed here are not necessarily those of any present or past employers. KTHXBAI

Feed: