Install KenLM Language Model

Language Model and KenLM Installations

Why this tutorial???

Each and Every blog gives some idea to improve our skills. So, this blog would help you to “Install KenLM Libraries” in Ubuntu. Before we go that, First we will see “What is Language Model”

Language modeling is the art of determining the probability of a sequence of words. This is useful in a large variety of areas including speech recognition, optical character recognition, handwriting recognition, machine translation, and spelling correction. Language models are widely applied in natural language processing, and applications such as machine translation make very frequent queries.

Language Model able to predict the next word in the sequence based on previous data. The best example for Language model is “Google recommend the next word based on your previous searches”. Google is using “BERT” Language Model, which is trained on several corpus.

KenLM Language Model helps in creating your own Scorer in Deepspeech.

mozilla/DeepSpeech

Deepspeech is one of the popular Speech to Text conversion engine which can run in real time or offline across devices.

Photo by Mert Talay on Unsplash

Steps to Install KenLM

Steps to Install KenLM Libraries

Step 1: Installing the Dependencies

sudo apt-get update
sudo apt-get install build-essential libboost-all-dev cmake zlib1g-dev libbz2-dev liblzma-dev
sudo apt-get install libboost-all-dev libeigen3-dev

Step 2: Clone the KenLM Git Repo

git clone https://github.com/kpu/kenlm

Step 3: Traverse to the Downloaded path and make directory

cd kenlm
mkdir build
cd build

Step 4: Compile and Install in Ubuntu

cmake ..
make -j 4
sudo make install

Your KenLM Bins is in the “/home/<username>/kenlm/build/bin”

Reference

kpu/kenlm

If you like this article and helpful to you, Kindly Clap as much as you can and share