2010-08-11
0.3

&kmyapplication; is an acoustic model manager.

KDE
kdeutils
Kapp
Simon
recognition
speech
voice
command
control
model
compilation
sam
accessibility
&FDLNotice;
2009-2010
Peter Grasch
Peter
H.
Grasch
peter.grasch@bedahr.org

# The &kmyapplication; Handbook

## Introduction

&kmyapplication; is a tool to create and test acoustic models. It can compile new speech models, use models created by Simon and produce models that can be used by Simon later on.

It is targeted towards people wanting more control over their acoustic model and provides much lower level access to the build process. &kmyapplication; is mainly geared towards speech professionals wanting to improve their acoustic model.

For more information on the architecture of the Simon suite please see the Simon manual.

### Background

This section will provide a bit of background information on the compilation and testing process.

#### Effective testing

One of the major features of sam is to test the generated acoustic models.

The basic testing procedure is to run recognition on samples where the transcription is already known and comparing the results. &kmyapplication; also takes the confidence score of the recognition into account to measure how robust the created system is.

Due to the way acoustic models are created, both the recognition accuracy and confidence will be highly skewed when the same samples are used both for training and testing. This is called "in corpus" testing (the samples used for testing are also in your training corpus).

While in corpus testing might tell you if the compilation process failed or produced sub par results it won't tell you the "real" recognition rate of the created model. Therefore it is recommended to do "out of corpus" testing: Use different samples for training and for testing.

For out of corpus testing simply split your prompts file in two parts: One used to compile the model and one to test the model. Of course the test set doesn't need to be very big to get a representative result.

If you don't have a lot of training data, you can also split the complete corpus in ten parts. Compile 10 models, each leaving out one part of the corpus. Then do then individual tests (always with the test set that was excluded during compilation) and average the results.

## Using &kmyapplication;

&kmyapplication; provides a graphical interface that is divided into five areas.

Main window Main window

The order of the tabs represents a complete sam workflow from creating a model to testing it.

### Input & output files

Define what files to work on. You can save and load this configuration by using the Save and Load buttons respectively.

The Save and Load menus only save the paths and options defined here. They don't save the associated files!

Input & output selection Input & output selection

If you want to compile and / or test the Simon model, you can use the Modify Simon's model option to load the appropriate files. Both scenarios and prompts will be serialized as appropriate.

When Simon compiles the model, it will automatically remove untrained words during serialization as well as adapting the prompts appropriately. This will also happen when selecting Modify Simon's model. If you do, however serialize them separately using Serialize scenarios and Serialize prompts this adaption will not be performed and you alone are responsible for the validity of the input files. If you provide file names directly this of course applies even more so.

Selecting a static model as the used model type will still compile the language model, just like Simon would do as well. It also copies the given base model input files to the output paths.

&kmyapplication; allows the user to provide two different types of prompts files. One prompts file to compile the model and test the model. For more information, please see the section on effective testing.

### Adapt scenarios

Simon stores the language model in scenarios. When you want to use them to compile the model, you first need to serialize them to files readable by Julius and the HTK.

Adapt scenarios Adapt scenarios

To do this, select Serialize scenarios button in the Input output tab. Simon's scenarios will also be serialized when using the Modify Simon's model options.

In this page you can find status information of the adaption and read detailed error messages if an error occurs.

### Create model

Here you can build the model using the input and output files defined in the input & output section.

The compilation process is identical to the one used by Simond. Unlike Simond, however, the full log of all called external programs, their output as well as progress information is shown regardless of whether the compilation was successful or not.

Create model Create model

Using this verbose output it is much easier to find issues with the input data.

### Test model

Here you can test the speech model.

&kmyapplication; will test the model set by in the Output files section in the input & output section.

During testing, Julius will run to recognize on the input files as set by the test prompts. Resulting word- and sentence errors will be counted as well as the overall robustness by analyzing confidence scores of the recognizer.

To test the model successfully you also need SoX installed and in your path.

While the test is running you can watch the current recognition in the test protocol.

Test model Test model

As soon as the test is finished, the test results will be automatically displayed.

### Test results

After a successful model test you can find a detailed report of the recognition accuracy here.

#### Scoring

To best reflect the recognition performance, &kmyapplication; uses multiple, ranked results for the tests.

A correctly recognized word or sentence will be scored with the confidence score the word achieved and the word will still be listed as "recognized" even though another recognition result might be ranked higher than the correct one.

If for example the sample "flowers" is recognized as "flower" (90%), "flowers" (70%), "apple" (12%) it will be marked as "recognized" with a score of 70%.

The displayed overall recognition rate is the average of the word recognition rate (average over each word) and the sentence recognition rate (average over each sentence).

#### Word

&kmyapplication; will list recognition accuracy for each word individually.

Word results Word results

If you have samples containing more than one word they will be segmented during recognition. Each word will be scored individually although the different words of course still influence each other.

#### Sentences

This section lists each prompt as "sentence".

Prompts that were recorded more than once are combined.

Sentence results Sentence results

#### Files

In the files section you can see the recognition results for each file. Each file will list the 10 most likely results in the details pane when you select it.

File results File results

When you identify problematic samples, you can re-record (or remove) them by selecting them and clicking on Edit sample.

You can sort the files by each column simply by clicking on the column header. This way it is very easy to find bad samples by sorting by recognition rate.

## Questions and Answers

In an effort to keep this section always up-to-date it is available at our online wiki.

## Credits and License

&kmyapplication;

Program copyright 2008-2010 Peter Grasch peter.grasch@bedahr.org

Documentation Copyright © 2009-2010 Peter Grasch peter.grasch@bedahr.org

&underFDL;

&underGPL;

## Installation

Please see our wiki for install instructions.

&documentation.index;