VoiceXML 2.1 Development Guide

| TOC

Audio Formats

The Voxeo platform supports a number of different audio formats for our VoiceXML platforms. When converting your sound files, you may want to keep in mind that ALL audio files eventually get converted when output by the phone to a 8bit, 8Khz format upon playback, so for better performance in your application, it is always best to have your files in this format to begin with. The supported sound formats for the Voxeo VXML platform are as follows:

8bit, 8Khz, u-law Wave files (Riff Format)
16bit, 8Khz, PCM Wave files (Riff Format)
8bit, 8Khz, u-law Sphere files
16bit, 8Khz, PCM Sphere files

Alternate Text-To-Speech

One of the features of VoiceXML is that it allows you to nest backup TTS within your audio element which will be played to the caller if the audio resource cannot be found. This is somewhat standard procedure for all commercially deployed VoiceXML applications. The syntax neccessary for implementing backup TTS is:



  <audio src="MyBogusFile.wav"> Here is some backup TTS. </audio>

But if we want to have some fine-grained control over our users bargein, then we can further nest the <audio> within a <prompt>:



  <prompt bargein="false">

  <audio src="MyFile.wav">

   Here is some backup TTS.

 </audio> 

 </prompt>

Audio Terminology

Amplitude - is basically the "volume" or loudness of a moment of audio.

Sample - is a recording of a "moment in time" of audio. Basically, each sample records the amplitude or volume of the audio at a moment in time. Audio recordings are made up of thousands and thousands of individual samples, each of which indicates the amplitude of the audio at that moment.

Sample rate - how fast audio is recorded or "sampled". An 8khz sample rate means 8,000 pieces or moments of audio amplitude information are recorded per second. 16khz means 16,000 per second.

Sample bits - how much information is in each sample. Typically, each sample has 8 or 16 bits of information.

Peak Amplitude - is the "peak" or loudest (or highest) volume/amplitude encountered during all the samples in an audio recording.

RMS Amplitude - is the average volume or amplitude in an audio recording.

CODEC or Encoding - indicates how the audio samples are represented in the sample bits, and/or how they are "compressed" to fit more information into the sample bits. For example, PCM encoding saves the amplitude values into the sample bits without any compression. uLaw or aLaw effectively store 16 bits of sample information into just 8 bits.
GSM, MP3, RealAudio, and other types of encoding store even more information into less bits.

Some Facts

All telephony uses 8khz sampling. 8000 samples per second is a universal constant in telephony.

Traditional telephony uses 8 bit samples with either uLaw or aLaw encoding. VOIP telephony commonly uses uLaw (also known as G.711), aLaw, GSM, or other encoding/compression such as G.723.1, G.729 to squeeze more audio into less bandwidth. FYI, future VOIP telephony may support "wideband" encoding that uses higher sample rates and encoding
formats to deliver the equivalent of Hi-Fi for telephony. We're already seeing "Hi Fi" type telephony audio on some high-end video conferencing solutions. However, to get back to reality and today, it's all 8khz ubit uLaw.

If you look at a microphone cable, you'll notice it has two wires. If you look at a speaker cable, you'll notice it also has two wires. Now look at a phone cable - phones have both a microphone and a speaker, but phone cables can have just two wires per phone line, and it works fine. But wait a minute? If every telephone has a microphone (2 wires) and a speaker (2 wires), that takes 4 wires. So how does it work with two? Simply put, phones combine or "mix" the input and output audio
onto one set of wires.

Because telephones combine input and output audio, telephones can and do create echo - just like what happens when you put a microphone too close to a speaker. Almost every call we make has echo; however, the human mind automatically "filters out" an echo that occurs less than 200ms or so from when we said something, and most telephony echoes come
back in under 200ms. So, our perception is that we don't here the echo. However, sometimes factors can combine to make the echo take more than 200ms to get back to us, and we start to here it. Anything that introduces latency or delay on a phone call can make the echo delay long enough to "hear it", including: calls on cell phones, which have more latency and thus more echo delay; calls to the other side of the world, where the audio takes longer to get there, and thus the echo takes
longer to get back; calls via satelite, etc.

Because echo can take more than 200ms on some types of telephony in some situations, telephony devices often have "echo cancelers". Echo cancelers are filters that attempt to "remove" the echo audio before the
audio is sent to our ears, (or to the recognition engines...more on that below).

Some Problems

When recording audio for use in telephony / IVR systems, the audio ultimately must be converted from bit rates other than 8khz and audio encodings other than 8bit uLaw - because 8bit ulaw is the only type of audio you can send over the phone network (at least here in the US). This conversion from higher bit rates and alternate encoding types can
occur in the IVR platform, but it's best for the conversion to be done in advance by IVR developers or prompt studios.

There are many audio tools out there that can convert bit rates and audio encoding, but few which do these conversions well.

Often after conversion, the resulting audio may sound too low; that is, the RMS amplitude - the average volume - is lower than callers are comfortable with. To fix this, many audio tools offer a "gain control" feature to boost the volume of audio.

Sometimes after conversion, the resulting audio may be TOO LOUD, that is, the RMS amplitude - the average volume - is higher than callers are comfortable with. Again, gain control can be used to fix these problems.

Sometimes, audio has POPS and CLICKS in it. These can result from a few things - bad recording, bad gain, or bad samples. If the audio was recorded with a microphone or telephone that is cheap or glitchy, those microphones and telephones can introduce pops and clicks. However, often, pops and clicks result from gain (ie volume increase) being "turned up" too high. When gain is too high, audio can "clip" as the peak amplitude in an audio recording exceeds the maximum volume that can
be used with the selected encoding format.

Sometimes, audio is so loud that the voice recognition engine can "hear" it. What the reco engine actually is hearing is the ECHO from an audio prompt. While our mind filters out echo under 200ms, the reco engine does not. The louder a prompt. the louder its echo will be. The echo can become so loud that the reco engine mistakes it as being a command or question spoken by the caller, and that's bad for your user-experience.

Some Solutions

Audio should always be converted to 8khz uLaw for use in the Voxeo platform... because one way or another, it's going to end up that way.

Programs like cooledit (now Audio Audition), Goldwave, and SoundForge can all be used to convert audio formats. However, SoundForge is the best - and also the most expensive at $399

While these programs can be used to convert audio, getting the right combination of audio sample rate, encoding, peak amplitude, and RMS amplitude in order to make prompts comfortable for users - but not so loud that the audio has pops or clicks or the reco engines hear echoes from the prompts - can be quite challenging. For these reasons, Voxeo always recommends that our customers use the services of professional IVR prompt recording companies like The Great Voice Co. to record their
prompts. In short, it seems like it should be easy to record your own high-quality audio prompts; but it is most assuredly not.

Ignoring Our Advice -- Recording your own prompts

If you don't plan on mixing and editing the prompts they record, it's best to just record at 8khz/16bit and then save or convert to 8khz/8bit-ulaw.

If you plan on mixing and editing prompts they record, it's best to record at 48Khz/16bit, and not 44Khz, (the down-sample to 8Khz works much better when you are dealing with something that is a multiple of 8k), and then save or convert to 8khz/8bit-ulaw when you are done mixing and editing their prompts.

Most definitely use the gain control tools in your audio software to adjust the RMS amplitude and peak amplitude of your audio recordings so that audio is loud enough to be heard by customers; but not so loud that prompt echoes will be heard by the reco engine; and not so loud that pops and clicks are heard as a result of clipping.

ANNOTATIONS: EXISTING POSTS

fliptone

2/14/2007 11:05 AM (EST)

Can I nest an <audio> tag inside an <audio> tag. I want to use this to play a custom prompt if one exists otherwise play the core prompt, like this:

<prompt timeout="5s">
<audio src="CustomWavs/Test.wav">
<audio src="CoreWavs/Test.wav" />
</audio>
</prompt>

voxeojeff

2/14/2007 12:50 PM (EST)

Hi fliptone,

The <audio> element can have alternate content in case the audio sample is not available:

If the audio file cannot be played (e.g. 'src' referencing or 'expr' evaluating to an invalid URI, a file with an unsupported format, etc), the content of the audio element is played instead. The content may include text, speech markup, or another audio element. If the audio file cannot be played and the content of the audio element is empty, no audio is played and no error event is thrown.

For example:

<?xml version="1.0" encoding="UTF-8"?>
<vxml version = "2.1">
<form id="F_1">
<block>
<prompt timeout="5s">
<audio src="PlayMeFirst.wav">
<audio src="ElsePlayMe.wav"/>
</audio>
</prompt>
</block>
</form>
</vxml>

This sample code will execute 'PlayMeFirst.wav' if it can be located, and will stop there. But, if it cannot be found, it will play 'ElsePlayMe.wav' instead, assuming it is there and can be played.

I hope this helps!

Cheers,

Jeff Menkel
Voxeo Corporation

fliptone

2/14/2007 2:11 PM (EST)

thanks. I tried that but did not hear the second prompt on the Voxeo SIP Phone. I do hear nested TTS. Maybe this is a bug in the SIP Phone implementation?

VoxeoBrian

2/14/2007 2:23 PM (EST)

Hello,

We tested this in our local install of prophecy over the SIP phone and did not experience the issues you are speaking of. I must mention that in order for the second audio to file to be hit, the first file must not be present or must fail in its fetch of the file. Try renaming the first audio file to a file that does not exist, that will ensure the second audio file is hit. Please go ahead and give this a try and let us know your findings, as always we are standing by to assist.

Cheers

Brian

fliptone

2/15/2007 6:28 PM (EST)

found the error. I had an old Prophecy version. Downloaded the latest and it works just as you said. thanks!

danielvinson

3/26/2007 5:52 AM (EDT)

Hi
When do you think you will begin supporting the MP3 format?
Regards
Daniel

VoxeoBrian

3/26/2007 11:24 AM (EDT)

Hello,

We currently do not have plans to offer support for .mp3 audio files. As with anything though, if the demand for the inclusion of this audio format becomes considerable, our engineering team may at that time incorporate such a feature.

Regards

Brian

hashakie

8/29/2007 5:01 AM (EDT)

Does Voxeo supports 3gpp file for audio playback?

VoxeoDante

8/29/2007 12:13 PM (EDT)

Hello,

I will just need a little more information before I can completely answer this question. To my knowledge there is not a .3gpp audio format. If I am mistaken, please correct me. I do know there is a .3gp format but it is for video not audio.

If there is a specific audio format you would like to know about please let me know which file extension interests you. If you were in fact asking about the .3gp video format, then we do not currently support that but plan to sometime in the future.

There are a few notes I would like to make about using audio for playback over the phone.

- All telephony uses 8khz sampling. 8000 samples per second is a universal constant in telephony.

- When recording audio for use in telephony / IVR systems, the audio ultimately must be converted from bit rates other than 8khz and audio encodings other than 8bit uLaw - because 8bit ulaw is the only type of audio you can send over the phone network (at least here in the US). This conversion from higher bit rates and alternate encoding types can
occur in the IVR platform, but it's best for the conversion to be done in advance by IVR developers or prompt studios.

- If you don't plan on mixing and editing the prompts they record, it's best to just record at 8khz/16bit and then save or convert to 8khz/8bit-ulaw.

Here is a link to the documentation we have on using audio for telephony applications.

http://docs.voxeo.com/voicexml/2.0/appendixf.htm

Please advise,
Dante Vitulano

voxeojeremy

12/7/2007 1:25 PM (EST)

It is not possible to play more than 15 minutes audio (TTS, several consecutive prompts, etc) in a single input action for items like <field>, <record>, etc.

So the work-around strategy should be the following:

An array of prompts for each topic should be created and assigned to prompts which are supposed to be played, then after each single prompt is played, the application should wait for no input for a short time (say 100ms) and then play the next slice.

Code snippet is below:

<var name="PromptSet" expr="new Array()" /> <var name="n_prompts"/> <var name="i_prompt" expr="-1" /> <var name="cur_prompt"/>

<script>
<![CDATA[
PromptSet [0] = p1.wav;
PromptSet [1] = p2.wav;
n_prompts = 1;

]]>
</script>
<block name="start_prompt_set">
<script>
<assign name="i_prompt" expr=" i_prompt + 1" /> <if cond=" i_prompt > n_prompts ">
<goto next="#DONE_PLAYING_ PROMPT_SET"/> </if> <assign name="cur_prompt" expr=" PromptSet[i_prompt]" /> <goto nextitem="wait_for_hot_interrupt"/>
</block>

<field name=" wait_for_hot_interrupt ">
<grammar>{whatever grammar you have}</grammar>
<prompt bargein="true" bargeintype="hotword" timeout="100ms">
<audio expr=" cur_prompt "/>
</prompt>
<noinput>
<goto nextitem="start_prompt_set"/>
</noinput
<filled>

{do whatever is done right now}

</filled>
</field>

Cheers!

Jeremy McCall
Voxeo Extreme Support

abhijeet

3/17/2008 2:30 AM (EDT)

How can I control the play of an audio file? For e.g. if I want to let the user pause, fast forward or rewind the messae or such audio control;on the pres of DTMF key.

voxeojeff

3/17/2008 9:43 AM (EDT)

Hello,

The ability to pause and resume playback of an audio file is not something that is currently possible in the VoiceXML specification. This can however be done with some clever server side logic that will rely on tracking the .wav bit files and manipulating it to move through the recording. While we don't have any examples of this being done, some of our customers have been able to accomplish this. Please let me know if this helps clarify things for you, as well if you have any additional questions.

Regards,

Jeff

F: All Things Audio

| TOC

| The Voxeo Audio Library