US8719016B1

US8719016B1 - Speech analytics system and system and method for determining structured speech

Info

Publication number: US8719016B1
Application number: US12/755,549
Authority: US
Inventors: Omer Ziv; Ran Achituv; Ido Shapira
Original assignee: Verint Americas Inc
Current assignee: Verint Americas Inc; Verint Systems Inc
Priority date: 2009-04-07
Filing date: 2010-04-07
Publication date: 2014-05-06
Also published as: US9401145B1

Abstract

A method for converting speech to text in a speech analytics system is provided. The method includes receiving audio data containing speech made up of sounds from an audio source, processing the sounds with a phonetic module resulting in symbols corresponding to the sounds, and processing the symbols with a language module and occurrence table resulting in text. The method also includes determining a probability of correct translation for each word in the text, comparing the probability of correct translation for each word in the text to the occurrence table, and adjusting the occurrence table based on the probability of correct translation for each word in the text.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority to U.S. Provisional Patent Application Ser. No. 61/167,495, and entitled “STRUCTURED SPEECH”, filed on Apr. 7, 2009, and Application Ser. No. 61/178,795, and entitled “SPEECH ANALYTICS SYSTEM”, filed on May 15, 2009, the contents of which are hereby incorporated by reference in their entirety.

BACKGROUND

Conversations within contact centers are typically more structured than everyday speech. The contact center conversations may contain a mixture of free conversation and structured speech. Structured speech is sequences that have a higher repetition rate than free speech. Structured speech may include scripts that are read word-for-word by agents, computer generated voice mail messages, interactive voice response (IVR) generated speech, and figures of speech.

Accuracy is also a concern when translating speech to text to generate transcripts of conversations in the contact center. When performing speech to text there are often errors in the conversion. This may be cause by noise on the line, speakers do not speak clearly, or transcription system itself has errors. In long texts, the probability of errors increases. Thus, it is difficult to determine agent compliance to scripts and to verify quality assurance.

SUMMARY

In accordance with some implementations described herein, there is provided a method for determining structured speech. The method may include receiving a transcript of an audio recording created by a text-to-speech communication processing system. Thereafter, analyzing text in the transcript to determine repetitions within the text, where the repetitions being indicative of structured speech. From the repetitions, a duration distribution of the repetitions may be determined to ascertain a first type of structured speech. The first type of structured speech many be interactive voice response (IVR) generated speech. A length of the repetitions may be determined to ascertain a second type of structured speech. The second type of structured speech may be scripts spoken by, e.g., agents in the contact center.

In accordance with some implementations, there is provided a method for converting speech to text in a speech analytics system. The method may include receiving audio data containing speech made up of sounds from an audio source, processing the sounds with a phonetic module resulting in symbols corresponding to the sounds, and processing the symbols with a language module and occurrence table resulting in text. The method also may include determining a probability of correct translation for each word in the text, comparing the probability of correct translation for each word in the text to the occurrence table, and adjusting the occurrence table based on the probability of correct translation for each word in the text.

This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing summary, as well as the following detailed description of illustrative implementations, is better understood when read in conjunction with the appended drawings. For the purpose of illustrating the implementations, there are shown in the drawings example constructions; however, the implementations are not limited to the specific methods and instrumentalities disclosed. In the drawings:

FIG. 1 illustrates a speech to text translation system;

FIG. 2 illustrates a method for translating speech to text in a speech analytics system;

FIG. 3 illustrates a method of determining a type of speech in accordance with a structure;

FIGS. 4A and 4B illustrate graphical representations of deviations from a particular sentence;

FIG. 5 illustrates a relationship of a number of mistakes to a percentage with respect identifying a sentence as being a script; and

FIG. 6 illustrates the communication processing system of FIG. 1 in greater detail.

DETAILED DESCRIPTION

The following description and associated drawings teach the best mode of the invention. For the purpose of teaching inventive principles, some conventional aspects of the best mode may be simplified or omitted. The following claims specify the scope of the invention. Some aspects of the best mode may not fall within the scope of the invention as specified by the claims. Thus, those skilled in the art will appreciate variations from the best mode that fall within the scope of the invention. Those skilled in the art will appreciate that the features described below can be combined in various ways to form multiple variations of the invention. As a result, the invention is not limited to the specific examples described below, but only by claims and their equivalents.

Structured speech has a different statistical behavior than free conversation. By understanding the statistical distinction, structured speech may be automatically identified based in Large-Vocabulary Continuous Speech Recognition (LVCSR) outputs from contact centers, given a set of transcribed calls. For example, a different retrieval criterion may be used for structured speech than free conversation. By exploiting the fact that structured speech includes long sentences, a high level of precision and recall may be obtained, in view of the following.

FIG. 1 is a block diagram illustrating a speech analytics system 100. Speech analytics system 100 includes audio source 102, audio source 104, communication network 106, communication processing system 108, recorder 109 and database 110. Audio source 102 exchanges data with communication network 106 through link 114, while audio source 104 exchanges data with communication network 106 through link 116. Communication processing system 108 exchanges data with communication network 106 through link 118, with database 110 through link 120, and with recorder 109 through link 122. Recorder 109 may also communicate with the network 106 over link 124.

Links

110, 114, 116, 118, 120, 122 and 124 may use any of a variety of communication media, such as air, metal, optical fiber, or any other signal propagation path, including combinations thereof. In addition,

links

110, 114, 116, 118, 120, 122 and 124 may use any of a variety of communication protocols, such as internet, telephony, optical networking, wireless communication, wireless fidelity, or any other communication protocols and formats, including combinations thereof. Further,

links

110, 114, 116, 118, 120, 122 and 124 could be direct links or they might include various intermediate components, systems, and networks.

Communication network

106 may be any type of network such as a local area network (LAN), wide area network (WAN), the internet, a wireless communication network, or the like. Any network capable of transferring data from one device to another device may operate as communication network 106.

The speech analytics system 100 may include recorder 109 that stores the speech from audio source 102 or audio source 104 for later retrieval by communication processing system 108 or other downstream devices or systems. The

audio sources

102 and 104 may be any source, such as a telephone, VoIP endpoint, mobile device, general purpose computing device, etc. The recorded speech is made up of a plurality of sounds that are then translated to text.

Communication processing system

108 may be any device capable of receiving data through communication network 106 from other devices, such as

audio sources

102 and 104, processing the data, and transmitting data through network 106 to other devices. For example, communication processing system 108 may include a processing system for processing data, a communication interface for receiving and transmitting data, a storage system for storing data, and a user interface. One example embodiment of communication processing system 108 is represented by the communication processing system 108 illustrated in FIG. 6 and described in detail below.

Communications processing system

108 receives speech made up of sounds from audio source 102, audio source 104 or recorder 109, and proceeds to convert the speech to text. First, communication processing system 108 uses a phonetic module to convert the sounds into symbols corresponding to the sounds. Next, communication processing system 108 uses a language module and occurrence table to convert the symbols into text. In addition, communication processing system 108 determines a probable accuracy for each word translated. This probability may be based upon the words proceeding or following the selected word. Finally, communication processing system 108 compares the probable accuracies for each word translated with an occurrence table and adjusts the occurrence table as indicated by the probable accuracies for each word.

In an example, the occurrence table may be based upon the occurrence of words in a test sample. There may be a variety of test samples and occurrence tables corresponding to different dialects, languages, or regional slang. There may also be a variety of test samples and occurrence tables corresponding to different domains such as banking, law, commerce, phone centers, technical support lines, or the like. When speech of a known dialect or domain is received, it is compared to a corresponding occurrence table, and the appropriate occurrence table is updated based upon the probable accuracy of the translation of the speech. Thus, the occurrence table for different dialects and domains are continually being updated as more speech is translated.

While FIG. 1 illustrates communication processing system 108 as a single device, other embodiments may perform the functions of communication processing system 108 in a plurality of devices distributed throughout communication network 108. For example, separate devices may be provided for each different method of speech to text translation, and their resulting transcriptions may then be transmitted to communication processing system 108 for compilation into database 110 and for generation of a database index. Still other examples may provide for translation of the audio data to text within communication processing system 108, while the function of creating the database index may actually be performed within database 110. FIG. 1 is simply representative of one possible structure for performing the methods described here for indexing a database.

In an example embodiment, communication processing system 108 receives audio data from

audio sources

102 and 104 through communication network 106. This audio data may utilize any of a wide variety of formats. The audio data may be recorded as .mp3 or .wav files or the like. Further, the audio data may include one or more conversations within a single data file or group of data files. In some embodiments, the audio data may be translated from speech to text by other elements (not shown) within communication network 106, and the translated text may then be provided to communication processing system 108.

Communication processing system

108 processes audio data received from

audio sources

102 and 104, producing an index of symbols found within the audio data. These symbols may include phonemes, words, phrases, or the like. The index of symbols may be stored in database 110 in some embodiments. Communication processing system 108 then processes the index of symbols searching for symbols that have a deviation in frequency of occurrence within a time period. This time period may be of any length. For example, communication processing system 108 may receive daily updates of audio data and search for symbols having a deviation in frequency of occurrence in comparison to the audio data received for the previous week. Other embodiments may use other periods of time in a similar method.

FIG. 2 illustrates a method of performing speech to text translation in a speech analytics system 106. Audio data containing speech made up of sounds is received from either audio source 102 or recorder 104 (operation 200). Speech analytics system 106 processes the sounds using a phonetic module producing symbols corresponding to the sounds (operation 202). Speech analytics system 106 then processes the symbols using a language module and occurrence table producing text (operation 204).

Speech analytics system

106 determines a probability of correct translation for each word in the text (operation 206). This probability may be based in part or in whole on the words proceeding or following the selected word. Speech analytics system 106 compares the probability of correct translation for each word in the text to an appropriate occurrence table (operation 208). This occurrence table may be selected based upon a number of factors such as dialect or language of the speech, and the domain in which the speech was obtained.

Speech analytics system

106 then modifies the occurrence table based on the probability of correct translation for each word in the text (operation 210). This modification may simply change the occurrence probability by a fixed percentage, or by a variable percentage based on the probability of correct translation of the given word, or any other of a wide variety of methods for modification.

Speech analytics system

In accordance with aspects of the disclosure, structured speech may be identified. Structure speech may include IVR, scripts, and figures of speech. With regard to IVR, this type of structured speech is repetitive. Thus, the communication processing system 108 can recognizing errors in a transcription by taking advantage of the repetitive nature of IVR speech. An IVR message may be, “Welcome to ABC Products customer service center. All of our representatives are busy assisting other customers.” Similarly, a voicemail system prompt may be, “You have reached the voice mail box of Jane Doe, please leave your message after the tone.”

Scripts are another type of structured speech, and are typically statements spoken by agents are, as required by law, certain situations (e.g., disclaimers), and in response to customer inquiries, etc. The scripts are spoken by many different agents with typically only minor modification and timing between the agents. An agent script may be, for example, “For security purposes can you please verify the last four digits of your social security number.” Scripts may have medium length sentences, but are repeated among conversations in the contact center.

Figures of speech are small to medium-sized sentences that people tend to say even thought they are not written text read aloud. The figures of speech are typically common phrases, such as “Oh, my!” They occur with some repetition, but are typically shorter than scripts. Similar to a script, figures of speech tend to have some repetition among conversations in the contact center, but are typically shorter in length and of lower frequency.

With regard to free speech, the order and length of words in is variable. Free speech typically does not repeat among conversations. An example of free speech is, “Well, you see, first click on start.”

The communication processing system 108 can make determinations of the type of speech by looking words within the transcript. For example, for IVR speech, if a predetermined number or percentage of words in an IVR recording are recognized (e.g., 9 out of 16), the communication processing system 108 can make a determination that a particular segment of the transcript is IVR speech. The communication processing system 108 may make a determination not to index each and every word of the recognized IVR speech.

FIG. 3 illustrates a method of determining a type of speech in accordance with a structure. Audio data containing speech made up of sounds is received from either audio source 102 or recorder 104 (operation 300). Communication processing system 108 processes the sounds using a phonetic module producing symbols corresponding to the sounds (operation 302). Communication processing system 108 then processes the symbols using a language module and occurrence table producing text (operation 304).

The communication processing system 108 can make analyze the text of the transcribed speech to make determinations of the type of speech (operation 306) by looking words within the transcript. For example, the communication processing system 108 may identify structured speech based upon repetitions. The communication processing system 108 may identify IVR speech based on durations and distributions. For example, FIG. 4A illustrates patterns of IVR speech. When a certain sentence, phrase, statement, etc. repeats over and over, it can be identified. As illustrated, phrases associated with IVR speech show little deviation. However, scripts spoken by agents may exhibit a higher degree of deviation, as shown in FIG. 4B. Thus, the two types of structured speech can be identified and separated.

The communication processing system 108 may make a determination (operation 308) that a particular segment of the transcript is IVR speech based on the duration distribution of the particular segment. As such, the IVR speech can be separated (operation 310). For example, the segment, “All of our representatives are busy assisting other customers” can be identified as IVR speech.

Using this knowledge, the communication processing system 108 may determine if a predetermined number of words in an IVR recording are recognized. In accordance with the determination, the communication processing system 108 may make a determination not to index each and every word of the recognized IVR speech.

After separating IVR phrases, scripts spoken by agents can be identified by, e.g., examining a length of the phrase. As noted above, scripts are read from text or are statements that agents are trained to say, (“A Federal Law and your decision will not affect your service”). Also as noted above, figures of speech are customary statement (“Hello, how can I help you?”). Because figures of speech tend to be a few words, whereas the scripts tend to be longer sentences, a phrase can be categorized as a script (operation 312) or a figure of speech (operation 314). Thus, figures of speech can be separated out from scripts based on length.

Those phrases that do not fall into the above structures are likely to be free speech (operation 316). Separating the structures may be useful, because for example, in scripts, the word “rebate” may have a different meaning than when it occurs in a figure of speech or free speech. Thus, as will be described below, when searching on the word “rebate,” a context (script, figure of speech, or free speech) may be included in the index and searched.

Identifying scripts within a contact corpus is useful for analysis purposes (operation 318). In some implementations, agent compliance may be determined. For example, it may be determined which agents do or do not strictly adhere to scripts that include, e.g., disclaimers. Contacts may be reviewed that should include a script, but do not. Agents may be ranked based on their compliance to scripts. In addition, identifying scripts may be used to determine which agents are more or less polite that others. Politeness may be analyzed to determine if agents who are more polite helping with customer retention, sales, etc. Yet further, identifying scripts may determine if agents are attempting to up-sell, and what the characteristics of the calls are in which up-selling is performed.

For contacts within the corpus, scores for a script can be determined by setting a minimum distance between the script and words in the contact. A script may be identified by looking for a word or group of words, a Boolean expression or weighting of words. Pattern matching may be performed if a number of errors are small. However, there is not a need to search each and every word in script for it to be correct. In some implementations, an order of the words may be used.

For example, as show in FIG. 5, a threshold number of mistakes (insertions, replacements, deletions) may be set, e.g., 18 to identify a percentage of sentences as being the script. Using this approach a higher recall and precision may be obtained because the script itself has a more accurate signature that looking for the each word by itself. For example, the sequence and/or timing of the words can be used.

In some implementations, scripts may be used as categories. For example, the communication processing system 108 may identify the scripts and output a list. The communication processing system 108 may evaluate each contact for the scripts it contains (e.g., binary output). A user may use a “Script Definition Tool” (SDT) to assign a script with a name, color, impact, etc. The user may assign the script with a type, such as a greeting, authentication, hold, transfer or closure. Additional types assigned can be legal, company policy, up-sale, politeness etc. Manual edit of scripts may be performed by the user who may edit the scripts list to focus on interesting scripts and perform “fine tuning.” Each script can be given a name, color, impact similarly to categories.

In some implementations, the communication processing system 108 may utilize scripts similarly to categories. For example, scripts may be used as a filter in a query. Since the script is binary, a “NOT” operator can be used for checking compliance. Scripts may be displayed, and impact and relevance determined for a specific query. In a player application, scripts may be marked within a contact (see, e.g., FIG. 5A).

In some implementations, scripts may be identified as part of a quality management and scorecards. A supervisor may obtain a list of sentences that each of his/her agents tends to use. The supervisor may ascertain the greetings/closure each agent uses. The supervisor may determine an agents first call resolution capabilities.

Scripts may be exported to database 110. From database 110, the scripts can be integrated with evaluation forms (QM) and Scorecards. Scripts compliance can be used in QM as metrics for evaluations, and for training purposes. Script adherence reports may be generated to determine which agents exceptionally use certain scripts. The reports may also surface scripts that have exceptionally low or high compliance. For each agent, a graph of his/her compliance to various scripts may be generated, as well as an overall scripts compliance graph for all scripts.

In some implementations, analytics may be performed to determine how the usage of an up-sale script contribute to sales (e.g., using meta data); whether agent politeness leads to better customer satisfaction (e.g., using categories); and whether a polite agent helps improve customer retention; whether complying to company policy has a positive effect on sales. Analytics may determine other aspects, such as what characterizes a specific agent group and what their common scripts are. In addition, it may be determined what characterizes good agents; and what are their common scripts (e.g., using QM data).

In the above, the user need not type in whole script when monitoring agents for compliance, QM, etc. Special identifiers can be added to the index and used to searching purposes.

FIG. 6 illustrates the communication processing system of FIG. 1 in greater detail. The communication processing system 108 may include communication interface 301, user interface 302, and processing system 303. Processing system 303 is linked to communication interface 301 and user interface 302. Processing system 303 includes processing circuitry 305 and memory device 306 that stores operating software 307.

Communication interface 301 includes components that communicate over communication links, such as network cards, ports, RF transceivers, processing circuitry and software, or some other communication devices. Communication interface 301 may be configured to communicate over metallic, wireless, or optical links. Communication interface 301 may be configured to use TDM, IP, Ethernet, optical networking, wireless protocols, communication signaling, or some other communication format—including combinations thereof. In this example, communication interface 301 is configured to receive audio data from recorder 104 or directly from audio source 102.

User interface

302 includes components that interact with a user. User interface 302 may include a keyboard, display screen, mouse, touch pad, or some other user input/output apparatus. User interface 302 may be omitted in some examples.

Processing circuitry 305 includes microprocessor and other circuitry that retrieves and executes operating software 307 from memory device 306. Memory device 306 includes a disk drive, flash drive, data storage circuitry, or some other memory apparatus. Operating software 307 includes computer programs, firmware, or some other form of machine-readable processing instructions. Operating software 307 may include an operating system, utilities, drivers, network interfaces, applications, or some other type of software. When executed by circuitry 305, operating software 307 directs processing system 303 to operate communication processing system 108 as described herein.

In this example, operating software 307 includes a phonetic module that directs processing circuitry 305 to translate speech to sounds, a language module that directs processing circuitry 305 to translate sounds to text, and an occurrence table that is used with the language module to improve the accuracy of the sounds to text translation.

The above description and associated figures teach the best mode of the invention. The following claims specify the scope of the invention. Note that some aspects of the best mode may not fall within the scope of the invention as specified by the claims. Those skilled in the art will appreciate that the features described above can be combined in various ways to form multiple variations of the invention. As a result, the invention is not limited to the specific embodiments described above, but only by the following claims and their equivalents.

Claims

What is claimed is:

1. A method for determining structured speech, the method comprising:

translating an audio recording into text using a text-to-speech communication processing system;

receiving a transcript of the audio recording;

analyzing text in the transcript to determine repetitions within the text, the repetitions being indicative of structured speech, wherein the text is analyzed using Large Vocabulary Continuous Speech Recognition (LVCSR);

determining a duration distribution of the repetitions to ascertain a first type of structured speech; and

determining a length of the repetitions to ascertain a second type of structured speech;

comparing probability of correct translation of the text-to-speech communication for each word in the text to an appropriate occurrence table, wherein the occurrence table is selected based upon dialect or language of the speech, and domain in which the speech was obtained; and

modifying the occurrence table based on the probability of correct translation for each word in the text.

2. The method of claim 1, further comprising determining a third type of structured speech when the length of the repetitions is greater than a predetermined threshold.

3. The method of claim 2, wherein the first type of structured speech is interactive voice response (IVR) generated speech, wherein the second type of structured speech is scripts, and wherein the third type of structured speech is figures of speech.

4. The method of claim 1, further comprising applying a distance threshold to determine contacts that contain the second type of structured speech.

5. The method of claim 4, wherein the distance threshold is approximately 18.

6. The method of claim 1, further comprising analyzing the second type of speech to determine one of compliance, quality management, and categories.

7. The method of claim 6, further comprising:

defining the second type of speech as a category; and

evaluating a contact to determine if the contact contains the category.

8. The method of claim 6, further comprising using the second type of speech as a filter in a query.

9. The method of claim 1, further comprising:

storing the second type of structured speech in a database; and

applying meta data to the second type of structured speech to annotate the second type of structured speech.

10. The method of claim 1, wherein modification of the occurrence table changes the occurrence probability by a fixed percentage or a variable percentage based on the probability of correct translation of a given word.

11. A method of determining scripts within speech associated with a contact center, comprising:

translating a speech into text using a text-to-speech communication processing system;

receiving a transcript of the speech;

analyzing the transcript to determine repetitions within the speech that are indicative of structured speech, wherein the text is analyzed using Large Vocabulary Continuous Speech Recognition (LVCSR); and

determining a duration distribution and length of the repetitions with the speed to determine which repetitions are scripts;

12. The method of claim 11, further comprising:

storing a corpus of contacts in a database; and

applying a distance threshold to determine the contacts that contain the scripts.

13. The method of claim 11, further comprising applying analytics to the scripts to categorize the scripts.

14. The method of claim 13, further comprising querying contacts using categories of scripts.

15. The method of claim 11, wherein modification of the occurrence table changes the occurrence probability by a fixed percentage or a variable percentage based on the probability of correct translation of a given word.