Our demonstration is a fairly simple example of creating a speech-enabled Web user control. It will allow the user to choose an employee from a drop-down list (or verbally say a name), and it will then present that employee's profile. The employee list is a fixed group of people stored in an XML file. As we work through this demonstration, notice that I first set up the control without any speech-enabled portions. Once I get the functionality working, I then add the speech plumbing. Doing things this way allows me to first focus on functionality and then the additional interaction.
Section 1 -- Basic Functionality
Step 1. Let's start by creating a C# ASP.NET Web application at the location http://localhost/SaltProfiles.
Figure 1.0 Creating the Project
Step 2. Once VS .NET sets up the project, we will create a new Web user control. This is done by right-clicking the "SaltProfiles" project in the Solution Explorer and choosing "Add", "Add New Web User Control". I gave mine the name "ProfileViewer."
Step 3. We need a drop-down list to display the current employees available. Add a standard "DropDown List" from the "Web Form" section in the Toolbox. Rename it "empDDL". Make sure that this is in the "ProfileViewer.ascx" designer.
Step 4. Add a panel to display the complete user profile, which contains a table with labels for the employees' characteristics. See Figure 1.1 for the final product.
Figure 1.1 The User Control without Speech Support
Step 5. Load up our empDDL list with all of the available employees in our Employee XML. Figure 1.2 contains the current Employee XML document.
Figure 1.2 EmpInfo.xml
The structure of this XML document will remain fixed and only "user" nodes will be added and removed, depending on new hires/fires, etc. Consequently, we can use the serialization feature within .NET Framework to automatically import this XML document into a useable class.
For more information on serialization, read my previous article on the topic at http://www.15seconds.com/issue/020903.htm. The deserialization in this demo is slightly different than what I show you in the serialization article, but the idea is just the same, taking an XML document and converting it to a specific Type (or class).
We need to use our XML document to generate a Type representation of our XML document. The XSD.exe tool that ships with the .NET Framework SDK allows us to do just that. It is available at the DOS command line. There are two commands that we need to perform with XSD.exe in order for it to give us a Type. The first is to generate a schema based on the given XML (EmpInfo.xml) document
xsd EmpInfo.xml
and then to take that generated schema and convert it into a Type
xsd EmpInfo.xsd /c
Figure 1.3 Using the XSD.exe in the Command console
After generating a Type, let's include that, along with our XML and XSD, into our project. This is done within the Solution Explorer by choosing "Show All Files" at the top of the Solution Explorer area, and then selecting all of "EmpInfo.cs", "EmpInfo.xml", and "EmpInfo.xsd", and right-clicking and choosing "Include In Project".
Take time now to explore both the generated XSD and CS file. Notice that in the CS file there are actually two generated classes, one to represent the actual user and the second to represent a collection of users. We will need to use both of these in our demo. One change I made in the CS file was to add our "SaltProfiles" namespace, just for better organization. You can review the EmpInfo.cs file included with the download for this article if you do not understand.
As I mentioned above, we are doing this so we can automatically deserialize the XML document into a Type (class), which is easier for us to use. We need to create a method that will allow us to perform that deserialization.
private userdata GetEmployeeData() {
SaltProfiles.userdata ud;
if(Session["ud"]!=null) {
ud=(SaltProfiles.userdata)Session["ud"];
} else {
System.Xml.Serialization.XmlSerializer serializer = new
System.Xml.Serialization.XmlSerializer(typeof(SaltProfiles.userdata));
System.IO.TextReader reader = new
System.IO.StreamReader(Server.MapPath(EmployeeData));
ud = (SaltProfiles.userdata)serializer.Deserialize(reader);
reader.Close();
Session.Add("ud", ud);
}
return ud;
}
In this code we use the session object for a caching mechanism. This could easily be changed to the application object or any other suitable mechanism. It is simply used to store the deserialized class so that we do not have to continuously perform the deserialization each time it's needed.
The main portion of the code above is within the "else" block of the code. It first creates a serialization object based on the type that we generated. Then it needs to read in the XML document, so it creates a TextReader. And, finally, the serializer uses the reader to pull out the userdata object. As you can see, it is very easy to create a fully typed class based on a simple XML document.
As an alternative, you could have easily used the XML processing classes within the Framework, along with XPath, and pulled the relevant data out of it. I prefer the above method, with no messy XPath to worry about.
With a collection of employees available for our application, let's use it to load up the drop-down list (DDL).
if(data==null) data = GetEmployeeData();
foreach(SaltProfiles.userdataUser user in data.Items) {
empDDL.Items.Add(new System.Web.UI.WebControls.ListItem(user.name, user.name));
}
Step 6. Once the user chooses a name in the drop-down list, we want it to automatically post back to the server and for the application to display the profile of that selected user. This is easily accomplished in a few quick steps. First, set the AutoPostBack property for the empDDL to "True". This will allow the list to automatically post back to the server. Next double click the drop-down list, which should take you to the event handler for when the Selected Index is changed.
Within our event handler we will first grab the text value for the selected item. This is done by using the "SelectedItem" property on the empDDL. We will then take that value and scan the list of employees we have cached in the session, and find the selected employee. Next, if we find a user, we will simply load up the value labels in our panel and make that panel visible. Here is the source:
private void empDDL_SelectedIndexChanged(object sender, System.EventArgs e) {
System.Web.UI.WebControls.ListItem selItem = empDDL.SelectedItem;
SaltProfiles.userdataUser user = FindUser(selItem.Value);
if(user!=null) {
Panel1.Visible=true;
nameLabel.Text=user.name;
System.DateTime bday = System.DateTime.Parse(user.birthdate);
System.TimeSpan ts = new System.TimeSpan(System.DateTime.Now.Ticks-
bday.Ticks);
this.ageLabel.Text=Convert.ToString(Math.Round(ts.TotalDays/365,0));
pictureImage.ImageUrl="/15seconds/SaltProfiles/images/"+user.image;
}
}
private SaltProfiles.userdataUser FindUser(string username) {
foreach(SaltProfiles.userdataUser user in data.Items) {
if(user.name==username) return user;
}
return null;
}
With that last piece added in, we now should have a functioning application. The drop-down list should populate with the employee list, and a user can choose which profile to display. Don't forget to click and drag your user control onto the Web Form designer so that it will be used on that form.
Section 2 -- Adding Speech
This section involves using the .NET Speech SDK so the user can optionally speak the name of the employee whose profile he wants to view.
It builds on the previous code and adds in speech functionality.
Step 1. In order to use the Speech SDK within VS .NET, we must first add the reference to the project. Right click the project, and choose "Add Reference", then hit the "Browse" button. Next, navigate to the folder in which you have installed the SDK. Mine is located at:
"C:\Program Files\Microsoft .NET Speech\SpeechControls\v1.0.2826.0\ Microsoft.Web.UI.SpeechControls.dll".
Make sure you choose the "Microsoft.Web.UI.SpeechControls.dll" and not the file "Microsoft.Web.UI.SpeechControls.ApplicationControls.dll". Since we are not using the Application Controls (described above), you do not need to add this reference.
Step 2. You should see a "Speech" section in the VS .NET Toolbox. Here are the new speech controls. Let's add a new QA control to our user control by clicking and dragging it onto our form. If you choose that QA control and view its properties, you will notice two links, one for "Property Builder ..." and the other "Manage Prompt Databases ...". View both of them and see what they allow you to do with the QA control.
Step 3. With a QA control available, let's set it up for use. First change the "OnClientComplete" and "OnClientListening" properties to "EndMultiModalInteraction" and "BeginMultiModalInteraction", respectively. These are the names of two JavaScript methods provided to you in the "speechInteraction.js" file and included with the download that was shipped with the SDK originally.
Expand the Reco section within the properties. Notice this is where we set up the speech-recognition properties, which will happen on the client. These are the settings I used:
BabbleTimeout=10000
EndSilence=1000
InitialTimeout=3000
MaxTimeout=30000
Mode=Automatic
StartElement=micIcon
StartEvent=onclick
All of these properties are listed in detail in the help that ships with the SDK. Please refer to it for a full explanation. The last two items are the important factors; I will explain them in Step 4.
The last step with the QA control is to add the SALT specification grammar tags. As mentioned above, we have a few different ways to load up the grammar, and one of these methods is to be able to specify the tags at runtime, meaning we can load up the grammar when the page loads. I have used the same EmpInfo.xml document that is available to me (an in-house corporate employee listing can be used from my company's internal documents, for example) and created an XSL stylesheet to transform the XML into the appropriate grammar tags we need.
Next, I have created a method that will apply the XSL with the XML to produce our grammar:
public string Transform(string xml, string xsl) {
System.IO.StringWriter sResult = new System.IO.StringWriter();
System.Xml.XmlDocument docXml = new System.Xml.XmlDocument();
System.Xml.XmlDocument docXsl = new System.Xml.XmlDocument();
System.Xml.Xsl.XslTransform xslTransform
= new System.Xml.Xsl.XslTransform();
docXml.LoadXml(xml);
xslTransform.Load(xsl);
xslTransform.Transform(docXml, null, sResult);
return sResult.ToString();
}
(See the download package for full source code.)
Final grammar output:
Step 4. Triggering the page to begin speech recognition is initiated by an icon. In our demo, a microphone icon is set up to capture the onClick event. <IMG id="micIcon" alt="" src="images/mic.ico">
Notice its "id" attribute is the same as the "StartElement" in Step 3, and it is its client id (not its server-side ID). Also notice that I did NOT specify an "onClick" attribute; the speech controls will do this for me.
Step 5. In the "Speech" section in VS .NET's Toolbox, click and drag a new "SemanticMap" onto our user control's designer and make sure its ID is "SemanticMap1". Choose the "SemItems" collection in the properties for this control. Figure 2.0 below shows the properties that I set for this control.
Figure 2.0 SemItems Properties
Notice the TargetElement is the server-side name for the drop-down list. This allows us to bind this SemanticItem to the drop-down list, more specifically to its "value" attribute.
Step 6. The last step is to bind the QA control to the SemanticItem control through the Answer Property within the QA control. In the properties for the QA control, there is an "Answer" Collection. Create a new answer with the following attributes:
Figure 2.1 Answer Properties
Notice that the property "SemanticItem" is the server-side name of the SemanticItem we previously added. Also notice the "XpathTrigger" is the node from within the grammar output that we are using to determine our rules for this answer. This allows you to use more than one grammar rule per XML file. Now that they are bound together, our control is finished. For the final product of the complete solution take a look at the solution within the downloadable source code.
