Le Post Infeeny

Les articles des consultants et experts Infeeny

Archives Mensuelles: novembre 2015

Exploring Microsoft Speech APIs

This article introduces the speech APIs, one of the services updated in Windows 10 and powered by the project Oxford. This article is divided into several parts:


I.Speech APIs features 

These APIs provide two major features :

  • Speech Recognition : Speech To Text (STT)

It converts spoken audio to text. The APIs can recognize audio coming from the microphone in real-time, or from an audio file.

  • Speech Synthesizer : Text To Speech (TTS)

It converts text to spoken audio when applications need to talk back to their users.

Microsoft Speech Platform

 

The Speech APIs are included in Windows 10 libraries, and also provided by the project oxford services, which require an Azure subscription.

II. Project Oxford Speech APIs 

These APIs are in a beta version, they work by sending data to Microsoft servers in the cloud, to use them we must have an Azure account. They also offer the Intent Recognition feature which can convert spoken audio to intent.

To use these APIs, follow the steps below:

  1. Using an Azure account, go to the Market Place to purchase the speechAPIs Service (which is for free 😉 ), then retrieve the primary or secondary key. Just in case you are using the Azure DreamSpark subscription, don’t be surprised if you don’t find this service. Unfortunately this type of account does not give access to Oxford Services.
  2. Download the Speech SDK of project oxford from here! if you are targeting another platform rather than Windows, have a look here you will find what you are looking for.

 

  • Speech To Text (STT):

The oxford version of Speech APIs offers two choice to make the STT:

  1. REST API
  2. Client library

When using the REST API, we only get one recognition result back at the end of the session, but in the case of a client library, we also get partial result before getting the final recognition.

Setting up speech recognition begins with the Speech Recognition Service Factory. By using this factory, we can create an object which can make a recognition request to the Speech Recognition Service. This factory can create two types of objects:

  1. A Data Recognition Client : used for speech recognition with data (for example from an audio file). The data is broken up into buffers and each buffer is sent to the Speech Recognition Service.
  2. A Microphone Recognition Client : used for speech recognition from the microphone. The microphone is turned on, and data is sent to the Speech Recognition Service.

When creating a client from the factory, it can be configured in one of two modes:

  1. In ShortPhrase mode, an utterance may only be up to 15 seconds long, As data is sent to the server, the client will receive multiple partial results and one final multiple N-best choice result.
  2. In LongDictation mode, an utterance may only be up to 2 minutes long. As data is sent to the server, the client will receive multiple partial results and multiple final results.

Also the client can be configured for one of the following several languages:

  • American English: « en-us »
  • British English: « en-gb »
  • German: « de-de »
  • Spanish: « es-es »
  • French: « fr-fr »
  • Italian: « it-it »
  • Mandarin: « zh-cn »

Now, time to code 😀 you can implement the code below in a WPF app


string stt_primaryOrSecondaryKey = ConfigurationManager.AppSettings["primaryKey"];

// We have 2 choices : LongDictation or ShortPhrase
SpeechRecognitionMode stt_recoMode = SpeechRecognitionMode.LongDictation;

// For a Speech recognition from a Microphone
MicrophoneRecognitionClient stt_micClient = SpeechRecognitionServiceFactory.CreateMicrophoneClient(stt_recoMode, "fr-fr",
stt_primaryOrSecondaryKey);

// For a Speech recognition from a data like wav file
DataRecognitionClient stt_dataClient = SpeechRecognitionServiceFactory.CreateDataClient(stt_recoMode, "fr-fr",
stt_primaryOrSecondaryKey);

Then we must subscribe some events to get the result of the recognition. The Microphone Recognition Client & Data Recognition Client have the same Events as follow:

  • OnConversationError : Event fired when a conversation error occurs
  • OnIntent : Event fired when a Speech Recognition has finished, the recognized text has
    been parsed with LUIS for intent and entities, and the structured JSON result is available.
  • OnMicrophoneStatus : Event fired when the microphone recording status has changed.
  • OnPartialResponseReceived : Event fired when a partial response is received
  • OnResponseReceived : Event fired when a response is received

Inside the events, we can do whatever we want, displaying the result in a textBox for ex. and more…


// Event handlers for speech recognition results
sst_micClient.OnResponseReceived += OnResponseReceivedHandler;
sst_micClient.OnPartialResponseReceived += OnPartialResponseReceivedHandler;
sst_micClient.OnConversationError += OnConversationErrorHandler;
sst_micClient.OnMicrophoneStatus += OnMicrophoneStatus;

// Data Client event from an audio file for ex.
sst_dataClient.OnResponseReceived += OnResponseReceivedHandler;
sst_dataClient.OnPartialResponseReceived += OnPartialResponseReceivedHandler;
sst_dataClient.OnConversationError += OnConversationErrorHandler;

Now how do we start or stop the speech recognition? It’s simple, we just need to make a method call


// Turn on the microphone and stream audio to the Speech Recognition Service
sst_micClient.StartMicAndRecognition();

// Turn off the microphone and the Speech Recognition
sst_micClient.EndMicAndRecognition();

To convert an audio file to text, it’s easy, we just need to convert the file into a byte array and send it to the server for the recognition, like shown below:


if (!string.IsNullOrWhiteSpace(filename))
using (FileStream fileStream = new FileStream(filename, FileMode.Open, FileAccess.Read))
{
int bytesRead = 0;
byte[] buffer = new byte[1024];

try
{
do
{
bytesRead = fileStream.Read(buffer, 0, buffer.Length);
// Send of audio data to cloud service.
sst_dataClient.SendAudio(buffer, bytesRead);
} while (bytesRead > 0);
}
finally
{
m_dataClient.EndAudio();
}
}

  • Text To Speech (TTS)

The TTS feature of project oxford can be used only through the REST API, and we have a complete example here.

The end-point to access the service is: https://speech.platform.bing.com/synthesize

The API uses HTTP POST to send audio back to the client. The maximum amount of audio returned for a given request will not exceed 15 seconds.

For any question about using this API, please refer to TTS through REST API documentation

III. Windows 10 Speech APIs

Windows 10 Speech APIs support all Windows 10 based devices including IoT hardware, phones, tablets, and PCs.

The Speech APIs in Windows 10 are represented under this two namespaces :

Requirement 

  1. Windows 10
  2. Visual Studio 2015
  3. Make sure that Windows Universal App Development Tools are installed in VS2015.

First of all we have to create a Windows 10 Universal application project in visual studio : New Project dialog box, click Visual C# > Windows > Windows Universal > Blank App (Windows Universal).

With Windows 10, applications don’t have the permission to use the microphone by default, so you must at first change the parameters of the universal application as follows:

Double click on the file Package.appxmanifest > Capablilites > Microphone > select the check box.

Note: The Windows 10 Speech APIs are using the languages installed in the Operating System.

  • Speech To Text (STT)

The STT feature using Windows 10 APIs works in online mode, if we want to make it available in offline mode we have to provide the necessary grammar manually.

To make this feature works we have 3 steps:

  • Create a SpeechRecognizer object,
  • Create an other object from SpeechRecognitionConstraint type and add it to the SpeechRecognizer object already created,
  • Compile the constraints.

SpeechRecognizer supports 2 types of recognition sessions:

  1. Continuous recognition sessions for prolonged audio input. A continuous session needs to be either explicitly ended or automatically times out after a configurable period of silence (default is 20 seconds).
  2. Speech recognition session for recognizing a short phrase. The session is terminated and the recognition results returned when a pause is detected by the recognizer.

Like shown in the code below


SpeechRecognizer speechRecognizer = new SpeechRecognizer();
// Here we choose a simple constraints scenario of dictation
var dictationConstraint = new SpeechRecognitionTopicConstraint(SpeechRecognitionScenario.Dictation, "dictation");
speechRecognizer.Constraints.Add(dictationConstraint);
SpeechRecognitionCompilationResult result = await speechRecognizer.CompileConstraintsAsync();

A continuous recognition session can be started by calling SpeechRecognizer.ContinuousRecognitionSession.StartAsync() method and can be stoped by calling speechRecognizer.ContinuousRecognitionSession.StopAsync(). The SpeechRecognizer.ContinuousRecognitionSession object provides two events :

  • Completed : Occurs when a continuous recognition session ends.
  • ResultGenerated : Occurs when the speech recognizer returns the result from a continuous recognition session.

We have another event tied to the speechRecognizer object, which is the HypothesisGenerated event, occurs when a recognition result fragment is returned by the speech recognizer.

The code below show how to start the recognition:


public async void StartRecognition()
{
// The recognizer can only start listening in a continuous fashion if the recognizer is urrently idle.
// This prevents an exception from occurring.
if (speechRecognizer.State == SpeechRecognizerState.Idle)
{
try
{
await speechRecognizer.ContinuousRecognitionSession.StartAsync();
}
catch (Exception ex)
{
var messageDialog = new Windows.UI.Popups.MessageDialog(ex.Message, "Exception");
await messageDialog.ShowAsync();
}
}
}

To stop the recognition :


public async void StopRecognition()
{
if (speechRecognizer.State != SpeechRecognizerState.Idle)
{
try
{
await speechRecognizer.ContinuousRecognitionSession.StopAsync();

TXB_SpeechToText.Text = dictatedTextBuilder.ToString();
}
catch (Exception exception)
{
var messageDialog = new Windows.UI.Popups.MessageDialog(exception.Message, "Exception");
await messageDialog.ShowAsync();
}
}
}

 

  • Text To Speech (TTS)

This feature is available in offline and online mode, to make it works we have to create a SpeechSynthesizer object, then we set the speech synthesizer engine (voice) and generate a stream from the speechSynthesizer.SynthesizeTextToStreamAsync method by passing the text we want to read in parameter.

To read the stream we have to use a MediaElement object, like shown in the code below:


SpeechSynthesizer speechSynthesizer = new SpeechSynthesizer();
speechSynthesizer.Voice = SpeechSynthesizer.DefaultVoice;
//Init the media element which will wrap tthe text to speech
MediaElement mediaElement = new MediaElement();
//We have to add the mediaElement to the Grid otherwise it won't work
LayoutRoot.Children.Add(mediaElement);

var stream = await speechSynthesizer.SynthesizeTextToStreamAsync("Hello World!");
mediaElement.SetSource(stream, stream.ContentType);
mediaElement.Play();

Managing voice commands using the STT and TTS features

We can make the applications implementing these APIs more interactive, by passing some commands using voice. Once the command is executed, the app will confirm this, using the TTS feature. To do that, we can use the STT events, like shown in the code below:


private async  void ContinuousRecognitionSession_ResultGenerated(SpeechContinuousRecognitionSession sender, SpeechContinuousRecognitionResultGeneratedEventArgs args)
{
// We can ignore the generated text using the level of the conversion confidence (Low,Medium,High)
if (args.Result.Confidence == SpeechRecognitionConfidence.Medium || args.Result.Confidence == SpeechRecognitionConfidence.High)
{
// The key word to activate any command
//ex. user says : text red, the key word is text and the command is red
string command = "text";

if (args.Result.Text.ToLower().Contains(command))
{
string result = args.Result.Text.ToLower();
string value = result.Substring(result.IndexOf(command) + command.Length + 1).ToLower();
//The generated text may ends with a point
value = value.Replace(".", "");
switch (value)
{
case "in line": case "line": case "in-line":
dictatedTextBuilder.AppendFormat("{0}", Environment.NewLine);
await dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
{
ReadSpecilaCommandToUser("Carriage return command is activated");
});
break;
case "blue":
await dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
{
TXB_SpeechToText.Foreground = new SolidColorBrush(Colors.Blue);
ReadSpecilaCommandToUser("Blue color command is activated");
});
break;
case "red":
await dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
{
TXB_SpeechToText.Foreground = new SolidColorBrush(Colors.Red);
ReadSpecilaCommandToUser("Red color command is activated");
});
break;
case "green":
await dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
{
TXB_SpeechToText.Foreground = new SolidColorBrush(Colors.Green);
ReadSpecilaCommandToUser("Green color command is activated");
});
break;
case "black":
await dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
{
TXB_SpeechToText.Foreground = new SolidColorBrush(Colors.Black);
ReadSpecilaCommandToUser("Black color command is activated");
});
break;
}
}
else
{
dictatedTextBuilder.Append(args.Result.Text + " ");
}

await dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
{
TXB_SpeechToText.Text = dictatedTextBuilder.ToString();
});

}

}

private async void ReadSpecilaCommandToUser(string text)
{
if (!string.IsNullOrWhiteSpace(text))
{
using (SpeechSynthesizer speech = new SpeechSynthesizer())
{
speech.Voice = SpeechSynthesizer.AllVoices.FirstOrDefault(item => item.Language.Equals(language.LanguageTag));
SpeechSynthesisStream stream = await speech.SynthesizeTextToStreamAsync(text);

mediaElement.SetSource(stream, stream.ContentType);
mediaElement.Play();
}
}
}

 


IV. Demo

The video below shows how to edit text by voice, the app is using the Windows 10 Speech APIs:

Going further

 

Speech APIs – Universal Windows Platform:

SpeechAPIs – Project Oxford: https://github.com/Microsoft/ProjectOxford-ClientSDK/tree/master/Speech

Project Oxford : https://www.projectoxford.ai

Le Windows Store for Business de Microsoft est enfin là !

Après plusieurs mois d’attente le « Business Store » est enfin disponible et accessible gratuitement avec quelques prérequis :

  • Avoir un Azure Active Directory.
  • Des postes clients sous Windows 10 uniquement (Desktop/Phone)
  • (Énormément de patience pour le déploiement…)

(Pour les clients sous Windows 8 et Windows 8.1, ou ceux qui n’ont pas d’Azure AD, nous avons une solution pour vous !)

Comment s’inscrire ?

La première étape se déroule sur https://businessstore.microsoft.com, l’administrateur général de l’AD se connecte avec le compte de l’organisation.

La création du store est immédiate, mais son déploiement prend du temps.

L’administrateur peut attribuer les droits à d’autres utilisateurs de l’AD de gérer le store (directement dans le BO du store) pour les actes d’achats, d’attributions de licences et de publications.

login.png

Les applications grand public :

Après « l’achat » (visiblement pour l’instant nous ne pouvons acheter que des applications gratuites), l’attribution de la licence peut être globale (au niveau de l’organisation), par groupe ou par utilisateur. Le déploiement prend plusieurs heures (+12h)

achat.png

 Les application LOB :

Le processus est long puisqu’il se passe en plusieurs actes :

  • Premier acte :
    • L’administrateur invite un éditeur LOB avec une adresse mail pour que celui-ci puisse attribuer à l’organisation une ou des applications. (Éditeur LOB = un simple compte de développement qui possède un accès au Windows Dev Center https://dev.windows.com)

lob.png

  • Deuxième acte :
    • L’éditeur publie ou met à jour une application en précisant dans la section « tarifs et disponibilités », le store d’entreprise qu’il souhaite viser (une même app LOB peut être diffusée sur différents Business Stores).
  • Troisième acte :
    • Après la validation de l’application par Microsoft (cette étape dure entre quelques heures à quelques jours), l’application est disponible coté BO du Business Store, et c’est à ce moment là que l’administrateur peut l’ajouter au catalogue, de la même manière qu’une application grand public (avec les droits qui vont avec et l’attente du déploiement qui prend aussi plusieurs heures)

Configuration du poste client (Desktop/Phone):

L’étape la plus simple se passe coté poste client où il suffit de se connecter ou d’ajouter (si ce n’est pas déjà le cas) le compte utilisateur rattaché à l’AD directement dans l’application du store. Un nouvel onglet va apparaitre avec la liste des applications attribuées que l’utilisateur pourra alors installer.

Sans titre.png

A vos déploiements !

#UWPXAML – Compiled Binding – Incremental rendering with x:Phase

Coming with Windows 10 are a lot of new features especially for XAML when creating a Universal Windows Platform application aka. UWP app. We can mention a few of them: new controls (as the RelativePanel, the SplitView and more); new tools for building responsive and adaptive UI (AdaptiveTrigger, Extension SDKs, etc.); and a new way to bind data to the UI, Compiled Binding.

This series on Compiled Binding will be composed of several parts:

 

Dealing with long list of data

You often have to work on apps that have a great deal of data. Those data are best displayed on list when they’re of same type, and you generally want to let your users scroll through it without noticeable performance issues.

To enable this, virtualized lists have been created.

Only a handful of items, that are visible to the user, are rendered at the same time while other items are simply not rendered at all. When the user scrolls through the list, rendered items are recycled to match where the user is in the list, thus keeping your memory usage low while still giving to your user what he needs.

This gives your users the illusion that your app is fast.

But sometimes, your items’ DataTemplate can be complex and when the user scrolls through the list, the CPU might not keep up and fully render new items in time. In that case, the list will display placeholder items until rendered items are ready. A quirky user experience may result from that.

To help you have more control on how to optimize the rendering of your items in those circumstances, Windows 8.1 added the event ContainerContentChanging allowing you to progressively render your items in a ListViewBase-derived control.
 

ContainerContentChanging event

The idea behind ContainerContentChanging is quite simple. By handling that event, you can progressively render the items that are visible to the user. This is done by using phases.

Let’s say that you have a list of books, each with a title, a subtitle and a description.
The most important information of your books is the Title property, so it must be displayed whenever possible. The Subtitle property is important but not crucial for the user experience while the Description property is not important.

With the ContainerContentChanging event, you can delay the rendering of the Subtitle and Description properties to later, when the CPU will be more available. The event is called multiple times if necessary, for each pair of container / data that need to be rendered.

Each time the event is raised for the same container / data pair is called a phase. The phase number is provided through the ContainerContentChangingArgs parameter, Phase property.

You can then check the phase number to decide which information should be loaded according to your importance scale.

There we can decide to let the Title property on Phase 0, the Subtitle property on Phase 1 and the Description property on Phase 2.

Phase 0 will be part of the first rendering of the item, while Phase 1 will be done once all current items’ Phase 0 have been rendered, and so on until all phases of all items are done.

Here’s an example that can be found on MSDN:
ListView and GridView UI optimization

XAML

<Page
    x:Class="LotsOfItems.MainPage"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    xmlns:lotsOfItems="using:LotsOfItems"
    mc:Ignorable="d">

    <Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
        <GridView ItemsSource="{x:Bind ViewModel.ExampleItems}" ContainerContentChanging="GridView_ContainerContentChanging">
            <GridView.ItemTemplate>
                <DataTemplate x:DataType="lotsOfItems:ExampleItem">
                    <StackPanel Height="100" Width="100" Background="OrangeRed">
                        <TextBlock Text="{x:Bind Title}"/>
                        <TextBlock Opacity="0"/>
                        <TextBlock Opacity="0"/>
                    </StackPanel>
                </DataTemplate>
            </GridView.ItemTemplate>
        </GridView>
    </Grid>
</Page>

C#

namespace LotsOfItems
{
    /// <summary>
    /// An empty page that can be used on its own or navigated to within a Frame.
    /// </summary>
    public sealed partial class MainPage : Page
    {
        public MainPage()
        {
            this.InitializeComponent();
            this.ViewModel = new ExampleItemViewModel();
        }

        public ExampleItemViewModel ViewModel { get; set; }

        // Display each item incrementally to improve performance.
        private void GridView_ContainerContentChanging(ListViewBase sender, ContainerContentChangingEventArgs args)
        {
            if (args.Phase != 0)
            {
                throw new System.Exception("We should be in phase 0, but we are not.");
            }

            // It's phase 0, so this item's title will already be bound and displayed.

            args.RegisterUpdateCallback(this.ShowSubtitle);

            args.Handled = true;
        }

        private void ShowSubtitle(ListViewBase sender, ContainerContentChangingEventArgs args)
        {
            if (args.Phase != 1)
            {
                throw new System.Exception("We should be in phase 1, but we are not.");
            }

            // It's phase 1, so show this item's subtitle.
            var templateRoot = args.ItemContainer.ContentTemplateRoot as StackPanel;
            var textBlock = templateRoot.Children[1] as TextBlock;
            textBlock.Text = (args.Item as ExampleItem).Subtitle;
            textBlock.Opacity = 1;

            args.RegisterUpdateCallback(this.ShowDescription);
        }

        private void ShowDescription(ListViewBase sender, ContainerContentChangingEventArgs args)
        {
            if (args.Phase != 2)
            {
                throw new System.Exception("We should be in phase 2, but we are not.");
            }

            // It's phase 2, so show this item's description.
            var templateRoot = args.ItemContainer.ContentTemplateRoot as StackPanel;
            var textBlock = templateRoot.Children[2] as TextBlock;
            textBlock.Text = (args.Item as ExampleItem).Description;
            textBlock.Opacity = 1;
        }
    }
}

As you can see, our DataTemplate contains three TextBlocks: one for the Title property, one for the Subtitle property and one for the Description property.

Title being the most important, the control is already databound in the DataTemplate while the others are simply not set and are even hidden from users with Opacity at 0.

Then we add an event handler to the ContainerContentChanging event of the GridView.

When the event is raised in GridView_ContainerContentChanging method, it is Phase 0, the book’s title is rendered.

x:Phase
 

To notify that we expect another phase to occur, we register an update callback through the ContainerContentChangingArgs.RegisterUpdateCallback method. In that callback, that will be called once all items’ Phase 0 are done, we retrieve the second TextBlock control to load its content, the book’s Subtitle, and display it by setting the Opacity to 1.

x:Phase
 

Once it’s done, we register for a third phase to display the book’s description.

x:Phase
 

If the user has scrolled past too many items and the list control wants to recycle containers before we have rendered all the phases, ContainerContentChanging is no longer called for these container / data pairs and the containers are recycled for other data, thus ending the rendering of those previous items.

While ContainerContentChanging is simple to use, its implementation can be quite tedious. Even more when you need to change the DataTemplate over the course of development, you found yourself re-implementing it over and over.

There comes to the rescue {x:Bind} and its x:Phase attribute.
 

Simplifying ContainerContentChanging with x:Phase

Based on the ContainerContentChanging event, x:Bind introduces a shortcut notation to use those rendering phases: x:Phase.

It uses the exact same principle that we’ve seen on the last example, except you don’t have to implement ContainerContentChanging. It’s done for you by code-generation when using {x:Bind}.

To use it, it’s really simple. Just set the x:Phase attribute on an {x:Bind} databound control.

XAML

<GridView ItemsSource="{x:Bind ViewModel.ExampleItems}">
    <GridView.ItemTemplate>
        <DataTemplate x:DataType="lotsOfItems:ExampleItem">
            <StackPanel Height="100" Width="100" Background="OrangeRed">
                <TextBlock Text="{x:Bind Title}"/>
                <TextBlock Text="{x:Bind Subtitle}" x:Phase="1"/>
                <TextBlock Text="{x:Bind Description}" x:Phase="2"/>
            </StackPanel>
        </DataTemplate>
    </GridView.ItemTemplate>
</GridView>

As we’ve seen with ContainerContentChanging, the higher the phase number, the later the control will be rendered. If no phase is set, it’ll be 0 by default.

Note: Those phases don’t have to be contiguous with x:Phase.

In this case, the CPU will try to render the book’s title, then the book’s subtitle and finally the book’s description instead of trying to render them all at once and failing to keep up with the scroll applied by the user.

When the control’s phase is not yet reached, they are set to Opacity=0 thus hiding them. When their phase is reached, the compiled binding is resolved and the controls are displayed.

The result is the same but it is less tedious to implement. You just have to set an attribute on the databound control as opposed to doing a whole implementation of the ContainerContentChanging event.

Later, if you need to change the DataTemplate, it will be way easier.

Just one thing, x:Phase needs x:Bind in order to work. If not present, x:Phase will simply be ignored. So if you need to delay the rendering of a not-databound control, you still have to use the ContainerContentChanging event, that also applies if you need more than just hiding a control for a time.
 

Going further

MSDN
« x:Phase attribute »
https://msdn.microsoft.com/en-us/library/windows/apps/mt204790.aspx

MSDN
« ListView and GridView UI optimization »
https://msdn.microsoft.com/en-us/library/windows/apps/mt204776.aspx

Channel 9
« Improving XAML performance »
https://channel9.msdn.com/Events/Windows/Developers-Guide-to-Windows-10-RTM/Improving-XAML-Performance

Build 2015
« Data Binding: Boost Your Apps’ Performance Through New Enhancements to XAML Data Binding » https://channel9.msdn.com/Events/Build/2015/3-635

#UWPXAML – Compiled Binding – Binding to events

Coming with Windows 10 are a lot of new features especially for XAML when creating a Universal Windows Platform application aka. UWP app. We can mention a few of them: new controls (as the RelativePanel, the SplitView and more); new tools for building responsive and adaptive UI (AdaptiveTrigger, Extension SDKs, etc.); and a new way to bind data to the UI, Compiled Binding.

This series on Compiled Binding will be composed of several parts:

 

{x:Bind} and events

As we’ve seen in the previous part « Part 1: Binding to data« , x:Bind is great when you want to bind various source of data to your UI. It does mostly the same things as classic binding but with better performance and compile-time checks.

x:Bind doesn’t stop here. It goes a step further than its cousin: it can also bind event handlers to events, regardless if it has been thought by the control’s author unlike {Binding}, which needs ICommand support to be built into the control.

All that in a natural way thanks to code generation introduced by compiled binding.
 

Before {x:Bind}

Before compiled binding, you had three ways to handle events in XAML.

1st Way: Register an event handler defined in your code behind directly to your control’s event in XAML. That’s the most common way of doing it as it’s how the XAML framework has been built.

<Button Click="CodeBehind_ButtonClicked" />

 

2nd Way: Use Command properties when available in conjunction with ICommand. This allows you to define your event handler outside the code-behind class, like in your ViewModel.

public class ViewModel
{
    private ICommand _clickCommand;
    public ICommand ClickCommand
    {
        get { return (_clickCommand ?? _clickCommand = new RelayCommand<object>(this.OnButtonClicked)); }
    }

    private void OnButtonClicked(object parameter) { ... }
}

Then in your XAML file, you just have to bind to it.

<Button Command="{Binding ClickCommand}" CommandParameter="HelloWorld" />

One limitation: the control must have built-in support for it. And well, apart from Button and its Command property, barely any other control have such thing.

For example, if you want to handle SelectionChanged of ListView, there isn’t any property allowing you to react to SelectionChanged event from your ViewModel without using the first way we saw.
 

3rd Way: Use Behavior to bind ICommand to any event.

<ListView>
    <i:Interaction.Triggers>
        <i:EventTrigger EventName="SelectionChanged">
            <command:EventToCommand Command="{Binding SelectionChangedCommand}" />
        </i:EventTrigger>
    </i:Interaction.Triggers>
</ListView>

To get around the previous limitation, Blend added an Interactions SDK that allows you, among other things, to create custom behaviors. These behaviors can be attached to controls, that way you can extend on the control’s behavior without having to create another control that inherits from the former.

You can create a behavior that subscribes to whatever event you need and let it call your ICommand, just like the Command property of Button does.

Behaviors with this purpose already exist in MVVM libraries such as EventToCommand from the well-known MVVMLight.
 

{x:Bind}, the Event Handler

Now in UWP apps, you have another way to do so: {x:Bind}.

Let’s say you have a ViewModel and you want to handle the SelectionChanged event of a ListView in a method named OnSelectionChanged, that is in your ViewModel.

public class MyPageViewModel
{
    public ObservableCollection<Book> Books { get; set; }

    public void OnSelectionChanged(object sender, SelectionChangedEventArgs args) { ... }
}

With {x:Bind}, all you have to do is just bind OnSelectionChanged of your ViewModel to the event you want to handle.

<ListView ItemsSource="{x:Bind ViewModel.Books}" SelectionChanged="{x:Bind ViewModel.OnSelectionChanged}" />

There, you just bound yourself an event! It’s that easy.

As you can see, to hook up an event to a method, you just need to use the same markup extension like you would with data binding. Behind, the compiler will add the required logic to call your method when the event is raised.

But when doing so, the compiler expects that the method you try to bind is public and respects one of these conditions:

  • Have the exact event signature
  • Have the same event signature, but parameters can be of a more generic type (up to object)
  • Have no parameter at all

Let’s take a look at how to meet these conditions when binding to the Click event of a button.

XAML

<Button Click="{x:Bind OnButtonClicked}" />

C#

// Exact event signature
public void OnButtonClicked(object sender, RoutedEventArgs args) { … }

// Same event signature, args' type is more generic (RoutedEventArgs --> EventArgs)
public void OnButtonClicked(object sender, EventArgs args) { … }

// Same event signature, args' type is now object
public void OnButtonClicked(object sender, object args) { … }

// No parameter
public void OnButtonClicked() { … }

Asynchronous methods are also supported, just set the « async » keyword on your method signature.

// Asynchronous event handler
public async void OnButtonClicked() { … }

Where can I put those methods, you may ask.

Well, you can put them in your code-behind class (this is x:Bind’s root context, remember?) or you can put them in your ViewModel! No need for ICommand nor EventToCommand behavior now. It just got a whole lot easier compared to what we were used to with classic binding, while using MVVM.

<!-- OnButtonClicked declared in code-behind (except when used in DataTemplate) -->
<Button Click="{x:Bind OnButtonClicked}" />

<!-- OnButtonClicked declared in the ViewModel -->
<Button Click="{x:Bind ViewModel.OnButtonClicked}" />

<!-- OnButtonClicked declared on a data model exposed by the ViewModel -->
<Button Click="{x:Bind ViewModel.LastReadBook.OnButtonClicked}" />

 

Moving on to the last part

We’ve seen pretty much everything x:Bind can offer, or have we?
There is one last neat feature that x:Bind brings: Incremental rendering in list controls.
To learn more about this, head to « Part 3: Incremental rendering with x:Phase« .

#UWPXAML – Compiled Binding – Binding to data

Coming with Windows 10 are a lot of new features especially for XAML when creating a Universal Windows Platform application aka. UWP app. We can mention a few of them: new controls (as the RelativePanel, the SplitView and more); new tools for building responsive and adaptive UI (AdaptiveTrigger, Extension SDKs, etc.); and a new way to bind data to the UI, Compiled Binding.

This series on Compiled Binding will be composed of several parts:

 

Databinding with {x:Bind}

Now that you know what compiled binding is and in what it differs from classic binding, let’s see how to use it in different conditions. If you don’t, see « Part 0: Compiled Binding, what is it? »
 

{x:Bind} over a simple value

To start this series of compiled binding examples, we will see how to bind to a single scalar property.

MainPage.xaml.cs

public sealed partial class MainPage : Page
{
    public string ApplicationTitle = "UWP Series - Compiled Binding";

    public MainPage()
    {
        this.InitializeComponent();
    }
}

Remember that I said x:Bind’s root context is the instance of control which we are currently declaring in its XAML file?

To bind to a value, we need to define it in the « .xaml.cs » file of the control with the public accessor as x:Bind doesn’t have access to protected/private fields. We don’t set the DataContext property because it’s not the root context of compiled binding.

This is also useful to avoid DataContext with compiled binding as it is of type Object and if you need to access some sub-properties of the type you put in DataContext, x:Bind will require that you define the real type in your path like this « {x:Bind Path=DataContext.(local:User.FirstName)}« , reducing the readability of your XAML.

So here, we define the ApplicationTitle string as a public field, but we can also declare it as a property.

Now that we have access to it with x:Bind in XAML, let’s see how to bind to it.

MainPage.xaml

<Page x:Class="UWPSeries.CompiledBinding.MainPage"
      xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
      xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">

    <Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
        <TextBlock Margin="8" Text="{x:Bind ApplicationTitle}" />
    </Grid>

</Page>

As you can see, it is the same as classic binding, we just have replaced the keyword Binding (also known as Markup Extension) by x:Bind and it works!

Compiled Binding
 

{x:Bind} versus the world

From now on, every code samples will use this Book class.

Book.cs

public class Book
{
    public string ImageUrl { get; set; }
    public string Title { get; set; }
    public string Author { get; set; }
}

 

{x:Bind} over a ViewModel

Using x:Bind with a ViewModel is the same as a scalar data, you can define your ViewModel as a public field/property in your « .xaml.cs » file, and then x:Bind will have access to it in the XAML file.

MainPage.xaml.cs

public sealed partial class MainPage : Page
{
    public Book LastReadBook = new Book
    {
        ImageUrl = "ms-appx:///Images/H2G2_UK_front_cover.jpg",
        Title = "The Hitchhiker's Guide to the Galaxy",
        Author = "Douglas Adams"
    };

    public MainPage()
    {
        this.InitializeComponent();
    }
}

MainPage.xaml

<Image Source="{x:Bind LastReadBook.ImageUrl}" />
<TextBlock Grid.Column="1" VerticalAlignment="Center">
    <Run Text="{x:Bind LastReadBook.Title}" />
    <LineBreak />
    <Run Text="{x:Bind LastReadBook.Author}" />
</TextBlock>

As you can see, we specify in the binding path the name of the public field we declared in the « .xaml.cs » file followed by a point to access its properties.

Compiled Binding
 

{x:Bind} in a DataTemplate

To use x:Bind in a DataTemplate, there is only one mandatory step. You need to declare to the DataTemplate what will be the type of the context to render. This is done by setting the attribute « x:DataType » on the DataTemplate to the name of the type you will use.

<DataTemplate x:DataType="local:Book">
    <Grid Height="150">
        <Grid.ColumnDefinitions>
            <ColumnDefinition Width="150" />
            <ColumnDefinition />
        </Grid.ColumnDefinitions>

        <Image Source="{x:Bind ImageUrl}" />
        <TextBlock Grid.Column="1" VerticalAlignment="Center">
            <Run Text="{x:Bind Title}" />
            <LineBreak />
            <Run Text="{x:Bind Author}" />
        </TextBlock>
    </Grid>
</DataTemplate>

One thing: in DataTemplate, the root context of compiled binding is the current item you are trying to render just like with classic binding. So here, when using « {x:Bind Title} », you bind to the current item’s title.

Compiled Binding
 

Special case : {x:Bind} in a ResourceDictionary

When you are creating a DataTemplate, you may want to put it in a separate ResourceDictionary to allow the reuse of this DataTemplate.

But when doing so with compiled binding, you get a build error stating « This Xaml file must have a code-behind class to use {x:Bind}. »

As I said in the previous part, compiled binding is achieved by doing code generation in the « .xaml.g.cs » file linked to your « .xaml » file where you are using x:Bind. But when putting a DataTemplate with compiled binding in a ResourceDictionary, as the dictionary doesn’t have a « .xaml.cs » file, also called a code-behind class, the compiler can’t associate the file generated to your resource dictionary.

To resolve this, you need to create that missing file, to create a partial class that inherits from ResourceDictionary and declare this class with « x:Class » in your ResourceDictionary’s XAML.

DataTemplateResourceDictionary.xaml.cs

public partial class DataTemplateResourceDictionary : ResourceDictionary
{
    public DataTemplateResourceDictionary()
    {
        this.InitializeComponent();
    }
}

DataTemplateResourceDictionary.xaml

<ResourceDictionary
    x:Class="UWPSeries.CompiledBinding.DataTemplateResourceDictionary"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="using:UWPSeries.CompiledBinding">

    <DataTemplate x:DataType="local:Book" x:Key="BookDataTemplate">
        <Grid Height="150">
            <Grid.ColumnDefinitions>
                <ColumnDefinition Width="150" />
                <ColumnDefinition />
            </Grid.ColumnDefinitions>

            <Image Source="{x:Bind ImageUrl}" />
            <TextBlock Grid.Column="1" VerticalAlignment="Center">
                <Run Text="{x:Bind Title}" />
                <LineBreak />
                <Run Text="{x:Bind Author}" />
            </TextBlock>
        </Grid>
    </DataTemplate>

</ResourceDictionary>

Tip: To simplify this step, you can use the file template « Blank Page » of Visual Studio like you would to create a new page, but once the appropriate files created (« .xaml » and « .xaml.cs »), you change the Page tags to ResourceDictionary tags. All will be set up correctly allowing you to use x:Bind.

Last step is to reference your ResourceDictionary to be used by your application.

As we have created a class to hold the generated binding logic, we can’t use the classic way of declaring ResourceDictionary:

App.xaml

<Application x:Class="UWPSeries.CompiledBinding.App"
             xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
             xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
             xmlns:local="using:UWPSeries.CompiledBinding"
             RequestedTheme="Light">

    <Application.Resources>
        <ResourceDictionary>
            <ResourceDictionary.MergedDictionaries>
                <ResourceDictionary Source="DataTemplateResourceDictionary.xaml" />
            </ResourceDictionary.MergedDictionaries>
        </ResourceDictionary>
    </Application.Resources>

</Application>

Instead, we declare it by instantiating it directly.

App.xaml

<Application
    x:Class="UWPSeries.CompiledBinding.App"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="using:UWPSeries.CompiledBinding"
    RequestedTheme="Light">

    <Application.Resources>
        <ResourceDictionary>
            <ResourceDictionary.MergedDictionaries>
                <local:DataTemplateResourceDictionary />
            </ResourceDictionary.MergedDictionaries>
        </ResourceDictionary>
    </Application.Resources>

</Application>

 

{x:Bind} over another control

x:Bind also has the capability to bind to another control inside the page/usercontrol you are working on, like ElementName and RelativeSource from classic binding.

This is easily done because as we saw in « {x:Bind} over a simple value« , {x:Bind} has access to the page instance. Setting a name to a control make it available in the page’s code-behind in a field of the same name. x:Bind can then use this named control in its path to bind to a property of it.

MainPage.xaml

<TextBox x:Name="InputTextBox" Text="I'm binded" />
<TextBlock Margin="8" Text="{x:Bind InputTextBox.Text, Mode=OneWay}" />

I set the binding as OneWay because x:Bind is by default OneTime and thus not updating when the user starts typing in the textbox.

Compiled Binding
 

Going further

MSDN
« Data binding in depth »
https://msdn.microsoft.com/en-us/library/windows/apps/mt210946.aspx

Channel 9
« Improvements to XAML data binding »
https://channel9.msdn.com/Events/Windows/Developers-Guide-to-Windows-10-RTM/Improvements-to-XAML-data-binding

Build 2015
« Data Binding: Boost Your Apps’ Performance Through New Enhancements to XAML Data Binding » https://channel9.msdn.com/Events/Build/2015/3-635
 

Next time on Compiled Binding: binding to events

Data binding with x:Bind is great like we’ve just seen, but what about event binding?
Head to « Part 2: Binding to events » to learn about how to use compiled binding with events.

#UWPXAML – Compiled Binding – What is it?

Coming with Windows 10 are a lot of new features especially for XAML when creating a Universal Windows Platform application aka. UWP app. We can mention a few of them: new controls (as the RelativePanel, the SplitView and more); new tools for building responsive and adaptive UI (AdaptiveTrigger, Extension SDKs, etc.); and a new way to bind data to the UI, Compiled Binding.

This series on Compiled Binding will be composed of several parts:

 

A quick refresh on Classic Binding

Before going into details about Compiled Binding, let’s get a refresh on “classic” binding.

If you ever have developed a XAML app (with WPF, Silverlight, Windows Phone or Windows 8 Store App), you’ll be most likely familiar with classic data binding.

It uses the {Binding} syntax to pass, observe and update an underlying data to the view easily, therefore simplifying separation of concerns: the view doesn’t have to know about the data, retrieve and update logic being handled by a binding source object created by {Binding} thus making it best used with the MVVM pattern.

Here’s a code sample using classic binding:

MainPage.xaml.cs

public sealed partial class MainPage : Page
{
    public MainPage()
    {
        this.InitializeComponent();
        this.DataContext = new Person() { LastName = "Doe", FirstName = "John" };
    }
}

MainPage.xaml

<StackPanel>
    <TextBlock Text="{Binding LastName}" />
    <TextBlock Text="{Binding FirstName}" />
</StackPanel>

FirstName and LastName here are called paths, they will be resolved at runtime, requiring a small overhead due to the use of reflection.
 

Introducing Compiled Binding

What is Compiled Binding?

Introduced with Windows 10, Compiled Binding is very similar to Classic Binding as it serves the same purpose but with some slight differences.

Compiled Binding is used with the {x:Bind} syntax as opposed to the {Binding} syntax of Classic Binding. It still uses the notifying interfaces (like INotifyPropertyChanged) to watch for changes but {x:Bind} is by default OneTime compared to {Binding} which is OneWay.

The reason for that is that most of the time, you just want to display a value on screen and don’t need it to change over the lifetime of your page. Setting binding to be OneTime by default, remove the need to watch for changes and thus reduce CPU and memory usage.

You can still use OneWay and TwoWay binding, you just need to set it like you normally would with classic binding:

MainPage.xaml.cs

public sealed partial class MainPage : Page
{
    public Person Person = new Person() { LastName = "Doe", FirstName = "John" };

    public MainPage()
    {
        this.InitializeComponent();
    }
}

MainPage.xaml

<TextBlock Text="{x:Bind Person.LastName, Mode=OneWay}" />
<TextBox Text="{x:Bind Person.FirstName, Mode=TwoWay}" />

The main difference, as the name implies, is that the binding is actually « compiled », the paths are resolved at compile-time rather than runtime like classic binding, this permits to have a better troubleshooting experience (mistyped or invalid paths are treated as build error instead of just outputting an error message in the console) and remove the overhead due to reflection usage, which are both great.

This compilation is achieved by code generation. When you use {x:Bind} instead of {Binding}, the compiler will add logic into the « .xaml.g.cs » file corresponding to your « .xaml » file.

Data binding with {x:Bind} has another feature that {Binding} doesn’t have: incremental rendering of the items in ListViewBase-derived controls like ListView and GridView. This is done by using both {x:Bind} and the attribute x:Phase on controls inside your DataTemplate.

<DataTemplate x:DataType="local:Person">
    <StackPanel>
        <TextBlock Text="{x:Bind LastName}" />
        <TextBlock Text="{x:Bind FirstName}" x:Phase="1" />
    </StackPanel>
</DataTemplate>

Based on the ContainerContentChanging event that came with Windows 8.1, x:Phase allows you to progressively render your DataTemplate inside long list of data while the user is scrolling fast.

As you can see in the example, just set the x:Phase attribute on an x:Bind databound element and its rendering will be delayed until the CPU will have more time for it. If no phase is set, it’s by default phase 0, so whenever the item can be rendered, the element will also be.

That way, you have control on what to display first instead of just showing a placeholder. Used effectively, this can greatly enhanced your app’s user experience.

And finally, {x:Bind} is also capable to bind event handlers to events.

With classic binding, this was only possible by using ICommand to bind ViewModel’s methods to a Button click. If you wanted to bind to an event with no corresponding built-in Command property, you had to use a behavior and Interactions library from Blend SDK, usually you had to use EventToCommand behavior from MVVM libraries like MVVMLight.

One important thing to note also is that while classic binding uses the property DataContext as a root source for evaluating binded paths, Compiled Binding on the other hand directly references the root control in which it is used. For example, when using compiled binding in a page like MainPage, {x:Bind HelloWorld} will reference MainPage.HelloWorld. A field, a property or a method named HelloWorld must exist on MainPage and must be public. This also works when using compiled binding in a UserControl.

We’ll see in the next section a more complete comparison between {x:Bind} and {Binding}.

NB: Compiled Binding is only available for Windows 10 Universal apps (UWP). There is actually no plan to port it to WPF or Windows 8 apps in the future.

 

{Binding} versus {x:Bind}

Here’s a non-exhaustive list of differences between classic binding and compiled binding:

Classic Binding Compiled Binding
Creation Runtime Compile-time
Syntax {Binding LastName} {x:Bind LastName}
Root path context DataContext Root control (Page, UserControl, etc.)
Default mode OneWay OneTime
Invalid path Silently output an error in the console at runtime Build error
INotifyPropertyChanged
INotifyCollectionChanged
Supported Supported
Converters Supported Supported
Duck typing Supported Not supported
Dynamic creation (in code behind) Supported Not supported

Compiled Binding is missing some features from classic binding like Source, UpdateSourceTrigger, TemplatedParent, etc. If you really need these features, you should stick with classic binding when necessary.

RelativeSource=Self is a special case, the keyword itself is missing in compiled bindings but you just need to rewrite your binding because {x:Bind} has access to the root control instance and so has access to every named control in that root control.

RelativeSource with classic binding:

<TextBlock x:Name="TestClassicBinding" Text="{Binding RelativeSource={RelativeSource Mode=Self}, Path=Name}" />

RelativeSource with compiled binding:

<TextBlock x:Name="TestCompiledBinding" Text="{x:Bind TestCompiledBinding.Name}" />

For a more complex use like binding to an unknown ancestor, stick with classic binding and RelativeSource.
 

Can I use both Compiled Binding and Classic Binding in a single project?

Of course you can!

As I said earlier, {x:Bind} only notifies the compiler to add some logic to handle passing a data to the UI that does not need usage of reflection at runtime like {Binding}.

Classic Binding is still available for UWP apps and is sometimes needed to achieve something Compiled Binding does not support such as Duck Typing, where you just know the property names but don’t know about the source object’s type (example : Property « Age » of classes Person and Wine).

It’s welcome when porting existing Windows 8 apps to Windows 10.

You can even use both bindings on a single control:

<TextBlock Text="{x:Bind Person.LastName}" Foreground="{Binding LastNameForegroundColor}" />

But Compiled Binding is currently only available for Universal Windows Platform projects.
 

When should I use Compiled Binding over Classic Binding?

Well, mostly anytime you can.

It really is a great way to enhance your app’s performance and simplify it a little (ICommand no more necessary) while remaining as easy to use as classic binding and as a bonus, troubleshooting becomes easier thanks to compile-time check of your paths!

Classic Binding is still there for you to use. It’s useful when you’re porting a previous Windows 8 app to Windows 10, or when you need advanced features only achievable without knowing the binded type at compile-time (for example, when you are dynamically loading XAML in your app or when your DataTemplate can be used with multiple types not having necessarily a common ancestor)
 

Binding to data

That’s all for introduction on compiled binding.
Head to « Part 1: Binding to data » to learn about how to use compiled binding with data.

Sécurité dynamique dans les cubes SSAS avec SQL Server 2012

Avec l’ouverture des données Power BI Self-Service vers les utilisateurs (vu dans plusieurs de mes missions) les métiers veulent gérer eux même les droits ou périmètres sur de nombreux utilisateurs finaux (>1000).

En effet dans certains domaines, nous avons une forte contrainte de confidentialité des données contenues dans les cubes SSAS. Les juridictions qui doivent s’appliquer peuvent être changées avec des cadences variables allant de la journée, à des cadences inférieures (temps réel). Pour cela, l’ensemble de la sécurité mise en œuvre est stocké dans la source de données du cube, qui détermine qui peut voir quoi.

A travers cet article, nous allons voir comment rendre dynamique une sécurité de cube SSAS basée sur l’appel à une procédure stockée.

Lire la suite