Microsoft codename “Cloud Numerics” lab (referred to as “Cloud Numerics” in the text that follows) is a numerical and data analytics library for data scientists, quantitative analysts, and others who write C# applications in Visual Studio. It enables these applications to be scaled out, deployed, and run on Windows Azure.

Getting Started

Learning More

 
Providing Feedback and Getting Support

Overview

Modes of Use

“Cloud Numerics” has the following primary modes of use:

• Running a Distributed or Serial Application Locally
• Running on Azure

Run a Distributed or Serial Application Locally

As outlined in the following figure:

Running locally using the Numerics.Runtime provides the following functionality:

• Access from .NET applications and calling routines to a library of numerical algorithms ranging from simple math to advanced statistics to linear algebra and signal processing.
• Distributed and non-distributed n-dimensional arrays and the operators and functions to manipulate them.
• Ability to spread out your computation as well as your data over many cores in Single Instruction Multiple Data (SIMD) fashion.
• Ability to invoke user-defined methods in parallel in Single Program Mulitple Data (SPMD) fashion using the Apply Framework.  
• A development environment where you can design, test, and iterate over your algorithms before you deploy and scale them out on Azure.

To leverage the "Cloud Numerics" libraries and distributed array data while running locally:

1. Follow the steps to run a sample program in the Getting Started section titled Run a Distributed Application Locally.

2. Modify the sample program, or copy a distributed example over the sample program, build it, and run it.

3. View the pre-compiled, example programs in the Cloud Numerics API reference and in the Simple Examples section of this wiki.

4. See the blog post titled Using Data for tips and code examples helpful for working with, casting, and loading arrays.

Running on Azure (with Numerics Runtime)

As outlined in following figure:

Running "Cloud Numerics" projects on Windows Azure provides the following functionality:

• Access from .NET applications and calling routines to a library of numerical algorithms ranging from simple math to advanced statistics to linear algebra and signal processing.
• Distributed and non-distributed n-dimensional arrays and the operators and functions to manipulate them.
• Ability to spread out your computation as well as your data in Single Instruction Multiple Data (SIMD) fashion.
• Ability to invoke user-defined methods in parallel in Single Program Mulitple Data (SPMD) fashion using the Apply Framework. 
• Ability to scale out Azure cloud compute and Azure SQL storage resources on demand.
• Access to large scale staging, backup, or production environment resources, which can access SQL Azure data.

To leverage the "Cloud Numerics" libraries using either serial or distributed array data while running on Azure:

1. Follow the steps to run a sample program in the Getting Started section titled Run a Distributed Application over an Azure Cluster.

2. Modify the sample program, or copy a distributed example over the sample program, build it, and run it.

3. View the pre-compiled, example programs in the Cloud Numerics function library reference help and in the section titled Simple Examples.

4. See the blog post titled Using Data for tips and example code helpful for working with, casting, and loading arrays, and creating arrays from Windows Azure blob storage.
 

Software and Platform Requirements

This section contains the software and platform requirements for using the "Cloud Numerics" libraries for the following modes of use:

• Local distributed mode
• Distributed mode on Windows Azure

OS Requirements (for Local Development System)

You must have one of the following operating systems installed on your local development system to use the "Cloud Numerics" Lab:

• Windows 7 Home Premium (x64)
• Windows 7 Professional (x64)
• Windows 7 Ultimate (x64)

Application and SDK Requirements (Installed on Local Development System)

These requirements satisfy both:

• Local distributed mode
• Distributed mode on Windows Azure

To run code that uses “Cloud Numerics” libraries on the Windows Azure cloud, ensure that you have the following software installed on your local development machine. Portions of this software, which you will install locally, will be bundled up as part of the package you deploy to your Windows Azure cluster.

Windows Azure-Specific Requirements

This section contains the additional environment-related requirements for using the "Cloud Numerics" libraries.

The following table lists the additional requirements you need to run your application using the “Cloud Numerics” distributed runtime libraries using the MPI-based local distributed libraries.

Install Cloud Numerics Software

After you satisfy the software and platform requirements, you can install the Microsoft codename “Cloud Numerics” software. Once you install the software to a local machine running Visual Studio, you can deploy and run the sample application supplied as part of the "Cloud Numerics" Visual Studio project deployment template.

The "Cloud Numerics" installer includes:

• Math, Statistics, and Signal Processing libraries as managed Dynamically Linked Library (DLL) files.
• DLLs for initializing and running jobs on Windows Azure used by the Math, Signal, and Statistics libraries.
• Associated IntelliSense files for the DLLs.
• A project deployment template and utility for deploying your application package to Azure.

Download the Lab Distribution Package

Once you are registered for the lab, you can download the software from:

http://connect.microsoft.com/

The registration confirmation e-mail contains a hyperlink to the latest “Cloud Numerics” lab distribution media. If you do not yet have a confirmation e-mail, you can get one by registering here.

Install the Software

After you satisfy the software and platform requirements, you can install the “Cloud Numerics” software.

To install the “Cloud Numerics” software:

1. Run the installer.

2. Accept the Microsoft Software License Terms.

3. Choose an installation directory.

4. Click the Install button on the Ready to Install screen. After you see the Installation Complete screen, restart your machine, and proceed to the next section titled Getting Started: Deploy and Run a Sample Application.

Guidelines for Developing Locally and Launching on Azure

This section outlines the typical programming model and sequence for writing distributed applications.

Get a Windows Azure Subscription

You will need a Windows Azure subscription ID when you use the "Cloud Numerics" Deployment Utility to set up your cluster's deployment configuration. You can find instructions on how to get an Azure subscription here:

Run a Distributed Application Locally

Run a Distributed Application over an Azure Cluster

This section details the sequence for how to:

• Configure and deploy a "Cloud Numerics" cluster on Windows Azure.
• Run your application or a sample application on the cluster.

The following table outlines the steps for both deploying "Cloud Numerics" to a cluster and running your distributed application. This process is detailed in the subsections that follow.

Summary of Steps

Step 1: Get an Azure Subscription ID

To get your Azure subscription ID:

1. Navigate to the following URL (https://windows.azure.com/) and sign in to your Windows Azure Management Portal.

2. In the left pane, click the Hosted Services, Storage Accounts & CDN tab.

3. In the Subscription and Services view (center pane) click your active subscription.

The main view displays.

4. In the Properties pane, scroll down and copy your Subscription ID. You will need your subscription ID to deploy and configure your Azure cluster.

5. Proceed to the next section titled Step 2: Create a Numerics Project and Launch the Azure Deployment.

Step 2: Create a Cloud Numerics Project and Launch the Cloud Numerics Deployment Utility

To Launch the "Cloud Numerics" Deployment Utility:

1. Open Visual Studio as Administrator.

2. From the File menu, select New Project.

The New Project window displays.

3. From the New Project window, click Visual C# in the Installed Templates pane, and then select Microsoft Cloud Numerics Application.

Solution Explorer displays a newly created project named MSCloudNumericsProject.

4. For this step you can either:

• Do nothing and use the sample application (Program.cs) underneath the MSCloudNumericsApp subproject.
• Replace the body of the sample application (Program.cs underneath the MSCloudNumericsApp subproject) with the code example in the section titled Simple Distributed Example.

5. From Solution Explorer in the Visual Studio IDE, right click the AppConfigure project and select Set as Startup Project.

6. Press the F5 key. (Please note, this might take a few minutes to build the deployment package.)

The "Cloud Numerics" Deployment Utility displays.

7. Proceed to the section titled Step 3: Configure Your Azure Service Account.

Step 3: Configure Your Azure Service Account

To configure your Azure service account from the "Cloud Numerics" Deployment Utility:

1. From the Azure Account tab, type in your Azure subscription ID you copied from the Azure Management Console.

2. Get a certificate (.cer) file for authentication of your remote machine with the Azure Management Console by either using an existing certificate file or by creating a new one. To get a certificate file, complete either of the following:

• Browse the file system and select an existing certificate file, or
• Click on the Create button (creates a new certificate).

a. Change the default friendly name in the Certificate Name text field to a unique and easy-to-identify name that you will recognize later.

b. Browse to the directory where you want to create the certificate file and specify a name for the output file.

c. Click OK.

	Note!
	If an alert box warns you are changing the credentials list on your local development machine, click Yes to accept the change and save your newly-created certificate. Make a note of the directory path where your certificate is created.

3. Upload the certificate file you created or referenced in the previous step to Azure.

a. Navigate to the Windows Azure Platform portal at http://windows.azure.com, and sign in.

b. Click the Hosted Services, Storage Accounts, & CDN tab.

The main browser pane opens to the Deployment Health view.

c. On the navigation pane, click Management Certificates.

The management view for your active Azure subscription displays.

d. Click the Add Certificate icon.

The Add New Management Certificate dialog displays.

	Note!
	If the Add Certificate icon is grayed out indicating that it is not enabled, you might have reached the limit of certificates you can upload for this account. The default setting for this limit is 10. To work around this, you can delete a single certificate. This will make the Add Certificate icon active (not grayed out) and enable you to add your certificate.

e. Browse to the place on your local development machine where you created the certificate (.cer) file, click Open to select it, and click OK in the Add New Management Certificate dialog to upload it to your Azure account.

You certificate has been uploaded to Azure. You can now continue filling out the fields in the Azure Account tab.

4. Add a service name to the Service Name text field.

You can add a service by either of the following:

• Type in a new service name in the Service Name text entry field.
• Type in the name of an existing service name associated with the current Azure subscription.

5. Select a data center location for your Azure cluster.

6. Click Next.

The Cluster in Azure tab displays.

7. Proceed to the next section titled Step 4: Configure Your Cluster.

Step 4: Specify Cluster Sign on Information and Specify Number of Nodes

To configure your Azure cluster through the "Cloud Numerics" Deployment Utility:

1. From the Cluster in Azure tab, create a cluster administrator user name or enter an existing name by typing it into the Name text field.

2. To add or create a password for this administrator, type it into the Password text fields.

3. If you want a different number of head nodes from the default, type the number in the Head text field.

4. If you want a different number of compute nodes from the default, type the number in the Compute text field.

5. If you want a different number of administrative nodes from the default, type the number in the Web FrontEnd text field.

6. Click Next.

The SQL Azure Server tab displays.

7. Proceed to the next section titled Step 5: Manage Your SQL Azure Storage Service.

Step 5: Manage Your SQL Azure Storage Service

To manage your SQL Azure Storage Service through the "Cloud Numerics" Deployment Utility:

1. From the SQL Azure Server tab, complete either one of the following:

• Create a new SQL storage instance.
• Use an existing storage instance selecting it from the drop down list.

2. From the Database Server Credentials pane, complete either one of the following:

• Click the Use the Same Username and Password as the Cluster to use your Azure credentials to authenticate to your Azure SQL storage instance. (Recommended.)
• Specify the username and password to use when you authenticate to your Azure SQL storage instance.

3. Click Next.

The Application Code tab displays.

4. Proceed to the next section titled Step 6: Create, Configure, and Deploy a Cloud Numerics Cluster.

Step 6: Create, Configure, and Deploy a Cloud Numerics Cluster

After you fill in the necessary fields in the Azure Account, Cluster in Azure, and SQL Azure Server tabs, you can use the "Cloud Numerics" Deployment Utility to create, configure, and deploy a new "Cloud Numerics" Cluster. If you have previously deployed your cluster, do not attempt to redeploy over an existing cluster.

To create, configure, and deploy your cluster.

1. From the Deployment Actions frame of the "Cloud Numerics" Deployment Utility, click the Configure Cluster button to create and configure your Windows Azure cluster.

The status bar indicates the gradual progress of the configuration step. When it is finished, the Configuration complete” message displays.

2. Click the Deploy Cluster button to deploy the necessary services on your Azure HPC cluster. This step might take more than 10 minutes.

The status bar indicates the progress of the deployment. This step includes installing the necessary software on the Azure cluster nodes. When it is finished, the “Deployment complete” message displays.

Step 7: Submit a Job and View its Status and Output

This procedure lists how to submit a compiled .NET application as a job to be run on the Azure HPC Scheduler. It consists of the following parts:

• Submit a Job
• View Job Status and Output from the Azure HPC Scheduler Web Portal

Submit a Job

To submit a job to be run on your Windows Azure cluster:

1. Specify a compiled application (.exe) to deploy to the cluster. For this application, you can specify the sample application provided with the "Cloud Numerics" project. You can find the project executable in your Visual Studio Projects directory.

If you used the default project name, the default directory path for this project executable is in your Visual Studio Projects directory.

C:\Users\<user_name>\Documents\Visual Studio 2010\Projects\MSCloudNumerics\MSCloudNumerics\MSCloudNumericsApp\bin\Debug\

2. From the Deployment Actions frame of the "Cloud Numerics" Deployment utility, click the Submit Job and View Status and Output button to submit the application and dependencies.

The status bar indicates the progress of the job submit step. When it is finished, the “Job successfully submitted” message displays.

You can also access Job status from either the:

• Azure Management console.
• Azure Admin Service using a web browser or web client using the URL:
  https://<your service name>.cloudapp.net.

Get Job Status from the Azure HPC Scheduler Web Portal

To access job status and application console output from the Azure HPC Scheduler web portal.

1. In a web browser, navigate to https://<your service name>.cloudapp.net

The Windows Azure Sample Application page displays in your browser.

2.  Click on Windows Azure HPC Scheduler Web Portal.

3. Enter your cluster user name and password when prompted.

When the page loads, the Windows Azure HPC Scheduler Web Portal Displays.

4. Click My Jobs..

When the web page loads, you will see the jobs submitted to the cluster scheduler in a table with the following column headings.

5. Find your job in the Name column, and click on it. If you used the default name created for you in the "Cloud Numerics" project, the job name is MSCloudNumericsApp.exe.



6. Click the Tasks tab and then click on the link under the Command line column that begins with "mpiexec" and ends with your application name.

Job results display in the Output frame.

Final Step: Get Job Results Data

You can get your results data by:

• Signing into the Azure console, selecting the storage account you created, and viewing your blob.
• Retrieving your results with the Blob Service REST API.
  (http://msdn.microsoft.com/en-us/library/windowsazure/dd135733.aspx)

Extracting and Viewing the Function Library Reference Help

You can download the Microsoft codename “Cloud Numerics” library and distributed runtime reference help to your local development machine from the Microsoft Connect portal (http://connect.microsoft.com/). The “Cloud Numerics” library and runtime reference pages are encapsulated in a single compiled HTML archive or .CHM file.

1. If you registered to participate in the Microsoft codename "Cloud Numerics" lab, sign in to the Microsoft Connect portal with your Windows Live ID. Otherwise, use this link to register.

2. Use a web browser such as Internet Explorer to navigate to the Microsoft codename "Cloud Numerics" documentation download page. If you already registered to participate in the lab, and you signed in to Microsoft Connect, you can use this URL.

http://connect.microsoft.com/site1267/Downloads/DownloadDetails.aspx?DownloadID=40550

The Microsoft Connect download page appears.

3. Download the package to your local development machine.

4. Read the README.txt file.

5. Double click the .CHM file to open it with Windows Help Viewer.

The “Cloud Numerics” library function help file displays.

Convert Existing C# Application to Use Cloud Numerics

Simple Examples

The following serial and distributed examples can be used, along with any of those provided in the library function reference help, in place of the sample application provided in your Microsoft codename “Cloud Numerics” Visual Studio C# project. The library function reference help is available by way of the Microsoft Connect portal (http://connect.microsoft.com/).

Simple Serial Examples

The following example operates on local (non-distributed) arrays created using the Numerics.Local.NumericDenseArrayFactory method.

using System;
using Microsoft.Numerics;
using Microsoft.Numerics.Local;
using Microsoft.Numerics.Mathematics;

namespace BasicMathModExample
{
    class BasicMathModExample  
    {
        public static void Main()
        {
            // Construct input arrays
            var values = 
            NumericDenseArrayFactory.CreateFromSystemArray<double>( new double [] {15.0, 
                                                                                   17.0, 
                                                                                   24.0, 
                                                                                   30.0});
            var divisors = 
            NumericDenseArrayFactory.CreateFromSystemArray<double>(new double [] {3.0, 
                                                                                  5.0, 
                                                                                  7.0, 
                                                                                  9.0});
            // Compute Mod function value and write result
            var modulus = BasicMath.Mod(values, divisors);
            Console.WriteLine(modulus); 
        }
    }
}

The following is a serial signal processing example. The "Cloud Numerics" signal processing library methods can be used with local arrays.

using System;
using Microsoft.Numerics;
using Microsoft.Numerics.Local;
using Microsoft.Numerics.Signal;
namespace IIRExample
{
    class IIRExample
    {
        public static void Main()
        {
            // Counting of signal
            int N = 100;

            // Generation of input singal
            var x = Microsoft.Numerics.Local.NumericDenseArrayFactory.UniformGrid<Double>
                    (N,-2 * Microsoft.Numerics.Mathematics.BasicMath.PI,2 * Microsoft.Numerics.Mathematics.BasicMath.PI);                  

            var inputSignal = Microsoft.Numerics.Mathematics.BasicMath.Sin(x);

            // Frequencies 
            double Freq = 0.25;
            double NoiseFreq = 0.1;

            // Generation of noise
            var t = Microsoft.Numerics.Local.NumericDenseArrayFactory.UniformGrid<Double>(N, 0, N-1);            
            var Noise = Microsoft.Numerics.Mathematics.BasicMath.Sin(2 * Microsoft.Numerics.Mathematics.BasicMath.PI * NoiseFreq * t / Freq) / 2;

            //Input signal with noise
            inputSignal += Noise;                        

            // Creating of low-pass IIR Chebyshev-I filter 
            int filterOrder = 8;
            double ripple = 0.001;
            double cutOffFreq = 0.5;
            var IIRFilter = IIR.ChebyshevI(filterOrder,FilterKind.LowPass,cutOffFreq,ripple);                

            //Applying IIR filter to the input noised signal 
            var outputSignal = Filtering.Filter(IIRFilter, inputSignal);                     
            Console.WriteLine("The filtered signal (IIR): {0}", outputSignal);            
        }
    }
}

Simple Distributed Example

The following example creates local arrays with Numerics.Local.NumericDenseArrayFactory, operates on them with the Kronecker product method, casts the result array to be distributed, and then operates on this distributed array. You can run this example either by using the local distributed runtime or on Windows Azure

using System;
using Microsoft.Numerics;
using msnl = Microsoft.Numerics.Local;
using msnd = Microsoft.Numerics.Distributed;
using Microsoft.Numerics.Mathematics;

namespace BasicMathExample
{
    class BasicMathExample
    {
        public static void Main()
        {
            // Demonstrate identity sin(x)*sin(x) + cos(x)*cos(x) = 1, for local real, local complex, and distributed arrays

            // Initialize distributed runtime 
            NumericsRuntime.Initialize();
            long n = 10;
            double minValue = 0;
            double maxValue = 5;
            var values = msnl.NumericDenseArrayFactory.UniformGrid<double>(n, 
                                                                           minValue,
                                                                           maxValue);

            var result = BasicMath.Pow(BasicMath.Sin(values), 2) +
                                       BasicMath.Pow(BasicMath.Cos(values), 2);

            Console.WriteLine("Identity check result: {0}", result);

            // Show that the identity holds for complex values too
            // Construct vector of complex numbers with specified phase and magnitude
            var phase = BasicMath.PI * msnl.NumericDenseArrayFactory.CreateFromSystemArray<double>(new double[] { -0.5, 0.0, 0.5, 1.0 });
            var magnitude = msnl.NumericDenseArrayFactory.CreateFromSystemArray<double>(new double[] { 0.25, 0.5, 0.75, 1.0 });
            var complexValues = magnitude.ConvertTo<Complex128>() * BasicMath.Cis(phase);

            // Apply formula
            var complexResult = BasicMath.Pow(BasicMath.Sin(complexValues), 2) + BasicMath.Pow(BasicMath.Cos(complexValues), 2);

            Console.WriteLine("Identity check for complex numbers");
            Console.WriteLine("Magnitude: {0}", BasicMath.Abs(complexResult));
            Console.WriteLine("Real Part: {0}", BasicMath.RealPart(complexResult));
            Console.WriteLine("Complex Part: {0}", BasicMath.ImaginaryPart(complexResult));

            // Show that the identity holds for large distributed array, 
            // by using distributed function overloads

            // Use Kronecker product to expand vector into distributed matrix
            n = 2000;
            msnd.NumericDenseArray<double> row = msnl.NumericDenseArrayFactory.UniformGrid<double>(n, minValue, maxValue).Reshape(new long[] { 1, n });

            msnd.NumericDenseArray<double> column = msnl.NumericDenseArrayFactory.UniformGrid<double>(n, minValue, maxValue).Reshape(new long[] { n, 1 });
            var distributedValues = Microsoft.Numerics.LinearAlgebra.Operations.KroneckerProduct(column, row);

            // Compute distributed sin function and compare to exact result of 1
            var distributedResult = BasicMath.Pow(BasicMath.Sin(values), 2) + BasicMath.Pow(BasicMath.Cos(values), 2);
            var maxError = ArrayMath.Max(BasicMath.Abs(distributedResult - 1));

            Console.WriteLine("Maximum error in distributed result: {0}", maxError);

            // Shut down distributed runtime
            NumericsRuntime.Shutdown();

        }
    }
}