Tag Archives: javascript

Programming well structured Javascript stored procedures for DocumentDB with Typescript and SystemJs

EDIT: code sample on github https://github.com/bpatra/StoredProcedureGeneration

If you are using DocumentDB you may had to write your own stored procedure. A stored procedure is a function written in Javascript that runs on the DocumentDB cloud infrastructure. It may reduce performance problem or make you execute some queries that are not supported yet through the REST API such as aggregate function.

A stored procedure should be registered as a single function of the form:

function myStoredProcedure(arg1, arg2){ /* you can place all the arguments you want here*/
    //The body of the function here

    //you set the response like this
    var context = getContext();
    context.setBody(myResult);
}

You can create function inside the myStoredProcedure function body. However, you cannot create other functions in the file otherwise DocumentDB will complain. This is quite annoying because you probably want to create independent and reusable pieces of code for testability or simply for the sake of readability. The problem comes from the fact that you cannot really use out-of-the box third party module management libraries such as requireJS, commonJS or SystemJS. This sounds no good. Is that means we are forced to inline all our code in a big not modular and un-testable Javascript file!?!?

The answer is no, in this blog post I will show you the solution we implemented at Keluro to overcome this problem. This solution is based on SystemJs and SystemJs-Builder to create a standalone function where all modules/class dependencies are embedded in the stored procedure single function which acts as our entry point. The code snippets presented in the following are extracted from the following git repository.

In this article, we will use Visual Studio as an IDE but it is not mandatory. Actually, it is only to simplify the options settings for compiling Typescript files, you can invoke the compiler manually exactly with the same set of options.

In the following, we will take an example where the stored procedure computes the sum of input arguments. Therefore, we create a Typescript class for the core of our stored procedure called Utilitary that contains a method called sumValues.

A typescript class used in DocumentDB stored procedure

A typescript class used in DocumentDB stored procedure

Then we create the entry point of the stored procedure in a Typescript file that uses the DocumentDB current context. We benefit from the typings from documentdb-server.d.ts that defines the IContext interface.

MyStoredProcedure typescript file executed by DocumentDB

MyStoredProcedure typescript file executed by DocumentDB

We compile the Typescript files has SystemJs modules and we redirect all generated Typescript in the directory out-js. As I told you, no need of VisualStudio to do this, you can achieve the same result by invoking the Typescript compiler with similar options.

ompiling options set in Visual Studio

ompiling options set in Visual Studio

If you look at the generated Typescript you will get something that looks like

System.register(["Utilitary"], function(exports_1, context_1) {
    "use strict";
    var __moduleName = context_1 && context_1.id;
    
     /* ETC */
});
//# sourceMappingURL=myStoredProcedure.js.map

If you try to run such a Javascript source code directly with DocumentDB you will get an error. These modules are meant to run with System.js and in the case of a stored procedure you cannot reference any third party library such as System.js. This is where SystemJs-Builder will come into play. SystemJs-builder will package everything so that your modules are independent of System.js and they can be used as a standalone file.

To invoke SystemJs-Builder you need to use Node.js. In the sample git repository, you will have to hit ‘npm install’ to restore the SystemJs-Builder Node package. When the Typescript files are compiled invoke the node script “documentdb_storedprocedure_builder.js” at the root of the repository. This script basically executes two tasks. First, it generates a standalone Javascript file with SystemJs-Builder. Secondly, because DocumentDB explicitly needs a file with one function and not an executable Javascript script, we wrap this code inside a function that will act as our entry point for the stored procedure.

We also retrieve the arguments passed to the storedprocedure procedure function in a variable called storedProcedureArgs that is kept in the global namespace. The resulting file is generated and put in the directory generated-procedure. Finally, this file contains what we needed: a standalone and executable by DocumentDB Javascript function. With this approach all Typescript classes can be reused for other stored procedures , for unit tests or anywhere in your Typescript code base.

There are probably thousands of alternatives to create testable and decoupled stored procedures. We liked this approach because it reuses the same tools that we were already using for our single page applications: Typescript and SystemJs. To conclude let me thank Olivier Guimbal who showed us some months ago how Typescript, SystemJs and SystemJs-Builder worked well together.

Implementing the OAUTH 2.0 flow in app for office sandboxed environment

EDIT: this approach is outdated. You can have a look at the sample we released and especially in the README section we compare the different approaches
https://github.com/Keluro/Office365-AddinWeb-SignInSample

In this post I will explain how I implemented the OAUTH 2.0 flow in the app for office sandboxed environment. The objective of this post is to discuss with others if the proposed implementation is satisfactory. Do not hesitate to criticize and propose alternatives.

Following the msdn documentation, the app for office model is the new model for

engaging new consumer and enterprise experiences for Office client applications. Using the power of the web and standard web technologies like HTML5, XML, CSS3, JavaScript, and REST APIs, you can create apps that interact with Office documents, email messages, meeting requests, and appointments.

Personally, I find this extremely promising, I even release my first app last year: Keluro web analytics that is distributed here on the store. I even wrote a small article dealing with app for office on the Keluro web blog.

Let us get straight to the point for this tech article. Apps for office aim at building connected service. For example, Keluro web analytics fetchs data from Google Analytics within MS Excel. I am working on a new project that must communicate with other Office 365 APIs. In all cases, you must use the OAUTH 2.0 flow to grant your app the right to communicate with those APIs. Sometimes there are some client libraries to leverage technicalities of the OAUTH flow. For my fist app, I used the Google Javascript API and for the second one I used the Azure Active Directory adal.js library.

However, there is problem. The apps for office run in a sandboxed environment that prevent cross domain navigation. This sandboxed environment is an iframe with sandbox attributes for the case of the office web apps. I asked the question for more information in this msdn thread. In all cases, the consequence is that the usual OAUTH 2.0 flow that redirects you to https://login.microsoft.com/etc. or https://apis.google.com to prompt the user for allowing access will not work. In addition, the pages served by those domains cannot be ‘iframed’ whether they are served to with X-Frame-Options deny or some js ‘frame busting’ is embedded.

I will get straight to the only working solution that I found. Actually, its the same recipe that I used for Google APIs and ADAL.

It looks like that the apps for office run with the following sandboxed attributes sandbox=”allow-scripts allow-forms allow-same-origin ms-allow-popups allow-popups”. Therefore, we are allowed to use popups and this popup is not restricted by CrossOrigin. What we will do is to open a popup and we will do the OAUTH flow in this popup, when its done we will get back the necessary information in the main app iframe. Here is a summary with a small graphic.

An overview of the solution found for the OAUTH2.0 in sandboxed env

An overview of the solution found for the OAUTH2.0 in sandboxed env

Then, you got it we will use a special callback.html file (that is basically blank and contains only javascript) as returned url. Here it its code.

<!--usual html5 doctype and meta-->
<body>
    <div id="oauthurl"></div>
     <script>
        var closeWindowCheck = function () {
        <!--transmit the hash of the return url to the parent iframe-->
            window.oauthHash =  location.hash;
            <!-- wait that the parent window give its approbation to close-->
            if (window.canclosewindow === true) { 
                window.close();
            }
        }
        var interval = setInterval(closeWindowCheck, 200);
    </script>
</body>
</html>

Then in the main iframe when it’s time to start the OAUTH flow, you have a piece of javascript that looks like the following

//this javascript lives in the sandboxed environment
//urlNavigate is the oauth2.0 url of the apis you are targeting with the parameters
//use callback.html as redirect uri.
var childWindow =window.open(urlNavigate,'Auth window', 'height=500,width=500');
var interval;
var pollChildWindow = function () {
    if (childWindow.closed === false) {
        var oauthHash;
        try {
           //this throws exception until the popup returns to
           //callback.html served in our domain.
            oauthHash = childWindow.oauthHash; 
        }catch(ex){}

        if (typeof oauthHash === &quot;string&quot;) {
            //childWindow.close(); // does not work with Chrome
            // ->; that is why we have the setInterval in callback.html javascript
            childWindow.canclosewindow = true; // allow the child window to close.
            var startWith = oauthHash.indexOf('#id_token');
            var currentRef = window.location.href.split('#')[0];
            //you get the id_token and other information from OAUTH2.0 
            //do whatever you have to do with it in your app.
            //YOUR LOGIC HERE
        }
    } else {
        clearInterval(interval);
    }
};                                                                                                                                          
interval = setInterval(pollChildWindow, 200);

I personally find this implementation not satisfactory because of the use of the popup (that may be blocked). What do you think of it ? Do you think of an alternative. I tried many things using postMessage API but it does not work with IE in sandboxed environment.

This is the implementation that is discussed in this ADAL.js issue. You may also find a setup of a sandboxed environment and some implementation attempt (without apps for office for simplicity) on my remote branch of my Adal.js fork.

Tools for javascript TDD with Visual Studio

In this post I describe the tools that I have selected for efficient development of javascript tests within Visual Studio. Indeed, if you are developing sites with ASP.NET or Apps for Office then you are more or less committed to use Visual Studio. Therefore, you probably do not want to use another editor for the javascript development.

What were my requirements when I chose the tools that I will describe to you in this post? Actually, I needed tools which can provide me the same comfort that I have while I am developing in TDD with .NET. Precisely, the test should be run individually (or at least individually for a given set). They should be run on the continuous integration build, which is teamcity in my case. The test runner may use any browser for executing the tests even if flexibility for using all kind of browsers would be appreciated. In addition, the test should be easily debugged and this is a very important aspect for me.

Let me detail a little this latter requirement. Indeed, I have read recently this blog post and its written that

Debugging the tests
Wiseman said: “If you need to debug unit test then something is wrong with it’s unitness.”
PS: If it’s really needed you can debug it in browser as you normally debug JavaScript code.

Even if the rest of the blog is brilliant, I do not agree at all with this sentence.  If you practice regularly TDD then you’ll see that you have to debug, that’s part of the game. If you never have to debug maybe it’s because you are asserting too much trivialities and not enough your own complex logic. In addition, if you have to create the webpage on your own, for debugging, that is not acceptable to me either. You should be able to put your breakpoint anywhere in your code and hit it in less than five seconds. We will see that the tool chosen permits such quick debugging and that’s the main reason why I am using it.

At the time of the writing, the popular Visual Studio Addin Resharper with its version 8 supports javascript tests with the frameworks Jasmine or QUnit. Unfortunately, the debugger cannot be properly attached while debugging the scripts. Sadly, I had to reject Resharper that I appreciate so much for .NET development.

Finally, the best tool that I found is Chutzpah (which means Audacity in Yeddish). Firstly, there is the Chutzpah runner which is a javascript test runner that uses the headless browser phantomJs, we will invoke it from the command line in the continuous build. Sedondly, there is the Chutzpah Visual Studio extension which enables quick runs and debugging sessions from Visual Studio. Both runner and Visual Studio extension is fully compatible with JS test framework Jasmine 2.0. that I have chosen. Note that I also use the mocking library sinonJs but I won’t discuss it there.

You may retrieve the code samples below on my github repository.

To illustrate these tools we will use a very simple mockup project and we start its description with the following VisualStudio solution.

The Visual Studio solution describing the example of this post.

The Visual Studio solution describing the example of this post.

As usual there are two projects: WebApplication1, the core project, containing the app folder with our javascript logic, especially the one that we would like to test: sut.js (for SystemUnderTest). There is also the test project, WebApplication1.Tests. The file testing the logic contained in sut.js is simply sutTests.js. The testing framework Jasmine is added to the solution by referencing the folder lib/jasmine-2.0.0..

We will take a very simple example, where the object app.sut has a function for computing the factorial of an integer. The code of sut.js can be found in the following snippet.

(Note that in app we have implemented a basic namespacing function such as this one)

Let us now focus on the test suite in the sutTests.js code.

This piece of code is very basic Jasmine syntax for a test suite. We assert the values of the factorial function for the two corner cases where the input is 0 and 1 and we also test the situation with n=5. Remark also that we have a test which asserts that an exception is thrown when a non positive value is passed to the function. We have not covered this situation with the snippet above, consequently, we expect this test to fail… Remark that all the needed references to js files are handled with ///<reference of Visual Studio. By the way, we benefit from Visual Studio Intellisense here.

The Javascript intellisense with Visual Studio

The javascript Intellisense with Visual Studio

In the next screen shot, we run the tests with the Visual Studio Test Explorer that is compatible with Jasmine thanks to Chutzpah.

Run all our test with the Visual Studio Test Explorer. As expected we encounter a failure.

Run all our test with the Visual Studio test explorer. As expected we encounter a failure.

If we were in the situation where we do not know what is going wrong (which is often the case) we would need to debug the test. Unfortunately, like Resharper, you cannot debug the test directly in VisualStudio, the debugger does not attach well. Even Chutzpah creator does not know how we would do that, so I believe we will have to wait to debug javascript test as easily as we debug .NET within visual studio. Then, we will have to debug with a web browser. However, Chutzpah creates the webpage for bootstrapping your failing test. So just by clicking in Open in browser you will have the web page loaded and a link to run the failing test within the browser (see screenshots below).

Chutzpah Visual Studio extension creates web page for running/debugging tests in the web browser.

Chutzpah Visual Studio extension creates web page for running/debugging tests in the web browser.

Then the debugging sessions happens in your browser. Reclicking the links reexecutes the tests.

Then the debugging sessions happens in your browser (here Chrome). Clicking again the links reexecute the tests.

Now, we do have all the material for editing efficiently tests in Visual Studio. Let us have a few words for running the test using the command line on the server. Personally, I embed the Chutzaph runner in a /build directory within my sources and executes the following Powershell script.

Here is what it looks when ran it an Powershell console with teamcity options.

The execution of all javascript tests of the solution with Chutzpah runner using Powershell

The execution of all javascript tests of the solution with Chutzpah runner using Powershell

To conclude, I would say that Chutzpah is a great project and if you need a simple and ready to use test runner I would recommend it. The only limitation for now  would be that the runner only supports phantomJS. However, you may use another another test runner for executing the tests with different browser and you can keep the Chutzpah Visual Studio extension for the development. One last important thing to note is the fact that Chutzpah 3.0 supports RequireJs.