Provisioning with the Client Object Model Part 2 – Getting Started and Provisioning Fields

Previous post: Provisioning with the Client Object Model Part 1 – Introduction

Until recently I thought of the client object model as a client side version of the server object model. Today I realize that thinking of CSOM this way is a mistake. The two models are very similar. However, the way you use them is very different. Because of this, differences that seemed very important when I looked did a direct comparison of the classes in the two API’s turned out to not be very important. For the most part this is because the way you create and access objects is fundamentally very different. This is not to say that CSOM doesn’t have any holes or couldn’t use some improvement. It does, it could, and I will point them out as I continue this series.

Thinking in batches and working in Layers

When you write provisioning code on the server you probably completely configure each object before you move on to the next. The following pseudo code illustrates what I mean.

For each list to create

    Create the list
    Bind the content type
    Configure the views

Next list

 

Because the code is running on the server, beyond the unavoidable expense of the operation itself, there is no penalty for persisting updates as you go. If you need pieces of information from some other object in the site, there is no penalty for accessing the object to get the information you need. For all practical purposes, in most situations such code is pretty straightforward and opportunities to make noticeable optimizations are few. I define the time to execute a potential operation on the server as:

[Time to perform server side operation]

When you are working on the client, every object access and every update is a potential round trip to the server and so it is easy to write inefficient code. The potential cost of a given server side operation becomes:

[Time to format request] + [Time to send request] + [Time to parse request] + 
[Time to perform server side operation] + [Time to format response] +
[Time to send response] + [Time to parse response]

Therefore the goal is to pay the expense of doing the server side operation as few times as possible for a given transaction. The client object model is designed to make this goal attainable by providing a few key features.

  • You can add operations to a batch of requests via ClientRuntimeContext.Load
  • You can submit a batch of operations with ClientRuntimeContext.ExecuteQuery
  • You can use an object without fetching it from the server it as long as you don’t try to read its properties

The flexibility of ClientRuntimeContext.Load and the ability to use an object without fetching it from the server are the things that make CSOM so fundamentally different from the server OM. It makes it possible to change the previous pseudo code to the following.

For each list to create
    Create the list

Next list

Submit the batch to the server

 

For each created list

    Bind the content type
Next list

Submit the batch to the server

 

For each created list

    Configure the views
Next list

Submit the batch to the server
			

 

The batches are determined by dependencies. A list must exist before a content type is added. A list must have all of its fields before configuring a view.

Provisioning Fields

The various Field classes were a significant source of concern for me when I compared the client and server API’s because there are a number of missing properties. It turns out that this is not an issue because you can manipulate all of a field’s settings via its SchemaXml property. The only wrinkle I have found is that there are a number of properties SharePoint wants you to set only when the field is in a List’s Fields collection, not in a Web or ContentType’s Fields collection. For this reason, the first step in provisioning the overall solution is to simply create, but not attempt to configure, each field. If a field is a lookup or metadata column it will require additional configuration, but I leave this until later in the overall process after the base list instances and/or term sets are created.

The easiest and best way to create a new field is via the FieldCollection.AddFieldAsXml Method. The XML this method takes as a parameter is exactly the same as the XML you would use to create the field via a feature. The primary thing that can go wrong when attempting this operation is an error if the field already exists. Because operations are submitted in batches the code must be defensive – you don’t want to be in a position where you must puzzle out which operation in a large batch is making the operation fail! Therefore the provisioning code must first get a list of the existing fields to prevent the error.

The pseudo code for the operation is simple.

Request the Web's Fields

Submit the batch to the server

For each field to create
    If the field does not exist

        Create the field from XML

Next field

Submit the batch to the server

 

As I said in the previous post, a primary goal is a good style that shares my favorite aspect of feature CAML, readability. For this reason, I declare the list of fields using literal syntax to create a Dictionary<string, string> where the key is the field’s StaticName and the value is its SchemaXml. The basic syntax is:

And a real example (cropped) looks like this:

The code to do the actual provisioning is simple.

Getting the Existing Fields

The check against the FieldDefinitions dictionary uses the value of the StaticName property and nothing else. Therefore the client should only request that property value when it requests the entire collection to minimize the message size. This is accomplished by the lambda expression passed as the second argument of the Load method as follows.

Note the call to ExecuteQuery. The FillExistingFieldsList method reads the StaticName property. Attempting to read StaticName before the call to ExecuteQuery will throw a PropertyOrFieldNotInitializedException. Attempting to refer to any other property of a Field after the call to ExecuteQuery will yield the same exception because the lambda expression in Load doesn’t include any other properties.

Optimizing the Query

One very powerful aspect of the lambda expression is that it can restrict the returned set by a property value. In my solution the fields are members of an App specific group or of hidden. I attempted to optimize the operation by modifying GetExistingFields as follows.

This ability is amazingly cool because it closes what I thought were a number of holes in CSOM related to fetching items. Examining the results in Fiddler shows that the response is ~33% smaller. However, it also shows that in this situation the call to ExecuteQuery took 53% longer than the unfiltered version. However, when the entire operation is timed, it the ‘optimized’ version is only 1.5% slower. Almost all of the time lost on the server was made up for on the client.

The clear take away is that you should not assume that reducing the response size via a query filter will result in better performance. However, minimizing the number of round trips is a safe bet. In my development environment I have 46 fields. If I changed the code to call ExecuteQuery after adding each field, the number of round trips increases from three to forty-eight and the overall time to complete increases by 82%. There is virtually no latency in my development environment, so in a production scenario where the App is hosted on a separate server, the difference would be dramatically greater.

Author: Doug Ware

Provisioning with the Client Object Model Part 1 – Introduction

I suppose there are many ways that you could get started writing CSOM and JSOM code, but provisioning is probably where most people will start when they create a SharePoint app for a traditional collaboration solution. After all, the implementation of these types of solutions usually starts with fields, content types, lists, and views. The provisioning of these things is the subject of this series of posts.

Client Side Object Model and C# versus the JavaScript Client Object Model

Given my recent series on JavaScript you might be surprised that all of the code in this series will be written in C#! My general opinion is that provisioning (and other server-type operations) should be done in a remote event receiver to process the App installed event and that user interface code should be written in JavaScript. In other words, both models have their place and an App will have some of both. Once you are comfortable with JavaScript moving back and forth between the API’s is easy because the differences are minor. If you’d rather have this code in JavaScript you should have no trouble converting it.

Unfortunately, remote event receivers don’t seem to work with the current beta bits. Because of this I am using a high-trust provider hosted site for the App in my development environment. The start page has some buttons I can click to run my code in-lieu of an event handler.

Goals

I have a few general goals that the code must meet for my satisfaction. These include minimizing the number of round trips between my App and the host web and maximizing reuse. I know I’ll need this code for many different projects! But, the main thing I want is a good style that shares my favorite aspect of feature CAML, readability. Because this code base is very much a work in progress I won’t be including a download as I usually do… at least not yet.

In the next post I’ll talk show you how I provision the easiest type of item to provision – fields.

Next Post: Provisioning with the Client Object Model Part 2 – Getting Started and Provisioning Fields

–Doug

Author: Doug Ware

Building Traditional SharePoint Collaboration Solutions with the App Model

The SharePoint store on the Office 365 preview is slowly filling with apps. To date the one thing all of the apps have in common is that none of them resemble the sort of traditional SharePoint collaboration solutions that I believe constitute a large majority of the solutions people build in-house. By ‘resemble’ I do not mean look and feel but rather functionality.

Traditional collaboration solutions

SharePoint has a huge number of features and people use it for all sorts of things, but the sweet spot has long been solutions that take the basic components of a Team site and apply them to solving specific business scenarios. Such solutions are made up of business specific content types, lists, libraries, and event handlers or workflow. They often involve branding and custom UI components, but the bulk of the functionality usually comes from SharePoint and much of the code (both declarative and procedural) in a given solution is devoted to configuring the site as opposed to creating truly custom components.

SharePoint is an attractive platform for a wide range of applications because it contains so much configurable functionality right out of the box. Skilled implementers can do some amazing things in a very short amount of time compared to traditional start-from-scratch development. The deep integration with Microsoft Office, especially with Outlook, makes it possible to easily integrate shared functionality delivered via the web with the desktop productivity applications business people use all day, every day.

None of the current marketplace apps make extensive use of these building blocks. Instead most of them add complimentary functionality, as is the case with the Nintex Workflow app, or integrate with some external service like HP ePrint.

App Web Confusion

When SharePoint 2010 was in beta I built a non-trivial sandbox solution to find out how well the sandbox model works. I am now in the process of doing the same thing with the new app model. I had a very hard time at the beginning because I was focused on the App Web as the location of my functionality. An App Web is a special web intended to host all or part of an app. However, an App Web is very special and it lacks most of the building blocks needed for a traditional solution including:

  • Navigation
  • User and group management
  • Rich client support
  • Site settings

It seems that an App Web is a good option for completely custom solutions like Nintex Workflow, to host App Parts like the Bamboo World Clock and Weather part, or as a landing page with instructions or configuration tools, but for a traditional solution it is completely inadequate. After flailing about for a significant amount of time I came to understand that a traditional solution requires the use of the host web, not the App Web.

Using the Host Web

A host web is a regular SharePoint site where the application is installed. Any regular SharePoint site can be a host web. An App can specify that it needs significant permissions to the user’s host web up to full control.

But there is a problem.

Although an App can contain features in a wsp (web solution package) file, the solution package and features are applied not to the host web, but to the App web. The only declarative features you can apply to the host web via an App are CustomAction and ClientWebPart. These allow you to add functionality to a site’s menus and to add App Parts, respectively. The implication of this is that to use the host web effectively you can’t use features but must use one of the client side API’s to do provisioning, CSOM, JSOM, or REST.

Living without Features

For my App I decided to use C# and CSOM for the provisioning code, but I was very skeptical that the client side object model contained enough functionality to do the job. I was pleasantly surprised to discover that it was. I am able to provision and configure fields, content types, lists, and pages with Web Parts. The only real issue I had was that you can’t specify a content type ID when you create a new content type.

I am even more surprised to tell you that I prefer this approach over CAML and that I find it to be more flexible. I built a generalized framework to provision these elements which will be the subject of further posts, and I greatly appreciated being able to add tracing and logging to the process. It made debugging much easier. I believe that I wound up with more code than I would have using XML, but not much more. I also appreciate the fact that should the App Web become an option, I will be able to simply change the target web and my code will still work.

On the other hand, I did have to a lot of work to move from features to code. Fortunately I can reuse the code so future projects will be fairly painless.

About the only thing my App lacks is workflow, but the Nintex Workflow App clearly demonstrates that the API’s are there.

Remote Event Receivers don’t Work in the Preview

…at least if they do I haven’t ever seen it. In the meantime I just have a "Provision" button on a page in a provider hosted site configured for high-trust. In my App the list forms are custom and so I have been able to live without event receivers – I can do the work on submission of the forms. However, for many potential applications this will be a major issue. If it is I suggest you simply wait until the next code drop before you try to implement anything where you simply must have event handlers.

Is it better than Sandbox?

Based on my experience thus far it’s no contest. The App model is far more flexible. The client API’s are larger and more complete than the sandbox object model and the ability to do work at a higher level of permissions than those possessed by the current user enables many more scenarios.

Is it better than Farm?

I don’t know yet, but if you are currently restricted to a sandbox model or do extensive sandbox development you might want to change simply to minimize the number of skills you have to maintain across your team. The downside is that the architecture is more complicated and has more moving parts.

Is this Approach Valid for the Office 365 Marketplace?

At the moment there is nothing explicit in the guidelines that prohibits this approach. At the same time, I doubt they want users to party on the host web and I doubt Microsoft will approve an App that does this. However, both Office 365 and on-premises installations support side-installs to support just this sort of scenario by requiring an administrator to enable the App. Regardless, most Apps of this nature won’t be built for public sale anyway.

–Doug

Author: Doug Ware

JavaScript for the C# Developer Part 8 – WTF is this?

Previous: JavaScript for the C# Developer Part 7 – Prototypal Inheritance

In JavaScript for the C# Developer Part 6 – Objects I promised to cover JavaScript’s very confusing treatment of the this keyword.

 

 

The this keyword

Many languages use the this keyword or something similar, e.g. Me in Visual Basic, to represent the current object instance. JavaScript does too (http://jsfiddle.net/DougWare/rX8JA/).

The preceding example shows that within an object this works exactly like you would expect based on experience with C#. In the object x, this is equal to x. In the object y, this is equal to y.

If only it were that simple

The way this works in the example is obvious, because whatsThis is defined inside both objects. It makes sense that this refers to the object. Unfortunately it is not always so obvious because of the flexibility JavaScript allows with functions. Consider this example (http://jsfiddle.net/kgJ87/).

This example shows three different object instances as the value of this within the function globalWhatsThis: x, y, and window. This is actually a byproduct of the syntax – from a runtime perspective; this example is essentially the same as the first example. The declaration of the whatsThis members in the two object literals assign the value of the globalWhatsThis variable (which is a function) to whatsThis. Even though the function is declared outside the objects, the runtime effect is that each object has its own copy of the function.

A very simple change to the object literals to invoke globalWhatsThis from the whatsThis member functions has a profound effect on the results (http://jsfiddle.net/DougWare/cZ6p7/).

DOM element events

It should be clear that using this inside a function that is not part of an object is dangerous because the functionality of the called function depends on the caller’s implementation. This problem is often encountered with DOM element events. Consider this example where a button’s onclick handler contains JavaScript to call a global function (attached to window) (http://jsfiddle.net/DougWare/w46yC/).

Contrast that example with one where the function is attached directly to the event (http://jsfiddle.net/DougWare/kdTDN/).

If you use the jQuery framework (or something similar) to manage events you won’t encounter this because the framework ensures this will be the element that raised the event. Either way, you should understand that this is the same syntactical illusion as before; the value of whatsThis (a function) is assigned to the element object’s onclick property and is therefore part of the button element object! If the function calls a function that is part of the window, this will be window (http://jsfiddle.net/DougWare/kSEwE/).

Bind

There are a number of ways you can control the value of this instead of just writing code that uses it and hoping for the best! Among these is the bind() method of the function object. Bind is available in most modern browsers, but it isn’t available in versions of IE before 9.0. However, you can add it if you need it or use a framework like prototype that includes it.

The bind() method takes an object as an argument and returns a new function bound to the object of your choice to use as this (http://jsfiddle.net/DougWare/e7fBy/).

Callbacks

The most common error I see with regards to this is its improper use in scenarios where the developer is attempting to share data between the function that sets up the callback and the callback function. The SharePoint CSOM documentation contains numerous examples of this type of error. Here is an example from http://msdn.microsoft.com/en-us/library/jj163201(v=office.15).aspx.

Every instance of this in the sample code refers to window and these are therefore global variables just waiting to cause issues when there are multiple instances of the callback waiting!

An easier to identify variation of this mistake is found in callbacks bound to event handlers. In this scenario, this is the DOM element but this is the window in the callback for reasons that you should now understand. Here is an example that demonstrates the issue with a couple of buttons that setup callbacks via the built-in setTimeout() function (http://jsfiddle.net/DougWare/vfR42/).

When you run this example and click the buttons, undefined displays twice after a short delay because this.someData is undefined – this refers to window.

Using bind() with callbacks

The previous example can be fixed to bind the button elements to the appropriate callback (http://jsfiddle.net/DougWare/KByPj/).

Keep in mind that bind takes any object as its argument. In the example I bound this (the button) but I could use any object I want to act as this in the callback.

–Doug

Author: Doug Ware