Office 365/SharePoint Sandbox Service Pages in Action – Autocomplete Text Boxes

A couple weeks ago I wrote about an Unorthodox Architecture for Services in SharePoint Sandbox Solutions and Office 365. This post is about an application of this architecture to create an autocomplete text box using jQuery UI. An autocomplete field looks like this:

It is a great alternative to the stock lookup field UI when the source list of the lookup field has a large amount of data. With hundreds of items the standard dropdown list just isn’t a good experience. The jQuery Autocomplete widget makes it easy to build a field that displays a list as the user types. In this example the JavaScript is simple and the underlying field is "single line of text". If you need to ensure the entry matches a value from the lookup list you’ll need to do some extra work. That said, here is the JavaScript:

source: function(request, response) {
url = L_Menu_BaseUrl + "/ServicePages/AutoComplete.aspx?l=Clients&f=Title&v=" + request.term;
$.getJSON(url, function(data) {
minLength: 1


The url in the above script is to a service page that contains a Web Part whose code is shown at the bottom of this post. The variable L_Menu_BaseUrl is declared and populated by SharePoint’s own JavaScript libraries and it contains the Web’s server relative URL. The Web Part reads the query string and runs a CAML query against the specified list to search for values that begin with the specified term. The result is written to the page as JSON by using the JavaScriptSerializer to serialize a List<string> of the specified field for each item in the result set.


//Inputs are
public class AutoComplete : WebPart
protected override void Render(HtmlTextWriter writer)
var items = GetItems();
if (items.Count > 0)
JavaScriptSerializer js = new JavaScriptSerializer();

private List<string> GetItems()
var items = new List<string>();
string listName = Page.Request.Params["l"];
string field = Page.Request.Params["f"];
string val = Page.Request.Params["v"];
uint max = 10;

//Parse the query string parameters and return an empty list if there is a problem
if (Page.Request.Params["max"] != null)
try { max = uint.Parse(Page.Request.Params["max"]); }
catch { } //ignore the error and return 10 items

if (string.IsNullOrEmpty(listName) || string.IsNullOrEmpty(field) || string.IsNullOrEmpty(val))
return items;

//Build and execute the query
string queryXML = string.Format(@
"<OrderBy><FieldRef Name='{0}' /></OrderBy><Where><BeginsWith><FieldRef Name='{0}' />
<Value Type='Text'>{1}</Value></BeginsWith></Where>", field, val);

string viewFields = string.Format(@"<FieldRef Name='{0}'/>", field);

SPList list = SPContext.Current.Web.Lists.TryGetList(listName);

if (list == null) return items;

SPQuery query = new SPQuery();
query.Query = queryXML;
query.ViewFields = viewFields;
query.RowLimit = max;
var listItems = list.GetItems(query);

//Put the returned items into the list
foreach (SPListItem listItem in listItems)
return items;

Author: Doug Ware

How Office 365 Made Me Hate My Windows 7 Phone

I had a hard time deciding on a title for this post. The experience I am going through is just ridiculous on so many levels and there is a lot of hate to go around.

SharePoint Online in Office 365 uses claims based authentication which is cool, and it supports federated authentication and trust to let you use your corporate credentials which is even cooler. Microsoft has been a leader in this space for a long time and everyone I know has a Live ID. However, for reasons that are unexplained, Office 365 doesn’t use Live ID. Instead it has its own federated system that gives users super long IDs like

Strangely enough SharePoint Online does support Live ID via the (also very cool) external user access features. Basically, you invite an external user and they can access your site using LiveID as long as the Live ID is a email address.

Yes! A platform that includes Exchange requires the use of for LiveID in spite of the fact that Live ID itself has no such requirement.

A couple weeks ago I tweeted this depressing but funny comic of various org charts including a spot-on depiction of Microsoft’s. I suspect that this chart explains Microsoft’s reasoning behind this idiotic requirement. Someone’s bonus was threatened by Exchange in 365, but by requiring they can keep the numbers up. (I freely admit and emphasize that this is pure speculation with no basis in fact).

I wanted to try this External User feature out, so (trusting fool am I) I changed my existing LiveID which I have been using happily for years to use a address.

Right away there were problems.

First, I noticed that Live Messenger now blocks most (but not all) of my contacts. That’s a really big problem that I have yet to solve.

Xbox Live worked like a champ, as did Zune. Both picked up the change and both were painless.

My Windows Phone 7? Well… You can’t change the Live ID on a Windows Phone once it is set up without resetting it to the factory defaults. Pure genius! I wonder if changing this is one of the 500 new features in Mango.

The moral of this story? Never change your Live ID because Microsoft does not have its act together and you will suffer greatly for it if you do.

Unorthodox Architecture for Services in SharePoint Sandbox Solutions and Office 365

These days we are spending a lot of time on a series of sandbox applications that will run on Office 365. If you are a follower of this blog you know I was more than a bit disappointed in the restrictions the SharePoint 2010 sandbox implementation imposes. However, if you want to play in the sandbox, you can either just say no to requirements that aren’t directly supported, or you can earn your money and get creative!

I’ve been getting creative. These applications are going to be great – that said, some elements of the architecture are unorthodox and even a little strange. This post details one of the elements.


Expose services to external clients that:

  1. Hide their underlying technology from the caller and hide the application’s underlying implementation from prying eyes
  2. Support all current web development stacks
  3. Perform well
  4. Are easy to create and consume

Client side technologies like the Client Object Model (JSOM) and SPServices are awesome. However, they fail goal #1. If page’s business logic is implemented in JavaScript it is available to any and all who care to look. Worse, they both encourage tight coupling between the client UI and SharePoint. Both support JavaScript, and the Client Object Model also supports managed (.NET) clients and Silverlight. However, both have minor learning curves for developers who mainly write C# and use the server object model.

What we really want is for the client to consume services via an abstraction that hides the details and helps minimize dependencies. Unfortunately, you can’t deploy traditional asmx or WCF services to the SharePoint Sandbox.

Before we get into the solution, please be aware that I am not saying that this architecture is better than JSOM, SPServices, or the REST API – I’m just saying it is a better fit for our particular needs for certain specific applications. It also illustrates an interesting technique that you can add to your own arsenal.


While it is not possible to deploy traditional Web services, we can come close. A Web service is just an http endpoint to which we can pass arguments and from which we can read a response. Conceptually, the only differences between an aspx page and a Web service are the calling convention and (usually) the content type (e.g. ‘text/html’ versus ‘text/xml’ or ‘application/json’). You can create an aspx page that works just like an asmx or WCF service if you want – you just have to handle the plumbing yourself.

Our applications have lots of JavaScript and JSON is the easiest format to consume. It is easy to create a page in the sandbox that returns JSON based on query string or form POST parameters. Here’s how!

serviceContainer.master Master Page

This is a very simple master page with a single ContentPlaceHolder that the content page overrides with the service web part.


Service pages are stored in a folder or library off the root of the web allowing a relative URL reference to serviceContainer.master. Each page uses SPUserCodeWebPart to load the a sandboxed Web Part that contains the actual implementation of the service.

A welcome byproduct of using a page in a library is the ability to use the standard SharePoint security model to control access to the page (and hence the service). This is a clear advantage for this technique compared to both the Client Object Model and SPServices.


The service Web part reads any required arguments from the query string and/or post parameters from the Page.Request and does its work returning the format of your choice. In the screen shot below it is returning a JSON string that contains the current Web’s Title. Because the part is stateless all of the work is done in Render().

The result looks like this:

It could just as easily be XML or any other type of content. The master page and the page that contain the Web Part are empty. What comes out of the Render method is the body of the response.

Note: Unfortunately because the Web Part lacks access to the outer page you can’t set the response content type properly. Having tested with IE, FireFox, Chrome, Safari, and Opera I can confirm that script can still consume the result without issues using jQuery.getJSON(). I have also emitted dynamically JavaScript and used the page as the src attribute in <script> elements. The response is always of type text/html. Be aware that this is a risk and may cause issues with some clients!


Testing shows that this solution provides performance that is comparable (and at times significantly better) than the equivalent Client Object Model code. The tests used a site on SharePoint Online (Office 365).


The service page was invoked via a .NET console application using the HttpWebRequest class. The code below shows the function that makes the call.

Fiddler shows that this call sent 1258 bytes and received 866 bytes in approximately 150 milliseconds.

Client Object Model

Two passes were tested with the JSOM from the same console application. The first initialized the ClientContext each time and the second reused an existing ClientContext. It turns out that instantiating the client context calls the sites.asmx service. Therefore each call resulted in two round trips. The important lesson learned here is that you should hold on to the context and not recreate it for the best performance. The code for the second version of the test method is shown below. To make the comparison fair, the code explicitly asks for only the Title property of the current Web.

Fiddler shows that this call sent 2022 bytes and received 724 bytes in approximately 225 milliseconds.

The results for one set of tests are as follows:

Round 1

10 calls to the sandbox page took 23544921 ticks

10 calls with the client object model took 39882812 ticks

10 calls with the client object model with one context took 22656250 ticks


Round 2

10 calls to the sandbox page took 21337890 ticks

10 calls with the client object model took 40322266 ticks

10 calls with the client object model with one context took 30048828 ticks


Round 3

10 calls to the sandbox page took 17460938 ticks

10 calls with the client object model took 43066406 ticks

10 calls with the client object model with one context took 22109375 ticks


Round 4 (With Fiddler On)

10 calls to the sandbox page took 15449219 ticks

10 calls with the client object model took 38222656 ticks

10 calls with the client object model with one context took 24550782 ticks


Round 5

100 calls to the sandbox page took 171845703 ticks

100 calls with the client object model took 473261719 ticks

100 calls with the client object model with one context took 240097656 ticks


In the first round the JSOM performed slightly better than the service page. In subsequent rounds the service page’s performance was significantly better. I speculate that the differences are due to unknown factors on my test Office 365 tenancy. Therefore, the only conclusion I am willing to stand by is that using this technique is at worst performance neutral but that it might be better.

The tests used a trivial scenario. In spite of the excellent batching capabilities found in the JSOM, real world scenarios often require multiple roundtrips where each operation depends on data from a previous operation that is not on the client. This architecture potentially offers much better performance by combining such operations into a single round trip.

Author: Doug Ware

Screen Scraped Authentication to Office 365 and SharePoint Online

Way back in April, Microsoft published a very good article by Robert Bogue entitled Remote Authentication in SharePoint Online Using Claims-Based Authentication. This article describes one way to get the FedAuth cookie required to talk to SharePoint online via any of its many interfaces. While the article is very informative and the associated code works, I didn’t like the implementation as it involved the web browser control and PInvoke (The former has a lot of overhead, the latter requires full trust).

So, I decided to try to do the same thing using tried and true screen-scraping techniques. I got it working and shortly thereafter I discovered this post and sample by Wictor WilénHow to do active authentication to Office 365 and SharePoint Online. Wiktor’s approach is more robust than my screen-scraping solution and it uses Windows Identity Foundation directly to use the active federation service provided by the STS is located at Wiktor’s approach is also more efficient and requires fewer round-trips to the server to do the job.

He also posted it a few days before I got mine working!

If Wiktor’s solution is so much better than mine, why am I posting this?

The reason is that Wiktor’s solution depends on Windows Identity Foundation and mine will work on any platform capable of dealing with HTTP and cookies – including Windows XP and various non-Windows development stacks. Although the sample is written in C# you should have no trouble converting it to the language of your choice. Hopefully, the sample will be helpful to a few folks. J

You can download the sample from here. It consists of a Visual Studio 2010 C# solution containing a console application and a class library. If you do not have Visual Studio (Express will work) and you are looking at this sample because you are using something other than .NET, the file you want to look at is O365AuthenticationScraper.cs. It is heavily commented and should be easy to follow.


Author: Doug Ware

SharePoint Security and the Object Model – Part 3

Authorization and Access Control – Evaluating Permissions

This is the third part of a three part post on security object model basics in SharePoint. The other posts are: SharePoint Security and the Object Model – Part 1 and SharePoint Security and the Object Model – Part 2. The first post covers users and groups, the second post covers the mechanics of defining and applying permissions. This post is about using and applying user, groups, and permissions to control the user’s access and experience in a site in the UI and in code.

Why is this Necessary?

Before I talk about using permissions to control access (which is very easy), let’s consider why you need to know anything beyond the first two posts in this series. In the first post, you learned how to work with users and groups. That post even covered how to add users to groups based on claims with SPClaim and SPClaimProviderManager. If you know how users and groups work, why not write code that checks permission based on user or group names?

I’d bet that most of you instinctively know why this is a very bad idea – users and groups change. Obviously you wouldn’t want to write your security code based on them. But, what about claims? If you are using claims you will be tempted to write code that checks for a claim via ClaimsIdentity.Claims or a custom claim manager. If you’ve seen my Claims Identity presentation, you may recall that I use an example where the doorman lets Mary enter because she has a claim that says she is over 21. Writing code that checks for the claim directly is just as big a mistake as checking for a specific user or group, but for slightly different reasons.

Checking for a claim directly creates a dependency between your code and Windows Identity Foundation. Depending on how you do it, it might also create a dependency on a specific identity provider. It also makes your code very inflexible and requires development if the rules change.

You should never perform authorization checks based on users, groups, or claims. Instead you should use permission levels and role assignments as discussed in part 2 of this series. To perform authorization checks you should always use the simple techniques discussed below for maximum flexibility. Although doing all the work discussed in parts one and two to enable these techniques may seem like a lot of work, it is the least amount of work that doesn’t create lots of trouble over time.


ISecurableObject defines a method named DoesUserHavePermissions. The abstract SPSecurableObject provides an implementation of this method that is overridden in the various securable object implementations, i.e. SPWeb, SPList, and SPListItem. The base definition of DoeasUserHavePermissions is simple (see part 2 for more on SPBasePermissions):

bool DoesUserHavePermissions(SPBasePermissions permissionMask)


It returns true or false given a permission mask for a given web, list, or item for the current user. For example:

bool hasPermission = SPContext.Current.Item.DoesUserHavePermissions(SPBasePermissions.EditListItems);
writer.Write("User can edit this item = " + hasPermission.ToString());


In older versions of SharePoint this method has the bizarre behavior of throwing UnauthorizedException
instead of returning false.
This alternative code produces the same result:

permissionMask == SPBasePermissions.EmptyMask || (itemToCheck.EffectiveBasePermissions & permissionMask) == permissionMask;

(itemToCheck is a valid instance of an SPSecurableObject)


SPSecurityTrimmedControl is a handy web control that provides a container for other controls and markup. You simply provide a permission mask for the PermissionString property and SharePoint takes care of the rest! You can use it on pages and in user controls and web parts. Unfortunately, this control is located in the Microsoft.SharePoint.WebControls namespace and is therefore unavailable to sandbox code. Code is the operative word – you can still use it on pages as long as it is in the page markup and not in a control. You can specify multiple permissions by separating the values with commas.

The documentation for this control is good and there are plenty of posts out there the describe its full functionality. The SDK article is here.

CustomAction Rights

The CustomAction feature element allows you to add items to most menus and also to the ribbon. It exposes a Rights attribute that specifies a set of rights that the user must have for the link to be visible, for example, "ViewListItems,ManageAlerts". Simillar functionality is exposed by the SPUserCustomAction class. It has a Rights property that accepts a permissions mask.

Author: Doug Ware