Category Archives: Sandbox/Office 365

Sandbox Solutions – Not Completely Dead

Recently Richard diZerega (whose blog you should be reading) wrote a post called SharePoint Sandbox isn’t Dead…UserCode is wherein he explained that the use of sandbox solutions for declarative provisioning via CAML with features is not deprecated but that the use of procedural code within the sandbox user code host is deprecated. This got some folks talking because none of the official Microsoft docs make this clear.

Today I was part of a large email thread where Iouri Simernitski from the Visual Studio Tools for Office team said the same thing. When asked if that was the official word he said that it was and that this fact was not secret or subject to any NDA’s – we should feel free to share this information publicly.

So, if you have a sandbox solution and you want to maintain it going forward in a supported way, keeping your modules, fields, content types, list instances, etc. in CAML is an option. You can migrate the procedural code to CSOM, JSOM, or REST as needed. It won’t be compatible with the Office 365 marketplace but it will be supported.

Author: Doug Ware

Using the People Picker Control in Sandbox Solutions / Office 365

The SharePoint sandbox does not allow you to use the Microsoft.SharePoint.WebControls namespace in your code. This means you can’t instantiate objects from classes in that namespace or automate existing object instances of classes in that namespace. Because of this restriction, the MSDN documentation for PeopleEditor you will see the following:

Sandbox is a Server Thing

As I wrote above, this restriction is enforced by the sandbox. Available in Sandboxed Solutions: No means you can’t use this class in your code. However, you are free to use any of the Microsoft.SharePoint.WebControls in your page markup. For example, if you create a custom master page it will almost certainly contain controls like CssRegistration and CssLink; both are members of Microsoft.SharePoint.WebControls. You can’t write code that uses these classes. Therefore you can’t call CssRegistration.Register() programmatically. You can use the control in markup assuming you can get the desired server side functionality declaratively via the control’s property attributes. Markup as follows is permitted in pages deployed to the sandbox.

<SharePoint:CssRegistration name="<% $SPUrl:~sitecollection/Style Library/~language/Themable/Core Styles/controls.css %>"

Pop Quiz!

If you want to use the PeopleEditor control in your sandbox solution can you use it in: a) an aspx page, b) a Web Part, or c) both?

The answer is a) an aspx page. However, to do anything with the selected value you will need a custom Web Part. The Web Part can’t use the PeopleFinder class, but it can read all of the page’s form fields. As it turns out, this is good enough with a little JavaScript glue to hold everything together on the client side.

Adding People Finder to a Page

First you must register the control namespace by including the following register directive (but it’s probably on your page already).

<%@ register tagprefix="SharePoint"
assembly="Microsoft.SharePoint, Version=, Culture=neutral, PublicKeyToken=71e9bce111e9429c" %>


Next include the control with the desired properties. In the example below the control allows the selection of a single user or SharePoint group. See the MSDN documentation for PeopleEditor for complete documentation.


On my custom task form the result looks like this:

Validating the Entry

Since automation of this control via the PeopleEditor class is forbidden, the page requires some JavaScript code to ensure that the entry is valid. It also must resolve the entry to make it easy for the page’s Web Part to get the value from a form field. The picker’s rendered state includes JavaScript that validates the user and the custom JavaScript takes advantage of the existing script. The only real trick is that the validation involves an asynchronous call to the server. The solution is to use a timer and observe the state of the picker’s markup. As usual I use jQuery to observe and manipulate the markup.

There are four states the markup manifests:

  1. An entry that is confirmed as invalid

  2. An entry that is confirmed as valid

  3. No entry

  4. An entry that is not confirmed as either valid or invalid

To detect the first state the script looks for descendants of the control that contain the isresolved attribute set to false. Conversely, for the second state the script looks for descendants of the control that contain the isresolved attribute set to true.

Assuming the control is inside a content placeholder named PlaceHolderMain as is usual for SharePoint content pages, the HTML ID attribute of the control with a server ID of assignedTo is ctl00_PlaceHolderMain_assignedTo. The check for the first two states is as follows:

if ($('#ctl00_PlaceHolderMain_assignedTo').find('*[isresolved="False"]').length > 0) {
return false; //Confirmed invalid
if ($('#ctl00_PlaceHolderMain_assignedTo').find('*[isresolved="True"]').length > 0) {
return true; //Confirmed valid


The rendered control contains a div element that is empty if the user hasn’t entered anything. To detect the third state, the script checks this div for an absence of children,

var entry = $(‘#ctl00_PlaceHolderMain_assignedTo_upLevelDiv’).html();
if (entry === "" || entry === "&nbsp;") {
return true; //Valid – no entry

The final state is the one that requires the picker to validate itself before form submission. This process starts by calling the WebForm_DoCallback function. While it runs the script uses a timer to check for the first or second state. The completed validation functions are shown below.

var checkingName = false;

function checkName() {
if (!checkingName) {
checkingName = true;
var arg = getUplevel('ctl00_PlaceHolderMain_assignedTo');
var ctx = 'ctl00_PlaceHolderMain_assignedTo';
WebForm_DoCallback('ctl00$PlaceHolderMain$assignedTo', arg,
EntityEditorHandleCheckNameResult, ctx, EntityEditorHandleCheckNameError, true);


function nameIsValid() {
if ($('#ctl00_PlaceHolderMain_assignedTo').find('*[isresolved="False"]').length > 0) {
checkingName = false;
return false; //Confirmed invalid
if ($('#ctl00_PlaceHolderMain_assignedTo').find('*[isresolved="True"]').length > 0) {
checkingName = false;
return true; //Confirmed valid
var entry = $('#ctl00_PlaceHolderMain_assignedTo_upLevelDiv').html();
if (entry === "" || entry === "&nbsp;") {
return true; //Valid - no entry
//Not confirmed and not blank, ask the server
//Wait half a second and check the markup to determine validity
setTimeout(ValidateAndSave, 500);
return false;


Getting the Result

If you submit the form after executing the above code, you can pick apart the form fields to read the entries. However, I prefer to use a bit more script to pull the valid name from the picker markup and stuff it into another field. This is cleaner and it decouples the Web Part code completely from the PeopleFinder. The following function returns the value from the picker in a valid, not empty state.

function getName() {
var data = $('#ctl00_PlaceHolderMain_assignedTo_upLevelDiv').html();
if (data !== "" && data !== "&nbsp;") {
var ret = $('#ctl00_PlaceHolderMain_assignedTo').find('div#divEntityData').attr('key');
if (ret !== undefined) {
return ret;
else {
return "";
return "";

In the Web Part I use the value directly without additional parsing.

SPUser user = web.EnsureUser(task.assignedTo);
SPFieldUserValue at = new
SPFieldUserValue(web, user.ID, user.LoginName);


Happy coding!

Author: Doug Ware

jQuery Plugin to Display SharePoint 2010 List Views

I’m working on a new line of products for SharePoint Online / Office 365. A few of the pages need to display list views whose locations are unknown at design time – the lists are created on the fly and can have any title and a single page displays a view based on what the user chooses. This means that I can’t use a data view Web Part because it is tied to a specific list and view.


My first attempt involved the use of the RenderAsHtml method of SPList and SPView. The code for this is super simple:

When I tested the Web Part, I couldn’t believe my eyes. It worked on the very first try!

Unfortunately, this was just my old friend SharePoint playing a fun joke on me. As soon as I clicked an item I saw the following error message:

Yes – This item is no longer available. It may have been deleted by another user.

The reason this error displays is that the HTML emitted by RenderAsHtml is for the V3 UI. It doesn’t match what the SharePoint 2010 JavaScript expects and the result is this error. I started digging into how I might make this sucker work. Along the way I learned a lot about the client side behavior. In fact, I learned so much that I decided a general solution based on RenderAsHtml was not going to fly.

Near as I can tell, the RenderAsHtml method in SharePoint 2010 is Microsoft’s little joke on us – it’s there to make you think you can get it to work if you just try hard enough.

jQuery Pluggin

My next step was to cry out in frustration and rage redouble my efforts and get creative. It occurred to me that a SharePoint site is just full of properly functioning list views. I thought ‘hmmm, jQuery Get()’. In jQuery the Get() method issues an asynchronous HTTP get for a page and gives you a string that contains the returned HTML. The final solution is a jQuery plugin extracts the view HTML and script from a list’s view page, e.g. AllItems.aspx, and inserts it into the current page.

Of course it’s not quite that simple – I’ll explain how it all works in the next post. In the meantime, you can get the plugin from here:

To use the plugin you will need to first include jQuery – I built this with jQuery 1.6.2 and I have not tested it with earlier versions!

Second: you will need to modify the site’s default master page to include a DIV that marks the end of PlaceHolderMain:

<div id="MSO_ContentDivEnd" style="display:none"></div> makes it easy for the pluggin to know where the view ends. You can probably modify the plugin to avoid this requirement, but I have a custom master page already and this was the easiest solution for me!

Third: Include the listViewPlugin.js. (very important!)

Fourth: Add an anchor element to the page where you want the view as follows:

<a href=[/listurl/viewPage.aspx] class="listView"></a>


The href is the URL to the view and class is either listView or libraryView depending on if the view is a list or a library, respectively.

Finally: add the script to load the view on document ready.

<script type="text/JavaScript">

    jQuery(document).ready(function($) {





The Result

Below you see the result in a Content Editor Web Part. Note that the menu works!

Filtering, sorting, and paging all work too!

Let’s filter by document type.

I’ve tested this on IE, FireFox, Chrome, and Safari. It works with basic list and library types. It does not work with Calendar or GANTT views – however, in the case of Calendar views I think the missing step is to link to the appropriate style sheet.


Author: Doug Ware

Office 365/SharePoint Sandbox Service Pages in Action – Autocomplete Text Boxes

A couple weeks ago I wrote about an Unorthodox Architecture for Services in SharePoint Sandbox Solutions and Office 365. This post is about an application of this architecture to create an autocomplete text box using jQuery UI. An autocomplete field looks like this:

It is a great alternative to the stock lookup field UI when the source list of the lookup field has a large amount of data. With hundreds of items the standard dropdown list just isn’t a good experience. The jQuery Autocomplete widget makes it easy to build a field that displays a list as the user types. In this example the JavaScript is simple and the underlying field is "single line of text". If you need to ensure the entry matches a value from the lookup list you’ll need to do some extra work. That said, here is the JavaScript:

source: function(request, response) {
url = L_Menu_BaseUrl + "/ServicePages/AutoComplete.aspx?l=Clients&f=Title&v=" + request.term;
$.getJSON(url, function(data) {
minLength: 1


The url in the above script is to a service page that contains a Web Part whose code is shown at the bottom of this post. The variable L_Menu_BaseUrl is declared and populated by SharePoint’s own JavaScript libraries and it contains the Web’s server relative URL. The Web Part reads the query string and runs a CAML query against the specified list to search for values that begin with the specified term. The result is written to the page as JSON by using the JavaScriptSerializer to serialize a List<string> of the specified field for each item in the result set.


//Inputs are
public class AutoComplete : WebPart
protected override void Render(HtmlTextWriter writer)
var items = GetItems();
if (items.Count > 0)
JavaScriptSerializer js = new JavaScriptSerializer();

private List<string> GetItems()
var items = new List<string>();
string listName = Page.Request.Params["l"];
string field = Page.Request.Params["f"];
string val = Page.Request.Params["v"];
uint max = 10;

//Parse the query string parameters and return an empty list if there is a problem
if (Page.Request.Params["max"] != null)
try { max = uint.Parse(Page.Request.Params["max"]); }
catch { } //ignore the error and return 10 items

if (string.IsNullOrEmpty(listName) || string.IsNullOrEmpty(field) || string.IsNullOrEmpty(val))
return items;

//Build and execute the query
string queryXML = string.Format(@
"<OrderBy><FieldRef Name='{0}' /></OrderBy><Where><BeginsWith><FieldRef Name='{0}' />
<Value Type='Text'>{1}</Value></BeginsWith></Where>", field, val);

string viewFields = string.Format(@"<FieldRef Name='{0}'/>", field);

SPList list = SPContext.Current.Web.Lists.TryGetList(listName);

if (list == null) return items;

SPQuery query = new SPQuery();
query.Query = queryXML;
query.ViewFields = viewFields;
query.RowLimit = max;
var listItems = list.GetItems(query);

//Put the returned items into the list
foreach (SPListItem listItem in listItems)
return items;

Author: Doug Ware

Unorthodox Architecture for Services in SharePoint Sandbox Solutions and Office 365

These days we are spending a lot of time on a series of sandbox applications that will run on Office 365. If you are a follower of this blog you know I was more than a bit disappointed in the restrictions the SharePoint 2010 sandbox implementation imposes. However, if you want to play in the sandbox, you can either just say no to requirements that aren’t directly supported, or you can earn your money and get creative!

I’ve been getting creative. These applications are going to be great – that said, some elements of the architecture are unorthodox and even a little strange. This post details one of the elements.


Expose services to external clients that:

  1. Hide their underlying technology from the caller and hide the application’s underlying implementation from prying eyes
  2. Support all current web development stacks
  3. Perform well
  4. Are easy to create and consume

Client side technologies like the Client Object Model (JSOM) and SPServices are awesome. However, they fail goal #1. If page’s business logic is implemented in JavaScript it is available to any and all who care to look. Worse, they both encourage tight coupling between the client UI and SharePoint. Both support JavaScript, and the Client Object Model also supports managed (.NET) clients and Silverlight. However, both have minor learning curves for developers who mainly write C# and use the server object model.

What we really want is for the client to consume services via an abstraction that hides the details and helps minimize dependencies. Unfortunately, you can’t deploy traditional asmx or WCF services to the SharePoint Sandbox.

Before we get into the solution, please be aware that I am not saying that this architecture is better than JSOM, SPServices, or the REST API – I’m just saying it is a better fit for our particular needs for certain specific applications. It also illustrates an interesting technique that you can add to your own arsenal.


While it is not possible to deploy traditional Web services, we can come close. A Web service is just an http endpoint to which we can pass arguments and from which we can read a response. Conceptually, the only differences between an aspx page and a Web service are the calling convention and (usually) the content type (e.g. ‘text/html’ versus ‘text/xml’ or ‘application/json’). You can create an aspx page that works just like an asmx or WCF service if you want – you just have to handle the plumbing yourself.

Our applications have lots of JavaScript and JSON is the easiest format to consume. It is easy to create a page in the sandbox that returns JSON based on query string or form POST parameters. Here’s how!

serviceContainer.master Master Page

This is a very simple master page with a single ContentPlaceHolder that the content page overrides with the service web part.


Service pages are stored in a folder or library off the root of the web allowing a relative URL reference to serviceContainer.master. Each page uses SPUserCodeWebPart to load the a sandboxed Web Part that contains the actual implementation of the service.

A welcome byproduct of using a page in a library is the ability to use the standard SharePoint security model to control access to the page (and hence the service). This is a clear advantage for this technique compared to both the Client Object Model and SPServices.


The service Web part reads any required arguments from the query string and/or post parameters from the Page.Request and does its work returning the format of your choice. In the screen shot below it is returning a JSON string that contains the current Web’s Title. Because the part is stateless all of the work is done in Render().

The result looks like this:

It could just as easily be XML or any other type of content. The master page and the page that contain the Web Part are empty. What comes out of the Render method is the body of the response.

Note: Unfortunately because the Web Part lacks access to the outer page you can’t set the response content type properly. Having tested with IE, FireFox, Chrome, Safari, and Opera I can confirm that script can still consume the result without issues using jQuery.getJSON(). I have also emitted dynamically JavaScript and used the page as the src attribute in <script> elements. The response is always of type text/html. Be aware that this is a risk and may cause issues with some clients!


Testing shows that this solution provides performance that is comparable (and at times significantly better) than the equivalent Client Object Model code. The tests used a site on SharePoint Online (Office 365).


The service page was invoked via a .NET console application using the HttpWebRequest class. The code below shows the function that makes the call.

Fiddler shows that this call sent 1258 bytes and received 866 bytes in approximately 150 milliseconds.

Client Object Model

Two passes were tested with the JSOM from the same console application. The first initialized the ClientContext each time and the second reused an existing ClientContext. It turns out that instantiating the client context calls the sites.asmx service. Therefore each call resulted in two round trips. The important lesson learned here is that you should hold on to the context and not recreate it for the best performance. The code for the second version of the test method is shown below. To make the comparison fair, the code explicitly asks for only the Title property of the current Web.

Fiddler shows that this call sent 2022 bytes and received 724 bytes in approximately 225 milliseconds.

The results for one set of tests are as follows:

Round 1

10 calls to the sandbox page took 23544921 ticks

10 calls with the client object model took 39882812 ticks

10 calls with the client object model with one context took 22656250 ticks


Round 2

10 calls to the sandbox page took 21337890 ticks

10 calls with the client object model took 40322266 ticks

10 calls with the client object model with one context took 30048828 ticks


Round 3

10 calls to the sandbox page took 17460938 ticks

10 calls with the client object model took 43066406 ticks

10 calls with the client object model with one context took 22109375 ticks


Round 4 (With Fiddler On)

10 calls to the sandbox page took 15449219 ticks

10 calls with the client object model took 38222656 ticks

10 calls with the client object model with one context took 24550782 ticks


Round 5

100 calls to the sandbox page took 171845703 ticks

100 calls with the client object model took 473261719 ticks

100 calls with the client object model with one context took 240097656 ticks


In the first round the JSOM performed slightly better than the service page. In subsequent rounds the service page’s performance was significantly better. I speculate that the differences are due to unknown factors on my test Office 365 tenancy. Therefore, the only conclusion I am willing to stand by is that using this technique is at worst performance neutral but that it might be better.

The tests used a trivial scenario. In spite of the excellent batching capabilities found in the JSOM, real world scenarios often require multiple roundtrips where each operation depends on data from a previous operation that is not on the client. This architecture potentially offers much better performance by combining such operations into a single round trip.

Author: Doug Ware

Screen Scraped Authentication to Office 365 and SharePoint Online

Way back in April, Microsoft published a very good article by Robert Bogue entitled Remote Authentication in SharePoint Online Using Claims-Based Authentication. This article describes one way to get the FedAuth cookie required to talk to SharePoint online via any of its many interfaces. While the article is very informative and the associated code works, I didn’t like the implementation as it involved the web browser control and PInvoke (The former has a lot of overhead, the latter requires full trust).

So, I decided to try to do the same thing using tried and true screen-scraping techniques. I got it working and shortly thereafter I discovered this post and sample by Wictor WilénHow to do active authentication to Office 365 and SharePoint Online. Wiktor’s approach is more robust than my screen-scraping solution and it uses Windows Identity Foundation directly to use the active federation service provided by the STS is located at Wiktor’s approach is also more efficient and requires fewer round-trips to the server to do the job.

He also posted it a few days before I got mine working!

If Wiktor’s solution is so much better than mine, why am I posting this?

The reason is that Wiktor’s solution depends on Windows Identity Foundation and mine will work on any platform capable of dealing with HTTP and cookies – including Windows XP and various non-Windows development stacks. Although the sample is written in C# you should have no trouble converting it to the language of your choice. Hopefully, the sample will be helpful to a few folks. J

You can download the sample from here. It consists of a Visual Studio 2010 C# solution containing a console application and a class library. If you do not have Visual Studio (Express will work) and you are looking at this sample because you are using something other than .NET, the file you want to look at is O365AuthenticationScraper.cs. It is heavily commented and should be easy to follow.


Author: Doug Ware

Publishing in Sandbox Solutions – Create a Page

I recently built a publishing solution for a client that is deployed as a sandbox solution on Office 365. It was a fairly typical publishing solution containing master pages, page layouts, web parts, and other assets. Content production (putting words and media on pages) took place after deployment of this solution, but we had an initial site map. Therefore the solution built the site with a feature event receiver that created subsites, added blank publishing pages with custom page layouts, and configured navigation.

In a farm based deployment the creation of a publishing page is easy thanks to members of the Microsoft.SharePoint.Publishing assembly like AddPublishingPage method of the PublishingWeb class. Unfortunately the Microsoft.SharePoint.Publishing assembly is not available in the sandbox. Fortunately, the contents of the assembly are not obfuscated and I was able to use ILSpy to understand and recreate AddPublishingPage in my own assembly.

First I need a string that represents the new page’s markup.

internal static class CompiledTemplateStrings


    internal static string templateRedirectionPageMarkup = 

@"<%@ Page Inherits='Microsoft.SharePoint.Publishing.TemplateRedirectionPage,
Microsoft.SharePoint.Publishing,Version=,Culture=neutral,PublicKeyToken=71e9bce111e9429c' %>

<%@ Reference VirtualPath='~TemplatePageUrl' %> <%@ Reference VirtualPath='~masterurl/custom.master' %>";



Then I need a class that creates a new SPFile from an array of bytes that represent the page content. Once created, the code sets the appropriate properties to make the new page an Article Page content type and set the PublishingPageLayout and Title. Finally, the file is checked in and approved.

internal class PublishingPageCreator


    private static byte[] pageAsBytes;

    private static readonly UTF8Encoding PageEncoder = new UTF8Encoding();

    internal static byte[] GeneratePageStreamAsBytes()


        if (pageAsBytes == null)


            byte[] bytes = PageEncoder.GetBytes(CompiledTemplateStrings.templateRedirectionPageMarkup);

            pageAsBytes = new byte[bytes.Length + 3];

            pageAsBytes[0] = 0xef;

            pageAsBytes[1] = 0xbb;

            pageAsBytes[2] = 0xbf;

            bytes.CopyTo(pageAsBytes, 3);


        return pageAsBytes;



    internal static SPListItem AddPublishingPage(string name, string title, string layout, SPFolder folder)


        SPFileCollection files = folder.Files;

        byte[] file = GeneratePageStreamAsBytes();

        SPFile newFile = files.Add(name, file);

        SPListItem item = newFile.Item;

        item["ContentType"] = "Article Page";

        item["ContentTypeId"] = "0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF3900242457EFB8B24247815D688C526CD44D";

        item["PublishingPageLayout"] = layout;

        item["Title"] = title; 


        newFile.CheckIn(string.Empty, SPCheckinType.MajorCheckIn);

        item.ModerationInformation.Status = SPModerationStatusType.Approved;


        return item;




Invoke AddPublishingPage as follows:

PublishingPageCreator.AddPublishingPage(pageName, "Page Title", rootWeb.Url + @"/_catalogs/masterpage/MyPageLayout.aspx, My Page Layout", pages.RootFolder);


Use your target site’s Url as rootWeb.Url. pages is an SPList instance of the Pages library.

You can read about an alternative approach using a module element in this excellent post:

Author: Doug Ware

Participating in the Page Rendering Pipeline in SharePoint 2010 Sandbox Solutions – SPUserCodeWebPart

If you have been reading this blog of late, you probably picked up on the fact that I am a bit disappointed by the limitations of sandboxed solutions. I recently updated my original post on the subject to include the following:

<This is mostly wrong>

Issue #4 – Almost No Server Side Rendering – Forget About Using ASP.NET

I thought I would try to work around my ribbon issue with a simple custom WebControl. Turns out that, since the sandbox assembly is installed in ProgramDataMicrosoftSharePointUCCache, the regular worker process (w3wp.exe) that renders pages can’t load the custom UI assembly because it has no knowledge of the UCCache location. The same applies to code behind assemblies for ASPX.

You can create WebParts (sorta/kinda) but they are also extremely crippled. See

</This is mostly wrong >

Well, I got to thinking… ‘I know that you can create sandbox solutions that include Web Parts, and SharePoint is loading them somehow. Maybe there is a way to get my controls onto the page!’

Turns out there is!!! I give you SPUserCodeWebPart. This little guy can load your assembly and its controls from the user code store.

Description="My WebPart" Title="Howdy y'all"
AssemblyFullName="MySandbox, Version=, Culture=neutral, PublicKeyToken=9120be3acbd2f74f"


Bear in mind that Web Parts are just user controls and you can use them even when the page is not a Web Part page. That means you can use them on regular ASPX pages and on master pages. This means that you can access controls and handle events on the page that contains this hook – as long as the controls are also contained in your Web Part. It seems that the way this works is that SharePoint generates a page within a page using the SPUserCodePage and SPUserCodeForm classes. The pattern I will be trying will be an ASPX page with a single SPUserCodeWebPart that wraps the control that implements the page logic. If I need to get at the outer shell I’ll emit JavaScript to do so.


Author: Doug Ware