Category Archives: SharePoint 2013 and Office 365 Apps

SharePoint 2013 App Web versus Host Web Redux

One thing I do not like about apps as they exist today is the reduced functionality of app webs compared to a normal web, e.g. a team site. To review, an app web is created as part of a SharePoint-hosted app and as part of hybrid provider-hosted app. I wrote about my complaints in more depth here: Building Traditional SharePoint Collaboration Solutions with the App Model. In that post I advocate the use of the host web as a deployment target to create a full-featured app without the restrictions found in the ghetto-web where everything is treated as a second-class citizen.

Not long after I wrote that post fellow MVP Chris O’Brien wrote this post: SharePoint apps: working with the app web, and why you should. In it he briefly discusses reasons why you might not want to deploy to the host web.

Enough time has passed and I’ve learned enough that I feel like the question is worth revisiting.

Why App Webs Exist

The app web concept really exists for one reason only – to protect the host environment from malicious JavaScript while integrating with a site collection in the host. An app web gets its security users and groups plus things like available content types and fields from the host web. If you use a tool like SharePoint Manager you can see that an app web is contained by the host site collection. The key thing is that it is on a different domain.

If the host web’s url is something like https://hostsite.sharepoint.com/ then the app web url will be something like https://hostsite-83fb00ac6b07a2.sharepoint.com/myapp.

This accomplishes two very important things:

  1. The cookies from the host web’s domain are not visible to the app web domain. This is very important because among the cookies for the host web’s domain is the user’s authentication cookie. Scripts can’t create a client context or issue REST messages against the host domain.
  2. The JavaScript code running in the pages from the app web are prohibited by default from issuing POST requests to the host domain.

Manage Web and Full Control

In my original post and in my offline conversations with Chris I pointed out that it was possible to provision to the host web as long as the app had Manage Web permissions to the host. This was true in the beta, but when the RTM bits became available, this was changed to require Full Control for any type of file that can contain JavaScript. So, you can add word documents and text files with modest app permissions, but a Web Part page or master page requires full control.

Consider that an app with manage web permissions can read every bit of content on a site, manage users and permissions, and even delete content without having full control. So, what good is full control!? What does the requirement to have full control achieve?

It protects the tenant from the following scenarios…

An Easy Way to Hack a Farm

If you have the ability to add JavaScript to a SharePoint site, you can hack your environment with a little social engineering and help from an admin. This applies to any site that allows users to add JavaScript and is the scenario that we fear most when discussing cross-site scripting exploits.

  1. Add some JavaScript that tries to do something only an admin can do, e.g. make a user a site collection or even a farm admin.
  2. Call an admin and complain about the site not working and ask them to check it for you.
  3. Use your newly granted permissions to perform evil deeds.

You should understand that this attack doesn’t require anything more than the ability to put script into a content editor web part or to edit pages in SharePoint Designer.

Another Easy Way to Hack a Farm

Install or get an admin to install a farm solution that uses RunWithElevatedPrivileges that uses the application pool identity to perform evil deeds. In many poorly configured environments this will be a domain admin.

This might sound far-fetched, but how hard would it be to put a couple cool free tools on a site like CodePlex just to phish for farms?

How the App Web Protects You

The protection and benefit of the App Web concept should now be clear to you – prevention of cross-site scripting exploits.

How you can Protect Yourself

Use separate accounts to administer your farm. Do not browse your sites using a farm or tenant level admin account.

Should I Deploy Solutions to the Host Web?

Clearly you are taking risks if you install an app that requires Full Trust and makes significant changes to the host web. Because of this, Microsoft does not allow apps that require this permission into the marketplace. If you are a vendor and you require this permission you will have to sell directly to your clients and use a side-install approach. My plan is to offer a marketplace compatible app which deploys to the app web and offer a more advanced version that is sold directly.

However, the risk you take with a Full Trust app is the same risk that you accept by using SharePoint Designer or Content Editor Web Parts, and it is less than the risk you assume by installing a farm solution. The key decision from a security perspective comes down to a simple question of trust. Do you trust the source of the app?

If the app is for internal use and you built it in-house, I see no reason to accept the fact that the app web strips so much functionality found in a host web. I feel the same way about trusted, reputable vendors.

P.S. A Note on Full Trust

You should be aware that an app’s permissions within a given context is constrained by the permissions of the user under normal circumstances. In order to escalate to a higher level of permission the app must either impersonate a known user or create the client context using an app-only policy. Full trust does not equate to constant god-mode processing.

Furthermore, an app principal always has full control over its app web regardless of its permissions on the host web. However, it is always the case that a user’s permissions constrain the app’s permissions when it is acting on behalf of a user. Even in an app web, normal code execution is subject to the user’s rights.

P.P.S. Bugs in RTM

The permissions required to perform many kinds of actions in the on-prem version are defective as of this writing. Many operations that should not require full control require full control. Office 365 works as documented. Hopefully the issues will be resolved in a hot-fix soon.

–Doug

Author: Doug Ware

An Architecture for Provider SharePoint 2013 Hosted Apps on Azure

For the past several months I’ve been working on a complex app and last month I wrote a post called The SharePoint 2013 App Model is better than Farm Solutions based on my experiences. This post led to much discussion online, in person, and via email. These conversations have made it clear to me that if I want to continue blogging about app architecture I need to provide more detail about my specific architecture.

There is a great deal of confusion and misinformation out there, and I don’t want to get into point-by-point debates based on generalities and assertions. This is especially easy to do when discussing apps because ‘app’ is a meaningless generic term. The architecture discussed here is by no means the only way to build an app. In fact, it has very little in common with a SharePoint hosted app or a provider-hosted app delivered via an app part.

Before I go any further I should remind you that traditional solutions are easier to build and that apps have a learning curve that can’t be discounted. At no time should you take my statement that apps are better as meaning that they are easier to write. Most of the pieces of the architecture I am about to describe are things I won’t have to write again for subsequent apps, however they did take many months of work to design, refine, and build.

Operational Goals

I believe that enterprise architectures should have clear operational goals that the system meets that are independent of user facing functionality. I’ve seen first-hand how a good architecture can yield serious competitive advantages. When I first began thinking about sandbox solutions that could be sold to Office 365 users I had a few questions that I wanted the architecture to answer.

  1. Entitlement – how do I deal with a customer who doesn’t pay but has the solution installed?
  2. Updateability – how do I keep (hopefully) thousands of instances of the app consistent with each other and how do I push new functionality?
  3. Protection of IP – if I give out a solution package or if I rely on JavaScript, how do I prevent trivially easy reverse engineering?
  4. Other devices – How can I design my code base to allow clients other than SharePoint? Is SharePoint a good platform for application services to a heterogeneous set of client technologies?

These are all concerns of a SaaS vendor more so than concerns applicable to in-house solutions although #2 and #4 could be critical to an in-house app. Either way, if you add these requirements into the mix, all of a sudden a farm or sandbox solution becomes potentially much harder to build and in the case of sandbox, you are dependent on a deprecated technology.

Strategies

To meet these goals I settled on certain strategies and guidelines based on my previous experience with large distributed systems and with SaaS; yes, I had a life before SharePoint!

Embrace web technologies

This one means that I want to align to the current state of the art for web based systems. I want to use the most common libraries like jQuery and I want to exchange data using JSON. I want to avoid putting UI code into any of my services and keep the UI in the browser.

Minimize technology stack dependencies between subsystems

Pieces of the system should be as atomic as possible and they should have absolutely minimal dependencies on the technology used to implement other subsystems. I should be able to move them around and re-platform them if necessary. Most importantly, the architecture should allow major changes in one subsystem (such as a new version of SharePoint) with minimal impact on the other subsystems.

Centralize

I dream of success and thousands of clients! I want as much centralized control as possible. Therefore my application services should centralize:

  • Entitlement
  • Versioning
  • Business logic
  • Data services
  • Instrumentation

This allows me to know when a client is having problems, to push updates as needed, and to handle many different scenarios like trial accounts, notifications specific to a client, and subscription based services. Furthermore, it protects my IP and makes piracy a non-issue.

Distribute

Ultimately I want to support a variety of client devices and I also want my clients to keep their data so that their information stays secure and private. Therefore the static user interface components, i.e. web pages, are distributed and the data stays in the client’s environment, i.e. in SharePoint lists and libraries.

High-Level Architecture

The following diagram shows the high-level architecture and data flow. Notice that my pages do not talk directly to SharePoint with the exception of list views and the other functionality gained simply by being a page in a SharePoint site. The data layer and application services are in a web role and a worker role in Windows Azure.

The pages depend on the services for processing and they do so by exchanging JSON via HTTP GETs and POSTs. There is no JavaScript CSOM or REST in the browser which protects my IP. It also means I can reuse my HTML and UI JavaScript with other types of client technologies. The list views are an exception. I am not using any listview web parts. Instead I am using my list view plugin. However, this is a dependency I am willing to accept because I want the app to look and feel exactly as it would if I wrote it as a farm solution.

Notice also that the JavaScript and the CSS are not stored in SharePoint, but are served by the application server. They are centralized. I can fix a bug in one spot and it immediately becomes available to every customer even if there are millions of installations. This also allows the system to serve page resources dynamically and enables such things as special prompts for a user whose subscription is about to expire.

It also means I can support any client technology that can consume HTML and JavaScript such as Windows 8 and IOS.

You might worry that such a system would incur a big performance penalty. In fact the communication overhead is not significant on Azure. There is very little latency between my Azure services and Office 365. I assume they are on the same network.

App Launch Flow

I wrote earlier about Managing Identity and Context in Low-Trust Hybrid SharePoint 2013 Apps. The mechanism I described requires a certain flow when the user launches an app. This flow is also the key to updating the pieces which are distributed to the client’s site. A conceptual version of this flow is shown below.

Whenever the user launches the app the centralized system checks the version of the user’s installation. If necessary, the app server can make changes or additions to the deployment and add new lists, libraries, or pages. Again, this works with one client or with millions of clients.

Conclusion

Hopefully this post goes a long way to frame the discussion about the pros and cons of apps compared to a farm solution. As I get closer to release of the monster I’ll be showing more concrete examples, but until then, if you need help or training with an app I’d love to hear from you!

–Doug

Author: Doug Ware

Adding Custom Actions with JavaScript to the Host Web from an App

You can add custom actions to the host web using CAML in an App package, but there is a serious restriction: you can’t invoke any JavaScript. Liam Cleary noticed this limitation on twitter not long ago.

I soon replied…

I had the code lying around and so I made a demo for him using Napa. One of Napa’s cooler features is the ability to publicly share app code. The code also illustrates the use of the cross domain library to access the host web from the app web.

You can view the full sample here: aka.ms/vtbx5z .

Happy coding!
–Doug

Author: Doug Ware

Managing Identity and Context in Low-Trust Hybrid SharePoint 2013 Apps

SharePoint 2013 Apps written for Office 365 and on-premises installations using Windows Azure Active Directory rely on OAuth for authentication of the app and users of the app against SharePoint. Conceptually, the way this works is fairly simple, but in practice it can be complex when you are starting from scratch because there is some infrastructure and related code that you will almost certainly need to build to manage the various pieces of the puzzle. The SharePoint App project templates in Visual Studio include a file named TokenHelper.cs which contains essential functionality for processing tokens and creating authenticated connects to SharePoint, but there are no out-of-box components for storing and managing the tokens with which TokenHelper helps.

This is a very high-level post that you can use as a basis for your solution to this problem. I included links to Microsoft’s documentation if you need more detailed information.

Initial Authentication Flow

Step 1 – App Redirect

When a user launches an app in SharePoint, the first step is an HTTP GET of SharePoint page named appredirect.aspx, _layouts/15/appredirect.aspx. The GET includes a single parameter, instanceId. AppRedirect uses the instanceId to get information from the app’s manifest that the page uses to build the URL and form POST for the next step in the process. This information includes the app’s ClientId, the app’s start page URL, and the query string parameters that should be passed to the start page.

ClientId is a GUID that represents the app’s identity. It is used by tokenhelper.cs along with the ClientSecret to generate the tokens the app will require to talk to SharePoint. These values are created when the app is registered with SharePoint.

Step 2 – App Start Page

The app redirect page issues a POST to your app’s start page. In a hybrid app, this page is on your application server and it redirects to the App Web once it is finished processing and storing the authentication data. The POST contains quite a bit of important information and (for the most part) this is your only chance to grab it!

If your manifest specifies the Visual Studio default tokens, {StandardTokens}, the query string will contain two critical pieces of information: SPHostUrl and SPAppWebUrl.

  • SPHostUrl is the url of the web where the app is installed.
  • SPAppWebUrl is the url of the app web assuming there is one. If this parameter is not present, the app does not include an app web.

The POST’s form body contains the rest of the information. The most important parameter is SPAppToken. SPAppToken is specific to low-trust configurations. If you are using a high-trust configuration the SPAppToken token is missing. This fact causes a great deal of confusion because all of the original SDK samples assume low-trust and break when they attempt to get this value in a high-trust configuration.

SPAppToken is an encrypted string that contains many more parameters with which we are concerned. TokenHelper.cs uses the Microsoft.Identity classes to decrypt SPAppToken with the ClientSecret. The decrypted and parsed SPAppToken contains several important pieces of information:

  • CacheKey – The cache key is an opaque (non-readable) string that is unique to the combination of user, app, and tenant: CacheKey = UserNameId + "," + UserNameIdIssuer + "," + ApplicationId + "," + Realm
  • AccessToken – The token required to create an authenticated connection to SharePoint, it expires in 12 hours.
  • RefreshToken – The token used to request a new AccessToken or RefreshToken, it expires in 6 months.

If you have these values, you can talk to SharePoint on behalf of the user.

Values you need to Persist and Why

In practice you will always need a mechanism to persist these key values. I use a combination of Windows Azure SQL Database for the tokens, the App Web’s property bag for the host url, and session cookies for my version of the CacheKey. This allows me to accommodate the following scenarios:

  1. Deep links – If a user enters my app via a link instead of from the app’s icon in the host site the traversal of appredirect and the app start page won’t happen. If there is a session cookie for the CacheKey everything is fine because the code can get the tokens from persisted storage which can be used to create a full context. However, if the cookie does not exist the app needs to redirect the user to appredirect to authenticate the app for the user. For the redirect to work the code needs the SPHostUrl and the instance id.
  2. Background processes – I use Azure worker roles for long running processes. To create a CSOM client context, the input to the job must include a valid cache key.
  3. Authentication to the provider hosted web site and services – My apps don’t actually store the cache key from SharePoint. They add additional information to the key and encrypt the result using a private certificate. The creation of this value requires the user to have successfully launched the app from SharePoint and the user can be considered authenticated. By storing this value in the database and in the session key I add a layer of protection that I build on as part of my services infrastructure.
  4. Privacy – My apps never store any information that can be used to identify an individual user without help from the SharePoint site. If someone with malicious intent gets the content of this database it will do them no good without also breaking the encryption, knowing the app identity, client secret, and app web url.

Figuring this all out took me much time and effort. Hopefully this post can spare you from going through that, but if you would like some consulting or training on the subject I would love to hear from you!

Happy coding!
–Doug

Author: Doug Ware

 

The SharePoint 2013 App Model is better than Farm Solutions

Almost seven months ago, after everyone got access to the public beta of SharePoint 2013 I wrote a post titled SharePoint 2013 Preview – Apps or Crapps?. My conclusion at the time was ‘too soon to tell.’ Since then I’ve been re-implementing a complex sandbox solution I originally wrote for the first version of Office 365. I never released the product because I was almost done building it when I found out there was to be a new model in SharePoint 2013. After many months I have a more or less complete architecture and a great deal of code.

Unfortunately, I am not a rich man and I can’t spend every day (or even most days) writing code for myself and I’ve kept busy building farm solutions for other people. The split time has helped me consider which I prefer from an objective standpoint as I’ve worked with the good and the bad of each. I like farm solutions and demand for my services to build them has only improved since I started building them full time in 2007. There is no pressure to switch to the app model in my consulting practice. I have customers that are using it for pilot initiatives, but the farm solution practice is strong enough that it is still not uncommon for us to turn away business because we are too busy.

Farm solutions have been very good to me, but I’ve had enough experience with the app model to now say that, generally speaking, it is objectively better for most scenarios.

‘Better’ is a Dangerous Word

What makes ‘better’ so dangerous is that it is usually subjective and even when it is objective it depends on how you frame the questions and weight the answers. So I should be clear that the question of which is easier is one I am discounting because it depends as much on the tooling as anything else and I will say right up front that, if your main goal is to build something quickly then a farm solution is the way to go. Visual Studio makes it easy to build and test farm solutions and it offers the most complete API with the best documentation and an internet full of blog posts for just about any situation you are likely to encounter.

On the other hand, if you want to build the most robust and scalable solution possible that provides the widest array of options with architecture that is most aligned with modern client-side techniques and with distributed cloud architecture, you should seriously consider an app. The problem is that there is a good deal of pain to suffer and work to do for your first non-trivial solution. One reason it’s taken me so long to make up my mind is that I’ve had to do lots of setup and write lots of plumbing code – work I won’t have to do on subsequent apps. The architecture and framework was definitely not trivial and represents a significant investment. Fortunately it isn’t a recurring cost.

The questions I will ask and answer as I make the case that apps are better than farm solutions are as follows:

(In this post)

  1. Which better supports modern web development techniques?
  2. Which has better scalability?
  3. Which is easier to version and maintain?

(In future posts)

  1. Which provides better experience for users?
  2. Which provides more flexibility, control, and security for farm owners?
  3. Which provides more flexibility, control, and security for software vendors?
  4. Which is easier to integrate with other systems?

     

One Other Qualifier

The app model allows integration of completely alien systems and platforms into SharePoint sites. You could make an app that embeds a video game based on HTML 5 into a SharePoint site via an App Part and serve it from a potato-powered web server running assembly language on a Z80 processor if you want. The app I am using as the basis for this article is a traditional SharePoint business solution consisting of lists, libraries, user interface, and automation based on said lists, libraries, and UI. It is an app that started life as a sandbox solution and that could easily be implemented as a farm solution.

Which better supports modern web development techniques?

This is actually a tie. You can (and in my opinion should) use the same UI development techniques with either model via HTML 5 and JavaScript. Farm solutions also give you the option to use traditional ASP.NET Web Forms with traditional server side rendering while the app model will force you to use HTML and JavaScript on the client. However, you can and should be using these client-side techniques with Web Forms in SharePoint 2013 – SharePoint 2013 does. Lists, libraries, and search are now rendered by JavaScript by default in 2013 and the simple fact of the matter is that there is much more energy going into browser based UI than in Web Forms because it’s much easier once you master the required techniques.

Which has better scalability?

On balance, apps provide better options for scaling than farm solutions because they force custom code to exist in the browser and/or on a server outside of the SharePoint farm. In either case the custom code automates SharePoint using a set of abstract web services. Farm solutions are all about running code directly in the farm. If this code is poorly written, it will affect the performance of the entire farm. However, even when the code is properly written, it runs on the farm and consumes resources. An app offloads this work to the browser and/or other servers.

On the other hand, the client-side API’s involve a (potentially big) network hop(s) with a corresponding time delay. If you write the code poorly or routinely move large amounts of data across the wire you will be faced with a slow system. Solutions that need large amounts of data from SharePoint will perform better as farm solutions. So, a timer job that reads data from a database to build a large PDF file for placement in a document library will work better, and indeed may only be possible, as farm solutions.

So, to be precise, apps provide better potential for scalability in many cases, but bad code is bad code.

Which is easier to version and maintain?

This question is easy to answer and the answer is unambiguous: apps are easier to version and maintain.

The downside to farm solutions is that they use a very powerful but comparatively rigid combination of file systems and a SQL databases in conjunction with lots of XML files. In order to use a solution, it must be installed on a farm and put files directly on each server. The XML files are features, site templates, list templates, and a wide variety of other artifacts. This system works really well unless the same solution is installed on multiple farms and used by many site collections and needs to change to accommodate new features or bug fixes. Furthermore, some things, like site and list templates, are very dangerous to change once sites and lists exist that use the templates.

The app model makes it possible to centralize and serve content to as many farms as is necessary and if you use CSOM to provision everything instead of XML you remove the coupling between concrete instances of sites and lists and their definitions. CSOM allows the creation of fields, content types, lists, web part pages, and other types of files. In my app everything, with the exception of one page, is provisioned by the application server running on Azure. The app server is capable of updating these items and of adding new items. It makes no difference how many farms use the app from an update perspective, each can be patched centrally whenever a user launches the app.

Here is a screen shot of the actual app package in Visual Studio 2012. It consists of the app manifest and a single module.

 

Here is a screen shot of the (not yet unveiled) app in an app web on Office 365. As you can see it has more than just one page!

A Note on Background Processing and Elevation of Privilege via Impersonation

Farm solutions provide a relatively easy facility for creation of background jobs via the SharePoint timer service. In my app background processing is done via Azure service bus and worker roles. The centralization of application services makes it easier to scale, monitor, diagnose and recover from problems because I don’t have to suffer through the lack of access ULS logs in Office 365 – logging happens on my end. However, to do this sort of work requires the ability to connect to the SharePoint site without a user doing the work in a browser.

As it turns out, this is pretty easy to do in an app but you need to store some information about the admin account that installed the app or allow the admin to specify an account. Apps use OAuth and there are a variety of tokens involved in getting a context to a site. In this case there are two tokens of interest, the access token used to connect which is good for twelve hours and the refresh token used to get a new access token which is good for six months. If you have these pieces of information for a user you can open a context as that user. The app model gives you some help in the form of the TokenHelper.cs class, but it is up to you to figure out how to persist and manage these values. The only thing the model does to help you with this is to provide an immutable cache key for each user along with the tokens so that you can store them without having to store information that identifies the user in human readable form.

In my app this is done on the start page which is in my application server, not in the app package. This start page captures and persists the tokens and keys when the user launches the app and then redirects either to the initial landing page which kicks of the provisioning process or to the app’s home page which only exists once the initial setup is complete. At that point I can act on behalf of the user from the application server as needed and I can put the complicated (and proprietary) code in my app server as c# instead of in the browser as JavaScript.

So, you can elevate privileges in a low-trust app via impersonation and do work on behalf of the user or run a scheduled process. It does not require a high-trust solution (but high trust makes it much easier). Most people think you can’t. There are many misconceptions about the app security model that I plan to touch on in the next post when I take a crack at answering the rest of the questions in my objective criteria.

Until then, happy coding!

–Doug

P.S. If you’d like help or training building an app, I’d love to hear from you.

Author: Doug Ware

High Trust Apps and Low Trust Apps, What does it all Mean?

If you are a SharePoint developer, the app model in SharePoint 2013 can be hard to get your head around. I’ve noticed that there is a great deal of confusion and misinformation floating around concerning security. The purpose of this post is to help you understand the meaning of the terms "High Trust Apps" and "Low Trust Apps".

If you’ve been working with the beta or the RTM bits and experimented with app development you have used the high trust model if you did your work locally with a provider hosted environment and you have used the low trust model if you’ve used Office 365 and Azure in an autohosted environment. In both cases you wrote code that executes in a web server outside of SharePoint. You used the CSOM, REST, and/or JavaScript API’s to talk to SharePoint.

Any JavaScript code that you wrote to execute in the browser runs under the user’s authenticated browser session. The terms high trust and low trust do not apply here; they apply to server to server communication and they concern the flavor of OAuth used for authentication and authorization between the external server and the SharePoint server.

To configure a development environment for high trust communication read this article which provides complete documentation for the setup… How to: Create high-trust apps for SharePoint 2013 using the server-to-server protocol (advanced topic).

You might be surprised to learn that you can also create low trust applications for local deployments without any cloud services. To set that up read this article… SharePoint Low-Trust Apps for On-Premises Deployments.

The low trust configuration article contains some excellent background on OAuth and the two flavors of trust directly supported out of the box, how to set it up, and what happens under the covers. However, none of the articles I’ve read have made it clear what the fundamental difference is, so here goes…

What is a High Trust App?

A high trust app uses the server to server protocol between your server and SharePoint. It is high trust because it can assert the identity of any user without knowing the user’s password and SharePoint will trust that it is working on behalf of the user. This does not mean that high trust apps can pretend to be an admin and do everything an admin can do! Every app has a manifest which specifies the permissions available to the app. No matter the user which the app asserts, the app cannot execute any operations above the permission level of the app itself.

This model is extremely useful when the app needs to do something on behalf of the user and it is much better than RunWithElevatedPrivileges which assumes the identity of the System account. If the app asserts a call as Jane Doe that modifies an item, the modified by field will say Jane Doe.

The app is further constrained by the permissions of the user it asserts. If an app has full control, but asserts the identity of a user with only read permissions, any attempt to work above the read permission level will produce an Access Denied exception. This is why the default app code for provider hosted sites looks like this:

The normal case is to act on behalf of the user who accessed the internal site. However, if you need to assert a more privileged identity you can without having to elevate and assume the identity of the app pool account. This is much better than the farm code most people write!

What is a Low Trust App?

A low trust app communicates with the SharePoint server using the app’s identity. It is not allowed or trusted to pretend to be someone else and it has the permissions defined in the app manifest. A low trust app is appropriate for Office 365 and other types of environments where the server that provides the functionality for the app and the server that hosts SharePoint do not share the same directory services or authentication mechanisms. In this case the OAuth trust relationship represents a static identity.

I hope that helps you understand the difference between these two models as well as what you are used to in farm development. If you understand this better than I do and would like to correct me, please comment so I can improve this post!

If you are waiting for me to write some more about provisioning, please be patient for a while longer. I’ve been busy and I want to make sure I test the code completely against the RTM bits before I write any more. I’ve already found a couple of differences.

–Doug

Author: Doug Ware

Provisioning with the Client Object Model Part 2 – Getting Started and Provisioning Fields

Previous post: Provisioning with the Client Object Model Part 1 – Introduction

Until recently I thought of the client object model as a client side version of the server object model. Today I realize that thinking of CSOM this way is a mistake. The two models are very similar. However, the way you use them is very different. Because of this, differences that seemed very important when I looked did a direct comparison of the classes in the two API’s turned out to not be very important. For the most part this is because the way you create and access objects is fundamentally very different. This is not to say that CSOM doesn’t have any holes or couldn’t use some improvement. It does, it could, and I will point them out as I continue this series.

Thinking in batches and working in Layers

When you write provisioning code on the server you probably completely configure each object before you move on to the next. The following pseudo code illustrates what I mean.

For each list to create

    Create the list
    Bind the content type
    Configure the views

Next list

 

Because the code is running on the server, beyond the unavoidable expense of the operation itself, there is no penalty for persisting updates as you go. If you need pieces of information from some other object in the site, there is no penalty for accessing the object to get the information you need. For all practical purposes, in most situations such code is pretty straightforward and opportunities to make noticeable optimizations are few. I define the time to execute a potential operation on the server as:

[Time to perform server side operation]

When you are working on the client, every object access and every update is a potential round trip to the server and so it is easy to write inefficient code. The potential cost of a given server side operation becomes:

[Time to format request] + [Time to send request] + [Time to parse request] + 
[Time to perform server side operation] + [Time to format response] +
[Time to send response] + [Time to parse response]

Therefore the goal is to pay the expense of doing the server side operation as few times as possible for a given transaction. The client object model is designed to make this goal attainable by providing a few key features.

  • You can add operations to a batch of requests via ClientRuntimeContext.Load
  • You can submit a batch of operations with ClientRuntimeContext.ExecuteQuery
  • You can use an object without fetching it from the server it as long as you don’t try to read its properties

The flexibility of ClientRuntimeContext.Load and the ability to use an object without fetching it from the server are the things that make CSOM so fundamentally different from the server OM. It makes it possible to change the previous pseudo code to the following.

For each list to create
    Create the list

Next list

Submit the batch to the server

 

For each created list

    Bind the content type
Next list

Submit the batch to the server

 

For each created list

    Configure the views
Next list

Submit the batch to the server
			

 

The batches are determined by dependencies. A list must exist before a content type is added. A list must have all of its fields before configuring a view.

Provisioning Fields

The various Field classes were a significant source of concern for me when I compared the client and server API’s because there are a number of missing properties. It turns out that this is not an issue because you can manipulate all of a field’s settings via its SchemaXml property. The only wrinkle I have found is that there are a number of properties SharePoint wants you to set only when the field is in a List’s Fields collection, not in a Web or ContentType’s Fields collection. For this reason, the first step in provisioning the overall solution is to simply create, but not attempt to configure, each field. If a field is a lookup or metadata column it will require additional configuration, but I leave this until later in the overall process after the base list instances and/or term sets are created.

The easiest and best way to create a new field is via the FieldCollection.AddFieldAsXml Method. The XML this method takes as a parameter is exactly the same as the XML you would use to create the field via a feature. The primary thing that can go wrong when attempting this operation is an error if the field already exists. Because operations are submitted in batches the code must be defensive – you don’t want to be in a position where you must puzzle out which operation in a large batch is making the operation fail! Therefore the provisioning code must first get a list of the existing fields to prevent the error.

The pseudo code for the operation is simple.

Request the Web's Fields

Submit the batch to the server

For each field to create
    If the field does not exist

        Create the field from XML

Next field

Submit the batch to the server

 

As I said in the previous post, a primary goal is a good style that shares my favorite aspect of feature CAML, readability. For this reason, I declare the list of fields using literal syntax to create a Dictionary<string, string> where the key is the field’s StaticName and the value is its SchemaXml. The basic syntax is:

And a real example (cropped) looks like this:

The code to do the actual provisioning is simple.

Getting the Existing Fields

The check against the FieldDefinitions dictionary uses the value of the StaticName property and nothing else. Therefore the client should only request that property value when it requests the entire collection to minimize the message size. This is accomplished by the lambda expression passed as the second argument of the Load method as follows.

Note the call to ExecuteQuery. The FillExistingFieldsList method reads the StaticName property. Attempting to read StaticName before the call to ExecuteQuery will throw a PropertyOrFieldNotInitializedException. Attempting to refer to any other property of a Field after the call to ExecuteQuery will yield the same exception because the lambda expression in Load doesn’t include any other properties.

Optimizing the Query

One very powerful aspect of the lambda expression is that it can restrict the returned set by a property value. In my solution the fields are members of an App specific group or of hidden. I attempted to optimize the operation by modifying GetExistingFields as follows.

This ability is amazingly cool because it closes what I thought were a number of holes in CSOM related to fetching items. Examining the results in Fiddler shows that the response is ~33% smaller. However, it also shows that in this situation the call to ExecuteQuery took 53% longer than the unfiltered version. However, when the entire operation is timed, it the ‘optimized’ version is only 1.5% slower. Almost all of the time lost on the server was made up for on the client.

The clear take away is that you should not assume that reducing the response size via a query filter will result in better performance. However, minimizing the number of round trips is a safe bet. In my development environment I have 46 fields. If I changed the code to call ExecuteQuery after adding each field, the number of round trips increases from three to forty-eight and the overall time to complete increases by 82%. There is virtually no latency in my development environment, so in a production scenario where the App is hosted on a separate server, the difference would be dramatically greater.

Author: Doug Ware

Provisioning with the Client Object Model Part 1 – Introduction

I suppose there are many ways that you could get started writing CSOM and JSOM code, but provisioning is probably where most people will start when they create a SharePoint app for a traditional collaboration solution. After all, the implementation of these types of solutions usually starts with fields, content types, lists, and views. The provisioning of these things is the subject of this series of posts.

Client Side Object Model and C# versus the JavaScript Client Object Model

Given my recent series on JavaScript you might be surprised that all of the code in this series will be written in C#! My general opinion is that provisioning (and other server-type operations) should be done in a remote event receiver to process the App installed event and that user interface code should be written in JavaScript. In other words, both models have their place and an App will have some of both. Once you are comfortable with JavaScript moving back and forth between the API’s is easy because the differences are minor. If you’d rather have this code in JavaScript you should have no trouble converting it.

Unfortunately, remote event receivers don’t seem to work with the current beta bits. Because of this I am using a high-trust provider hosted site for the App in my development environment. The start page has some buttons I can click to run my code in-lieu of an event handler.

Goals

I have a few general goals that the code must meet for my satisfaction. These include minimizing the number of round trips between my App and the host web and maximizing reuse. I know I’ll need this code for many different projects! But, the main thing I want is a good style that shares my favorite aspect of feature CAML, readability. Because this code base is very much a work in progress I won’t be including a download as I usually do… at least not yet.

In the next post I’ll talk show you how I provision the easiest type of item to provision – fields.

Next Post: Provisioning with the Client Object Model Part 2 – Getting Started and Provisioning Fields

–Doug

Author: Doug Ware

Building Traditional SharePoint Collaboration Solutions with the App Model

The SharePoint store on the Office 365 preview is slowly filling with apps. To date the one thing all of the apps have in common is that none of them resemble the sort of traditional SharePoint collaboration solutions that I believe constitute a large majority of the solutions people build in-house. By ‘resemble’ I do not mean look and feel but rather functionality.

Traditional collaboration solutions

SharePoint has a huge number of features and people use it for all sorts of things, but the sweet spot has long been solutions that take the basic components of a Team site and apply them to solving specific business scenarios. Such solutions are made up of business specific content types, lists, libraries, and event handlers or workflow. They often involve branding and custom UI components, but the bulk of the functionality usually comes from SharePoint and much of the code (both declarative and procedural) in a given solution is devoted to configuring the site as opposed to creating truly custom components.

SharePoint is an attractive platform for a wide range of applications because it contains so much configurable functionality right out of the box. Skilled implementers can do some amazing things in a very short amount of time compared to traditional start-from-scratch development. The deep integration with Microsoft Office, especially with Outlook, makes it possible to easily integrate shared functionality delivered via the web with the desktop productivity applications business people use all day, every day.

None of the current marketplace apps make extensive use of these building blocks. Instead most of them add complimentary functionality, as is the case with the Nintex Workflow app, or integrate with some external service like HP ePrint.

App Web Confusion

When SharePoint 2010 was in beta I built a non-trivial sandbox solution to find out how well the sandbox model works. I am now in the process of doing the same thing with the new app model. I had a very hard time at the beginning because I was focused on the App Web as the location of my functionality. An App Web is a special web intended to host all or part of an app. However, an App Web is very special and it lacks most of the building blocks needed for a traditional solution including:

  • Navigation
  • User and group management
  • Rich client support
  • Site settings

It seems that an App Web is a good option for completely custom solutions like Nintex Workflow, to host App Parts like the Bamboo World Clock and Weather part, or as a landing page with instructions or configuration tools, but for a traditional solution it is completely inadequate. After flailing about for a significant amount of time I came to understand that a traditional solution requires the use of the host web, not the App Web.

Using the Host Web

A host web is a regular SharePoint site where the application is installed. Any regular SharePoint site can be a host web. An App can specify that it needs significant permissions to the user’s host web up to full control.

But there is a problem.

Although an App can contain features in a wsp (web solution package) file, the solution package and features are applied not to the host web, but to the App web. The only declarative features you can apply to the host web via an App are CustomAction and ClientWebPart. These allow you to add functionality to a site’s menus and to add App Parts, respectively. The implication of this is that to use the host web effectively you can’t use features but must use one of the client side API’s to do provisioning, CSOM, JSOM, or REST.

Living without Features

For my App I decided to use C# and CSOM for the provisioning code, but I was very skeptical that the client side object model contained enough functionality to do the job. I was pleasantly surprised to discover that it was. I am able to provision and configure fields, content types, lists, and pages with Web Parts. The only real issue I had was that you can’t specify a content type ID when you create a new content type.

I am even more surprised to tell you that I prefer this approach over CAML and that I find it to be more flexible. I built a generalized framework to provision these elements which will be the subject of further posts, and I greatly appreciated being able to add tracing and logging to the process. It made debugging much easier. I believe that I wound up with more code than I would have using XML, but not much more. I also appreciate the fact that should the App Web become an option, I will be able to simply change the target web and my code will still work.

On the other hand, I did have to a lot of work to move from features to code. Fortunately I can reuse the code so future projects will be fairly painless.

About the only thing my App lacks is workflow, but the Nintex Workflow App clearly demonstrates that the API’s are there.

Remote Event Receivers don’t Work in the Preview

…at least if they do I haven’t ever seen it. In the meantime I just have a "Provision" button on a page in a provider hosted site configured for high-trust. In my App the list forms are custom and so I have been able to live without event receivers – I can do the work on submission of the forms. However, for many potential applications this will be a major issue. If it is I suggest you simply wait until the next code drop before you try to implement anything where you simply must have event handlers.

Is it better than Sandbox?

Based on my experience thus far it’s no contest. The App model is far more flexible. The client API’s are larger and more complete than the sandbox object model and the ability to do work at a higher level of permissions than those possessed by the current user enables many more scenarios.

Is it better than Farm?

I don’t know yet, but if you are currently restricted to a sandbox model or do extensive sandbox development you might want to change simply to minimize the number of skills you have to maintain across your team. The downside is that the architecture is more complicated and has more moving parts.

Is this Approach Valid for the Office 365 Marketplace?

At the moment there is nothing explicit in the guidelines that prohibits this approach. At the same time, I doubt they want users to party on the host web and I doubt Microsoft will approve an App that does this. However, both Office 365 and on-premises installations support side-installs to support just this sort of scenario by requiring an administrator to enable the App. Regardless, most Apps of this nature won’t be built for public sale anyway.

–Doug

Author: Doug Ware