I believe that the most common types of apps people will build will be either provider-hosted or hybrid apps that include important server-side components. These server side components will manipulate content inside of SharePoint based on actions performed in the browser and on-demand based on scheduled jobs or in response to input from other parts of the overall system.
How this communication takes place and the overall architecture is of primary importance in any distributed app architecture and it is something on which I spent the most time figuring out as I built out my current reference architecture. I tried many options before settling on a final strategy. This post is about the various options, strengths, weaknesses, and my current implementation.
I’ve built many distributed systems over the years using a variety of technologies. If the experience has taught me anything it’s that it is critical to minimize dependencies between subsystems – this includes dependencies between user interfaces and application services.
In any distributed system there may be operations that take more time than a typical page load. This is especially common in SharePoint solutions because provisioning is comparatively slow. The third goal is that the architecture must support long running transactions.
Sometimes an operation will face problems. When this happens the system must be robust enough to identify the issue and compensate to recover. The fourth goal is that the architecture allows complete control over the manipulation of data and other artifacts.
To sum up the goals, the application services need to support an arbitrary set of UI technologies and ensure data integrity for short running and long running transactions.
Remote Event Receivers
I was very excited by the concept of remote event receivers when I first learned about apps, but I had a very hard time getting them to work because of bugs in the beta and quirks in the early Visual Studio tools. I am happy to report that both problems were addressed and now it is pretty easy to create and handle remote events.
The data flow for a remote event receiver looks like this:
Note the question marks: remote event receivers are not reliable. If your listener is not available to handle the event or the call is not successful, your app won’t handle the event and SharePoint won’t retry the notification. Worse yet, there is no direct mechanism available to the client to determine the ultimate success or failure of the event.
Finally, the use of remote event receivers makes the application server reliant on SharePoint for part of what should be atomic operations and creates a dependency between the app services and SharePoint. I still envision scenarios where remote events are helpful, but they will need to be augmented by background jobs that sweep up any missed events.
Verdict: Remote event receivers do not meet the goals.
Others have suggested that workflows are good alternative because the workflow engine can deal with transient errors in service calls, but that approach fails to minimize dependencies between my tiers and cedes the critical functionality of one subsystem to another… not good. But I freely admit that I don’t know enough about this approach to dismiss it generally. In my case, lack of time to investigate the possibility was the biggest reason I didn’t investigate this option.
At this point I realized that I needed to put all of my data processing into my application server and orchestrate my transactions from there.
Web Proxy / Remote Endpoints
Clearly what I needed to do was call services on my app server from the browser that could take charge of a complete transaction. The challenge here is that the app server is always on a different domain and that cross-site scripting restrictions exist. Fortunately SharePoint includes a couple facilities to enable this sort of interaction. One facility, the SharePoint Web Proxy, is for scenarios where the page is in SharePoint and the service is in another domain, the second, the cross domain library, is for the inverse and is primarily for communicating with SharePoint from pages contained in App Parts.
In my architecture the pages are stored in SharePoint and so I began working with the web proxy. The web proxy requires RemoteEndpoint elements in your app manifest that identify domains you are allowed to call. Calls are sent using the SP.WebRequestInfo object to SharePoint which forwards the request to the actual service.
I had a few problems with this approach. The first is that it is inflexible and creates a dependency between my app manifest and my services. What if I want to move the services around? The location is baked into the manifest. However, the deal-breaker was that the Web Proxy enforces certain timeouts and other settings that you can neither view nor change. It is possible for the Web Proxy to report an error to the client in cases where the application server received and processed the message.
I first encountered this problem with a long transaction before the worker role was in place – the timeout appears to be 30 seconds, but I also had transient issues that I can’t explain. Sometimes it just reported a failure even though the request made it to my server and was processed successfully. The web proxy is a black box and there appears to be no facility for troubleshooting.
Verdict: The Web Proxy does not meet my goals.
Cross Domain Library
The cross domain library uses an HTML 5.0 feature called postMessage via SP.RequestExecutor.js. I quickly realized that this was the mechanism I needed to use, but that I couldn’t use the cross domain library itself because it goes from a remote page to a SharePoint service and I needed to go from a SharePoint page to a remote service.
Verdict: Not applicable
Custom postMessage Implementation and JSONP
In the end I wound up building a custom postMessage implementation for my complex flows. My services are implemented as generic ASP.NET handlers. My pages in SharePoint use a custom function that adds an iframe to the page which receives the script that ultimately calls the custom service and proxies the reply back to the SharePoint page.
This mechanism meets all of my goals and it has an added advantage in that it allows me to implement additional security. Complete control over the communication lets me to easily share my context token cookie, support deep links, and implement a digest to prevent man in the middle attacks.
For simpler interactions, like the service that supports my autocomplete fields I am simply using jQuery with JSONP.
Author: Doug Ware