man thinking

SharePoint Apps - The Good, the Bad, and the Pot of Gold at the End of the Rainbow

TackleSharePointDevelopment.jpg
I recently appeared on the SharePoint Podshow and I was asked what features I liked the most and the least in SharePoint 2013. I answered both questions with 'the App model'. I suppose because I am mainly interested in SharePoint 2013 as a development platform this is the thing I care about the most. So it evoked feelings of dislike initially, and then a gradual appreciation and ultimately enthusiasm for this new development model. Let me explain my thinking:

The History

I didn't build code solutions for the versions of SharePoint prior to the 2007 version; frankly the custom development story was too painful. But in SharePoint 2007 there was a huge improvement and there was really just one server-side development model to worry about. Then, with SharePoint 2010, Microsoft introduced the Sandboxed Solutions model. They were very keen that developers should build sandboxed solutions instead of farm solutions; so keen that they made it the default in Visual Studio. But the fact is that the sandboxed solutions approach didn't really cut it, and they are now to be deprecated (or at least the user code service is). Oops. So, unless you have already built one and need to support it, I think we can safely forget about sandboxed solutions as a way of running custom code.

Now with SharePoint 2013, in addition to traditional farm solutions and the deprecated sandboxed solutions, we have a new option: the SharePoint 'App' model. And Microsoft are making the same heavy hints that we should all use the App model, just as they did three years earlier for sandboxed solutions.

The SharePoint App comes in two main flavours: the SharePoint-hosted and the cloud-hosted versions. In neither case do we get to write any code to run directly on the SharePoint server. This is a seismic change in the world of SharePoint development. And as developers we have been getting a fairly consistent message from Microsoft over the last two releases of SharePoint: 'we don't trust you to run code on our server'. As an experienced SharePoint developer I took that rather personally.

What's the big problem with server-side SharePoint development?

The fact is businesses always want custom solutions. Everybody's business is special of course, to them. But it is impossible to come up with completely generic solutions, especially with such a broad canvas as SharePoint, which will satisfy everybody. Even single-task applications like Word and Excel have become large and complicated as the developers have attempted to satisfy every requirement. This was never going to be possible out-of-the-box for something with the breadth of SharePoint and its many workloads.

There are degrees of customization of course, and the key is knowing at what level to apply them. Sometimes people build custom solutions when the same results could have been achieved using the out-of-the-box features, and that typically happens because the person making the decision, who may be a developer or more likely a business analyst, doesn't have full knowledge of what is built in to the platform. Often the amount of development effort is underestimated. But even so, most organizations I deal with have ended up building custom code.

The fact is that the SharePoint server-side object model was never particularly easy to write code against. As with many API's there is potential confusion about who is responsible for cleaning up objects that have been created by the framework. If it was all .NET code it wouldn't be a huge problem because the memory manager would deal with it. But the SharePoint API is a thin wrapper over COM objects that can represent megabytes of memory and other expensive resources so the garbage collector has no clue that these need disposing. So that is probably the biggest single thing that brings SharePoint servers down. And there are many other bear traps and pitfalls that experienced SharePoint developers are well aware of.

But we know what we are doing, right?

Now of course people like you and I know about this and we have learned all the quirks and what to build and what not to build and when to dispose of objects and when not to dispose. But in many cases SharePoint solutions have been built by developers who are not SharePoint specialists. They are very likely to be ASP.NET developers who are asked to do a bit of SharePoint. Knowing that SharePoint is built on top of ASP.NET they would assume, not unreasonably, that they would use the same practices as for their other web development work. Since they are unaware of the many pitfalls there is a good chance they will fall into them, or they may just build custom solutions that they shouldn't be building because it is already built into the platform. Or they might just go about it the wrong way, for example hosting an ASP.NET site inside the page layouts by means of a page-sized user control (yes, I have seen a large project built using this technique).

Even when custom development is done well it can be a problem when it comes to upgrading to the next version of SharePoint and that's another reason why Microsoft would prefer that we didn't use the server-side object model. And finally there is the Office365/SharePoint online cloud story playing out where Microsoft themselves are responsible for maintaining the platform and have service level agreements in place. In this case they don't want to take chances with people running code, which is understandable. They want a viable no-code (i.e. no code on the SharePoint server) model for the cloud and to make on-premise solutions supportable.

Why should we move to the App model?

This is tough for professional SharePoint developers who have spent years perfecting their SharePoint skills because now Microsoft is telling us we should use JavaScript on the client side instead. Isn't JavaScript a sort of front-end hackers' in-browser scripting language that was written over a weekend? In my talk on SharePoint Apps at a SharePoint Saturday I used a "downfall meme" spoof video to illustrate the frustration some of us felt initially when we first thought about moving towards client-side development.

But although it is true that JavaScript was produced in a hurry, and has a name that doesn't make sense, it has matured a great deal since it was introduced. A number of patterns and practices have emerged to get around some of the shortcomings and support a more professional development model. And in particular the browser implementations have become faster over the years so that it can now be considered a suitable language for writing applications as opposed to just being a scripting language. But the best part is that it now has some great libraries with a lot of community support which is reminiscent of the situation with Perl about a decade ago. And as HTML5 and CSS3 have become more powerful we can build a lot of slick UI features with minimal code.

We have been moving in the direction of using more client-side code anyway because users are nowadays expecting more interactivity with less page turns. The SharePoint App seems to me like a natural extension of that, and in fact the only things about the SharePoint-hosted app model that are new in 2013 are the improved REST and CSOM API which now have much greater coverage, and the packaging and deployment mechanism. Everything else we can do with SharePoint 2010 if we choose to.

So I think Microsoft is moving in the right direction with this and unless we adopt more of a client-side development model we risk being left behind by the rest of the web development world.

What if you need server-side code?

So SharePoint-hosted client-side code is great for a large number of scenarios, and I think the majority of web parts I have written, rendering and interacting with lists, could have been delivered using this approach. But sometimes you need to have code running server side, and the proposed App Model solution for this case is the Cloud-hosted App which is going to be either an Azure worker (auto-hosted) or some other platform (provider-hosted). This could be something like Ruby on Rails or Node.js, possibly running on a non-Windows server (I'm thinking a Raspberry Pi - it might have scalability issues but that's your problem now). Or more likely it will be an ASP.NET site, either Web Forms or MVC; ASP.NET developers will be right at home. This is a move towards the service development model like Facebook or Twitter or Bing where you wouldn't really expect to have code running on their servers. So SharePoint becomes another service that our application consumes and interacts with. But there's a pretty complicated security story around all this (authentication, firewalls, etc. - AC has an article that goes into depth on some of the limitations of the App model). This is much less of an issue for the SharePoint-hosted (i.e. client JavaScript) approach because everything is running in the security context of the SharePoint page. I think that security, along with the responsibility to deal with scaling of the external server, will be the key inhibitors that determine how quickly the cloud-hosted app model gets adopted.

So I think that in the short term at least we will continue to build farm solutions in those situations where we need server-side code. But I think the SharePoint-hosted App Model will become increasingly important and we should try to build these wherever possible, particularly in cases where we would otherwise have built a conventional web part. Microsoft has done a good job of packaging SharePoint Apps and App Parts (which are like client-side web parts). So maybe the right approach is to use the App model where we can, componentize our applications and deploy just those parts of the solution to the farm where it makes sense. But we might need to re-think the way we architect the overall solution to do as much as possible with client-side code.

What skills do developers need to develop SharePoint Hosted Apps?

I suggest that as SharePoint developers we should focus on the SharePoint-hosted app model for the short term and learn how to use JavaScript properly. We need to learn the patterns to get around some of the weaknesses of the language, a legacy of its fairly frantic initial development, and particularly the preponderance of global variables and the lack of encapsulation that we have become accustomed to in 'classical' languages like C++, C# and Java (meaning languages that have classes - I don't mean old like FORTRAN, or classy like Lisp). Then we need to get up to speed on some of the JavaScript libraries that can really help, like jQuery of course, and Bootstrap that gives you a jump start in building pages with HTML5 and CSS3, and Knockout that gives you some data binding, to name a few.

When it comes to communicating with the server you have two choices: the JavaScript Client-Side Object Model (CSOM) and REST (a backronym for REpresentational State Transfer); a repackaging of the HTTP protocol as the latest new thing, and a move away from the complexity of web services standards and back to basics. You also need to learn about OData which is used to form the requests, and JSON which is the preferred way of returning the data when using JavaScript.

Personally, I am finding I need to limit what I learn because I still have to keep this farm solution detail in my head, which is already full. I need to learn languages and techniques I can re-use. And for this reason, whilst CSOM might initially seem the most attractive as a SharePoint developer because it has some similarities to the server-side model, I think REST, JSON and OData give us a set of tools that we are going to be able to re-use the most in the future. I'm not ignoring the cloud-hosted app model either; it's just not as high on my list of priorities as the client-side technologies.

InterestingApps.jpg The pot of gold at the end of the rainbow

Wouldn't it be great if we could find a technology that allowed us to build lots of applications for different platforms from the same code base? Well, I realise after decades of running across fields that you never get to that pot of gold. But it is realistic to try and limit the number of languages and APIs we need to learn by choosing technologies that we can re-use. So this combination of JavaScript, REST, HTML5 and CSS3 is a good investment because as well as SharePoint Hosted apps we can use it to build excellent web applications (either using conventional SharePoint development or other web platforms), good Windows 8 apps, reasonable Windows Phone apps, and with PhoneGap possibly even extend that to other mobile platforms. And we can also build Windows 8 and Windows Phone apps that talk to SharePoint, and that is significant because people are browsing web sites using tablets and phones almost as much as PCs and that is set to cross over in the next year or so. People want apps on their devices, not just web sites; they don't want to browse Wikipedia on their phone, they want the Wikipedia app! So these platforms are increasingly important.

What about the SharePoint App Store?

It's difficult to predict how important the store is going to be as a way of delivering SharePoint Apps. You can see what they were thinking - Windows Phone Store, Windows 8 Store, Windows Azure Store; why not a SharePoint Store? But I wonder if an app store is the right channel for business-to-business solutions, even for small businesses, and whether the whole concept of an app store won't be forever associated in the minds of consumers with a low price point. Usually when people are depending on something for their business they want to have a relationship with the vendor, so it's really not the same as buying an app for your phone. On the other hand it could turn out to be a great way of finding customers. It will be interesting to see what happens.

Conclusions

I think that for enterprise customers we'll continue to build farm solutions 'on-prem', which may not be literally on premises because the hardware might be in a third-party data centre or increasingly in a hosted VM like Azure VM or one of the other vendors such as Rackspace or Fpweb. But we should use the app model where we can, and that's a good thing because we can't afford to keep building SharePoint sites that look like SharePoint sites. It's not because we think Microsoft wants us to move to the cloud or because we can't trust our developers, but because users expect a Web 2.0 experience and we need to use client-side technologies to deliver that.

Office 365 is a great solution, especially for smaller businesses, but it limits what we can do in terms of customisation. Over time we'll probably learn how to make better use of the cloud-hosted model and the platform will continue to evolve and improve to support this.

But the thing that has me most excited is the possibility of using these new tools, along with ever improving tooling support in Visual Studio, to develop applications on many platforms. We will still build farm solutions for SharePoint and managed code solutions for Windows 8 and Windows Phone, but we now have another option that brings everything together - web, phone, Windows and SharePoint development. That pot of gold at the end of the rainbow looks tantalizingly close!

Resources

If you want to learn more about JavaScript, HTML5 and CSS3 an excellent starting point is a set of video tutorials that are actually intended for Windows 8 developers that are hosted at the Microsoft Virtual Academy. Day one of this course focusses on the technologies described above. The second day is more about the WinJS libraries which are only relevant if you are planning to do Windows 8 development or take the exam 70-480 (Programming in HTML5 with JavaScript and CSS3) as I did. Andrew Connell has a recent article that discusses some of the challenges in implementing solutions particularly with the cloud-hosted model. Chris O'Brien has a series of articles about SharePoint Apps which are a great introduction, and Jeremy Thake also blogs on this topic. A couple of excellent books are JavaScript, The Good Parts by Douglas Crockford, and for getting started with SharePoint app development there is SharePoint 2013 Application Development by Scot Hillier and Ted Pattison. Both are slim volumes that shouldn't take too long to read.