On Thursday 26th November Alan Cossey and I will be doing a joint presentation at the UK Access User Group National Seminar demonstrating the new features in Access 2010 (Alan) and Access Services in SharePoint 2010 (me). 

I have to say there’s some really clever technology  at work here and if you are working with Access now and maybe already hooking up SharePoint lists to Access 2007 then the new features in Access 2010/SharePoint 2010 are worth a look.  One thing you will notice very early on – Access macros have a new lease of life!

Booking info can be seen here and you can download the full agenda details here.

Recently I’ve been involved in the development of a SharePoint 2007 solution using InfoPath and Forms Services.  The purpose of the solution was collection of employee questionnaire data using one of several forms designed as a multi-page survey.  InfoPath was chosen because the process needed to make use of SharePoint workflows, which are not supported with survey responses.  Also, the survey form needed to behave differently depending upon whether an employee or their manager was completing it.

I wanted to use this post to document some of the things I learnt during the development process.

Only Use Custom Code if Rules and Views Can’t Do the Job: You can do an awful lot with InfoPath Rules and Views to give the user a good experience when completing a form.  You also have the added advantage that when publishing your browser-compatible form to SharePoint Forms Services, you don’t have to deploy it as an Administrator-approved form.  Over at EnduserSharePoint, Paul Galvin has posted a useful starter template for InfoPath which includes views for handling successful submission and discarding the form.  And there are other useful InfoPath tips on Paul’s blog.

“Wizard” Style Forms Work Well Using Views: Using [Next>] and [<Back] buttons combined with views, you can create a nice “wizard” style user interface for your form.  Here’s a nice post from the MS InfoPath team on this topic.

But You can Only Validate a Maximum of 5 Fields Using a Rule: This is one of the issues which required use of custom code.  On our “wizard” style form, there are 7 questions per page, and the user should not be able to move to the next page until all 7 have been answered.  So we ended up putting validation code on the “click” event of each “[Next>]” button.  This tutorial from DevExpertise explains how it’s done.  It ended up as one of the reasons for a custom code requirement – I guess it might also be possible to to this using XPath, but as we also ended up needing code for another feature (see below), we used custom code here.

If You Want to Pass URL Parameters to your Browser Based Form, You Need to Write Some Code: For this solution, a manager could be assigned multiple questionnaires to complete (one for each person they are managing).  So we wanted to pass the ID of the employee into the new form, then this would be used to look up employee details on a secondary data connection.  Turns out that you can only read URL (querystring) parameters using code.  Here’s a post from the InfoPath Team showing how to pass/retrieve these parameters.

Use Site Columns for Promoted InfoPath Fields: Our solution required two different forms to be available as content types in the same forms library.  If both of these forms use the same set of promoted columns, your life will be a lot simpler if you first create the columns you want to promote as a set of site columns specific to the solution.  If you don’t, and you deploy the forms to different servers, you can end up with multiple “Microsoft InfoPath” site columns with same name but different IDs.  Also, we wanted to promote some InfoPath text to SharePoint multiline text columns.   This doesn’t seem to be possible using the default “Create New Column in this library” option but can be done using site columns.

Ensuring your Submitted Forms Have Unique Names: You need to ensure that the submitted form has a unique name.  Here are couple of items you might find useful – Hannah Scott: Submit InfoPath forms to SharePoint with Unique File Name and  SharePoint User Group UK Submit InfoPath with unique filename using SharePoint Library ID.

Centrally Managed Data Connections Make it Easy to Switch from Dev to Test to Production: Converting your SharePoint data connections to centrally managed ones makes it much simpler to move your solution between servers or web applications.  To make this work you will either first need to use a data connection library for data connections, then upload these using Central Admin, or use this UDC File Authoring Tool (which is also an InfoPath template).  More information here and here.

If You Are New to InfoPath, Read Paul Culmsee’s “Humble Leave Form” Series”:  Paul Culmsee of CleverWorkarounds fame has written an amusing and useful series on InfoPath called A Tribute To The Humble Leave Form.  There are 7 parts to the series so far, all based around a leave form for the Springfield Nuclear Plant.  In part 5 Paul discusses using the UserProfile web service to get more useful info about the user completing the form. 

And More Information on Getting User Information Without Code: Here are a couple more posts on getting user information from Itay Shakury and Clayton Cobb.

I hope others embarking on InfoPath/Forms Services projects find some of this useful!

In my previous post on SharePoint 2010 I mentioned Windows SharePoint Services 4.0.  Turns out that this is now going to be called SharePoint Foundation 2010.  Also, looking at What’s New In SharePoint Foundation 2010 it seems that it will include Business Connectivity Services.

Oh, and Bjørn Furuknap has already registered the domain sharepointfoundation2010.com.

Last week the Microsoft SharePoint Conference 2009 was held in Las Vegas, and following Steve Ballmer’s Keynote address, the NDA was lifted.  Very soon after came lots of blog posts and tweets about all of the new features.

Here at WorkerThread Towers we’ve had access to the preview bits of both Office 2010 and SharePoint 2010 since July and I’ve been doing my best to try out all of the new features in both.  I’ve also had a look at SQL Server 2008 R2, and spent a some time understanding how this integrates with SharePoint 2010. 

Before I get started, the clue is in the name: SharePoint and Office 2010 will be released some time in 2010.  Expect a public beta in November and I guess at least one more refresh of that (release candidate or similar) before it is released to manufacture.  But it’s still going to be worth spending some time before release understanding the new features.

As there’s a lot of stuff already out in the blogosphere, and I’m sure much much more to come, I thought I would use this post for two purposes – first to point you to some interesting summary posts on new features, and secondly to talk briefly about some of the new areas that will help develop (almost) code-free solutions.

So, to get a quick overview of the new features, take a look at these posts:

SharePoint Team Blog – Describing SharePoint 2010 in 1 Sentence,8 Categories and 40 Feature Areas

Joel Oleson – SharePoint 2010 Top 10 Features and Resources 

SharePoint Experts – Favourite Technologies Unveiled

Arpan Shah – Welcome to SharePoint 2010

SharePoint 2010 at Microsoft.com

And below is my quick list of favourite new stuff, with particular reference to features that involve the Office 2010 applications.   Actually before I start, I want to mention that some of the new stuff, useful though it is, should really have been there in earlier releases in my opinion.  For example, you can finally have unique constraints on list columns, SharePoint Designer Workflows can be list-independent, etc.  All of this is going to be very useful, just a shame we had to wait so long.  Also, for this post I won’t go into too much detail here on new features for developers using Visual Studio 2010 with SharePoint 2010 there is some really good new stuff here too – take a look here.

List Enhancements: many more configuration options such as not allowing duplicate values in columns, better column validation rules and options for “referential integrity” between a column and its lookup item (such as “cascade delete” so that deleting the “parent” record will delete its associated children).  You can also include more than one column in a lookup list – for example if you had a contact last and first name both can be displayed in the drop-down.

Access 2010 and Access Services: I’m really interested in this.  In a nutshell it is possible to use Access 2010 to develop an application using tables, queries, forms, reports and macros and then publish the whole lot to a SharePoint 2010 server running Access Services.  Now let’s not get too carried away – you wouldn’t want to use this for many of the very sophisticated Access apps that are out there, but it still looks to be a worthwhile technology to allow rapid deployment of a web application being used to capture data, produce reports etc.  There is a new Access show on MSDN Channel 9 where you can see a video explaining more.  Also I will be presenting on this subject at the UK Access User Group National Seminar on 26th November.

Business Connectivity Services: SharePoint 2007 has the Business Data Catalog, which provides a method to surface read-only “line of business” data in SharePoint as lists, searchable content etc.  In SharePoint 2010 this has been replaced by Business Connectivity Services and this can now be configured as a read-write data source.  To the user it can look like any other SharePoint list, and be used in DataViews, custom applications etc.  Under the covers it can be talking to a SQL Server database or similar.  Data from the connection can also be cached using a SQL CE database local to the SharePoint server.  It’s also possible to surface the BCS data in other client applications such as Outlook.  Powerful stuff, although it will definitely require tight control if we are to avoid performance bottlenecks when querying/updating line of business databases.  Read more on this here.  One other point – I believe I heard somewhere (but can’t remember where right now) that the original Business Data Catalog features will be available in Windows SharePoint Services 4.0 (the free version).

SharePoint Designer 2010: Like the current 2007 version, SharePoint Designer 2010 will remain free of charge, but worth noting that it can only be used with SharePoint 2010 servers (or WSS 4.0 I guess?).  For me, the best new features here are around workflows – you can create workflows in SharePoint Designer 2010 which are re-usable across different lists or sites.  Also, several new workflow conditions and actions which will make life easier, around the areas of list permissions particularly.  Here’s an initial post from the SharePoint Designer Team, and some info on the new conditions and actions.

InfoPath 2010: Lots of new design features here, many of which make it easier to build attractive and intuitive forms for use with Forms Services.  Also, you can use InfoPath 2010 to customise the design of the edit/display/new forms associated with SharePoint lists, instead of using SharePoint Designer.  Speaking as someone who has had “issues” more than once with SharePoint Designer-customised forms this will come in very handy.  More details from the InfoPath team here .

Visio 2010 and Visio Services: Microsoft’s own SharePoint 2010 demo videos include a look at one of new Visio/SharePoint features – you can create a Visio diagram which connects to backend data (in the Microsoft demo it’s a manufacturing process) then publish the data-connected diagram to a SharePoint 2010 site using Visio Services.  You can get a flavour of this here.  Also, Visio 2010 can be used as the starting point for creation of a SharePoint Designer Workflow.  The SharePoint Workflow template in Visio 2010 can be used to visualise what the workflow should achieve, then the finished article can imported into SharePoint Designer 2010 as a new workflow.  Here is a blog post from Phil Wicklund with some more detail.

Project 2010 Integration with SharePoint (and SharePoint 2010 Project Tasks Lists): Often, when I show users Project Task lists in SharePoint 2007, I’m immediately asked about task dependencies and how these can be incorporated.  Well the good news is that SharePoint 2010 task lists know a lot more about dependencies.  And with Project 2010 you can synchronise a task list between SharePoint and Project – something that many Project users will like a lot.

SharePoint Workspace 2010: This is what Groove has turned into.  Its a desktop client application that lets you cache copies of SharePoint site content on your desktop (or most likely laptop), work on the data, then re-sync with SharePoint. When you do this it’s possible to select which content on a site you actually copy offline, so again quite a useful tool.  Here’s a post from the SharePoint Workspace Team to explain more.

Business Intelligence, SQL Server 2008 R2 and PowerPivot: Microsoft evangelists have referred to SQL Server 2008 R2 as the “Business Intelligence Release”.  Definitely that’s where the emphasis lies regarding new features.   One much-publicised feature is PowerPivot (previously Gemini) an add-in for Excel 2010 which lets Excel users manipulate very large data sets in “slice and dice” mode, and publish the analyses up to SharePoint.  And if you want to integrate SQL Server Reporting Services with SharePoint, it looks like SQL Server 2008 R2 will make this a lot easier for admins – certainly the preview release allows you to install your SharePoint 2010 components, then install SQL Server 2008 R2 and select “Analysis Services with SharePoint Integration” as the configured option. 

Finally, don’t forget that SharePoint 2010 is totally 64 bit technology and definitely needs a fair amount of memory to run properly.  If you are planning to develop, test or demo on a laptop and want to use Virtual Machines, this means either installing Windows Server 2008 with Hyper-V or using VMWare Workstation or Virtual Box.  Alternatively you will be able to install SharePoint 2010 onto a 64-bit Windows 7 machine for development purposes, and indeed Tom Rizzo did just this in his demo at the SharePoint conference.

Oh, and SharePoint 2010 now has the much loved/much hated Office style ribbon…..

If you’re a UK-based IT professional working with Microsoft SQL Server as a developer or administrator then you should definitely check out the forthcoming SQLBits conference being held at the Celtic Manor Newport from 19th to 21st November 09.  As usual with SQLBits the Saturday event is free of charge, and looks like it will have an impressive line-up of sessions (right now you can vote for the ones you would like to see – including “Elementary Welsh for DBAs”!).

But wait, there’s more!  Thursday 19th November is a training seminar day which includes seminars on the new SQL Server 2008 R2 features from Donald Farmer, and an introduction to MDX from Chris Webb.  Friday 21st November is devoted to SQL Server 2008 and R2 and will include talks on the new BI features, Reporting Services, Master Data Services and lots of DBA-specific stuff.  Note that for the Thursday and Friday sessions you need to pay a fee, but these look very reasonable and if you book by 30th September you can get an early-bird rate.

Here’s some Elementary Welsh for DBAs - "Mae’r data-bas yn ddirwy, mae’n rhaid i fe bod y rhwydwaith" which apparently means “the database is fine, it must be the network”…

I got this link via the BI Monkey blog and thought it was well worth noting.  10 Essential SQL Tips for Developers is a good set of basic but important SQL points that all developers should fully understand.  It’s often the case that developers don’t pay enough attention to how well-crafted SQL can make a big difference to application performance and reliability.

A recent project I worked on called for data captured into a SharePoint list using InfoPath Forms Services to be ultimately imported into a SQL Server database for use with a set of performance reports. 

Given that InfoPath solutions can be designed to submit data to SQL Server (generally via a web service) you may ask why this wasn’t used for the solution.  Well, we wanted to do more stuff with the captured data in SharePoint, using workflows and task assignments before the data finally got approved and loaded into SQL Server.

In summary, the data was going to be submitted using InfoPath and Forms Services as promoted columns in a document library.  Once the appropriate people had reviewed, actioned and submitted the data it would be moved to a separate list, and any new items then needed to be loaded into SQL Server.  From there, Reporting Services would be used to generate and distribute nice looking reports.

So I took a look around to see what tools I might use to get the data into SQL Server with minimum coding effort, and ideally without installing any custom components on the SharePoint server.  I was aware of a set of SharePoint Designer Actions on CodePlex which included an Execute SQL Action.  For me the problem with this was that I was fairly sure I wouldn’t be allowed to install these on the production SharePoint server.

Also on CodePlex is a set of SQL Server Integration Services Community Samples, including SharePoint List Source and Destination Components.  For this project these components seemed more suitable, particularly as we would only need to import new items into SQL Server at the end of each day, and Integration Services would let us schedule a daily import.  And there’s even a MSDN article showing you how to do it.

To show how it works, I created a test SharePoint list called Responses which looks like this (let’s assume this data is coming in via InfoPath)

And on SQL Server, I created a table which looks like this:

Once the List Source and Destination components have been installed into Integration Services, I can create a Data Flow Task to read from SharePoint and write to SQL Server:

And map columns like so -

Of course when I run this I only want to import new rows.  The way I’ve handled this is to store the ID of the SharePoint list items in the SQL Server table and execute a SQL task to get the Max ID currently in the table like so –

SELECT ISNULL(MAX(ResponseID),0) AS LastResponseID  FROM Responses

As the SharePoint List Source component has a CAMLQuery property which I can update with a property expression at runtime, I can set it to something like this –

<Query><Where><Gt><FieldRef Name="ID" /><Value Type="Counter">1</Value></Gt></Where></Query>

In fact in the Integration Services Property Expression Editor I created an expression like this (LastResponseID is the output from my SQL Query):

"<Query><Where><Gt><FieldRef Name=’ID’ /><Value Type=’Counter’>" + (DT_STR, 4, 1252)  @[User::LastResponseID] +"</Value></Gt></Where></Query>"

So now if I put all of this together, I have an Integration Services Package which -

- Gets the last imported ID from the SQL Server table

- Queries the SharePoint list for items with an ID greater than the last one imported

- Loads any new list items into the table

Making it easy to get my list items -

Into my SQL Server table -

And I can schedule this to run each night, all without installing any new stuff on the SharePoint server or spending any money on new components!

I know there are already quite a few SharePoint-related blogs mentioning this video, but for any WorkerThread clients who haven’t seen it yet, it’s a nice little overview of what SharePoint is about and very useful when you are talking anyone who still doesn’t get it -

WordPress won’t seem to let me embed the video I’m afraid, so go and spend 3 minutes taking a look at SharePoint in Plain English.

And here’s another one on how to put the Share into SharePoint.

Update – Sharon at Joining Dots found the video on YouTube so I can embed it now:

Far too frequently I go to a client site, take a look at their SQL Server setup and notice that one or more of their SQL Server scheduled jobs is failing.  Now we all know that job monitoring should be part of a DBA’s daily checklist, but still things seem to get missed, particularly if there are multiple servers to monitor, a stream of user departments asking for assistance, etc etc.   Not good, especially if the job is a critical operation and you only find out when you need to do a restore.

So I often suggest that they download SQL Job Manager from Idera.  This is a free tool which lets you set up a single view of all jobs running on your SQL Server installations and very quickly see/fix any failure or contention problems.  Well worth a look.

And if you are doing SharePoint admin, they have a free SharePoint Performance Monitor as well.

I’ve been really busy over the past few weeks, so haven’t had much time for new blog posts.  So I thought I would roll up some related news into one post, as all of the information is about Business Intelligence, data and dashboards.

First, if you are interested in how browser-based dashboards can be used to show a company’s financial data, take a look at Microsoft Investor Central.  Here’s a snippet showing what it looks like right now:

If you visit the page you will see it makes heavy use of Silverlight to let you zoom in on and manipulate the charts.  Thanks to Patrick Husting for the original heads up on this.

And while on the subject of dashboards, can I also recommend this demonstration PerformancePoint site from Richard Lees, which has loads of examples using Analysis Services, data mining techniques and PerformancePoint scorecards.

Last week Microsoft made all sorts of announcements about Office 2010.  I’ve been taking a look at Excel 2010, and in particular the new data presentation options.  It looks like this release will have some worthwhile new Business Intelligence features.  As well as Gemini, which will I believe be available as a separate add-in, this will be the first release which will let you create Sparklines, those little word-sized graphics which I first blogged about in 2007.

Here’s how they look:

You can read more about how they work in Excel 2010 over on the Microsoft Excel Team blog here and even more here.

Finally, I’d like to mention the Guardian Newspaper’s Data Store.  This is a fairly unique exercise by a national newspaper to make datasets they have created, frequently to support news stories, publicly available.  Most of these can be accessed as Google spreadsheets, also downloaded.  As well as being interesting datasets in their own right, they are incredibly useful if you want to populate demonstration dashboards, load sample lists, or create training exercises.  You can keep up to date on what they are doing on their blog or on Twitter.