Now Excel pivot tables and charts aren’t perfect, certainly not cutting-edge.  But if the infrastructure is in place then using Excel 2010 as a client for your SSAS cubes and publishing workbooks or workbook items to SharePoint 2010 is an excellent “self-service BI” entry-point.  In my opinion Excel 2010 is still the best choice for validating the quality and accuracy of the data in your cube and I often get users who know what to expect from the data to play a major part in this task.  And of course Excel Services pivot tables and charts can become part of the bigger BI picture later as a part of PerformancePoint dashboards. 

However you still need to make it as easy as possible for your “self-service BI” contributors to find their data and publish it.   This is what I’ll cover here.  Note that I’m not intending this post as a guide to creating Excel reports with SSAS Cubes and SharePoint (see links at bottom of post for various walk-through examples and tutorials).  My main objective is to show how to put together a SharePoint environment which your Excel Services user population (both contributors and consumers) are comfortable using.

Before Your Contributors Get Started

To kick off, let’s briefly cover some things that need to be in place before you let your Excel Services contributors loose (all of which assumes you are working with Office 2010 and SharePoint 2010):

You need SharePoint Server 2010 Enterprise – I mention this because I still frequently see the “which SharePoint version do I need” question asked online.   Excel Services is a feature of SharePoint Server 2010 Enterprise, so if you have SharePoint Foundation or SharePoint 2010 Standard you are out of luck.

You need Office 2010 Professional or higher – many of the Office 2010/SharePoint 2010 integration features aren’t available in Standard Edition.

Decide where your Office data connection files will be located in SharePoint – all of your published Excel workbooks that connect to SSAS cubes should use an Office data connection (ODC) file stored in data connection library.  If you are using the default SharePoint Business Intelligence Centre site collection template then you will already have a Data Connections library in place.  If you are using your own site you will need to create a Data Connections library – and don’t forget that data connection files created in these will by default require approval once created and uploaded, so make sure that this isn’t overlooked.

Decide where your Published Excel workbooks will be stored –  I normally create a specific document library for these (and as you will see later, may well enable content types).  

You must configure Trusted File Locations and Trusted Data Connection Libraries in your SharePoint Excel Services application – This is frequently overlooked and needs to be planned and configured in advance.  If things aren’t configured correctly here then your Excel workbooks may not open in the browser (trusted file location not configured) or refresh data connections (trusted data connection libraries not configured).   TechNet has more details on this here and there is a good troubleshooting guide available for download here.

Assign appropriate permissions to the Excel document library in SharePoint – you need to assign Contribute permissions to the group of users who will be allowed to publish workbooks (this gives them permissions to view, add, update and delete workbooks).   Typically I recommend other users be assigned View Only permissions within the library as this ensures they can view the published workbook items in a browser but only download a snapshot containing values and formatting (users with next-highest Read permission can open the workbook in Excel and view all underlying details).  If your Excel contributors are also creating/editing dashboard pages containing Excel Web Access components then you will need to assign Design permissions to them.

Assign appropriate permissions in the Analysis Services Cube – if you are using Integrated Windows Authentication to connect to your cube, you need to do more than grant appropriate SharePoint permissions.  All users connecting to the cube via Excel and SharePoint will also need to be given permissions to access the cube itself.  I would recommend that when the cube is designed in Business Intelligence Developer Studio, a “cube readers” role is set up and an Active Directory group is assigned the appropriate permissions, then all that is needed is to add existing and new users via Active Directory.

Making Data Connections Easier

I’m assuming here that your Excel contributors are already familiar with pivot tables and charts and may already have connected to a cube themselves.   However, connecting using an Office Data Connection file stored in SharePoint may well be new to them and the process of creating a new analysis and re-using the existing data connection isn’t that intuitive.   Here are some suggestions:

Create the Data Connection files for them, before they get started – you won’t normally need a lot of data connection files (typically one for each cube or cube perspective) so I would absolutely recommend getting the data connection files created, published and approved before your contributors get to work.  Here is a Microsoft Office article on creating and using Office Data Connections – it dates back to Excel 2007 but most of the instructions still apply.  The article does talk about using a “DataConnections” web part.  Just to clarify, what they mean here is view of your data connections library which in the article is called DataConnections.  Below is an example in SharePoint 2010 where a view of approved ODC files in my  DataConnections library is shown on a site page (click image to open full-size) -

The default action that will take place when clicking on either of the links is for a new Excel workbook to be opened with an empty pivot table created, using the stored data connection you clicked.

Consider using Content Types in your Excel document library – this is another way of making it easier for your contributors to create new cube-connected Excel reports.  There are already a great many online tutorials on how to create and use SharePoint content types (here for example).  What I’m suggesting here is that the document library which will be used for your Excel workbooks has “allow management of content types” enabled.   Each content type added to the library can then use a different Excel document template which already contains a data connection pointing to the appropriate Office Data Connection file that you’ve already uploaded.   Your contributors then just need to select the appropriate new document from within the library:

then create a new pivot table in the workbook and select “Use an External Data Source”:

and choose the existing connection in the workbook:

So the contributor creating the workbook doesn’t need to find the connection file, and you have the added advantages of being able to add other metadata columns to your content type and of course re-use in other libraries.  Additional metadata columns are particularly useful if your library is likely to contain a large number of Excel documents – they will allow you to create custom list views of your reports by type, department, owner etc.  Something else you may find useful here is enabling content rating on your document library.

Make Sure Published Workbook Items Refresh

One other quick point to note before I finish this post – I frequently see published workbooks which don’t show the most up to date data when opened in the browser.  It’s important to ensure that workbook connection properties have “Refresh data when opening the file” checked, otherwise you will be looking at stale data:

Links for those Getting Started

Here are some links you might find useful if you are just getting started with Analysis Services,  Excel and SharePoint –

Getting Started with Business Intelligence in SharePoint Server 2010 (well worth downloading the PDF or Visio document)

Excel Services in SharePoint 2010 Dashboard Improvements

Excel Services Data Refresh Troubleshooting Flowchart

Connecting to Cubes and External Data with Excel in SharePoint (excellent blog post from John White)

Excel Services Module of SharePoint Server Business Intelligence Insights Training – good info in this video series and you can download the accompanying Lab02 here.

That’s it for now – I hope the points I’ve covered help you with a successful SharePoint BI solution.   If you’re interested in more posts on this topic leave a comment and I’ll see what I can do.

There has been a big buzz around all of the new Business Intelligence features coming along with the next release of SQL Server, currently codenamed Denali.   This release is going to have a huge number of new features, not least of all two different versions of the Analysis Services engine – the original multidimensional OLAP one and a new Tabular mode one.   Marco Russo has a brief but useful explanation of this in his blog post here.

As Marco says, the new Tabular mode engine is going to make great use of Data Analysis Expressions or DAX.  Learning DAX will be something all BI analysts and developers working with the Microsoft set of tools will want to do and his PowerPivot book is a good place to start.  

So you may ask, why am I reviewing a book on MDX, which is the multi-dimensional expression language used with the older multidimensional model?  Well, as exciting as all of the upcoming technology is, I think MDX is going to be around for a good while yet.  We don’t yet have a release date for Denali and many of the new features are going to need a lot of planning, particularly if you want to use the new reporting features such as Crescent along with SharePoint.  In the meantime, there is an opportunity to make use of the current SQL Server release to build very effective BI solutions with Analysis Services which will still work well once Denali is available. 

The book’s author, Tomislav Piasevoli is a SQL Server Analysis Services MVP based in Croatia.   He has a lot of real-world experience with Analysis Services, cube development and MDX and that certainly comes through in this book.  All of the cookbook “recipes” are well documented with a full set of downloadable examples.  And the technical reviewers for the book are among the most experienced in the world, including Marco Russo, Chris Webb and Greg Galloway.  This certainly shows through in the sample code – I’ve tried out a good few of these already and all work as described which unfortunately isn’t the case with some other books out there.  All examples use the good old AdventureWorks database samples in their 2008 R2 guise.  So its possible to work through everything in the book by installing the trial version of SQL Server, available as a free download, or maybe purchasing the developer edition which has all the features of SQL Server Enterprise.

As the title suggests, this is a “cookbook” covering lots of MDX solutions or “recipes”.  Tomislav’s technique is to lay these out as in a standard format: Getting ready, which sets up the initial query for the data you will use; How to do it which shows you a worked example with MDX code; How it works, describing what the code is doing and finally There’s more which will point you to other examples of how the technique might be used.

Each chapter of the book covers a specific set of problems or challenges which you might encounter when developing a multi-dimensional Business Intelligence solution and provides appropriate solutions.   You can take a look at the areas covered in the table of contents over at PackPub.  I haven’t tried out everything covered in the book – in the same way as it’s unlikely you would cook every dish in a real cookbook unless you are Julie Powell and you want a book/film deal.  Here are some of the topics I’ve been impressed with so far -

Working with Time – lots of examples on adding time intelligence to your cubes and MDX queries along with several examples of calculating today’s date.  As Tomislav says, this is one of the most frequently asked questions in online searches and forums.

Concise Reporting – How to use a set to make your analytical queries more compact and concise.  Examples here cover topics like isolating best or worst members, displaying important members plus “others” and “totals” rows and finding exceptions.

This isn’t a book for anyone completely new to MDX but is definitely an excellent resource if you have already started to work with it want good real-world advice.   MDX isn’t an easy language to master and I think regardless of your experience level you will find something of value here.   Well worth buying.   And if you are completely new to MDX I would suggest starting with the Stairway to MDX series on SQLServerCentral.com (registration required).

If you are a Kindle user the book is available from Amazon and you can download a sample from there to get a better feel for how the cookbook and its recipes are structured.

Let’s consider first where Data Sync Studio (I’ll shorten the name from now on) is positioned, and how it might help with data synchronisation requirements.   If you are working with Microsoft SQL Server you will know that one of its very useful components is Integration Services,  an excellent tool for moving high volumes of data between applications.  If you are populating a data warehouse, loading database tables with data from an external source, cleaning, merging or de-duplicating data and you are a SQL Server customer, you need to get familiar with Integration Services.  But of course not everyone needing to move data around has access to Integration Services and even if they do, it may be the case that the data providers they need to work with aren’t catered for, either officially or unofficially.   I’ll look at two of those data providers here, namely SharePoint lists, and Active Directory Users, both available in Data Sync Studio.   You’ll see though at the bottom of this page there are many more, including the usual Microsoft database formats, Microsoft Exchange and CRM as well as Oracle, MySQL and OData.

Installing the Application

Data Sync Studio is a standalone .Net application which comes in 32-bit and 64-bit versions.   I’ve been trying it out on a Windows Server 2008 R2 virtual Machine, but you could just as easily install on a modern Windows desktop PC running Windows 7, Vista or even XP.   It is licensed per developer rather than per server.  Once you create a synchronisation project, if it is something which needs to be run on a scheduled basis then you will need to have a licenced copy of the application installed on the machine (desktop or server) which runs the scheduled job.

At the time of writing (May 2011) a single user licence of Data Synchronisation Studio currently costs 450 GBP / 720 USD.  More information on pricing and licensing options are shown on the product home page.

Importing or Exporting SharePoint Data

Data Sync Studio allows you to use a SharePoint list as either a data source or destination and you can synchronise data in both directions.  This means that you can fairly easily connect your list to a SQL Server table and keep the two in sync.   Of course you should first be sure this is the best approach to solving your business problem.  There are other methods to consider here, not least of all Business Connectivity Services, but BCS is definitely not the answer for everyone as it’s quite difficult to get right and again will need other tools to set up correctly – at the very least access to SharePoint Designer 2010.   Also I’ve often come across situations where data needs to be captured in SharePoint then later moved to an external database such as SQL Server for additional analysis, reporting or even archiving.

I took the example I used in my article on using Integration Services with SharePoint data.  I created the same list in SharePoint 2010 and a destination table in SQL Server 2008 R2.   In Data Sync Studio the SharePoint list became Data Source A and the SQL Server table Data Destination B:

First, I moved all of the current SharePoint list items to SQL Server.   This is easily accomplished using the “Compare A>B” button, which gives me a summary of the differences between source and destination.  I can then use the “Synchronise” button to push these changes to SQL Server.   Updates, additions and even deletions from the SharePoint list can also be handled and sync’d to the SQL Server destination:

And if you want, the sync can be bi-directional – you can specify which synchronisation options are used by setting properties in the source and destination:

Data Sync Studio also includes a “Dynamic Columns” feature which lets you create custom code to express columns.  These can range from fairly simple examples such as concatenating multiple columns (for example first and last name) or performing lookup or validation tasks.  One example provided is a “SharePoint Lookup Helper”.  You would use this if you were importing data into a SharePoint list and during import you needed to match a source value to the ID of an item in a separate lookup list.  Note that this facility is available via a C# programming interface so definitely requires .Net expertise.  You can read more about this here.

I also tried using the SharePoint data provider with Office 365 but couldn’t get this to work as it wouldn’t connect.  Simego tell me they are actively looking at this and I think it would be useful.   I can imagine for example a scenario where external users submit data to an Office 365 application and the data is sync’d with a corporate application or reporting system.

Working with Active Directory Data

Data Sync Studio includes LDAP Active Directory Users as a data provider which can be used to import this data into a database  or SharePoint list:

I tried this out, loading specific columns of the AD user list into a SharePoint list.   I could certainly see some uses for this.   Simego say they already have clients using this to populate a staff or team directory in SharePoint.   Also, although the Active Directory User data provider doesn’t allow inserts or deletes (a good thing) it does allow you to update configured columns.   This could be useful – it’s often the case that when an organisation implements SharePoint they want to get best value from searchable information in Active Directory, but fields related to job title, department, contact phone numbers etc aren’t populated.   This could provide a nominated user (maybe a member of the HR department) with a way of updating the fields in a list exported to SharePoint and then syncing the updated fields back to Active Directory.

It’s probably worth noting that bi-directional synchronisation is also possible with the SharePoint 2010 User Profile service, but again, this will present you with lots of configuration challenges and is definitely not for the faint-hearted.   Data Sync Studio may be a viable alternative here.

Creating a Project Schedule

Data Sync Studio lets you create a schedule for your synchronisation project using a dialog within the Studio itself:

Once you have selected dates and times to schedule a run and entered login credentials, a Windows scheduled task is created automatically for you.   A separate “Data Synchronisation Run Tool” is also provided within which you can schedule multiple projects to run in sequence and it’s also possible to execute a synchronisation project from the command line.

Summing Up

Data Synchronisation Studio is well worth a look if you have a need to move data between line of business applications and your requirement can’t be addressed by SQL Server Integration Services, either because you aren’t able or allowed use it or because the appropriate data providers aren’t available.   Certainly it won’t take the place of Integration Services when you are moving very high volumes of data to or from SQL Server and Simego don’t sell it as a replacement.   The fact that several of the data providers included currently aren’t in Integration Services could mean it pays its way quite quickly (as well as the two mentioned above I could also see it being valuable for Exchange data and for Microsoft CRM for example).   It’s also possible to create your own data providers and make these available as add-ins.   Simego provide some example add-in providers showing how this might be done, via a C# project.

You should be aware though that this isn’t an end-user tool.  It’s really aimed at developers with some coding experience, someone happier with developer tools such as Visual Studio, Integration Services or Business Intelligence Developer Studio.   Also, although the application comes with a help file and lots of online screenshots, it did take me a little while to understand the user interface and perhaps one or two additional “walk-through” sessions in the help file would be useful.   Having said that, you might also find what you are looking for in these blog posts.

Overall then my evaluation experience was a positive one.  It’s also good to see a home-grown (UK) development company coming up with a tool like this (and even spelling synchronisation without a “z”!).

1. If you are allowing anonymous comments on your SharePoint blog, it’s much more difficult to manage these than it is in WordPress, so you need to try fiddling around with the comments list columns and probably want to add something extra to weed out spam bots.  It’s also not ideal if you want to capture simple non-intrusive information about who is commenting.  You can see lots of examples of this among SharePoint bloggers (take a look for instance at the comments entry form for any of the posts on the current SharePoint Product Group blog).
2. The Akismet spam filter that WordPress provide does a pretty good job of catching spammers without any intervention from me.
3. I noticed that even FPWeb bloggers host their blogs on WordPress rather than using SharePoint !

One thing that I (and several other WordPress bloggers) have noticed recently is the appearance of automatically generated links which click through via viglink.com.   From what I can see, these are being generated by WordPress themselves as a way of tracking (and somehow monetising?) popular links, but would be interested to hear if anyone knows more about this.

I’m also keen to have a change from the current WordPress theme (Contempt at the moment), so if anyone has a suggestion for a replacement which looks good with the WorkerThread colour scheme let me know.

So – I have some new posts in progress and these will be appearing here shortly…

One of my favourite places for interesting dataset downloads is the UK Guardian’s Datablog.  They have everything from Expenses submitted by Members of Parliament to Bafta Nominations.  Recently I noticed this post about the data collected London Bicycle Hire Scheme.  Transport for London has provided a downloadable file of all bike hires during the first 5 months of the scheme.

If you take a look on the Guardian site you will see that there have already been some interesting visualisations done using the data, like this one on Tableau Public.  The source download contains over 1.4 million rows and for each ride you can see starting and finishing location as well as dates and times, which bike was used and  duration of the hire.  So I thought, how well would this work in  PowerPivot in Excel?

I downloaded the full ZIP file of data – over 156mb text file once unzipped, with a separate file describing the column names.  If you want the column titles in place before loading into PowerPivot (so they can be used as is without the need for any renaming) you will need to use a text editor capable of loading the source Raw Data file.  Something like NotePad++ works well here.

Firing up Excel and the PowerPivot window, first step is to get the data loaded.  The data has a tab character Column Separator and if you have added the column headers into the top row make sure you checked the box – otherwise just rename the columns once imported -

Worth bearing in mind of course that 1.4 million rows is too big to load into Excel directly where you have a row limit of 1 million rows.  But 1.4 million rows is fairly trivial for PowerPivot, which can handle much higher data volumes.

First thing I like to do once the data has been imported is take a roam around it to understand what’s on offer and whether any adjustments or cleaning needs to happen.  You will see that the start and end time columns need to have their format changed.  On load they show as dates (30/12/1899) and we need to set them to a time format to make them useful.

You will also see Start and Finish Station formats, such as “Queen Street,Bank”.  I thought it might be more interesting if the start and finish stations could be split into the area and individual bike docking station.  Area and docking station are in most (but not all) cases separated with a comma.  So we can create calculated columns to split up the data to left or right of the comma with the addition of some conditional logic to handle any data which doesn’t  contain a comma.  Most of this works pretty much the same way as it in Excel and there is a good article on splitting data into separate columns here.

I’m already getting some useful data, but lets take things a little further -

We have Start and End dates for each ride, so we can follow PowerPivot best Practice and create Date dimensions for each, as linked Excel tables.  You will find some very useful articles online on how to do this – I would particularly recommend this excerpt from the invaluable book by Marco Russo and Alberto Ferrari (or buy the actual book – well worth it).  Once I have my date dimension joined to the source data I can easily get to day of the week, calculate weekdays or weekends etc and if I want I can use DAX Time Intelligence Functions.  I also created calculated columns to show for hour of the day between 0 and 23 and a simple AM or PM column based on that hour.

Next I took a look at adding a linked table which could be used to “band” the journey times.  The table I created in Excel looked like this:

and I used the CALCULATE and FILTER functions described here to create a banding column in my data.

Interesting point to note here – take a look at the size of my saved Excel file containing the loaded PowerPivot data and compare it to the size of the source text file I loaded:

43,885 KB in PowerPivot compared to 159,653 KB for the original text file!

Now I can start exploring the data by creating some pivot tables and charts.  This Pivot Chart shows total bike hires by hour of the day with a “starting area” slicer and the table below shows the area in which the bike hires ended up, also linked to the same filter:

And this one uses the calculated “banding” column to group journey times by month and starting location:

At this point you can easily find out a lot more – average journey times, which bikes got used the most, maybe some “ratio to parent” calculations to see which docking stations are used most in each area.

To sum up, for me this reinforced how quickly you can get some interesting analyses out of large data sets with one of the most widely used desktop tools out there.  If you are interested in trying this for yourself the first thing I would suggest is getting hold of this book (or if you are not a technical book person, maybe get yourself booked on one of the PowerPivot Workshops).

If you try working with the bike data yourself, either in PowerPivot or one of the other analysis or visualisation tools leave a comment on the blog.   And hopefully Transport for London will make even more data available – maybe I can try a full year’s worth next time.

The RSS Viewer web part is a good way of adding interesting content to SharePoint site home pages.  There are lots of good quality RSS feeds out there – for example BBC News, various Microsoft Technical resources and many diverse trade publications (or even this blog).   It’s pretty easy to set up once you have chosen a useful feed.  Here’s an example which has been configured to use the TechNet News feed:

Clicking on an article heading will reveal some more detail:

Then if you then click on the “More…” link, you will open the source article in the current window.  Now, I’ve found this is not always ideal and you often want to open the article in a new window, without leaving your SharePoint site page.  To make this happen, it’s necessary to make a change to the “More…” hyperlink.  You do this by editing the web part and clicking on the XSL Editor button to open the text editor:

This exposes the XSL styles which define how the RSS feed is displayed (if you’re looking for XSL syntax info by the way, take a look online for articles by Marc D. Anderson – he’s published some excellent stuff).

One particular line is of interest to us.  It’s in the section “RSSMainTemplate.description” and is the line defining the “href” tag to open the article when clicking on “More…”:

<a href=”{ddwrt:EnsureAllowedProtocol(string(atom2:link/@href))}”>More…</a>

To make the article open in a new window, you need to add a target=”_blank” attribute like so:

<a href=”{ddwrt:EnsureAllowedProtocol(string(atom2:link/@href))}” target=”_blank”>More…</a>

(by the way I prefer to copy the entire XSL into a better text editor to make my changes, then paste the whole lot back into the web part XSL Editor afterwards rather than working in the web part editor)

Once you’ve done that, apply your changes and you will find that your RSS feed articles will open in a new window when you click on the More… link.

And so to My Question -

In some circumstances I’ve noticed there can sometimes be problems with the RSS Viewer web part loading correctly – it seems to get stuck in “Loading” mode without ever completing.  I’ve checked around to see if anyone else is experiencing this with SharePoint 2010 but haven’t found any definitive information.  There’s a school of thought that this could occur when Alternate Access Mappings have been set up for the web application, although I’m not sure why this would be.  So if anyone can shed light on this please leave a comment!

I’ve been spending a good bit of time with PowerPivot in recent months, primarily as part of a proof of concept exercise for a client.  Like many others I have to say I started looking at it with a high degree of scepticism.  Surely I thought, this thing goes against all of the principles we’ve been trying to drum into users, particularly the importance of building a well designed Unified Dimensional Model with calculated members in our Analysis Service cube – then we get that “one version of the truth” that we’re all after when we build a Business Intelligence solution.  PowerPivot on the other hand is a free add-in for Microsoft Excel 2010 which lets you load very large amounts of data into memory work and with it in a new desktop version of Analysis Services called VertiPaq. So you build your models, including all the calculations, inside of Excel.

The thing is, the big majority of analysts who need to pull together and work with large sets of data often need to do so in response to rapidly changing requirements (and frequently, changing data sources).  And of course they spend most of their day using Microsoft Excel.  Show PowerPivot to someone who spends their time creating complicated Excel VLookups to build reports and pivot tables against data loaded from database extracts and you will quickly see their eyes light up.  And in all of the time I’ve spent working with Microsoft Analysis Services (since the first release in 1998) I’ve never seen as much excitement online about a BI tool as I’ve seen recently for PowerPivot.  Do a search on twitter for the #Powerpivot hashtag to see what I mean.

PowerPivot in Excel 2010

So if you have Excel 2010 you can download the PowerPivot add-in from Microsoft’s PowerPivot site here.  Note that there are two different versions of the download, one each for the 32-bit and 64-bit versions of Office 2010.  Right now most users are likely to be running the 32-bit version I suspect.  There are very good reasons for this – if you install 64-bit Office 2010 you may have problems with existing add-ins which are 32-bit only, and there is also an issue with SharePoint datasheet views with 64-bit Office.  On the other hand, if you are planning to use PowerPivot with very large data sets you will almost certainly need to install the 64-bit version of Office 2010 – as it says in this TechNet Installation Note – “the 64-bit version of PowerPivot enables you to work with up to 4GB of data in memory, and the 32-bit version enables you to work with up to 2GB of data in memory”.  There is some guidance from Microsoft on 64-bit Office 2010 usage here.

Once you are up and running, you’re going to be impressed with the performance you get, even on large datasets.  Data can be loaded into PowerPivot from a good variety of sources – SQL Server, Analysis Service cubes, other OLEDB/ODBC data sources, text files, Excel data, also interestingly SQL Server Reporting Services, Atom data feeds and the Azure DataMarket.  Again, this is where lightbulbs start coming on over Excel Users’ heads because PowerPivot lets you create joins between disparate data sources.  So providing there is a matching column, it’s quite easy to create a relationship between line of business data, maybe residing on SQL Server, and other data you might have acquired from an online service such as Azure DataMarket.

To get the best out of PowerPivot, you need to learn how to use Data Analysis Expressions (or DAX for short).  One of the reasons Excel users are likely to warm to PowerPivot more than they have to Analysis Services cubes is that DAX calculations are closer to the type of functions they may already be using in Excel.  That’s not to say DAX doesn’t also require some effort to learn, it definitely does if you want to master it.  DAX has some great date calculation features built-in by the way, so very good for year-to-date, or same-period-last-year type reports.

So what are the downsides when you’re using PowerPivot with Excel?  For me the same issues have cropped up whenever I’ve been shown it to clients.  First, there is no easy way to automate the refreshing of your data from within Excel.  Right now the only option you have is to open the PowerPivot window and manually refresh one or more of your connections.  Because PowerPivot is a separate add-in you can’t automate it via VBA right now either.  If you publish your model to PowerPivot on SharePoint 2010 then you can get the server to schedule refreshes for you but if you don’t have this luxury then right now manual refresh is the only option.  There is an interesting post from Ken Puls on some of the bits he would like in the next release.

Something else that concerns me slightly is the need for governance when defining and using complex DAX calculations in Excel models.  As I’ve already said, a key benefit of the dimensional model, with all of your custom calculations defined in the cube, is that there is a single definition.  So if your company has a complex ratio calculation, or way of defining utilisation percentages or whatever then the possibility that two different analysts use different calculations (and arrive at different results) is significantly reduced.  DAX formulae can get pretty complex so ideally you would want to store “templated” definitions somewhere.   Over on PowerPivot Pro there is an interesting post on how you might use NotePad++ to create and troubleshoot the more complicated formulae.  Perhaps something similar could be used as a reference guide for sharing complex DAX calculations.

PowerPivot in SharePoint 2010

So as I’ve mentioned above, as well as being an add-in for Excel 2010, PowerPivot is also an option for SharePoint 2010 in conjunction with SQL Server 2008 R2.  Once you have PowerPivot running on a server in your SharePoint 2010 farm you can publish your models to “galleries” and your users can view your models in their browser, without even the need for Excel.  Again, a lot of work has gone into the SharePoint bits here, making good use of SilverLight to browse through published documents.  SharePoint administrators can also monitor usage of each model, identifying resource utilisation, which items have been opened most frequently and so on.  This would then be used to see which PowerPivot models might be suitable candidates for re-development as a true dimensional model in an Analysis Services cube.

PowerPivot on SharePoint 2010 can be an expensive proposition though.  You will need SQL Server 2008 R2 Enterprise Edition on the PowerPivot Server.  Your SharePoint 2010 farm will also need Enterprise Client Access licences.  And the server running PowerPivot will need to be pretty substantial (lots of memory, multi-core etc).  Here’s an MSDN document on recommended configurations.  I know we keep hearing the phrase “BI for the Masses” but these masses need to have pretty deep pockets.  Also note that the PowerPivot Analysis Services “VertiPaq” engine will be a separate instance to any existing traditional Analysis Services instance you might be running, almost certainly requiring its own server.

Getting Started

Here is some recommended online reading to get you started:

Loads of useful information on the PowerPivotPro and PowerPivot FAQ sites

Marco Russo’s blog (one of the authors of the book recommendation below)

The DAX articles on Chris Webb’s BI blog

Book Recommendations

If you want an in-depth understanding of how to use PowerPivot and DAX in Excel 2010 I would wholeheartedly recommend PowerPivot for Excel 2010 by Marco Russo and Alberto Ferrari.  Excellent explanations of how it all works, best practices and lots of examples.  You can download sample chapters here.

If you are interested in the SharePoint 2010 side of things, take a look at Professional PowerPivot for Excel and SharePoint which has useful information the PowerPivot architecture and how it’s all put together in a SharePoint farm.

Summing Up

This is already a long post and I haven’t even mentioned the next release of SQL Server (currently code named Denali but my spell checker keeps suggesting “denial”).  From the information recently released understanding how to use PowerPivot is going to stand you in good stead when the new release is available.  Here’s a good overview of the “Denali” roadmap.  I’ll try to follow up this post with news about Denali BI features when more information is available. 

For my own part I’m very excited about the possibilities that PowerPivot opens up.  Given that we are only in version 1 most of what I’ve seen is very impressive and I would definitely recommend taking it for a test drive.  And your Excel “power users” will definitely like what they see.

Let me know what you think – can you see a use for PowerPivot in your organisation?  Is “self service BI” a step in the right direction?

Users of Microsoft Project will be aware that you can easily create summary tasks to act as a container which groups sub-tasks together.  On the Project 2010 ribbon you will see “indent” and “outdent” options – the picture below shows “Indent Task” which will indent selected tasks and make them subtasks of the nearest preceding task at a higher level:

You can see this in action in the sample project below, where the three tasks concerned with mobile development have been indented and made subtasks of Phase 2. 

So, what happens when we sync this with our SharePoint 2010 Project Tasks –

As you can see, I have a new Summary task item for Phase 2, and if  click on the hyperlink to drill down further, I get to see the subtasks:

This is all made possible because a SharePoint 2010 Project Task list actually includes a second “summary task” content type:

Which is actually a folder with appropriate project task metadata – and Project 2010 uses this when synchronising.  Here is list view of the same project task list, which shows the Phase 2 folder more clearly:

So there you go Christophe – and thanks for all your efforts on PathToSharePoint!

In my previous post I wrote about the Project Task List in SharePoint 2010.   Using it in conjunction with Microsoft Project 2010 makes it even more useful.  For example, you can use the nice Project 2010 features to create a project, add resources, set predecessors etc (all standard Microsoft Project stuff), then use a new feature you can find under the “Save and Send” option on the File menu:

From here you can either sync with an existing SharePoint task list, or create a new task list in the site you specify.  It’s also possible to start from a SharePoint Task list and create your project from there, using this option on the “New” menu:

The synchronisation process will also assign resources, as long as these already exist in your SharePoint user list and in the same format.  It’s important to note that the data in your SharePoint Task List is Manually scheduled.  In Microsoft Project 2010 you will see two scheduling options on the ribbon:

Auto Schedule:

And Manually Schedule:

Now, it’s quite likely that if you start by creating your project with Project 2010, everything will be Auto-Scheduled, because that means that when durations or actual completion dates change, other dependent items in the project will be re-scheduled.  If you are auto-scheduling, then when you sync with the SharePoint list, you will see this message:

which is telling you that everything will be switched to manual scheduling when sync’d.  So after a sync you will need to re-set auto-scheduling to see the impact of any updated tasks (by the way, if there are any synchronisation conflicts you will see a conflict resolution dialog where you can decide whether you keep the SharePoint version or the Project 2010 version).

Project 2010 also provides a “Manage Fields” option:

where we can add other Microsoft Project fields to our SharePoint list like this:

And once added you will see them as new columns in the SharePoint list:

This is ideal if you want to use other features in SharePoint to provide current project status information, perhaps on a dashboard where you might use SharePoint charts or KPIs.  Once you are syncing between Project and SharePoint you might also want to place other restrictions on the SharePoint list for standard users – perhaps only allow data in specific columns to be amended, or make sure only project owners or administrators can add or delete tasks.   The new InfoPath 2010 SharePoint list form capabilities might come in handy here.

In summary, if you are planning to use SharePoint 2010 Project Task lists, then getting a copy of Project 2010 for your project owners or managers is likely to be a good investment.  It will allow you to publish and maintain project tasks to your assigned resources, who can easily update status and completion dates from the SharePoint site without the need to provide Project 2010 licences for everyone else involved.

I’ve also started looking at the new Silverlight SharePoint Gantt Chart from IntelliGantt and how it might work in this scenario.  I’ll post more about this once I’ve tried it out on some sample project sites.

It’s been a good while since the last post here – I’ve been really busy on client projects, as well as trying to learn as much as possible about SharePoint 2010 now that the released product is available.  I thought I would try to cover some of the things I’ve found out while working with the 2010 SharePoint and Office products in this and future posts, kicking off with how Project Tasks now look and behave.

The Project Tasks List in SharePoint 2007 / WSS 3.0

One of the standard list templates in both SharePoint 2007 and WSS 3.0 is called “Project Tasks”.  You can find it in the “Tracking” section when creating a new list:

It comes with a default “Gantt View” already set up, so that when you add entries to the task list, you get to see them like this:

When I show this to clients new to SharePoint they get very enthusiastic about managing project tasks via a web application.  Then they ask about task dependencies – in other words can the tasks be linked so that the Gantt shows how each one is linked to its predecessor or successor tasks – same as you would be able to create if you were using Microsoft Project.  Unfortunately SharePoint 2007 Project Tasks can’t do this “out of the box”.  There are several 3rd party options available, such as SharePoint Task Master from Bamboo Solutions and of course Microsoft’s own Project Server.

Improvements in SharePoint 2010

The Project Tasks list is still there in SharePoint 2010 (all versions, including Foundation) and in the “Tracking” you will see this icon:

In 2010, the first thing you will notice once you create a new list from this template is that the Gantt view has changed so it now looks a lot more like the one you would see with Microsoft Project:

And as you’ll notice from the screenshot above, it’s possible to define predecessors for tasks when adding or editing them:

So far so good, but there are a few other things you need to know.  First, updating one of the tasks does not automatically re-schedule its predecessors.  You can consider a SharePoint 2010 Project Tasks list as using manual scheduling, not the clever auto-scheduling you get in Microsoft Project.  Here you can see the duration of the “Define Scope” task has been extended, but nothing happens to the start date of successor tasks:

The other point to bear in mind is that it’s possible to set as task as it’s own predecessor, without SharePoint giving you any kind of error:

Then you can end up with a task that looks like this on the Gantt

This is something that isn’t permitted in full-blown Microsoft Project, where you will get an error like this if you try it:

So overall some nice improvements. It doesn’t stop there though.  Microsoft Project 2010 adds more – namely the ability to synchronise with your SharePoint list.  More on this in part 2….