It is no secret that Microsoft is making a big push for Office 365 and SharePoint Online. I’m all for it for many organizations. The way I look at it, I see many of my customers spending 70 percent of their calories “running” SharePoint and only 30 percent on getting return on investment (ROI), building meaningful solutions for users, proactively reaching out, and instigating engagement. If we can flip those numbers and spend the time often spent managing the platform for the latter activities, I’m on board. I know it’s not possible for everyone to migrate yet, but we’re getting closer to full parity between on premises and SharePoint Online, and the security and compliance story offered by Office 365 gets continually better as time goes by.
[RegUserOnly]I recently had a customer sign up for Office 365, ready to abandon their on premises SharePoint farm only to be shocked when she read the following statement in the TechNet guide Prepare to Migrate to SharePoint Online: Office 365 does not provide SharePoint content migration support for customers.
Though it may sound shocking at first, the only way this really differs from SharePoint on premises is that in SharePoint Online there is no database attach option. I assured her that we could get her content into their new tenant, but that it just requires some planning and extra considerations.
This blog is a response to that interaction, and is meant to be:
- a project approach/roadmap, based on experiences in the field
- an introduction of primary considerations
- an introduction of common pain points
- designed to give you the confidence to approach an endeavor of this sort
This blog is not:
- a technical discussion about setting up and configuring SharePoint Online
- a specific recommendation for your environment
Tactical (the easy part?)
As is typical with SharePoint, the technical part is the easy part (and it’s usually not that easy). A migration to SharePoint Online is no different. The tactical and technical part of this (decision-wise) is pretty easy, though. You have roughly three choices:
- Manually upload your content into SharePoint Online
- Design and develop a custom tool to automate the migration
- Purchase commercial off the shelf (COTS) migration software
Of course there are pros and cons to each approach. Here are some of the main considerations:
I’ve used each of these methods for migrations in the past. The decision will come down to the requirements. Is metadata such as the “date” and “modified by” stamps important? Then manual is not an option as you will lose those with this method. But if it’s not necessary for you to retain this information and you have a small amount of content, moving your data manually could work just fine.
If you have a specialized way you need to perform the migration, a custom tool could make sense as long as you have access to quality developers and the budget for the project.
Commercial software would certainly be the easiest option and will boast the widest range of features, but it comes with a cost, of course.
As you plan, be sure to look hard at your requirements, budget, and access to labor. The decision should become clear, but that decision is only the beginning!
Why are migrations hard?
As a consultant, migration projects can be some of the most challenging. They are notoriously underestimated and usually more complex than initially thought. They require a great deal of process in place for training, transitioning, communicating with the business, and user acceptance testing (UAT). In a way, migration projects are a collection of lots of smaller projects all happening at the same time.
When people talk about migration, they are often only thinking about a small part of the entire project and not all of the steps that need to take place before and after the data is moved. Migration is much more than what you see on the surface. There are typically many different facets to a migration lurking beneath the surface of the project that we don’t fully understand until we start digging around.
What’s so special about SharePoint Online?
Migrating to SharePoint Online is still a SharePoint migration project. Many (maybe most) of the project tasks do not change. But I will concede that there are definitely some special considerations to be taken. For example:
- No native content migration support. As mentioned before.
- Customizations that are in your current, on premises environment may not easily refactor to SharePoint Online. For example, they may include workloads that aren’t available, use full-trust code, or require access to the file system – none of which are allowed in SharePoint Online.
- Not all workloads are available. If your SharePoint users need to leverage a particular feature or workload that’s only available on premises (PerformancePoint Services, for example), an alternate solution may need to be found.
- You must understand compliance concerns. There can be compliance concerns for certain types of data in the cloud, especially a multi-tenant host. Microsoft is adding new levels of rigor around this daily, but it’s still something that is likely to come up, and time and attention will need to be planned for in this area.
- Content migration speed. The speed to actually move content from point A to point B may just take longer. This could impact how you schedule your migration.
- Will you need a development, testing, or staging tenant?
- Will you end up with a hybrid architecture? How will you manage that?
- Should you first stage on premises before doing a final migration up to SharePoint Online?
What Roles are required?
As I mentioned earlier, migrations are complex, and most will have many moving parts with various required roles. I’ve listed some of the typical roles below.
- Executive Sponsor – This effort is going to require time from users. You need support from the top of the organization to get their time. Also, we never really want SharePoint projects to be IT led, but rather IT facilitated. If you’re working with a company that tracks billable hours, you may need a cost center to bill their hours against. This is going to require executive sponsorship.
- Project Manager – This is obvious. You need a really strong one, as there are lots of moving parts to this type of project. Development, design, interaction with users, as well as lots of interviews and meetings will be required throughout the project.
- Business Analyst – This is probably the most important role of this project, in my opinion. This person will help with understanding what content there is and what can be archived. They will also help understand how users use their existing systems (so that these tasks can be mapped to SharePoint Online) and what improvements can be made along with the migration.
- Information Architect – On the non-technical side, this person is responsible for creating a logical site map. On the technical side, the individual needs to take that logical site map and figure out what infrastructure is required behind the scenes to make it all happen.
- Communications plan, change management, evangelism. These are all key to having a successful outcome from the users’ standpoint. Even if we move the content over properly, if we fail in the adoption of the new system, the project as a whole has failed.
Note, these are all roles not people. As you know, we tend to wear multiple hats in projects, and the individuals who fill one or more of the roles will depend on what is available at your specific organization.
With that background out of the way, let us walk through the primary steps of the processes to migrate content to SharePoint Online.
Four Steps to Migrate Content to SharePoint Online
This is certainly a “boiled down” view of the steps – there are obviously more tasks if we want to be more granular, but these tend to be the fundamental high-level steps required.
- Content analysis
- Establishing Requirements
- Metadata and Structure
It’s worth pointing out that these don’t necessarily have to be completed linearly either. Certainly, you can start establishing requirements from the outset and that can overlap with the content analyses effort. With that said, let us drill into each of these a bit.
Step 1: Content Analysis
This is really systems, content, people and behavior analysis – but that’s not very catchy.
For starters, understanding the number of content owners is extremely important. Even with a lot of content, if you have a small number of content owners, things can usually move relatively fast. Whereas if you have a smaller amount of content, but many content owners, it will often slow you down. That’s due to the number of user interviews required. Spinning up new conversations with each set of content owners can result in a large amount of overhead throughout your project.
Below are some key questions to help determine what the level of effort will be.
To be honest, it’s difficult to impossible to estimate the effort required for a migration until you’ve done at least some of this. Customers often do not like to hear that, but it’s true. If I’m pressed to make an estimate, I try to break the project up into a discovery and then migration. At end of discovery, we can provide a real estimate. If not, I’ll just be making wild assumptions. For example, I’ll assume we’re using a migration tool, there’s no restructuring involved, no improvements to the functionality – just moving from point A to point B.
The output of the content analysis will likely be a spreadsheet (or group of spreadsheets) that will be used to architect the migration. The spreadsheets typically include fields similar to these – the most important of which are bolded:
- Page Name
- Page Link
- Quality/structure issues
- Source content location
- Modified by: Username
- Point of contact
- Additional approvers
- Status (Keep, Update, Obsolete/Delete)
- Restricted access
- Destination in New Site
- Template (in new site)
- New Page Type
Some of the common challenges you may face in this phase are:
- being unable to find “owners” of certain content
- the sheer volume of content can be overwhelming and time consuming to address
- it’s sometimes difficult to get the time and attention of business users
- depending on what system you’re migrating from and what tools you have handy, automated reporting can be difficult or impossible to attain
With the content analysis done, we’re now ready to move onto requirement gathering.
Step 2: Establishing Requirements
Where do requirements come from? In my experience, they should come from many sources within the organization. A good start is the following:
From the business, you’re likely to hear about emerging requirements from leadership. Things like:
- “greater ability to disseminate information to users”
- “address compliance concerns”
- “we need a new HR information management system”
IT will likely be focused on aspects like:
- Manageability concerns
- Site provisioning
- User management
- Need to consolidate systems
- Compliance responsibilities
- Service Level Agreements (SLAs)
Note: IT should only have technical requirements, as it is not IT’s job to guess what users want. Business requirements should come from the business and the users themselves.
This is where the majority of the requirements will likely come from. Expect to hear:
- How users interact with data
- Most urgent needs
- What already works well
- What doesn’t work well
- What will bring business to a halt if it’s changed throughout the migration
These are some of the groups we tend to invite to the user community meetings. Yours will vary by vertical (legal, hospitality, and oil/gas each have their own groups), but this is a starting point.
- Human Resources
- Corporate Communications
- Records Management
An excellent guide for running the interviews can be found in the “Sample Stakeholder Interview Guide” compiled by SharePoint consultant Susan Hanley.
Understanding the systems themselves can provide requirements. For example, if the system that we’re migrating from can do one particular thing, maybe the new system can too.
The content itself will provide us with clues for requirements, too. Factors such as the following will influence requirements in the new SharePoint Online site:
- Current metadata
- Types of content (pages, documents, lists, “rooms”, “spaces”, etc.)
- Content size, structure
- Application-like solutions
Having access to reporting tools and their output is very important. They can help us determine the areas of heaviest traffic, where search queries were failing, what types of errors came up and much more. These can be great influencers for the next system design.
From Interviews to Requirements
The process of getting from interviews to requirements looks something like this:
- We conduct our interviews and take copious notes.
- A Business Analyst will pour over the notes from the user interviews and extract the themes.
- Some will be feature requests (“I wish we had”)
- Some are challenges and pain points (e.g. “Search doesn’t work”, “I know there are compliance rules, but I don’t know what they are”, “Too much tagging”, “It takes 10 clicks to do things I do twice a day”)
- The things we hear most frequently (balanced with what executive sponsors want) then become our priorities
- Map those priorities to SharePoint Online
- Aspects that map cleanly become requirements.
- Others go into a gap analysis, and we either purchase a tool, build something, or push off to a future phase of the project.
Step 3: Metadata & Structure
Through the lens of migration, what we’re really doing in this stage is creating the receiving containers for our migrated content. We need a structure that will receive this new content cleanly and make the system usable and manageable. This really falls under the heading of “Information Architecture.” The primary factors that need to be addressed are:
- Page Architecture
- Site Templates
- Site Structure
There is tons of overlap and dependencies among these topics, and how I usually end up grouping them together is the following:
Page architecture will drive site template design
Page architecture usually comes first and prioritizes content on the pages based on what we heard in the interviews. From there, we will usually create a design mock-up, including a mobile design if applicable. Then I think about the infrastructure (lists, pages, content types, etc.) we need to make everything that’s being inferred on the page happen. Along with content types, security, and everything else, these become my new site templates.
Site structure informs navigation and vice versa
Metadata and search go hand in hand. Let’s be clear about something: It is not IT’s job to guess where things should go! The way to create an intuitive, highly usable site is to make these decisions based on data! Here’s one way to do that:
- Start card sorting and other usability testing. This is how we know where users expect to find things, such as whether users call types of content by different names.
- You’ll end up with data that will tell you empirically that, for example, 80 percent of users expect to find user handbook in a particular place. If that’s the case then, of course, you will want to put it there, or at least add a link to the content there. Now we have a usable experience, and we know that based on data! If you don’t take these kinds of steps, you’re simply guessing.
You can utilize a “natural” out of the box (OTB) navigation, where the sites roll up automatically and the navigation reflects the actual site structure, trimming out anything the current users don’t have access to. In this case, your site hierarchy will be defined somewhat by how you want your navigation to look. So your logical and physical hierarchies are the same. But if you’re going with a different navigation model (manually managed links, metadata driven or custom global navigation) your actual site hierarchy may differ from the logical.
Let’s also recognize that what you see in the navigation doesn’t necessarily reflect what is actually built out in the site architecture. Sometimes they’re the same, but often not.
To my mind, the main considerations for choosing a navigation/site structure model are:
- Ease of management
- SharePoint boundaries and limitations
- Technical constraints (wanting the same navigation across site collections, for example)
It’s worth touching on tagging briefly. Most organizations know that the more uniformly and consistently their content is tagged, the better search can work and the more information they have to leverage for features like workflows and content roll ups. Most also realize that burdening their users with lots of required metadata fields is a sure way to drive them away from the system! Here are a few tips for addressing metadata/tagging as part of the migration project:
- Don’t go crazy with metadata fields
- Every field should have a specific purpose
- Drives a workflow
- Enables aggregation/roll-ups
- Used in search
- Drives content lifecycle
- Every field should have a specific purpose
- Whenever possible, utilize auto-tagging
- Default values (native capability)
- Custom code
- Third party tool
Step 4: TESTING
With SharePoint Online, we still have to conduct testing on the new system, but I think the focus is adjusted slightly as compared to testing for an on premises system. The areas I focus on for SharePoint Online migrations are:
- User Validation
- Is all expected content available?
- Can users can still do their work?
- Do refactored user solutions still work?
- Are workflows working?
- Custom Development
- Performance and load testing not as important with SharePoint Online. For custom solutions, though, testing is very important
- Load test anywhere you’re pushing up against published boundaries (large lists, content size, etc.)
- Perform usability tests – informs navigation changes and changes for future iteration
When it’s not SharePoint from which we’re migrating
You may be migrating from a non-SharePoint source such as LiveLink, eRoom, SiteCore, eDocs, Lotus Notes, or any number of additional systems. In those situations, there are additional considerations to keep in mind, such as:
- Authentication – is the source system’s authentication and security compatible with SharePoint Online’s?
- User mapping – If the source system user IDs differ from those configured in SharePoint Online, how will they map so that Access Control Lists (ACLs) and permissions work properly.
- Container mapping – Does the source system have content containers that map logically to SharePoint’s sites, lists/libraries, etc.?
- Feature mapping – Does the source system include functionality that cannot be addressed with SharePoint Online?
- Leave plenty of time/budget for change management and training – this is a huge part of having a successful outcome!
Don’t forget governance!
Some people may be sick of hearing about governance, but it’s hugely important. A customer once shared with us an analogy comparing governance to the bumpers bowling alleys sometimes put in place for children’s bowling events. In a way, that’s kind of what governance enforcement is: Preventing users from throwing gutter balls.
Here you can utilize much of what you learned from carrying out user interviews and content analysis. It’s highly likely that much of the mess you uncovered in that stage had to do with lack of governance. What did you discover in the content analysis?
- Is there site sprawl?
- Does the organization have a need for a provisioning/decommissioning process?
- Has there been a lack of content accountability?
- Should content owners/stewards be listed on sites (as part of the site templates)?
- Is there a lot of stale outdated content?
- Is there a need for a content lifecycle/archiving mechanism?
- Was any non-compliant content identified?
- Is there a lot of duplicate content?
As with some of the other steps I’ve mentioned, this part of the process should be led by the business. IT can create governance mandates, but the first executive who says “no” can cause the whole plan fall apart.
Make sure the governance is built into training, communications, site templates, and mechanisms for enforcement, otherwise our governance document will be nothing more than a nice binder on the shelf.
I hope this has been useful for those pondering a migration to SharePoint Online. You should now have a good understanding of one process for running a migration project. Please find a number of related resources compiled below.
- SharePoint Online Compliance
- Hybrid Scenarios
- SharePoint Online Boundaries & Limits
Very nice blog post Paul.
Here is a link to my presentation which I created earlier this year for the Office 365 Connect in Harlem http://bit.ly/migrateSharePoint