I’m back fresh from Seattle where I presented a couple of sessions at SharePoint Fest Seattle and, both in sessions as well as casual conversation, people were talking a lot about “hybrid.” And with good reason. It’s well understood that many customers will not be able to move their SharePoint deployments fully to Office 365. For some, there are compliance regulations or concerns around the cloud. For others, their deployments may not move easily due to heavy customization or workloads that are not yet available in Office 365.
So what is Hybrid anyway?
A hybrid deployment, as it pertains to SharePoint, is the state of hosting some content and/or services running “on premises” while some other content and/or services are running in Office 365. The messaging from Microsoft is that if you’re a customer that is unable to move everything to Office 365, a hybrid architecture is a great way to take advantage of the latest innovations from Office 365 (Delve and the Office Graph, for example) while remaining on premises.
One of the key scenarios for hybrid deployments has been integrating Search between on-premises and Office 365 environments, and things have just gotten a lot more interesting with recent announcements.
Hybrid Search Historically
Up until now, the hybrid search story has been good, but not great. Essentially, customers with content stored on premises and in Office 365 would manage search for the two environments separately. Then they could tie the two systems together by federating search results such that when searching from on premises, a user could see Office 365 results, and vice versa. This scenario is described in detail in this TechNet article.
The big gaps with the federated model are:
- The results for the two environments are displayed separately.
- The queries and results are evaluated separately.
For number 1 above, this isn’t a deal-breaker, but it’s not an ideal user experience. For example, if I perform a search on premises, my results for Office 365 would appear either in a result block or on a separate page in the search center. In some cases, this could be desired, but it offers little flexibility as it’s the only option.
For number 2 above, the searches are not integrated at all – the architecture simply allows you to display a second set of search results. So if your organization has taken the time to really tune your search results and you care about search ranking and relevancy, there’s no way to apply a single rank profile across all of your results. They’re each evaluated separately. This may not be a big deal for some, but for customers who have invested heavily in search, this could easily be a deal breaker.
What’s new with cloud hybrid search?
Microsoft Senior Technical Product Manager Bill Baer was the keynote speaker at the SharePoint Fest Seattle, and one of the exciting parts of his talk was the announcement that a preview version of the cloud hybrid search feature has been released for SharePoint 2013.
Cloud hybrid search is a new architecture where an organization’s on premises content can be crawled and stored as part of its Office 365 search index – thereby creating a single, unified index which contains both on-premises and Office 365 search results, along with any external content that the organization may be crawling such as file shares or line-of-business applications. As a long-time search nerd, this is very exciting news to me! The diagram below describes what the scenario looks like at a high level.
This new scenario is made possible primarily by the introduction of a new on-premises search service application called the Cloud Search Shared Service Application (SSA). Think of the Cloud Search SSA as the “crawling” half of a search engine. Instead of processing and indexing the crawled content like normal, the Cloud Search SSA pushes that crawled information up to Office 365 to be indexed along with the native Office 365 content.
Once the Cloud Search SSA has done its job and you have the unified index living in Office 365, users searching from Office 365 get search results from both on-premises and Office 365! You can also optionally configure your on-premises environment to serve up all of your search results from Office 365 so that when a user performs a search from the on-premises environment, O365 will respond and provide all the search results from both on premises and O365. So, instead of seeing results separated (again, by result blocks or separate pages), users will see a single set of results comprised of both on-premises and Office 365 content which has been evaluated, ranked, and sorted as a single set.
How do I set up cloud hybrid search?
Here are the necessary steps to follow in order to set up cloud hybrid search in your environment:
- Use Active Directory to synchronize user accounts between your on-premises and Office 365 environments.
- Spin up a new Cloud Search SSA from your on-premises SharePoint 2013 farm.
- Configure server-to-server trust between your on-premises environment and your Office 365 tenant.
- Crawl some on-premises content using the new Cloud Search SSA.
- Optionally, configure your on-premises environment to utilize the Office 365 search index.
For more detailed instructions on how to do this, please visit Microsoft’s site.
What about Delve and the Office Graph?
Earlier in the blog I stated that Hybrid architectures were all about providing cloud capabilities in an on-premises farm. So does this mean you now get Delve and the Office Graph capabilities for your on-premises environments? Yes! Well, sort of.
You will not be able to access Delve in your on-premises environments. However, when you access Delve from Office 365, you will see your on-premises content there! There’s a catch, though. The on-premises content will not be displayed automatically as a content card the way Office 365 content is. The on-premises content will only be available when a manual search is executed from within the Delve interface.
Key Considerations
For customers who are unable to move to Office 365 due to compliance concerns, cloud hybrid search may not be an option. Because the on-premises content (or an indexed representation of it) is being sent to Office 365, this may be an unacceptable scenario. Microsoft does state in its documentation that the “content metadata is encrypted when it’s transferred to the search index in Office 365”. However, I encourage customers with concerns about this scenario to work with their Microsoft account managers to understand this in more detail to make sure it satisfies whatever standards must be met by their organizations.
In addition, there are a number of limitations, such as the inability to support connectors that require custom security trimming as well as lack of support for some advanced on-premises features such as Custom Entity Extraction and use of the Content Enrichment web service. Please refer to Microsoft’s documentation for a complete list of unsupported features and scenarios.
Key Benefits
The following is a summary of key benefits offered by the new cloud hybrid search feature:
- No more separated search results in hybrid environments
- You’re always accessing the latest and greatest search platform for your environment, because Office 365 is automatically updated with no customer intervention or upgrades required
- Find on-premises content in Delve
- Retire the disk space required to store your on-premises SharePoint search index, which can bring about a major savings in overall cost of ownership
Availability
The following is a summary of when and how cloud hybrid search is available to customers:
- This new capability is available as a preview in SharePoint 2013 via the August Cumulative Update (CU)
- It is also available in the SharePoint 2016 Preview which was released on Monday, August 24.
- Though you can install this in your on-premises environments, as stated in Microsoft’s documentation, the results will not start showing up in Office 365 until September 7, 2015
- Again, this is in preview only at this point, so now is the time to “kick the tires,” but don’t deploy to your production environments just yet
Wrap-Up
I hope this has been a helpful introduction to the new cloud hybrid search scenario. I, for one, am thrilled about the promise of this unified set of search results across my on-premises and Office 365 environments! I’ll keep the community updated as I put this scenario through its paces! For more information, please see the following resources:
- Microsoft documentation: The full documentation with everything you need to know as well as set-up scripts.
- Ignite Session on Next Gen Hybrid Search: This on-demand session from Microsoft Ignite 2015 goes into a lot of detail on how the new Cloud Search SSA works.
Happy crawling! Have a question about cloud hybrid search? Leave a comment on this blog post and we’ll be happy to answer!
This will be a great feature improvement for hybrid deployments, thanks Paul for your very informative analysis !
Thanks for your comment Derek. I agree! This is an update I’ve been waiting on for a long time 🙂
I really enjoyed this explanation and roundup of new hybrid SharePoint functionality. I follow this with great interest because I believe the future of the hybrid model is going to rely on architectures such as these that take full advantage of the strengths of on-prem and cloud to jointly host these workloads. My follow up question is related to performance. Separating the crawl makes sense, and taking advantage of the storage and compute power of the cloud makes sense, but are there any concerns with network performance in both the indexing and query response from this model?
Edmund, these are good and very important questions!
In general, Office 365 customers need to go into deployments with their eyes wide open with regard to performance. I’ve seen a number of customers assume that because their content is being hosted in O365 that performance is Microsoft’s concern, only to realize later that their network is insufficient. Microsoft provides some good resources for organizations to prepare themselves, for example the Network Planning and Performance Tuning guide. This Office Mechanics video about Optimizing Network Connectivity with O365 is also definitely worth a look.
Regarding crawling and querying over significant distances, yes that’s always a concern and an architectural challenge I’ve faced many times in global search deployments. In general you want the crawl to be local, which is what Microsoft has implemented here (we must remember crawling is different than indexing even though they’re often referred to interchangeably). And assuming your concern about performance has been mitigated, the remote queries should perform just fine.
Thanks for the questions!