SharePoint 2013 Custom Search - Part 3 - Display Templates

This is the final part in a three-parter on leveraging SharePoint 2013 custom search and a business solution, this follows directly on from SharePoint 2013 Custom Search - Part 2 - Result Sources and Query and will focus on the display of results.

SharePoint 2013 makes heavy use of display templates throughout the platform. Microsoft have now made it easier than using XSLT (which is a relief!) to display search results and in a similar way to the way the masterpages are controlled, you simply upload a html/javascript version of your display template and SharePoint - via the use of jiggery-pokery and magic - creates the necessary .js file! Awesome!

Ok, so lets get to it! We saw before we 

SharePoint 2013 Custom Search - Part 1 - Managed Properties

This is the first in a three part blog which will focus on leveraging SharePoint 2013 Custom Search in a real world business solution, part one is creating managed properties, part two looks at getting these as part of result sources and querying, and finally part three brings it together with a bit of custom styling using display templates.

OK, so Microsoft have changed the way they have implemented Search in SharePoint 2013 - in quite a massive way if you didn't already know. 

Alongside some of the larger changes, there are a few smaller ones which aren't immediately obvious.
Once upon a time to configure managed properties, one had to venture into the 'Metadata Properties' section in 2010:

The game has changed a little, and this option is now embedded in 'Search Schema' in the 2013 world:

If you've already added managed properties in 2010 I hate to tell you how to suck eggs, but I'll cover it anyway as there are a few more changes, I'll point these out as we go on...

A few ground rules on managed properties and crawling -

Rule 1: You have to map a managed property to a crawled property

Rule 2: A crawled property is not a crawled property until its been crawled (kind of makes sense, but the next rule will clarify)

Rule 3: A crawled property is not registered as crawled unless there is some data against it

Rule 4: You have to run a full crawl before data in a crawl property is crawled

Hopefully that all makes sense, but basically, to add a managed property; create the column in SharePoint, add an item with data against that column, run a full crawl and the crawled property will be visible in central admin.

There will be a few of you reading this know your stuff that will be like 'hang on a minute, why do I need to map a crawled property to managed one in SharePoint 2013?'

Well the answer is - you dont! SharePoint 2013 has added a neat feature which creates a managed property automatically on a full crawl for any columns that contain data. However, I prefer to create mine manually for a few reasons.

Take the following two managed properties:

They both actually crawl the same value and will return the same thing, however the automatically created one (at the top of the screenshot above) you will see has a somewhat messy name appended with OWSTEXT. In addition, if you wanted this property to be searchable (which is extremely likely) you would have to edit the property as its created with only query and retrieve attributes.

So to create a managed property (after the rules above have been followed) choose 'Add New Managed Property' in the resulting screen give it a meaningful name (I would create this without spaces to make this easier to reference in javascript / html / xsl later):

Select the attributes that you wish this property to have:

Map this to a crawled property (you can map this to more than one crawled property!):

You are now able reference the values from crawled properties in your solutions using the managed property name. To read about how to achieve this as part of a business solution take a look at the second part in this series:

SharePoint 2013 Custom Search - Part 2 - Result Sources and Query

Configuring Apps for SharePoint 2013

Let’s take a look at making use of the one of the new features of SP2013, Apps. Whether you’re planning to use downloadable apps across your organisation or a developer creating your own, both require a correctly configured app infrastructure.

MS recommend that you have a separate domain name for Apps that is not a child of the domain in which your servers are sitting. They also say you need to ‘purchase’ a domain name, what they fail to say is this is only applicable if your publishing apps externally. If, like me you’re configuring this within the walls of your corporate network, then simply creating a new domain is possible without making a purchase in the real world.

Important! – This isn’t creating a new domain as you may imagine, because that does sound overkill right? Actually it’s about creating a domain (zone) in DNS. This then needs to have a wildcard alias. This is so your App namespace can eventually be ‘AppName.DomainName’.

Head over to your DNS console and create a new Forward Lookup Zone, this should be of type ‘Primary’ and replicate to all DNS servers in the domain.

Now this is present, we now need a new Wildcard Alias for this zone. Right click on the new domain you just created and choose CNAME, add an * to denote wildcard in the Alias Name section. In FQDN, add in the full URL of your SharePoint site.

OK, were done now for DNS, let move onto to configure SharePoint. We need two service applications created to deal with the App infrastructure, App Management Service Application and Subscription Settings Service Application, and their associated proxies.

As for App management, not really anything out of the ordinary here. Central Admin – Service Applications – New – App Management Service

Give the Service Application a decent name, its own database and app pool and let it create an App Management Service Application Proxy.

When done this will appear in the list of Service Applications

Now for the harder bit, the Subscription Settings Service Application cannot be created in Central Admin. PowerShell only for this one folks.

Open up the SP Management Shell on of your 2013 servers, run the following to set a variable with the desired managed account, obviously his account must be present in managed accounts in central admin – security:

$account = Get-SPManagedAccount "ServiceAccountName" 

OK, now let’s create a new application pool using the account we set earlier and store to a variable.

$appPoolSubSvc = New-SPServiceApplicationPool -Name “Subscription Settings Service Application Pool” -Account $account

And now to bring it all together, creating the service application using the variable we created before, we’re adding the name of the Service Application here and giving it a database name:

$appSubSvc = New-SPSubscriptionSettingsServiceApplication -ApplicationPool $appPoolSubSvc -Name “Subscription Settings Service Application” -DatabaseName SharePointSubscriptionSettingsService

And finally, the proxy for the Settings Service Application, we don’t get the option of giving it a name here.

$proxySubSvc = New-SPSubscriptionSettingsServiceApplicationProxy -ServiceApplication $appSubSvc

Worthwhile heading to Central Admin – Security – Service Accounts and checking that your new Pool is present with the correct managed account attached.

Also we should have our new Service Application and proxy in the Service Application list

Now, head over to Application Management – Manage Services on Server and make sure the ‘App Management Service’ and (this one is harder to spot) the ‘Microsoft SharePoint Foundation Subscription Settings Service’ has been started.

Now give the box a cursory IISRESET

Head to Central Admin - Apps and click on Configure App URL’s

A tell tale sign here that you’ve configured everything correctly! If there is an error, you haven’t!

On this screen we set supply SharePoint with the app domain we created earlier in DNS. Also you need to go for a prefix for the apps. I’ve gone for SP, but you can go for anything you want.

Time to create an App Catalog Site. Choose ‘Manage App Catalog’ from the Apps page in Central Admin.

Change the web application to the one in which you wish to create the site collection, and select ‘Create a new app catalog site’

Hit next and at the next screen you create a site collection just like any other. The only difference being adding the ‘End Users’ i.e. group of people that can see the apps in the store.
You can add AD groups in here or individuals. If this is for all your users, then add ‘Everyone’, that should cover it.

Okay, so now we run into a bit of a gotcha. Although everything is configured correctly, and you can head to the app catalog and add apps. If you attempt to use an app you've added you will get a HTTP 404. This is due to our wildcard domain we created in DNS. The app URL will be correctly produced, with the prefix-appid.domain which in-turn resolves to your SharePoint box but obviously IIS is not recognising this host header and knowing which web application to route it to.

The solution to this is to create an 'empty' web application on port 80 without a host header. This will then receive any unrecognised requests and know what to do with them. Give this a name of something descriptive, I've called mine 'SharePoint App Host', on port 80 and leave the host header blank.

Important! - Make sure the app pool you choose has access to the web application databases you're using for your apps. Otherwise you will get random access issues. So you want to choose the same app pool as your other web applications, or, if creating a new one (as I am in the screenshot below) choose the same security account you used for the other pool..

You'll notice in the screenshot above that I've called the database 'temp' this doesn't matter as when the web application has been created, you can go to Databases - Manage Content Databases and remove this.

Now you should be able to add, use and develop apps!

Using the power of SharePoint 2013 Search in your 2010 environment

So consider the scenario, you have a SharePoint 2010 environment running your corporate intranet, mySites, team sites etc. You want to make the move to 2013. Migration from team sites is going to take some planning and also some time, heartache, sweat and tears. As for the intranet that is simply going to be an absolute pain to move. 
So although migration is on the cards, you want to make the most of the some of the new features 2013 provides. For example the new search, continuous crawl, deep refinement and greater relevancy. Basically the kind of stuff people want from a search engine. Well you can, and on top of that you can do this so your users don't notice a change, except of course for better results!
What we can do is create a trust between your 2010 and 2013 farms, publish your 2013 Search Service Application and consume it in 2010 using the original 2010 Search Center, and here's how...

Within your 2013 Farm Search Administration, first check the default access account.

Back in 2010, this account needs adding into the User Policy to the web application(s) to be crawled with the ‘Full Read’ permission. (It’s worth pointing out here, both farms are on the same domain. Otherwise domain trust needs to be setup and you have to set searchadforests people picker property etc etc, this is a different blog for a different day)

Now add a new Content Source in 2013, here I have called mine ‘SharePoint 2010 Sites’ (you could of course add these URL’s to the existing Local SharePoint Sites content source, but you may want to scope these out later so I recommend separating them.

You can add schedules for full and incremental crawls or enable continuous crawling at this stage, but again you can come back to this later. When your happy with the settings, kick off a full crawl. 

Check in crawl log that you’re getting successes on the newly added content source.

Also check some results are coming back from the new content source when running a search in an SP2013 Search Center.

OK, time to add some trust between the farms. First up we need to dip into Powershell to get some certificates. Now, the way this works is certificates need transferring from the publishing farm to the consuming farm and vice versa, creating a two way trust. There are two certificates in question, the Root Certificate and the Security Token Service (STS) Certificate.
Just to make things a little complicated, both farms need each other’s Root Certificate, however only the publishing farm needs the STS Certificate of the consuming farm. SharePoint does try to explain this in the ‘Manage Trust’ dialogue located in Central Admin – Security.

On an SP2010 server, fire up a SharePoint Powershell window and run the following to grab the Root Certificate (the folder these are copied to needs to be created otherwise the command will error).

$rootCertificate = (Get-SPCertificateAuthority).RootCertificate

$rootCertificate.Export("Cert") | Set-Content C:\Cert\SP2010Root.cer -Encoding byte

And now for the STS Certificate

$stsCertificate = (Get-SPSecurityTokenServiceConfig).LocalLoginProvider.SigningCertificate

$stsCertificate.Export("Cert") | Set-Content C:\Cert\SP2010STS.cer -Encoding byte

Copy these off to somewhere safe on your local machine, or the machine you’re launching your Central Admin sessions from.

Before closing the Powershell window on the 2010 box, we may as well grab the farm GUID while we’re here, all will become clearer later on! Run the following and copy the resulting GUID it provides.

$farmID= Get-SPFarm

$farmID.ID

Head over to an SP2013 box, again fire up Powershell and run the script to get the root certificate, again copy this to your machine. Notice I’ve named these appropriately so we choose the right ones later!

$rootCertificate = (Get-SPCertificateAuthority).RootCertificate

$rootCertificate.Export("Cert") | Set-Content C:\Cert\SP2013Root.cer -Encoding byte

Right, now to apply these certificates in the respective farm’s Central Administration. Let’s get 2010 out the way first. Head to Security – General Security - Manage Trust, we’re going to add a new trust relationship, so yep, you guessed it, hit the new button. We want to give this trust a name, this is only really for you as an admin so pick something meaningful, I’m going for ‘SP2013 Farm Trust’.
Next, in the Root Certificate for the trust relationship section, click browse and locate the 2013 Root Certificate you copied earlier. Finally, leave the STS Certificate section blank.

Hit OK, and we’re done. Now for SP2013 CA. Again, same as before Security – Manage Trust – New.
This time I’m going for SP2010 Farm Trust as the name and applying the root certificate we extracted from the 2010 farm. This time though, we need to tick the box ‘Provide Trust Relationship’, provide if you want a Token Issuer Description, and add the 2010 STS Certificate in the final section of the Establish Trust Relationship dialog.

Hit OK, and that’s trust configured. Now we need some permissions. Still in SP2013 Central Admin head to Application Management – Manage Service Applications.
Find the ‘Application Discovery and Load Balancer Service Application’ highlight it, and choose ‘Permissions’ from the ribbon

Remember the GUID of the SP2010 farm we collected earlier? Now’s the time to make use of it.
Paste it into the picker in the Connection Permissions dialog.

And choose ‘Add’, this should resolve and give a bit more detail in the name, tick the full control checkbox and hit OK.

Repeat this for your Search Service Application also.

When done, leave the Search Service Application highlighted and we’re now going to publish this, so in the ribbon you’ll see a Publish button.

Hit this. In the resulting dialog choose your connection type (my farms are both internal and so not SSL secured I’ll be going for http), tick the ‘Publish this Service Application to other farms checkbox’, we’ve already added trust so no need to click the ‘add a trust relationship’ link. You will see a published URL, get a copy of this from urn through to svc. In the description text, put something useful in here.

Now let’s see the fruits of our labour and connect to the published Search Service Application from our 2010 farm! Head to 2010 CA, Application Management – Manage Service Applications. Choose ‘Connect’ form the ribbon and select the ‘Search Service Proxy’ option.

Paste in the published URL copied from the SP2013 publish operation.

Choose Ok and this will tick over for a moment or two and then hopefully tell you that there is a Search Service Application at the address you specified. Click on it to highlight it and the OK button will become active.
Just a quick note on the Add this service application’s proxy to the farms default proxy list. I’m not going to go into the whys and wherefores of Service Application proxy’s, but in some instances it may not be best to add this to the default at this stage. For example, in my circumstances, I have an up and running 2010 Search Service Application service our users, I plan to do a scheduled switch over to the 2013 search engine at some stage, but not right now. For this reason, I will be deselecting this box and at a later date editing the default proxy to remove the existing search and add this one. So just something to consider..

Anyway, after hitting OK, you get the chance to give this connection a name, again choose something meaningful.

Following this, and all being well, you should get a lovely successful message!

Finally we need to associate this with your web application(s) that you are running your searches from. There are a few scenarios here depending on your setup.

I mentioned before I didn't add my connection to the default proxy group as I have an existing SSA. If this is your case, then you can create a new proxy group in SP2010 (or use custom) and associate your new 2013 Search Connection leaving off your old one, and then in turn associate the desired web application(s) with the new proxy to use the new search.

In the case you added this to the default proxy group, one of the following will apply:
- You do not have an existing SP2010 Search, in which case you’re good to go.
- You have an existing 2010 Search. This will still be the default search provider. To make your web applications switch to the new 2013 one, head to 2010 Central Admin – Service Applications – Configure service application associations, click on the ‘default’ Application Proxy Group and you will see both your Search Service Applications ticked, notice the original one has the word [default] next to it, to set the other one active, click the [set as default] link

You could of course un-tick one of the Search providers here as well, which would do the same thing.

The settings made in the Configure Service Application Associations dialog are applied immediately and you can make searches from the respective providers and see the results!

There you have it, SharePoint 2010 powered by the much superior SharePoint 2013 Search engine.