SharePoint 2013: Programmatically set crawl cookies for forms authenticated web sites

August 29, 2015 - 07:54, by Steven Van de Craen - 0 Comments

Last week I was having difficulties in crawling forms authenticated web sites. When configuring a crawl rule to store the authentication cookies the login page was returning multiple cookies with same name but different domain.

Add Crawl Rule

This gave issues in a later stage (during crawl) because all cookies would be sent along with the request, the target system had issues correctly identifying us due to these “duplicate” cookies.

You can easily check the request information that is sent during crawl by starting fiddler on the crawl server and configuring the proxy settings to http://localhost:8888 (default Fiddler settings).

 Search Proxy Setting

In the end we chose for an alternate method of configuring the cookies, namely through PowerShell. This gave us the ultimate flexibility to configure exactly the cookies we wanted to pass along with the crawl requests.

asnp microsoft.sharepoint.powershell -ea 0 | Out-Null $ssa = get-spenterprisesearchserviceapplication $crPath = 'http://authenticatedwebsite*' # Get or create crawl rule $cr = Get-SPEnterpriseSearchCrawlRule -SearchApplication $ssa | ? { $_.Path -eq $crPath } if ($cr -eq $null) { $cr = New-SPEnterpriseSearchCrawlRule -Path $crPath -SearchApplication $ssa -Type InclusionRule -AuthenticationType CookieRuleAccess -FollowComplexUrls $true } # Set cookie credentials $cr.SetCredentials('CookieRuleAccess', 'myUser=crawlUser; myPwd=crawlPassword', 'http://cookie-set-via-powershell')


SharePoint 2013: Some observations on Enterprise Search

August 13, 2015 - 16:44, by Steven Van de Craen - 0 Comments

I’m doing some testing with the Enterprise Search in SharePoint 2013 for a customer scenario and here are some observations…

Content Source as Crawled Property

The “Content Source” name is out of the box available as Managed Property on all content in the search index

This makes it possible to create Result Sources that aggregate content from different Content Sources similar to Search Scopes back in the old days.

SharePoint 2013 Search Query Tool #1

Meta elements (HTML <meta> tags) as Crawled Properties

Information from meta elements in web pages is extracted into crawled properties

Consider the following example:

Simple website with meta tags

After crawling this website with SharePoint 2013 Search it will create (if new) or use (if existing) a Crawled Property and store the content from the meta element. The Crawled Property can then be mapped to Managed Properties to return, filter or sort query results.

SharePoint 2013 Search Query Tool #2

Query string parameters as Crawled Properties

Query string parameters are not extracted into Crawled Properties

This was actually a request from the customer in order to be able to provide additional information regarding documents (on their website) into the search index. As I suspected it isn’t possible out of the box but you could definitely do it using Content Enrichment.

The “OriginalPath” is available as an input property for Content Enrichment and contains the exact url used for indexing the document:

Content Enrichment Input Properties

With Content Enrichment it is pretty straightforward to look for predefined query string parameters and then map them to output properties.

$ssa = Get-SPEnterpriseSearchServiceApplication $config = New-SPEnterpriseSearchContentEnrichmentConfiguration $config.Endpoint = 'http://cews:818/ContentEnrichmentService2.svc' $config.InputProperties = 'OriginalPath', 'ContentSource' $config.OutputProperties = 'MyParam1', 'MyParam2' $config.DebugMode = $false $config.SendRawData = $false Set-SPEnterpriseSearchContentEnrichmentConfiguration –SearchApplication $ssa –ContentEnrichmentConfiguration $config

More information on Content Enrichment from my SharePoint Saturday presentation:


SharePoint: Portal navigation limited to 50 dynamic items

August 12, 2015 - 11:07, by Steven Van de Craen - 0 Comments

I was looking into an issue where the Navigation Settings page wouldn’t show all subsites in the treeview. When reproducing it was limited to 50 dynamic items.

Navigation Settings

The treeview component is a Microsoft.SharePoint.Publishing.Internal.WebControls.HierarchicalListBox which connects to the active Microsoft.SharePoint.Publishing.Navigation.PortalSiteMapProvider. This has a property “DynamicChildLimit” that can be explicitly configured in the web.config.

// Microsoft.SharePoint.Publishing.Navigation.PortalSiteMapProvider public int DynamicChildLimit { get { int? num = this.dynamicChildLimit; if (num.HasValue) { return num.GetValueOrDefault(); } if (this.Version < 14) { return 50; } return 0; } set { this.dynamicChildLimit = new int?(value); } }

The active provider used is “GlobalNavSiteMapProvider”, defined in web.config as

<add name="GlobalNavSiteMapProvider" description="CMS provider for Global navigation" type="Microsoft.SharePoint.Publishing.Navigation.PortalSiteMapProvider, Microsoft.SharePoint.Publishing, Version=, Culture=neutral, PublicKeyToken=71e9bce111e9429c" NavigationType="Global" EncodeOutput="true" />

I tried specifying Version=”14” but then it defaulted to 20 items, a path I didn’t further investigate. So I just explicitly specified the DynamicChildLimit=”100” and that fixed the issue.

<add name="GlobalNavSiteMapProvider" description="CMS provider for Global navigation" type="Microsoft.SharePoint.Publishing.Navigation.PortalSiteMapProvider, Microsoft.SharePoint.Publishing, Version=, Culture=neutral, PublicKeyToken=71e9bce111e9429c" NavigationType="Global" EncodeOutput="true" DynamicChildLimit="100" />

SharePoint: Users cannot create new subsites

July 13, 2015 - 17:02, by Steven Van de Craen - 0 Comments


First day after my vacation and I got presented with a nice situation at one of our clients. Most users trying to create new subsites would get “Sorry, this site hasn’t been shared with you” and the site would NOT get created. Troubleshooting this showed that during site provisioning the “SiteFeed” Feature would throw an exception which would roll back the site creation. The relevant lines in the ULS logs pointed towards the “Following” (Social) of the newly created site:

FollowedContent.FollowItem:Exception:Microsoft.Office.Server.UserProfiles.FollowedContentException: ItemDoesNotExist : Item does not exist.     at Microsoft.Office.Server.UserProfiles.SPS2SAppUtility.GetPersonalUrl(UserProfile& profile)     at Microsoft.Office.Server.UserProfiles.SPS2SAppExecutionContext.InitializeForProfile()     at Microsoft.Office.Server.UserProfiles.SPS2SAppExecutionContext.EnsureInitialized()     at Microsoft.Office.Server.UserProfiles.FollowedContent.FollowItem(FollowedItem item, Boolean isInternal)

Could not follow the url https://sharepoint/newsub

Leaving Monitored Scope (Event Receiver (Microsoft.Office.Server.UserProfiles, Version=, Culture=neutral, PublicKeyToken=71e9bce111e9429c, Microsoft.Office.Server.UserProfiles.ContentFollowingWebEventReceiver)).


The newly created site could not be added to the user’s Social list on his MySite due to an Access Denied on adding the item to the list.


The MySites were recently migrated from SharePoint 2010 (classic mode) to SharePoint 2013 (claims mode). MySites work by setting the ‘owner’ as Primary Site Collection Administrator, but the conversion to claims had erased all the classic-mode Site Collection Administrators from the migrated site collections. Querying for the Site Collection Administrators would just return empty which is of course not good.

(Blank) Site Collection Administrators


I whipped up a PowerShell script that would loop all MySites and report any sites with missing or non-matching Site Collection Administrator. Toggling the ‘report only’ flag in the script will correct the situation.

asnp Microsoft.SharePoint.PowerShell -ea 0 | Out-Null cls $reportOnly = $false Write-Host "ReportOnly:" $reportOnly $mySites = Get-SPSite -Limit ALL | ? { $_.RootWeb.WebTemplate -eq "SPSPERS" } Write-Host "Found" ($mySites.Count) "MySites" $mySites | % { $s = $_ $w = $s.RootWeb $owner = $w.EnsureUser($w.Title) $primaryAdmin = $w.SiteAdministrators | Select -First 1 if ($primaryAdmin -eq $null) { Write-Host -ForegroundColor Red ($s.ServerRelativeUrl) ": no primary SC admin. Owner should be" $owner if ($reportOnly -eq $false) { $owner.IsSiteAdmin = $true $owner.Update() } } elseif ($owner.IsSiteAdmin -eq $false) { Write-Host -ForegroundColor Yellow ($s.ServerRelativeUrl) ": primary SC admin (" $primaryAdmin ") does NOT match owner (" $owner ")" if ($reportOnly -eq $false) { $owner.IsSiteAdmin = $true $owner.Update() } } else { Write-Host -ForegroundColor Green ($s.ServerRelativeUrl) ": primary SC admin (" $primaryAdmin ") matches owner (" $owner ")" } $s.Close() }

After correcting all sites should turn up green.

 All MySites corrected


SharePoint 2013: Open PDF files in client application

June 26, 2015 - 16:23, by Steven Van de Craen - 0 Comments

SharePoint 2013 has this quirk going for quite some time where clicking on a PDF file opens it in the browser no matter what you configure in the list settings or client application. One of our customers migrated to SharePoint 2013 and suddenly they lost this functionality. Together with my colleague Elio Struyf we proposed a small customization that would add the “Open in client” option to the fly out menu for PDF documents.

Open in client in fly out menu

The customization itself is devised as a SharePoint Sandboxed Solution that can be installed and activated per Site Collection on either an on-premises environment or Office 365.

Activate (Sandboxed) Solution

The functionality can be enabled or disabled via the corresponding Site Collection Feature.

Site Collection Features

The customisation can be found on GitHub: Ventigrate/SharePoint_PDFOpenInClient

SharePoint 2013: Enable 'Change Item Order' Ribbon Action

June 4, 2015 - 17:00, by Steven Van de Craen - 0 Comments

My customer reported an issue where a ‘Links’ List didn’t have the option ‘Change Item Order’ enabled. This was for a list created in SharePoint 2007 and migrated to SharePoint 2013.

When creating a new Links List in SharePoint 2013 it would have this enabled, so it wasn’t related to Site or Site Collection Features. Creating new views in the migrated list showed the same ‘issue’.


With PowerShell it quickly becomes obvious that the View didn’t have the ‘Orderable’ flag set. It is not possible to change this through SPView.OrderedView because it is read-only, however you can set this ‘property’ on the XsltListViewWebPart that’s on the View Form as follows:

asnp Microsoft.SharePoint.PowerShell -ea 0 | Out-Null cls # Get the View page $w = get-spweb http://sharepoint/site/site $f = $w.GetFile("http://sharepoint/site/site/Lists/Links/AllItems.aspx") Write-Host "STARTING" $f.ServerRelativeUrl if ($f.Exists) { $man = $f.GetLimitedWebPartManager('Shared') $wp = $man.WebParts | ? { $_ -is [Microsoft.SharePoint.WebPartPages.XsltListViewWebPart] } | Select -First 1 if ($wp -ne $null) { # Make the view 'Ordered' by expanding the ViewFlags enum $wp.ViewFlags = [Microsoft.SharePoint.SPViewFlags]($wp.ViewFlags.ToString() + ', Ordered') $man.SaveChanges($wp) } else { Write-Host "Web Part not found on page" } $man.Dispose() } else { Write-Host "Page not found" } Write-Host "DONE"

Afterwards users will be able to set the order of the items…


When you create new Views either base them on an existing (‘fixed’ View) or run the above script on them to enable the functionality.

Moving site collections from explicit managed path to wildcard managed path

March 14, 2015 - 07:41, by Steven Van de Craen - 1 Comments

At my current project where we’re upgrading from SharePoint 2007 to SharePoint 2013 we’re running into a high number of managed paths on a Web Application. The recommended limit according to TechNet is 20 but they’re well above that with around 70 managed paths. An internal study showed that a lot of them were Explicit Managed Paths where there was no real requirement to have it that way and they could easily be transformed into a (single) Wildcard Managed Path.

Explicit Managed Path Wildcard Managed Path


It turns out that it is a rather painless operation to do:

  1. Unmount the Content Database with the Site Collection on the Explicit Managed Path
  2. Delete the Explicit Managed Path (ensure that a corresponding Wildcard Managed Path exists)
  3. Mount the Content Database with the Site Collection and it will make use of the Wildcard Managed Path

With this in mind the following PowerShell script automated this. It doesn’t ensure the Wildcard Managed Path since I wanted to control if the Explicit Managed Path should be transformed or not. In this case it will log a warning and skip that one.

### Script will loop explicit managed paths and restructure so the site collections are inside a wildcard managed path in order to reduce # (explicit) managed paths cls asnp Microsoft.SharePoint.PowerShell -ea 0 | Out-Null $logFile = "C:\temp\managedpath_restructure.txt" $trial = $false $waUrl = "http://intranet" $emps = Get-SPManagedPath -WebApplication $waUrl | ? { ($_.Type -eq "ExplicitInclusion") -and ($_.Name -like "*/*") } $emps | % { $emp = $_ $wmpName = $emp.Name.Split("/")[0] $wmp = Get-SPManagedPath -WebApplication $waUrl | ? { ($_.Type -eq "WildcardInclusion") -and ($_.Name -eq $wmpName) } if ($wmp -eq $null) { # SKIP $log = [string]::Format("{0} - {1} - No wildcard MP '{2}'. Skipping.", (Get-Date), $emp.Name, $wmpName) Write-Host -ForegroundColor Yellow $log $log >> $logFile } else { # RESTRUCTURE $log = [string]::Format("{0} - {1} - Restructuring explict MP to wildcard MP '{2}'.", (Get-Date), $emp.Name, $wmpName) Write-Host -ForegroundColor Green $log $log >> $logFile # Get matching sites' content database $cdbs = @() Get-SPWebApplication $waUrl | Get-SPSite -Limit ALL | ? { $_.Url -like ("*/" + $emp.Name) } | % { $s = $_ $cdb = $s.ContentDatabase if (-not($cdbs.Contains($cdb))) { $cdbs += $cdb } $s.Close() } # Dismount content databases $cdbs | % { $cdb = $_ $log = [string]::Format("{0} - {1} - Dismounting content database '{2}'.", (Get-Date), $emp.Name, ($cdb.Name + " @ " + $cdb.Server)) Write-Host -ForegroundColor Green $log $log >> $logFile if (-not($trial)) { Dismount-SPContentDatabase $cdb -Confirm:0 } } # Remove explicit managed patch $log = [string]::Format("{0} - {1} - Removing explict MP.", (Get-Date), $emp.Name, $wmpName) Write-Host -ForegroundColor Green $log $log >> $logFile if (-not($trial)) { Remove-SPManagedPath $emp -WebApplication $waUrl -Confirm:0 } # Mount content databases $log = [string]::Format("{0} - {1} - Mounting content database '{2}'.", (Get-Date), $emp.Name, ($cdb.Name + ' @ ' + $cdb.Server)) Write-Host -ForegroundColor Green $log $log >> $logFile if (-not($trial)) { $db = Mount-SPContentDatabase ($cdb.Name) -DatabaseServer ($cdb.Server) -WebApplication $waUrl -Confirm:0 } } } Write-Host "DONE"


SharePoint Publishing Feature activation failed

December 16, 2014 - 15:19, by Steven Van de Craen - 0 Comments

Last week I ran into an issue while reactivating the Publishing Feature on webs in a migrated (Dutch) site collection. If you have ever upgraded localized SharePoint 2007 Publishing sites this should sound familiar to you. What happens is that while in SharePoint 2007 the Pages library was called “Pages”, since SharePoint 2010 this is also localized to “Paginas” (Dutch), “Zeiten” (German), etc.

There’s an override property “__PagesListName” that is checked by SharePoint to get the correct list name in case it differs from what’s expected. And in case of data migrations it will be different (“Pages”) than expected (“Paginas”).

Microsoft has published a KB article ( on it with a PowerShell script to set the override property. This fixes most issues with navigation not displaying correctly, but it seems that at least in SharePoint 2013 this still gives problems with Publishing Feature reactivation.

Publishing Feature activation failed. Exception: System.IO.FileNotFoundException: <nativehr>0x80070002</nativehr><nativestack>OWSSVR.DLL: (unresolved symbol, module offset=0000000000052B3E)

at 0x000007FF151F2B3E OWSSVR.DLL: (unresolved symbol, module offset=00000000000112BD)

at 0x000007FF151B12BD (unresolved symbol, module offset=00000000000A3DA9)

at 0x000007FF15C33DA9 </nativestack>

at Microsoft.SharePoint.Library.SPRequestInternalClass.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder)

at Microsoft.SharePoint.Library.SPRequest.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder)

at Microsoft.SharePoint.SPWeb.GetList(String strUrl)

at Microsoft.SharePoint.Publishing.Internal.Store.GetListByUrl(SPWeb web, String listUrlName)

at Microsoft.SharePoint.Publishing.Internal.AreaProvisioner.Provision()

at Microsoft.SharePoint.Publishing.PublishingFeatureHandler.<>c__DisplayClass3.<FeatureActivated>b__0()

at Microsoft.Office.Server.Utilities.CultureUtility.RunWithCultureScope(CodeToRunWithCultureScope code)

at Microsoft.SharePoint.Publishing.PublishingFeatureHandler.FeatureActivated(SPFeatureReceiverProperties receiverProperties).

The AreaProvisioner.Provision() uses a private field pagesListUrl which seems to be initialized to Microsoft.Office.Server.Utilities.PageUtility.DefaultPagesListName, which in turn reads its information from the localized “osrvcore“ resource value (C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\Resources)




I’ve experienced this issue in SharePoint 2013 with Service Pack 1 (15.0.4569.1000) but also November 2014 Cumulative Update (15.0.4667.1000). It may be fixed with any later CU but don’t hope on it.

One workaround would be to temporarily(!) change the value of List_Pages_UrlName in to “Pages”, (re)start the process that’s doing the Feature reactivation (powershell or IIS worker process), reactivate the Feature and retrace your steps as to undo the change (don’t forget the process restart). This will allow you to reactivate the Publishing Feature on those Dutch/localized migrated Publishing sites.


SharePoint: Rendering inside iframes

October 31, 2014 - 16:35, by Steven Van de Craen - 32 Comments

This post is a revision of an old blog post on rendering Excel Services in an iframe on a different domain. This is prohibited because a HTTP response header X-FRAME-OPTIONS: SAMEORIGIN is added to the response. The issue isn’t limited to Excel Services but is applicable to any SharePoint-hosted page that you want to visualize in an iframe.

Consider the following:

  • SharePoint 2013 will always render the X-FRAME-OPTIONS header, even for regular pages. Adding an AllowFraming control to the page fixes that, but doesn’t cover all situations
  • You can’t add the AllowFraming control to Office Web Apps or InfoPath Forms Server (“FormServer.aspx”)
  • Clicking on (pdf) documents in a Document Library in the iframe will fail to load them because the document is a different request
  • You have a basic “integration” between different systems (like Dynamics CRM) and SharePoint content that uses iframes

This content cannot be displayed in a frame


This is a HttpModule that can be activated per Web Application by Web Application Feature and will ensure that all pages will render inside an iframe. The module will set values that will prevent SharePoint from trying to inject the header in the first place, but for some exceptions (Office Web Apps 2010, XLViewer 2013) it is still required to actually remove the header at the end of the request.

Please visit the Codeplex Repository to read more about this addon and for installation instructions:

Importing a Summary Links Web Part: List does not exist

October 23, 2014 - 15:23, by Steven Van de Craen - 2 Comments


Consider the scenario where you have a Summary Links Web Part (part of the SharePoint Publishing functionality) configured on a page and you want to import the preconfigured Web Part on a different page on a different site. If you try this you’ll get “List does not exist”:


Note that importing the Web Part in the same site (same or different pages) works just fine.


This is because the Summary Links Web Part references the list that contains the page where the Web Part resides on. If you open the .webpart file in a text editor you’ll see ListName and ListId containing the GUID of that list. So it can be the “Site Pages” library, the “Pages” library, or any Document Library that has Web Part Pages.


You can verify this by navigating to the following URL (note to replace the actual GUID): http://sitename/_layouts/listedit.aspx?List=GUID

Bonus question: what is the value when the Summary Links Web Part is on the “default.aspx” of a site? Answer:



So what’s the solution? Just remove the ListName and ListId elements (or their values) from the exported .webpart file and you’ll have no issues importing it to other sites.


<< Previous    Next >>