Issue
I recently examined a SharePoint Foundation 2013 environment where all Search Boxes had gone missing overnight. Also, when browsing to the Search Center I received an error. The ULS logs showed the following error:
System.InvalidProgramException: Common Language Runtime detected an invalid program.
at Microsoft.Office.Server.Search.WebControls.SearchCommon.GetUserAdvancedLanguageSettingsUrl()
at Microsoft.Office.Server.Search.WebControls.ScriptApplicationManager..ctor()
at Microsoft.Office.Server.Search.WebControls.ScriptApplicationManager.GetCurrent(Page page)
at Microsoft.Office.Server.Search.WebControls.SearchBoxScriptWebPart.OnInit(EventArgs e)
at System.Web.UI.Control.InitRecursive(Control namingContainer)
at System.Web.UI.Control.AddedControl(Control control, Int32 index)
at System.Web.UI.WebControls.WebParts.WebPartManager.WebPartManagerControlCollection.AddWebPartHelper(WebPart webPart)
at System.Web.UI.WebControls.WebParts.WebPartManager.WebPartManagerControlCollection.AddWebPart(WebPart webPart)
at Microsoft.SharePoint.WebPartPages.SPWebPartManager.AddWebPartWithRetry(WebPart webPart)
at Microsoft.SharePoint.WebPartPages.SPWebPartManager.CreateWebPartsFromRowSetData(Boolean onlyInitializeClosedWebParts)
at Microsoft.SharePoint.WebPartPages.SPWebPartManager.LoadWebParts()
at Microsoft.SharePoint.WebPartPages.SPWebPartManager.OnPageInitComplete(Object sender, EventArgs e)
at System.EventHandler.Invoke(Object sender, EventArgs e)
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
KB2837628 and/or KB2850058
It took some investigating but we managed to track down the Windows Updates that seem to cause this issue: http://support.microsoft.com/kb/2837628 and http://support.microsoft.com/kb/2850058. Installing either on a SharePoint Foundation 2013 will break the search infrastructure. Note that these updates CANNOT be uninstalled.
Impact
Other users have also reported the exact same issue and cause on the MSDN forums, but no fix is currently available. The impacted environment had a localized (Dutch) version of SharePoint Foundation 2013 running, other cases also mostly talk about localized versions, so it might be that the English SKU is unaffected.
All versions (both English and localized) of SharePoint Foundation 2013 are impacted.
» http://social.msdn.microsoft.com/Forums/en-US/905c6aac-2ad9-4099-b4a1-d4cdd6ff4b23/kb-2837628-removes-search-box?forum=sharepointsearch
» http://social.msdn.microsoft.com/Forums/sharepoint/en-US/3b403821-42d2-4139-8afb-7c9312b57bd4/search-box-disappeared-foundation?forum=sharepointsearch
» http://blogs.technet.com/b/stefan_gossner/archive/2013/10/26/october-2013-cu-for-sharepoint-2013-has-been-released.aspx (Comments section)
» http://social.technet.microsoft.com/Forums/sharepoint/en-US/4714d906-5bbe-4eca-b666-b0e65367a0a3/update-kb2850058-breaks-search?forum=sharepointsearch
Reproduce
Initially I was unable to reproduce this issue as the aforementioned update wouldn’t install (“The expected version of the product was not found on this system”), but ultimately I did find how it probably ended up on impacted systems.
- Install SharePoint Foundation 2013
- Install SharePoint Server 2013 March Public Update
- Install KB2837628 and/or KB2850058
So basically if you run the Server PU (and maybe this applies to CU’s as well) on a Foundation it will update without issues, but then Windows Updates will start rolling out SharePoint Server 2013 updates. If this is an unsupported scenario they should have a blocking mechanism to avoid installing Server PU’s or CU’s onto a Foundation.
Solution: KB2760625
The October 2013 or December 2013 Cumulative Update do NOT fix this issue. For now avoid installing this update altogether. It isn’t supposed to wind up on Foundation installations.
I’m currently working together with Microsoft PSS to get to the bottom of this and -hopefully- a hotfix.
An hotfix has been released and confirmed to fix the issue: KB2760625 (http://support.microsoft.com/kb/2760625). After installation the Configuration Wizard needs to be run.
This hotfix should be incorporated into the April 2014 Cumulative Update, so once that is available there shouldn’t be a need to install this specific hotfix separately.
Update (12/12/2013)
There’s are reports that KB2850058 has the same impact, so be careful! but this appears to be false. This update is related to Outlook 2007 Junk Email Filter. I can now confirm this (17/12/2013).
» http://social.technet.microsoft.com/Forums/sharepoint/en-US/4714d906-5bbe-4eca-b666-b0e65367a0a3/update-kb2850058-breaks-search?forum=sharepointsearch
I have also logged this as a support case with Microsoft, let’s hope this can clear the mist. I’ll update my findings here and on the above forums.
Update (13/12/2013)
It appears that English installations of SharePoint Foundation 2013 are also impacted.
I’m still working with PSS to get this resolved.
I haven’t been able to reproduce the issue on my dev environment. I tried WS2012 and WS2008R2 but KB2837628 seems unwilling to install (“The expected version of the product was not found on the system.”). If anyone has reproducible steps please let me know.
Update (14/12/2013)
Managed to reproduce the issue. See above for the reproduce steps.
I verified KB2850058 on my installation. It wouldn’t install because I didn’t have Outlook 2007 on it, but it appears to be unrelated since it is an update for the Outlook 2007 Junk Email Filter. Most likely both updates were installed on the environment of the user that reported this. I had my numbers screwed up.
Update (16/12/2013)
I tried the recently released December 2013 Cumulative Update but no luck.
Microsoft PSS is still examining the issue. To be continued.
Update (17/12/2013)
After getting notified that KB2850058 was really also causing this issue, I retraced my steps and I had gotten the KB numbers wrong. So we have both KB2837628 and KB2850058 causing the same issues. Thanks Lee and Matt for informing me on this via the comments below!
Update (23/12/2013)
I don’t expect a solution to be provided in the next few weeks :(
Update (4/02/2014)
I finally have an update to share after weeks of back-and-forth emailing regarding the issue. I’m absolutely not happy with the answer and have expressed my sentiment accordingly. Still, I doubt that will make an impression on anyone. Here’s the update;
The Product Group is acknowledging the problem but declared that they can only fix it over the April 2014 CU as there will be no February 2014 CU.
This means for the affected customer that cannot wait until April need to reinstall SharePoint in order to address the error in a supported manner. Some other customers tried to replace binaries with older versions, we haven’t yet received the feedback if this worked out for them – we are asked to discourage this kind of experiments anyway.
Update (5/02/2014)
A final word from the escalation engineer was that this is an acknowledged bug, but due to the release cycles of hotfixes and updates this issue cannot be fixed any sooner than the April 2014 Cumulative Update. Since this is well over two months away from now you’d be best in reinstalling your environment and avoid those specific Windows Updates, or live without Search until then.
Update (4/03/2014)
Service Pack 1 for SharePoint Foundation 2013 does not fix this issue. Furthermore, the KB for Service Pack 1 was amended with a notice to postpone installing SharePoint Foundation 2013 Service Pack 1.
Important A known issue in SharePoint Foundation 2013 SP1 can affect the functionality of the Search WebPart. We encourage you to limit production installations of SharePoint Foundation 2013 SP1 until a fix is available. SharePoint Server 2013 is not affected by this issue.
http://support.microsoft.com/kb/2850035
Update (20/03/2014)
A hotfix has finally been released: http://support.microsoft.com/kb/2760625! Make sure to run the Configuration Wizard afterwards.