After Windows Updates ran on a Windows Server 2008 R2 with WSS3 SP3 we ran into a few issues caused by the Configuration Cache. This post shows you lessons learned and recommendations in case you have similar issues, especially if the Configuration Wizard crashes the entire SharePoint Farm.
Many Errors but no useful details
After about 80 Windows Updates there were the following errors:
- http://intranet.contoso.com showed HTTP 404
- http://intranet.contoso.com/default.aspx showed: InvalidOperationException: Operation is not valid due to the current state of the object (Stack Trace: OpenWebInternal, SetThreadCulture, ValidateDatabaseSchemaCompatibility, PreinitializeServer, PostResolveRequestCacheHandler, …)
- SharePoint Central Administration worked normally
The detailed error message indicates to run the SharePoint Products Configuration wizard. So we did. But the Configuration Wizard failed with many errors related to the Database:
- ExecuteNonQuery requires an open and available Connection. The connection’s current state is closed.
- Database state cannot be changed while other users are using the database ‚SharePoint_Config_4829eeec-f431-4665-b10c-09ac21b35094‘
ALTER DATABASE statement failed
The SharePoint_Config DB stuck in Single User Mode and the SharePoint Farm was completely down. Central Administration showed “Cannot connect to the configuration database”, running the Configuration Wizard showed “Failed to detect if this server is joined to a server farm.” but all Windows Services were up and running. Also credentials IIS and the SQL Server were still correctly.
After a restore of the SharePoint_Config DB from a state before the Windows Update at lease Central Administration ran again. I ran psconfig -cmd upgrade -inplace b2b -wait -force which resulted in the same issue without any more details.
Don’t use the -force Parameter
I took a look at the documentation of psconfig and ran it without the -force attribute. I got the crucial message which was in no other Logfile before:
An update conflict has occurred, and you must re-try this action. The object SPConfigurationDatabase Name=SharePoint_Config_4829eeec-f431-4665-b10c-09ac21b35094 Parent=SPDatabaseServiceInstance Name=Microsoft##SSEE is being updated by NT AUTHORITY\SYSTEM, in the PSCONFIG process, on machine MYSERVERNAME. View the tracing log for more information about the conflict.
Solution
This leaded me to details about an update conflict has occurred, and you must re-try this. The Resolution to clear the Configuration Cache definitely fixed the issue even on WSS 3.
Here you’ll find more about what Is a Configuration Cache and Why Do I Care About It?
Lessons learned
- After Software Updates ran and the Configuration Wizard crashes the farm, restore the Farm Configuration DB might help
- Clear the configuration cache on servers you get curious errors about the SharePoint Config
- Run psconfig from the command line instead the GUI but don’t use the force attribute
Detailed Error Message of the SharePoint Content Application:
[InvalidOperationException: Operation is not valid due to the current state of the object.]
Microsoft.SharePoint.Library.SPRequestInternalClass.OpenWebInternal(String bstrUrl, Guid& pguidID, String& pbstrRequestAccessEmail, UInt32& pwebVersion, String& pbstrServerRelativeUrl, UInt32& pnLanguage, UInt32& pnLocale, String& pbstrDefaultTheme, String& pbstrDefaultThemeCSSUrl, String& pbstrAlternateCSSUrl, String& pbstrCustomizedCssFileList, String& pbstrCustomJSUrl, String& pbstrAlternateHeaderUrl, String& pbstrMasterUrl, String& pbstrCustomMasterUrl, String& pbstrSiteLogoUrl, String& pbstrSiteLogoDescription, Object& pvarUser, Boolean& pvarIsAuditor, Int32& plSiteFlags) +0
Microsoft.SharePoint.Library.SPRequest.OpenWebInternal(String bstrUrl, Guid& pguidID, String& pbstrRequestAccessEmail, UInt32& pwebVersion, String& pbstrServerRelativeUrl, UInt32& pnLanguage, UInt32& pnLocale, String& pbstrDefaultTheme, String& pbstrDefaultThemeCSSUrl, String& pbstrAlternateCSSUrl, String& pbstrCustomizedCssFileList, String& pbstrCustomJSUrl, String& pbstrAlternateHeaderUrl, String& pbstrMasterUrl, String& pbstrCustomMasterUrl, String& pbstrSiteLogoUrl, String& pbstrSiteLogoDescription, Object& pvarUser, Boolean& pvarIsAuditor, Int32& plSiteFlags) +429
Microsoft.SharePoint.SPWeb.InitWebPublic() +1091
Microsoft.SharePoint.SPWeb.get_Locale() +142
Microsoft.SharePoint.WebPartPages.Utility.SetThreadCulture(SPWeb spWeb, Boolean force) +95
Microsoft.SharePoint.WebPartPages.Utility.SetThreadCulture(SPWeb spWeb) +58
Microsoft.SharePoint.CoreResource.GetString(ResourceGroup rg, String name, Object[] values) +1081
Microsoft.SharePoint.SPResource.GetString(String name, Object[] values) +53
Microsoft.SharePoint.Administration.SPContentDatabase.ValidateDatabaseSchemaCompatibility() +534
Microsoft.SharePoint.SPSite.PreinitializeServer(SPRequest request) +99
Microsoft.SharePoint.SPWeb.InitializeSPRequest() +548
Microsoft.SharePoint.SPWeb.EnsureSPRequest() +75
Microsoft.SharePoint.SPWeb.get_Request() +74
Microsoft.SharePoint.WebControls.SPControl.SPWebEnsureSPControl(HttpContext context) +613
Microsoft.SharePoint.WebControls.SPControl.GetContextWeb(HttpContext context) +41
Microsoft.SharePoint.ApplicationRuntime.SPRequestModule.PostResolveRequestCacheHandler(Object oSender, EventArgs ea) +595
System.Web.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +80
System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +171
2sicgeeks 