Hi Lawrence,

Thanks for your answer.

I should say this thread with the issue synch. error "outofmemoryexeption" is resolved after reducing the approvals on upstream server according to your recommendation.

I am still fighting wiht my next problem deleting the four computer groups on all 20 replica server.

I could till now delete successfully these on half of the replica servers with costing too much time. The way only to unapprove the updates out of these groups could not help me, I have to decline the updates causing these problem one by one according to the logfile. And this is a big job.

I had to rebuild the replica WSUS by some of them too.

I have now a third problem, which I will create a new thread for that.

Thanks a lot.

kailar

It seems now I really have to rebuild all rest 17 replica servers one by one.

Given that we know the replicas are out of sync with the upstream -- a locally created Target Group with a different GroupID may just be one symptom of a larger collection of not-yet-identified issues, the more reliable course of action is definitely to reinstall the replica servers with new databases. (You can keep the existing content store, so no need to re-download files.)

After installing the new replica and synchronizing to upstream, run the Server Cleanup Wizard with the "Delete unneeded files.." option only (on the replica).

Lawrence Garvin, M.S., MCITP:EA, MCDBA, MCSA
SolarWinds Head Geek
Microsoft MVP - Software Distribution (2005-2012)
My MVP Profile: http://mvp.support.microsoft.com/profile/Lawrence.Garvin
The views expressed on this post are mine and do not necessarily reflect the views of SolarWinds.

Unfortunatlly that was the only one server on which I could delete the problem computer groups.

I have tried with the same procedure on other five downstream servers due to delete the false computer groups, but it does not work. That groups can not be deleted. And there is no updates approved for these groups anymore.

Well as we see WSUS upstream server is not able to just overwrite the DB onto his downstream servers.

It seems now I really have to rebuild all rest 17 replica servers one by one.

kailar

Have finally got one of them to run the synchronization without error.

As I said above it was not possible to remove the computer group GIG... and the approved view showed further many client as not approved (amazing). I made these declined. After that I could synch. that server as replica. I'm going to try with all other rest to do so on comming monday to see what happens.

Hi Lawrence,

I have tried on two of the downstream servers to delete the problem computer groups.

1. Made the replica server as autonomous downstream server
2. Set all updates for that groups to not approved
3. Tried to delete the groups but did NOT work with none of the both servers. Error: Unexpected error occurs must reset server node.(http://social.technet.microsoft.com/Forums/en/winserverwsus/thread/4e57e3ad-a8f8-4da4-8bec-410b0180bc4c) did not help in this case.

The very amazing thing on  these servers with the Approved viewing as you see in screen shot, it shows many like ~1200 “Not approved” updates. In such filtered view there must not be shown any updates as Not approved, right?

(Sorry it is not allowed to insert a screen shot in this site. My account must be verified, but how?)

Do you think I must start to reinstall new wsus with all 18 replica servers? This would be a very long work.

Thanks,

kailar

There are four computer groups which are causing probably this problem. These groups have different ID's on upstream and downstream servers.

Hier one of that groups GIG_Clients.

GIG_Cients by upstream:
ID: 42F13AF7-A2BB-438F-8F3B-E935D094514A

GIG_Cients by dowstream:
ID: A1180ED0-8A2A-4A69-8467-6A9548C14415

So when I remove the approvals for that group GIG_Cients (with ID: 42F13AF7-A2BB-438F-8F3B-E935D094514A) on the upstream server and at end delete that group from upstream server, what would happen? The downstream server tries further to delete it's GIG_Clients group with ID: A1180ED0-8A2A-4A69-8467-6A9548C14415 which is not known by the upstream server.

So how do you think to solve this problem?

Lawrence Garvin, M.S., MCITP:EA, MCDBA, MCSA
SolarWinds Head Geek
Microsoft MVP - Software Distribution (2005-2012)
My MVP Profile: http://mvp.support.microsoft.com/profile/Lawrence.Garvin
The views expressed on this post are mine and do not necessarily reflect the views of SolarWinds.

Hi Lawrence,

Thank you for the answer.

I assume this might not work.

There are four computer groups which are causing probably this problem. These groups have different ID's on upstream and downstream servers.

Hier one of that groups GIG_Clients.

GIG_Cients by upstream:
ID: 42F13AF7-A2BB-438F-8F3B-E935D094514A

GIG_Cients by dowstream:
ID: A1180ED0-8A2A-4A69-8467-6A9548C14415

So when I remove the approvals for that group GIG_Cients (with ID: 42F13AF7-A2BB-438F-8F3B-E935D094514A) on the upstream server and at end delete that group from upstream server, what would happen? The downstream server tries further to delete it's GIG_Clients group with ID: A1180ED0-8A2A-4A69-8467-6A9548C14415 which is not known by the upstream server.

The second problem Group which tries another downstream server to delete, does NOT exists by upstream server any more.

So all four computer groups by the downstream servers are not existing by the upstream server any more.

So how do you think to solve this problem?

Thanks,

kailar

I am trying to delete the target group on downstream server manually following the way told in the link below.

When this works I would have to do that on all 18 downstream servers too.

http://social.technet.microsoft.com/Forums/en/winserverwsus/thread/4e57e3ad-a8f8-4da4-8bec-410b0180bc4c

No, you don't need to do anything on the downstream servers.

In the cited thread, the issue was that the group could not be deleted from the upstream server because there were still existing approvals for that group. The UI fails to identify this problem. The fix was to create a custom view so that those approved updates could be identified and the approvals removed for that group, thus allowing the group to be deleted from the upstream server.

In the instant case it looks like you're getting timeout errors because, before you can delete the target group, the approvals must be removed. The WSUS server is attempting to remove those approvals, but it's hitting a timeout error -- probably because of the number of approvals. If you removed the group from the upstream server with approvals assigned, then the replica server must first remove those approvals before it can remove the group. The timeout is actually occuring because of the number of update approvals that need to be removed.

2013-01-17 08:58:06.166 UTC Error WsusService.29 CatalogSyncAgentCore.ProcessTargetGroups

DeploymentSync: could not delete target group a1180ed0-8a2a-4a69-8467-6a9548c14415:Timeout expired.

The timeout period elapsed prior to completion of the operation or the server is not responding.
Change:Deleted deployment(Install) of Windows XP Update Package, October 25, 2001 by NT AUTHORITY\NETWORK SERVICE UpdateID:1847AE6D-8360-4E5A-9D15-DA82B24DEFFD Revision Number:100 TargetGroup:GIG_Clients
Change:Deleted deployment(Install) of Critical Update, November 19, 2001 by NT AUTHORITY\NETWORK SERVICE UpdateID:57A02FF3-31B4-4BE8-BCEE-33D74EBA9A18 Revision Number:100 TargetGroup:GIG_Clients
Change:Deleted deployment(Install) of Remote Assistance Connection by NT AUTHORITY\NETWORK SERVICE UpdateID:4B62AA32-52F6-4C15-B739-1C8E696EB33E Revision Number:100 TargetGroup:GIG_Clients

In this case, the better approach is:

1. Remove the approvals for the GIG_Clients group from the UPSTREAM server, using the procedure described in the cited thread. Note: Depending on the number of updates actually approved for that group, it may take more than one cycle. If you try to remove the approval for too many updates at once, you'll get a timeout error when the database fails to report the task as completed before the console gets tired of waiting. I would suggest unapproving no more than a hundred updates at a time.
2. After each cycle of update approval removals, synchronize ALL downstream servers so they get the update approval removals.
3. Once all of the update approvals have been removed for the GIG_Clients group, and those approval change events synchronized to the replica servers, THEN you can remove the group from the UPSTREAM server, and that Delete Group event will be replicated to the downstream servers.

You cannot add/remove groups from a replica server! You can only change computer group memberships on a replica server (when using server-side targeting)

Lawrence Garvin, M.S., MCITP:EA, MCDBA, MCSA
SolarWinds Head Geek
Microsoft MVP - Software Distribution (2005-2012)
My MVP Profile: http://mvp.support.microsoft.com/profile/Lawrence.Garvin
The views expressed on this post are mine and do not necessarily reflect the views of SolarWinds.

I am trying to delete the target group on downstream server manually following the way told in the link below.

When this works I would have to do that on all 18 downstream servers too.

http://social.technet.microsoft.com/Forums/en/winserverwsus/thread/4e57e3ad-a8f8-4da4-8bec-410b0180bc4c

On just one server it takes very long time. I give you a feed back.

Thanks,

kailar

Once the upstream server is healthy, the downstream servers will be perfectly happy campers.

Hi Lawrence,

I thik I have reduced and made the DB more healthy as before. All 18 replica server have still synch. problem but as told before the error is different now. The 'OutOfMemoryException‘ error is gone. The new error is "Unknown".

I fight with this issue since very long but I don't get any positive result.

The synchronization result "Unknown" has the  following result in logfile same as above.

2013-01-17 08:55:31.070 UTC Info WsusService.29 CatalogSyncAgentCore.ReplicaSync DeploymentSync: new anchor: 1020369,2013-01-17 08:55:02.939
2013-01-17 08:55:31.070 UTC Info WsusService.29 CatalogSyncAgentCore.ReplicaSync DeploymentSync: target groups count: 60
2013-01-17 08:55:31.070 UTC Info WsusService.29 CatalogSyncAgentCore.ReplicaSync DeploymentSync: deployments count: 59306
2013-01-17 08:55:31.070 UTC Info WsusService.29 CatalogSyncAgentCore.ReplicaSync DeploymentSync: deleted deployments count: 0
2013-01-17 08:55:31.070 UTC Info WsusService.29 CatalogSyncAgentCore.ReplicaSync DeploymentSync: accepted EULA count: 5
2013-01-17 08:55:31.070 UTC Info WsusService.29 CatalogSyncAgentCore.ReplicaSync DeploymentSync: Hidden update count: 3553
2013-01-17 08:55:31.132 UTC Info WsusService.29 CatalogSyncAgentCore.ReplicaSync DeploymentSync: processing target groups
2013-01-17 08:58:06.166 UTC Error WsusService.29 CatalogSyncAgentCore.ProcessTargetGroups DeploymentSync: could not delete target group a1180ed0-8a2a-4a69-8467-6a9548c14415: Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
Change:Deleted deployment(Install) of Windows XP Update Package, October 25, 2001 by NT AUTHORITY\NETWORK SERVICE UpdateID:1847AE6D-8360-4E5A-9D15-DA82B24DEFFD Revision Number:100 TargetGroup:GIG_Clients
Change:Deleted deployment(Install) of Critical Update, November 19, 2001 by NT AUTHORITY\NETWORK SERVICE UpdateID:57A02FF3-31B4-4BE8-BCEE-33D74EBA9A18 Revision Number:100 TargetGroup:GIG_Clients
Change:Deleted deployment(Install) of Remote Assistance Connection by NT AUTHORITY\NETWORK SERVICE UpdateID:4B62AA32-52F6-4C15-B739-1C8E696EB33E Revision Number:100 TargetGroup:GIG_Clients

....

Change:Deleted deployment(Install) of 814033: Critical Update by NT AUTHORITY\NETWORK SERVICE UpdateID:359D3F60-B1AD-4CA7-BCDF-2EFE91227462 Revision Number:100 TargetGroup:GIG_Clients
Change:Deleted deployment(Instal
   at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.ProcessTargetGroups(ServerSyncTargetGroup[] groups)
   at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.ReplicaSync()
   at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.ExecuteSyncProtocol(Boolean allowRedirect)
   at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.CatalogSyncThreadProcess()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.runTryCode(Object userData)
   at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
2013-01-17 08:58:06.182 UTC Error WsusService.29 CatalogSyncAgentCore.ExecuteSyncProtocol System.Data.SqlClient.SqlException: Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
Change:Deleted deployment(Install) of Windows XP Update Package, October 25, 2001 by NT AUTHORITY\NETWORK SERVICE UpdateID:1847AE6D-8360-4E5A-9D15-DA82B24DEFFD Revision Number:100 TargetGroup:GIG_Clients

...

2013-01-17 08:58:06.198 UTC Error WsusService.29 CatalogSyncAgentCore.UpdateSyncResultAndGenerateReportingEvent Failed when call SubscriptionEvent.GenerateEvent(): System.ArgumentException: "ReportingEvent is invalid. Size 1256078 of field ExtendedData.MiscData[i] exceeded the maximum size of 409600."

I have tried again to reduce and decline more update.

Did again Cleanup Wizard and reindexed the upstream server. Several time synch. did not help.

Unapproved: 3501
Approved: 1063
Declined: 1093
Computers: 1515
Computer groups: 58

Is there any idea what else could I do?

Thanks,

kailar

After reducing the approved updates the error with 'OutOfMemoryException‘ is finally gone but I have still synch. problem on all 18 downstream servers though after several times of synchronization. Only 2 downstream server can synchronize now.
I cannot reduce the approved updates anymore. All approved updates have already ‘needed’ status.

Unapproved: 3397
 Approved: 1133
 Declined: 1091
 Computer groups: 58

The synchronization results are:
“Unknown” after 90% synch.
"Failed" with sqlexcetion: Timeout expired

I find in many issues with the error  “could not delete target group ….”
Like here:

2013-01-14 14:32:09.143 UTC Error WsusService.25 CatalogSyncAgentCore.ProcessTargetGroups DeploymentSync: could not delete target group a1180ed0-8a2a-4a69-8467-6a9548c14415: Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
Change:Deleted deployment(Install) of Windows XP Update Package, October 25, 2001 by NT AUTHORITY\NETWORK SERVICE UpdateID:1847AE6D-8360-4E5A-9D15-DA82B24DEFFD Revision Number:100 TargetGroup:xxx
Change:Deleted deployment(Install) of Critical Update, November 19, 2001 by NT AUTHORITY\NETWORK SERVICE UpdateID:57A02FF3-31B4-4BE8-BCEE-33D74EBA9A18 Revision Number:100 TargetGroup:xxx
Change:Deleted …
…
Change:Deleted deployment(Instal
   at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.ProcessTargetGroups(ServerSyncTargetGroup[] groups)
   at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.ReplicaSync()
   at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.ExecuteSyncProtocol(Boolean allowRedirect)
   at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.CatalogSyncThreadProcess()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.runTryCode(Object userData)
   at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
2013-01-14 14:32:09.159 UTC Error WsusService.25 CatalogSyncAgentCore.ExecuteSyncProtocol System.Data.SqlClient.SqlException: Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.

It is amazing that WSUS is not able to manage and synchronize the ~5400 updates not even now with ~1900 approvals these are only "critical" and the "security updates" not more.

Because having 1000+ approved updates on a WSUS server is not a practical scenario. I doubt it was ever tested. I've certainly never had a thousand approved updates on a WSUS server!

You're testing it now. It doesn't work. I'm not surprised. Why do you find that you need to have 1900 approved updates?

What would happen when more products and classifications would be activated to manage?

Well, I suspect that most of us, as we've been doing for the past seven years, willremove the approvals for updates no longer needed. Over the past seven years, through the addition of several instances of Windows, and almost every Microsoft application, I've yet to see more than 500 approved updates on my WSUS server at any given time (except when I've been lax about removing the old approvals).

But, part of the difference here is that I do it on a monthly basis, so the effort of removing approvals is [a] trivial, and [b] doesn't send the Server Sync process off in a quagmire of work to do within a time constraint. You're trying to cram three years worth of maintenance effort into a single sitting -- it ain't working.

I must fight further with 20 downstream servers:-/

No, you only need to fix the upstream server. Once the upstream server is healthy, the downstream servers will be perfectly happy campers. There is nothing wrong with the downstream servers except they're trying to duplicate the unhealthy state of your upstream server.

Unapproved: 2609
Approved: 1889
Declined: 1057

Lawrence Garvin, M.S., MCITP:EA, MCDBA, MCSA
SolarWinds Head Geek
Microsoft MVP - Software Distribution (2005-2012)
My MVP Profile: http://mvp.support.microsoft.com/profile/Lawrence.Garvin
The views expressed on this post are mine and do not necessarily reflect the views of SolarWinds.

I have set ~2600 updates to NOT APPROVALS and tried to synch. Every time crashes the synchronization with different results on different server like:
“Unknown” after 90% synch.
"Failed" with sqlexcetion: Timeout expired
"Failed" OutOfMemoryException: a.s.o.

It is amazing that WSUS is not able to manage and synchronize the ~5400 updates not even now with ~1900 approvals these are only „critical“ and the „security updates” not more. What would happen when more products and classifications would be activated to manage? I must fight further with 20 downstream servers:-/
Well now I try further to synch. the servers manually several times. See what happens till tommorrow. I give you a feedback.

Unapproved: 2609
Approved: 1889
Declined: 1057

The only allowed classification updates of my upstream sever are the „critical“ and the „security updates“. These updates are being approved automatically on the patch day only to some pilot computer groups.

Okay. That explains why you have 5400+ approved updates.

Now you need to go UNAPPROVE the ~5000+ updates that do not need to be approved anymore.

I did decline 1008 updates.

That's a good start. My guess would be there's at least another 4000 that can also be changed. On a WSUS server with only Security Updates and Critical Updates there are likely not more than 400-500 updates that actuallyneed to be approved to maintain current patching requirements.

Of course..... converting all of those 4000+ updates from Approved to Declined, will cause yet another problem (the introduction of the Declines), and likely the synchronization will continue to fail (as we've seen several times in recent months as a result of mass declinations of thousands of updates on an upstream server).

At this point, it is critical that you not Decline those updates, but simply set them to NOT APPROVED. Once that is done, try the synchronization again; however it's still even possible that the simple removal of the approvals may be problematic, and it may require you to reBuild the downstream replica servers (which, actually, would probably be much faster than trying to continue to fight this situation).

Lawrence Garvin, M.S., MCITP:EA, MCDBA, MCSA
SolarWinds Head Geek
Microsoft MVP - Software Distribution (2005-2012)
My MVP Profile: http://mvp.support.microsoft.com/profile/Lawrence.Garvin
The views expressed on this post are mine and do not necessarily reflect the views of SolarWinds.

Hi Lawrence,

unfortunately I did not hear from you, so I did try further steps reducing all superseded updates on the upstream server. I did decline  1008 updates. After that I tried many times to synchronize the downstream server with this but all time got the same error.

Actual updates of the upstream server:

All Updates: 5461
Unapproved: 0
Approved: 4453
Declined: 1008

Hello Lawrence,

thanks for your answer.

The only allowed classification updates of my upstream sever are the „critical“ and the „security updates“. These updates are being approved automatically on the patch day only to some pilot computer groups.

After a while the rest of all computer groups get the updates approved manually.

Due to solve my problem I had reduced the declined updates last days, which did not help that. But my issue already exists before this action.

So please tell me why is this point so strange for you.

Upstream or master server 2008 sp2 x64:
All Updates: 5459
Approved: 5459

Lawrence Garvin, M.S., MCITP:EA, MCDBA, MCSA
SolarWinds Head Geek
Microsoft MVP - Software Distribution (2005-2012)
My MVP Profile: http://mvp.support.microsoft.com/profile/Lawrence.Garvin
The views expressed on this post are mine and do not necessarily reflect the views of SolarWinds.

Hi,

The problem I have is, the downstream servers cannot synchronize to their upstream server. Every time gets the synchronization with 0%  the error 'OutOfMemoryException'…
Nevertheless the same problem exists with the version 3.2.7600.226 too.
The Automatic Approvals is activated for several computer groups, which during this process occurred no error.
I assume the failure occured as soon as I approved many updates to all 60 computer groups.

We have one upstream w2k8 server and 22 downstream w2k3 server as replica.
All servers are locating in different geographical locations and have this problem.
The master upstream server has no synchronization problem to the MS update server.

These steps I have already done without any effect.
wsusutil reset
Dbcc checkdb
Reindexed DB
Reduced declined updates
Restarted IIS, BIT, SQL server, wsus service

Upstream or master server 2008 sp2 x64:
RAM 8GB
Wsus version: 3.2.7600.251
All Updates: 5459
Approved: 5459
Unapproved: 0
Declined: 0
Computer groups: 60
Internal database

Downstream server 2003 R2 sp2 x86:
RAM 4GB
Wsus version: 3.2.7600.251
All Updates: 5459
Approved: 5459
Unapproved: 0
Declined: 0
Internal database

2012-12-30 16:18:39.967 UTC Info w3wp.10 ThreadEntry ISAPIRuntime.ProcessRequest
2012-12-30 16:18:39.967 UTC Change w3wp.10 AdminDataAccess.StartSubscriptionManually Synchronization manually started
2012-12-30 16:18:40.811 UTC Info w3wp.22 SusEventDispatcher.TriggerEvent TriggerEvent called for NotificationEventName: ConfigurationChange, EventInfo: ConfigurationChange
2012-12-30 16:18:40.827 UTC Info w3wp.23 SusEventDispatcher.DispatchManagerWorkerThreadProc DispatchManager Worker Thread Processing NotificationEvent: ConfigurationChange
2012-12-30 16:18:40.827 UTC Info w3wp.21 SusEventDispatcher.RegisterEventHandler RegisterEventHandler called for NotificationEventName: ConfigurationChange
2012-12-30 16:18:43.108 UTC Info w3wp.17 SusEventDispatcher.TriggerEvent TriggerEvent called for NotificationEventName: ConfigurationChange, EventInfo: ConfigurationChange
2012-12-30 16:18:43.108 UTC Info w3wp.13 SusEventDispatcher.TriggerEvent TriggerEvent called for NotificationEventName: ConfigurationChange, EventInfo: ConfigurationChange
2012-12-30 16:18:43.108 UTC Info w3wp.38 SusEventDispatcher.DispatchManagerWorkerThreadProc DispatchManager Worker Thread Processing NotificationEvent: ConfigurationChange
2012-12-30 16:18:43.124 UTC Info w3wp.37 SusEventDispatcher.DispatchManagerWorkerThreadProc DispatchManager Worker Thread Processing NotificationEvent: ConfigurationChange
2012-12-30 16:18:43.124 UTC Info w3wp.38 RevisionIdCacheChangeNotificationDispatcher.InternalEventHandler Get event ConfigurationChange from dispatchmanager
2012-12-30 16:18:43.608 UTC Info WsusService.7 SusEventDispatcher.TriggerEvent TriggerEvent called for NotificationEventName: CatalogSyncAgent, EventInfo:
2012-12-30 16:18:43.608 UTC Info WsusService.16 SusEventDispatcher.DispatchManagerWorkerThreadProc DispatchManager Worker Thread Processing NotificationEvent: CatalogSyncAgent
2012-12-30 16:18:43.608 UTC Info WsusService.7 SusEventDispatcher.TriggerEvent TriggerEvent called for NotificationEventName: ConfigurationChange, EventInfo: ConfigurationChange
2012-12-30 16:18:43.608 UTC Info WsusService.16 SusEventDispatcher.DispatchManagerWorkerThreadProc DispatchManager Worker Thread Processing NotificationEvent: ConfigurationChange
2012-12-30 16:18:43.624 UTC Info WsusService.28 ThreadEntry ThreadHelper.ThreadStart
2012-12-30 16:18:43.624 UTC Info WsusService.28 EventLogEventReporter.ReportEvent EventId=382,Type=Information,Category=Synchronization,Message=A manual synchronization was started.
2012-12-30 16:18:43.655 UTC Info WsusService.16 WebServiceCommunicationHelper.ConfigChangedHandler Update server configuration has changed. Reporting service: stats.update.microsoft.com, SyncFromMU: False
2012-12-30 16:18:43.671 UTC Info WsusService.28 CatalogSyncAgentCore.ExecuteSyncProtocol Server ID is da9aa6fb-ea7f-4929-862a-7d6e9eae1212
2012-12-30 16:18:44.530 UTC Info w3wp.25 SusEventDispatcher.TriggerEvent TriggerEvent called for NotificationEventName: ConfigurationChange, EventInfo: ConfigurationChange
2012-12-30 16:18:44.546 UTC Info w3wp.36 ThreadEntry ThreadHelper.ThreadStart
2012-12-30 16:18:44.546 UTC Info w3wp.36 SusEventDispatcher.DispatchManagerWorkerThreadProc DispatchManager Worker Thread Processing NotificationEvent: ConfigurationChange
2012-12-30 16:18:44.546 UTC Info w3wp.36 ChangeNotificationDispatcher.InternalEventHandler Get event ConfigurationChange from dispatchmanager
2012-12-30 16:18:44.546 UTC Info w3wp.26 Client.OnConfigurationChange Creating a new ClientImplementation because the DB configuration changed
2012-12-30 16:18:44.546 UTC Info w3wp.26 ClientImplementation..ctor Initializing ClientWebService ProcessID = 3700, Process Start Time = 12/30/2012 1:54:47 PM, Product Version = 3.2.7600.251
2012-12-30 16:18:45.827 UTC Info w3wp.22 SusEventDispatcher.TriggerEvent TriggerEvent called for NotificationEventName: ConfigurationChange, EventInfo: ConfigurationChange
2012-12-30 16:18:45.842 UTC Info w3wp.18 ThreadEntry ThreadHelper.ThreadStart
2012-12-30 16:18:45.842 UTC Info w3wp.18 SusEventDispatcher.DispatchManagerWorkerThreadProc DispatchManager Worker Thread Processing NotificationEvent: ConfigurationChange
2012-12-30 16:18:45.842 UTC Info w3wp.21 SusEventDispatcher.RegisterEventHandler RegisterEventHandler called for NotificationEventName: ConfigurationChange
2012-12-30 16:18:46.202 UTC Change WsusService.28 AdminDataAccess.ExecuteSPSetConfiguration WSUS configuration has been changed
2012-12-30 16:18:46.202 UTC Info WsusService.28 CatalogSyncAgentCore.SyncConfigUpdatesFromUSS Category Sync: Filter: <filter SyncAnchor="716667,2012-12-30 16:10:03.702" GetConfig="1"></filter>
2012-12-30 16:18:46.764 UTC Info WsusService.28 CatalogSyncAgentCore.SyncConfigUpdatesFromUSS Need 0 config updates, 0 are new
2012-12-30 16:18:46.764 UTC Info WsusService.28 CatalogSyncAgentCore.SyncConfigUpdatesFromUSS Category Sync: New Config Anchor: 716667,2012-12-30 16:19:50.501
2012-12-30 16:18:46.796 UTC Info WsusService.28 CatalogSyncAgentCore.ExecuteSyncProtocol Catalog Sync: Filter: <filter SyncAnchor="0,2000-01-01 00:00:01.000"><languageList><language delta="1" value="1033"> </languageList></filter>
2012-12-30 16:18:48.124 UTC Info w3wp.17 SusEventDispatcher.TriggerEvent TriggerEvent called for NotificationEventName: ConfigurationChange, EventInfo: ConfigurationChange
2012-12-30 16:18:48.124 UTC Info w3wp.13 SusEventDispatcher.TriggerEvent TriggerEvent called for NotificationEventName: ConfigurationChange, EventInfo: ConfigurationChange
2012-12-30 16:18:48.139 UTC Info w3wp.40 ThreadEntry ThreadHelper.ThreadStart
2012-12-30 16:18:48.139 UTC Info w3wp.40 SusEventDispatcher.DispatchManagerWorkerThreadProc DispatchManager Worker Thread Processing NotificationEvent: ConfigurationChange
2012-12-30 16:18:48.139 UTC Info w3wp.41 ThreadEntry ThreadHelper.ThreadStart
2012-12-30 16:18:48.139 UTC Info w3wp.40 RevisionIdCacheChangeNotificationDispatcher.InternalEventHandler Get event ConfigurationChange from dispatchmanager
2012-12-30 16:18:48.139 UTC Info w3wp.41 SusEventDispatcher.DispatchManagerWorkerThreadProc DispatchManager Worker Thread Processing NotificationEvent: ConfigurationChange
2012-12-30 16:18:48.624 UTC Info WsusService.7 SusEventDispatcher.TriggerEvent TriggerEvent called for NotificationEventName: ConfigurationChange, EventInfo: ConfigurationChange
2012-12-30 16:18:48.624 UTC Info WsusService.24 SusEventDispatcher.DispatchManagerWorkerThreadProc DispatchManager Worker Thread Processing NotificationEvent: ConfigurationChange
2012-12-30 16:18:48.655 UTC Info WsusService.24 WebServiceCommunicationHelper.ConfigChangedHandler Update server configuration has changed. Reporting service: stats.update.microsoft.com, SyncFromMU: False
2012-12-30 16:18:49.546 UTC Info w3wp.25 SusEventDispatcher.TriggerEvent TriggerEvent called for NotificationEventName: ConfigurationChange, EventInfo: ConfigurationChange
2012-12-30 16:18:49.546 UTC Info w3wp.42 ThreadEntry ThreadHelper.ThreadStart
2012-12-30 16:18:49.561 UTC Info w3wp.42 SusEventDispatcher.DispatchManagerWorkerThreadProc DispatchManager Worker Thread Processing NotificationEvent: ConfigurationChange
2012-12-30 16:18:49.561 UTC Info w3wp.42 ChangeNotificationDispatcher.InternalEventHandler Get event ConfigurationChange from dispatchmanager
2012-12-30 16:18:49.561 UTC Info w3wp.26 Client.OnConfigurationChange Creating a new ClientImplementation because the DB configuration changed
2012-12-30 16:18:49.561 UTC Info w3wp.26 ClientImplementation..ctor Initializing ClientWebService ProcessID = 3700, Process Start Time = 12/30/2012 1:54:47 PM, Product Version = 3.2.7600.251
2012-12-30 16:18:50.842 UTC Info w3wp.22 SusEventDispatcher.TriggerEvent TriggerEvent called for NotificationEventName: ConfigurationChange, EventInfo: ConfigurationChange
2012-12-30 16:18:50.858 UTC Info w3wp.43 ThreadEntry ThreadHelper.ThreadStart
2012-12-30 16:18:50.858 UTC Info w3wp.43 SusEventDispatcher.DispatchManagerWorkerThreadProc DispatchManager Worker Thread Processing NotificationEvent: ConfigurationChange
2012-12-30 16:18:50.858 UTC Info w3wp.21 SusEventDispatcher.RegisterEventHandler RegisterEventHandler called for NotificationEventName: ConfigurationChange
2012-12-30 16:18:54.936 UTC Warning w3wp.7 SoapUtilities.CreateException ThrowException: actor = http://<server>/ClientWebService/client.asmx, ID=8e430128-1b6b-4fb5-951c-de5424619d33, ErrorCode=ConfigChanged, Message=, Client=415558c3-b27e-4bef-a596-7160f44e2ad7
2012-12-30 16:19:07.077 UTC Info WsusService.28 CatalogSyncAgentCore.ExecuteSyncProtocol Catalog Sync: New Anchor: 716667,2012-12-30 16:19:51.610
2012-12-30 16:19:07.561 UTC Info WsusService.28 CatalogSyncAgentCore.ExecuteSyncProtocol Need 48297 updates, 0 are new
2012-12-30 16:19:07.561 UTC Info WsusService.28 CatalogSyncAgentCore.ExecuteSyncProtocol Total actually inserted updates: 0
2012-12-30 16:19:07.561 UTC Info WsusService.28 CatalogSyncAgentCore.ReplicaSync DeploymentSync: starting full sync
2012-12-30 16:19:07.561 UTC Info WsusService.28 CatalogSyncAgentCore.ReplicaSync DeploymentSync: anchors: "" to "716667,2012-12-30 16:19:51.610"
2012-12-30 16:19:10.358 UTC Warning w3wp.5 SoapUtilities.CreateException ThrowException: actor = http://<server>/ClientWebService/client.asmx, ID=c2b65e86-03c2-47c1-ab45-7e1d592d3915, ErrorCode=ConfigChanged, Message=, Client=5ddd6843-b615-44c4-92a8-ff5761e6fb79
2012-12-30 16:19:18.139 UTC Info w3wp.8 ThreadEntry _TimerCallback.PerformTimerCallback
2012-12-30 16:19:18.139 UTC Info w3wp.8 ServerImplementation.UpdateCache Database change occured; check if we need to update cache.
2012-12-30 16:21:23.499 UTC Warning w3wp.1 SoapUtilities.CreateException ThrowException: actor = http://<server>/ReportingWebService/ReportingWebService.asmx, ID=b59fe272-0909-42c6-81f9-de4d35696f57, ErrorCode=ConfigChanged, Message=, Client=d2b2c050-2ebf-4d18-a5c7-540aaa77f929
2012-12-30 16:21:23.514 UTC Error w3wp.1 WebService.ReportEventBatch Exception occured in ReportEventBatch: Fault occurred
   at Microsoft.UpdateServices.Internal.Reporting.WebService.ReportEventBatch(Cookie cookie, DateTime clientTime, ReportingEvent[] eventBatch)
   at System.RuntimeMethodHandle._InvokeMethodFast(Object target, Object[] arguments, SignatureStruct& sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)
   at System.RuntimeMethodHandle.InvokeMethodFast(Object target, Object[] arguments, Signature sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean skipVisibilityChecks)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Web.Services.Protocols.LogicalMethodInfo.Invoke(Object target, Object[] values)
   at System.Web.Services.Protocols.WebServiceHandler.Invoke()
   at System.Web.Services.Protocols.WebServiceHandler.CoreProcessRequest()
   at System.Web.Services.Protocols.SyncSessionlessHandler.ProcessRequest(HttpContext context)
   at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
   at System.Web.HttpApplication.ApplicationStepManager.ResumeSteps(Exception error)
   at System.Web.HttpApplication.System.Web.IHttpAsyncHandler.BeginProcessRequest(HttpContext context, AsyncCallback cb, Object extraData)
   at System.Web.HttpRuntime.ProcessRequestInternal(HttpWorkerRequest wr)
   at System.Web.HttpRuntime.ProcessRequestNoDemand(HttpWorkerRequest wr)
   at System.Web.Hosting.ISAPIRuntime.ProcessRequest(IntPtr ecb, Int32 iWRType)
2012-12-30 16:22:27.608 UTC Warning w3wp.7 SoapUtilities.CreateException ThrowException: actor = http://<server>/ClientWebService/client.asmx, ID=a0f46442-1628-4693-840c-6ab9fe923558, ErrorCode=ConfigChanged, Message=, Client=d2b2c050-2ebf-4d18-a5c7-540aaa77f929
2012-12-30 16:22:33.233 UTC Error WsusService.28 ServerSyncCompressionProxy.GetWebResponse GetWebResponse exception:
   at Microsoft.UpdateServices.ServerSync.ServerSyncCompressionProxy.GetWebResponse(WebRequest webRequest)
   at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
   at Microsoft.UpdateServices.ServerSyncWebServices.ServerSync.ServerSyncProxy.GetDeployments(Cookie cookie, String deploymentAnchor, String syncAnchor)
   at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.WebserviceGetDeployments(String anchorMin, String anchorMax)
   at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.ReplicaSync()
   at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.ExecuteSyncProtocol(Boolean allowRedirect)
   at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.CatalogSyncThreadProcess()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.runTryCode(Object userData)
   at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
2012-12-30 16:22:33.233 UTC Error WsusService.28 WebServiceCommunicationHelper.ProcessWebServiceProxyException ProcessWebServiceProxyException found Exception was an Enexpected Type. Action: No Retry, Fail. Exception Details: System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at Microsoft.UpdateServices.ServerSync.ServerSyncCompressionProxy.GetWebResponse(WebRequest webRequest)
   at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
   at Microsoft.UpdateServices.ServerSyncWebServices.ServerSync.ServerSyncProxy.GetDeployments(Cookie cookie, String deploymentAnchor, String syncAnchor)
   at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.WebserviceGetDeployments(String anchorMin, String anchorMax)
   at Microsoft.UpdateServices.Internal.WebServiceCommunicationHelper.ProcessWebServiceProxyException(SoapHttpClientProtocol& webServiceObject, Exception exceptionInfo)
   at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.WebserviceGetDeployments(String anchorMin, String anchorMax)
   at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.ReplicaSync()
   at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.ExecuteSyncProtocol(Boolean allowRedirect)
   at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.CatalogSyncThreadProcess()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.runTryCode(Object userData)
   at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
2012-12-30 16:22:33.233 UTC Error WsusService.28 CatalogSyncAgentCore.ExecuteSyncProtocol System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at Microsoft.UpdateServices.ServerSync.ServerSyncCompressionProxy.GetWebResponse(WebRequest webRequest)
   at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
   at Microsoft.UpdateServices.ServerSyncWebServices.ServerSync.ServerSyncProxy.GetDeployments(Cookie cookie, String deploymentAnchor, String syncAnchor)
   at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.WebserviceGetDeployments(String anchorMin, String anchorMax)
   at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.ReplicaSync()
   at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.ExecuteSyncProtocol(Boolean allowRedirect)
   at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.ExecuteSyncProtocol(Boolean allowRedirect)
   at Microsoft.UpdateServices.ServerSync.CatalogSyncAgentCore.CatalogSyncThreadProcess()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.runTryCode(Object userData)
   at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
2012-12-30 16:22:33.233 UTC Info WsusService.28 CatalogSyncAgentCore.UpdateSyncResultAndGenerateReportingEvent CatalogSyncThreadProcess: report subscription Undefined error class or Success
2012-12-30 16:22:33.233 UTC Info WsusService.28 EventLogEventReporter.ReportEvent EventId=386,Type=Error,Category=Synchronization,Message=Synchronization failed. Reason: Exception of type 'System.OutOfMemoryException' was thrown..
2012-12-30 16:22:33.249 UTC Info WsusService.17 CatalogSyncAgent.WaitUntilSyncFinishedOrCancelled Agent signalled done.
2012-12-30 16:22:33.249 UTC Info WsusService.17 CatalogSyncAgent.SetSubscriptionStateWithRetry Firing event SyncFailToStart...
2012-12-30 16:22:33.249 UTC Info WsusService.17 CatalogSyncAgent.WakeUpWorkerThreadProc Found no more jobs. CatalogSyncAgent quits but will run rollup before terminating ...
2012-12-30 16:22:33.264 UTC Info WsusService.17 CatalogSyncAgent.UpdateServerHealthStatusBasedOnError ServerHealth: Updating Server Health for Component: CatalogSyncAgent, Marking as Not Running
2012-12-30 16:22:34.577 UTC Info WsusService.7 SusEventDispatcher.TriggerEvent TriggerEvent called for NotificationEventName: RollupAgent, EventInfo:
2012-12-30 16:22:34.577 UTC Info WsusService.27 ThreadEntry ThreadHelper.ThreadStart
2012-12-30 16:22:34.577 UTC Info WsusService.27 SusEventDispatcher.DispatchManagerWorkerThreadProc DispatchManager Worker Thread Processing NotificationEvent: RollupAgent

I hope you could help me.

Thanks,

kailar

Hi Lawrence,

Thanks for your answer.

I should say this thread with the issue synch. error "outofmemoryexeption" is resolved after reducing the approvals on upstream server according to your recommendation.

I am still fighting wiht my next problem deleting the four computer groups on all 20 replica server.

I could till now delete successfully these on half of the replica servers with costing too much time. The way only to unapprove the updates out of these groups could not help me, I have to decline the updates causing these problem one by one according to the logfile. And this is a big job.

I had to rebuild the replica WSUS by some of them too.

I have now a third problem, which I will create a new thread for that.

Thanks a lot.

kailar

It seems now I really have to rebuild all rest 17 replica servers one by one.

Given that we know the replicas are out of sync with the upstream -- a locally created Target Group with a different GroupID may just be one symptom of a larger collection of not-yet-identified issues, the more reliable course of action is definitely to reinstall the replica servers with new databases. (You can keep the existing content store, so no need to re-download files.)

After installing the new replica and synchronizing to upstream, run the Server Cleanup Wizard with the "Delete unneeded files.." option only (on the replica).

Lawrence Garvin, M.S., MCITP:EA, MCDBA, MCSA
SolarWinds Head Geek
Microsoft MVP - Software Distribution (2005-2012)
My MVP Profile: http://mvp.support.microsoft.com/profile/Lawrence.Garvin
The views expressed on this post are mine and do not necessarily reflect the views of SolarWinds.