Wednesday, 4 November 2009

Public Folder Mayhem Exchange 2010

I have been working at a costomer on a Exchange 2010 implementation in an existing Exchange 2007 environment.

All went pretty smooth until we came to the public folder replication.

For some reason we can't replicate public folders.
We even could not replicate public folder hierarchy.

The HUB Transport server dropped the update message with the error:

554 5.6.0 STOREDRV.Deliver.Exception:ObjectNotFoundException;
Failed to process message due to a permanent exception with message The Active Directory user wasn't found.


In the event log we received an event:
Error
4-11-2009
12:52:23
MSExchange Store Driver
1020
MSExchangeStoreDriver

The store driver couldn't deliver the public folder replication message "Status Request (PublicFolder@domain.local)" because the following error occurred: The Active Directory user wasn't found..


For some reason the HUB Transport server is unable to resolve the recipient.

We ran into brickwalls over and over again and finally descided to scratch Exchange 2010 completely and start all over again....

Removing the last exchange 2010 mailbox server in your environment is a bit different than it was in exchange 2007.

Exchange 2010 uses so called arbitrary mailboxes.
These mailboxes can't be retrieved by the normal command Get-Mailbox.

You have to use:

get-mailbox -Arbitration

Name Alias ServerName ProhibitSendQuota
---- ----- ---------- -----------------
SystemMailbox{1f05a... SystemMailbox{1f0... exch-ut-mbx unlimited
SystemMailbox{e0dc1... SystemMailbox{e0d... exch-ut-mbx unlimited
FederatedEmail.4c1f... FederatedEmail.4c... exch-ut-mbx 1 MB


So to remove these mailboxes you have to use:

get-mailbox -Arbitration Remove-Mailbox -RemoveLastArbitrationMailboxAllowed -Arbitration

Then all mailboxes are removed and you can uninstall exchange 2010.

But because you removed some system mailboxes you cannot install exchange 2010.

It will fail because the federation mailbox and discover mailbox are missing.

To fix this issue you have to run setup.com /prepareAD.
Even if you already had prepared the AD you need to rerun this command.
The PrepareAD will create these accounts again so you can re-install Exchange 2010.

So back to my initial problem.
The public folder replication issue.

In our first configuration we had three servers.
In the main AD site:
1 HUB / CA server
1 Mailbox server

In a remote AD site:
1 Mailbox / HUB / CA server

To make sure PF replication is working we decided to begin with just one server running all roles.

Installation went smooth and all was up and running.
Then we configured the E2010 public folder database and setup replication.

NOTHING!!!
Same error, same problem.

Until now we have not figured out what the problem is and how we can fix it.

We see a couple of things:

  1. Exchange 2007 deliveres the message to the Exchange 2010 HUB server
  2. The event entry saying message could not be delivered.
  3. The error:

    554 5.6.0 STOREDRV.Deliver.Exception:ObjectNotFoundException; Failed to process message due to a permanent exception with message The Active Directory user wasn't found.
  4. in the connectivity logs we also see:

    08CC2B6692C30788,MAPI,mbx.domain.local,>,Starting delivery
    08CC2B6692C30788,MAPI,mbx.domain.local,>,Connecting to server mbx.domain.local session type Public Folder
    08CC2B6692C30788,MAPI,mbx.domain.local,>,Failed to connect to server mbx.domain.local
    08CC2B6692C30788,MAPI,mbx.domain.local,-,Messages: 0 Bytes: 0 Recipients: 0

At this point we are still trying to solve this issue.

I will keep you posted.


53 comments:

  1. Please do. You're not the only one in this situation! I've had to remove all mail-enabled public folders and substitute mailboxes until it can be solved.

    ReplyDelete
  2. Yes, the same is happening to us!

    ReplyDelete
  3. I have exchange 2010, and i delete the public folder, and created from the beging, and when i tried to send an email to a mail enabked public folder, i always get this NDR:

    #554 5.6.0 STOREDRV.Deliver.Exception:ObjectNotFoundException; Failed to process message due to a permanent exception with message The Active Directory user wasn't found. ##

    kindly advice, if anyone get this issue.

    thx in advance,

    ReplyDelete
  4. Yes I get the same problem Microsoft are you listening ?

    ReplyDelete
  5. Same Problem here....

    ReplyDelete
  6. In my case, I am trying to move PF replicas from EXC 2007 to 2010. Same error.

    ReplyDelete
  7. Called microsoft PSS no results either. still waiting from the escalation team.. keep you posted.

    ReplyDelete
  8. Same problem with moving Public folder from Exch 07 to Exch 2010. Cannot even move hierachy.

    ReplyDelete
  9. Anyone find a solution for this issue? I Have the same problem and I don't find anything :(

    ReplyDelete
  10. Public Folder support in Exchange 2010 is utterly abysmal. The only leverage I have to persuade my clients to go Exchange versus Google Apps is public folders, and every client I've migrated to Exchange 2010 has experience all-out failure on public folders.

    ReplyDelete
  11. Microsoft finally resolved my public folder not replicating..between 2007 and exchange 2010... escalation senior trace the issue. i have some lingering objects in active directory from exchange 2003 environment, there was a empty container/objects cn=First Administrator Group
    "Servers" which an empty container microsoft engineer deleted that object and there goes start replicating. first i was told to install the exhange 2003 admin tools. but did not display the server folder so end up deleting from adsiedit.msc .. hope this helps for everyone.. took a week to fix the issue..microsoft said that they will publish the article later on.

    ReplyDelete
  12. Thank you!!!
    Now it's possible to replicate between an exchange 2007 to 2010.

    ReplyDelete
  13. Does that mean I can delete the entire old Exchange 2003 Administrative Group?

    ReplyDelete
  14. This comment has been removed by a blog administrator.

    ReplyDelete
  15. "Does that mean I can delete the entire old Exchange 2003 Administrative Group?"

    I am wondering the same

    ReplyDelete
  16. I've just experienced the same issue. Replication between the 2007 and 2010 public folder hierarchy wouldn't start until I removed that empty servers folder with adsi edit. This is clearly something they need to fix.

    ReplyDelete
  17. Unbelievable. We are told that we aren't supposed to delete the Administrative Group even after Exchange 2003 is gone. I have the same problem and I will try the fix and ignore the advice about keeping the AG.

    Thanks.

    ReplyDelete
  18. My mistake. I thought I understood that the fix was to delete the entire Administrative Group because of the way the above post wrapped. For anyone else that may have misunderstood like me, it's only the Servers folder under an Administrative Grup that should be deleted, and only if it's empty.

    ReplyDelete
  19. We have exactly the same problem at a customer right now. I've been on the phone with Microsoft all day long, i'll post any solutions here as soon as i got them.

    ReplyDelete
  20. I had the problem when sending mail to a mail-enabled public folder. The problem started at around the time I applied the following modifications to the Exchange 2010 server:

    ****************
    (From http://ericsweblog.spaces.live.com/blog/cns!5C97571A12812235!189.entry)

    In the HOSTS file, changed line
    ::1 localhost
    to
    # ::1 localhost

    Open your regedit on Exchange server and add under:

    HKLM\SYSTEM\CurrentControlSet\Services\MSExchangeSA\Parameters\

    new multi string value (REG_MULTI_SZ) called NSPI Target Server. Inside it write down your local site Domain Controller(s). Next thing you need to do is go down to your Domain Controller(s) and open up the registry go down to:

    HKLM\SYSTEM\CurrentControlSet\Services\NTDS\Parameters\

    and add the following new multi string value (REG_MULTI_SZ) called NSPI Interface protocol sequences. Inside it write this: ncacn_http:6004, after this is done reboot your Domain Controller(s) and Exchange server.

    ****************

    The fix the issue, I deleted the empty CN=Servers folder from the old Exchange 2003 Admin Group.

    ReplyDelete
  21. Same solution worked for us too!

    ReplyDelete
  22. Hi, I am having the same problem, but I had a new install of Exchange 2007, pretty much unconfigured, no previous 2003. The client then decided to use Exchange 2010 so I did an uninstall of exchange 2007 and then an install of 2010 on the same server. I am now getting this error, the servers container isnt empty so cant delete it. The replica's all look ok, and t it is only the Outlook 2003 that cant get the OAB and the error is logged in the event viewer that there are no replica's. I dont get it and am about to pull out the rest of my hair.

    ReplyDelete
  23. Hi, just wanted to leave a short note for anyone who is trying to migrate Ex2003 to Ex2010. If you run into problems with public folders, DON'T unter any circumstances delete the "First Administrative Group" hive in AD.

    Our PFs were neither administrable nor usable for users after the deletion. We needed MS technical support to fix the issue. Now we're back to usability via Outlook but still no luck with administration of PFs.

    Long story short: Deleting the First Admin Group does NOT apply to migration scenarios where 2003 is involved!

    ReplyDelete
  24. ...and it worked also for me :-)

    many thanks

    ReplyDelete
  25. Deleting the empty servers container solved the issue for me. Good catch!

    ReplyDelete
  26. Re: Long story short: Deleting the First Admin Group does NOT apply to migration scenarios where 2003 is involved!

    Same problem here. Got the folders working in all Outlook versions, but killed the OWA access.

    ReplyDelete
  27. Same problem about mail enabling public folders, great. after 4 days find this post, Thank you very much.

    ReplyDelete
  28. David Eriksson25 May 2010 at 11:29

    just about 1 minute from calling microsoft when nothing else helped.

    Thank you very much!

    ReplyDelete
  29. I'm with you. What a bunch of BS.

    ReplyDelete
  30. The Microsoft Exchange team has posted an article regarding this issue in which they announce a fix will be scheduled in one of the upcoming rollups.

    To read the article see:
    Public Folder Replication Fails Due To Empty Legacy Administrative Group
    http://msexchangeteam.com/archive/2010/05/05/454821.aspx

    Hopefully this will indeed solve these issues.

    ReplyDelete
  31. Wow Seriosly. Microsoft should have had an KB article on this. I have been killing myself for 2 days wondering why it wasnt working. Sure enough as soon as I removed the server folder in adsiedit boom replication worked.

    ReplyDelete
  32. What is this folder isn't emty? Then i can't delete it without destroying my working exchange. What can then be done? Nothing works here...

    ReplyDelete
  33. Could I ask where is this server container you guys are refering to?
    Have the same problem, but cant find an empty conatainer.

    ReplyDelete
  34. Launch ADSIEdit.msc and navigate to CN=Configuration , CN=Services, CN=Microsoft Exchange, CN=, CN=Administrative Groups, CN=, CN=Servers.

    ReplyDelete
  35. Launch ADSIEdit.msc and navigate to CN=Configuration , CN=Services, CN=Microsoft Exchange, CN=, CN=Administrative Groups, CN=, CN=Servers.

    ReplyDelete
  36. Just FYI you need to
    Launch ADSIEdit.msc and choose "Configuration" as the "naming context" to see the CN=configuration. Took me a couple of mins to realise ;)

    ReplyDelete
  37. Migrated from Exch 2003 to 2010, last 2003 exchange server uninstalled...

    Removed the entire First Admin Group (prior to reading DON'T remove the entire First Admin Group) instead of just the Servers; solved this problem for us with no side effects (yet).

    ReplyDelete
  38. Had exactly the same issues, and was tearing my hair out for weeks before I finally found the 'solution' named above.

    Still can't see my PFs in Outlook, but I'll give it a while to see if it'll replicate over the weekend.

    Utterly abysmal situation that MS has let happen. Surely this is a routine operation for people with E2k3? Did they not test this?!

    ReplyDelete
  39. Thanks for your post and welcome to check: here
    .

    ReplyDelete
  40. According to Exchange team plans have changed. There will be no fix released for this issue.

    ReplyDelete
  41. Your post to cancel the CN=Server container on the Legacy First Administrative Group has work to me too.
    I have the same error after decommissioning the Exch 2003 a Exch 2003->Exch 2010 transition. Thank you.
    Enrico

    ReplyDelete
  42. Still a valid fix. Thanks for the help!

    ReplyDelete
  43. For me it was:

    Launch ADSIEdit.msc and navigate to CN=Configuration , CN=Services, CN=Microsoft Exchange, CN=, CN=Administrative Groups, CN=First Administrative Group, CN=Servers.

    Thank you!!!!!!

    ReplyDelete
  44. Excellent! worked for me as well!

    thanks!

    ReplyDelete
  45. I seem to be having the same problem although I cannot find the above error 1020 anywhere but I cannot replicate PF's between co-existing Ex2007-2010 environment which is setup and working correctly, there was a previous Ex2003 environment in place prior to this.

    Can someone please confirm this with me? So I look adsiedit.msc and go to CN=Configuration, CN=Services, CN=Microsoft Exchange, CN=nameofmydomain, CN=Administrative Groups and in here I have 3 CN's listed. One is CN=Exchange Administrative Group (FYDI.....) which has all my Ex2007 and Ex2010 servers listed in CN=Servers but the other 2 CN's (CN=Headoffice, CN-Branchoffice) both have the empty CN=Servers folder. Do I delete both of these empty CN=Servers folders?

    Thank you for you help.

    ReplyDelete
  46. Ok I have answered my question. I did find the 1020 error (i was looking on the wrong server) and since then I deleted both the empty CN=Servers containers and now PF's are replicated and working on exchange 2010. Thank you so much for this article it solved my problem.

    ReplyDelete
  47. this worked for me too.
    thanks all

    ReplyDelete
  48. unbelievable: delete the empty CN=Server folder worked form me too. No restart just instantly working! Delete ... not even 5 seconds later sent a test-email and itwas there!
    Thanks all.

    ReplyDelete
  49. I deleted the empty CN=Servers containers and now everything is working.

    Also the Microsoft article that was referenced helped too.
    Public Folder Replication Fails Due To Empty Legacy Administrative Group
    http://msexchangeteam.com/archive/2010/05/05/454821.aspx

    Thanks for the Assistance.

    ReplyDelete
  50. Hey - I know you've got a ton of these, but just wanted to throw in yet another thank you for posting this. This was my final hurdle to overcome before my migration was complete. Thankyou!

    ReplyDelete
  51. Hello Dude,

    Public folders, introduced in the first version of Microsoft Exchange, are designed for shared access and provide an easy and effective way to collect, organize and share information with other people in your workgroup or organization. Thanks a lot!

    ReplyDelete
  52. Removing the Empty Server Container worked for me also.

    Thank you !!

    ReplyDelete
  53. Thank you, thank you, thank you. Removing the Empty Server Container worked for me too. I have recently removed the last Exch 2003 server from our environment (what a mission that was too!!!! - 2 user accounts (created & never used) still pointing to the old 2003 Exchange server - took me ages to find the problem before I could un-install exch2003).
    This fix for getting our mail enabled public folders to receive Email would have taken me for ever if I hadn't foud your post.

    Delete empty Server container!!!!! Who would have thought!! Come on MS!!

    ReplyDelete