What Happened
Recently when bringing a client site up to modern security levels an environmental power outage suddenly caused an unexpected failure in Microsoft Exchange emails being delivered to client BlackBerry devices through the installed BlackBerry Enterprise software.
Identification
Besides the aforementioned 5003 error returned in the system event log service start failure, the first indication of a problem that email is not being delivered to the BlackBerries. On the server the biggest indication is that the "BlackBerry Controller" service will not start. It also leaves a mess of errors in the application log (events 10000, 20000, 50091, and 50093 were seen).
Why It Fails
By default the BlackBerry Server uses ports 4070 and 4071 to communicate. The Microsoft update lets those ports be locked by the DNS server service preventing BlackBerry communication.
The Emergency Fix
For a short term fix to get things working immediately you can sometime stop the DNS Server service, start the BlackBerry Controller successfully then restart the DNS Server service. Note that the next time there is a restart it will fail again until the true fix is completed.
The Complete Fix
To permanently fix the service start problem you must reserve a pair of ports for the BlackBerry server and set the server to use those ports. In the steps below we will be using the default 4070 and 4071 ports, change them as needed.
Reserve Ports
Microsoft KB 812873 details how to reserve a port, but it is generally as simple as adding a 4070-4071 to the
multi-string valuedHKLM\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\ReservedPortskey.
Set BlackBerry Ports
In the case of the BlackBerry server ports you must change the following two registry keys:
- HKLM\SOFTWARE\Research In Motion\BlackBerry Enterprise Server\Agents\SysLogHost
- HKLM\SOFTWARE\Research In Motion\BlackBerry Enterprise Server\Logging Info\Mailbox Agent\SysLogHost
The value for the above keys should be the string (not multi-string!) value localhost:4071,localhost:4070
Completing the Fix
To complete the fix you will have to reboot the server, in the meantime the emergency fix listed above will keep email moving so the clients will not notice any problems.