How to remove registered services if a client is shutdown unexpectedly

    I register some services from one client, but for some reason, the client may be killed or shutdown unexpectedly, like by system reboot, but on the web UI of broker, I can still see the registered service are there for about one hour, since the client has gone, there may be some problems, next time when I register the same services, they will be duplicated, so is it possible that the registered services are removed immediately once the client is not there?

    Or is there any approach that can be used to remove these services from another client? so that I can cleanup the old registered services when the client is restarted.


    thank you

    The broker has some logic to clear out registered services when the client which performs the registrations gracefully disconnects. For cases, however, where the client does not gracefully disconnect (i.e., fails to send a TCP RST or FIN packet to the broker), the broker may not know that the client is no longer connected and, therefore, keeps the services in its registry.


    The services are likely remaining registered for up to one hour because of the default TTL configured during the service registration. You could reduce the time needed to detect the disconnect by using a smaller value for service TTL field. See the documentation for the ServiceRegistrationInfo.ttl field in the OpenDXL Python Client SDK for more information. Note, however, that this would result in the client using more network bandwidth to perform service re-registrations.


    Once the client performing the service registrations has disconnected from the broker, the next request made which the broker attempts to route to the disconnected client should fail with an "unable to locate service for request" error. At that point, the service registration should be removed from the broker - even if the service TTL has not yet expired.


    In the future, it would be nice to have some "retry" logic built into the broker to handle this case. For example, if the same service were registered from two different clients where 1 client is disconnected and 1 client is connected and the broker fails when attempting to route an incoming request to the disconnected client, it would be nice to have some logic to automatically "retry" the request to the other client (which could succeed in providing the response because it is still connected). This is something that may be considered for the future but some more work would need to be done to think though the right design approach.


    Another thing that you may want to look into if you have not already is whether your applications are written in such a way that when they are shutdown more gracefully that they first attempt to unregister services and disconnect from the fabric. When you use the OpenDXL Bootstrap Python project to generate the basic structure for your application code, the generated __main__.py file contains logic to register a signal handler which allows the application to be stopped before the main thread exits and the process dies. When the application is stopped, an attempt should be made to unregister any services which have been registered and disconnect the client from the DXL fabric. Assuming that a system reboot were able to first signal the application to exit, this cleanup work should allow the broker to immediately unregister the client's services.