Re-try pattern using Akka actor / ask pattern
Hi there...
I thought of sharing a relatively simple gist of Retry akka pattern I came up recently. It could be useful in case Client actor sends request to another Service provider actor and request may have to be retried due to different reasons (network failure in distributed environment, service down or other...). To increase resilience Client attempts to re-try operation until either response is received or after number of retries exceeded Client simply gives up.
The pattern is quite generic and I thought it would be cool to have a pluggable re-tries for akka actors.
The implementation assumes following:
1) The service provider actor is supposed to respond to a Client actor. (it is not just fire and forget scenario)
2) Client attempts to send multiple requests: Each request is given an individual Timeout and requests are retried after Retry interval
3) Each request completes as Time out (service did not respond) or actual response message from Service provider. Timeouts are simply logged, while the response is propagated to the original sender actor and the Retry proxy is stopped.
4) In case of total failure - after retries exceeded exception sent to an original sender actor.
If you need to implement retries here is how you could do that.
Example 1. Existing code using tell:
Example 2. Or using the akka ask pattern:
Above will be re-tried 10 times with individual time out of 1 sec per try and with 100 milliseconds interval between tries.
Gist below:
And several test cases here...
I thought of sharing a relatively simple gist of Retry akka pattern I came up recently. It could be useful in case Client actor sends request to another Service provider actor and request may have to be retried due to different reasons (network failure in distributed environment, service down or other...). To increase resilience Client attempts to re-try operation until either response is received or after number of retries exceeded Client simply gives up.
The pattern is quite generic and I thought it would be cool to have a pluggable re-tries for akka actors.
The implementation assumes following:
1) The service provider actor is supposed to respond to a Client actor. (it is not just fire and forget scenario)
2) Client attempts to send multiple requests: Each request is given an individual Timeout and requests are retried after Retry interval
3) Each request completes as Time out (service did not respond) or actual response message from Service provider. Timeouts are simply logged, while the response is propagated to the original sender actor and the Retry proxy is stopped.
4) In case of total failure - after retries exceeded exception sent to an original sender actor.
If you need to implement retries here is how you could do that.
Example 1. Existing code using tell:
worker ! someWorkCan be modified to include re-tries as:
context.actorOf(ReTry.props(tries = 2, retryTimeOut = 1000 millis, retryInterval = 100 millis, worker)) ! SomeWorkUsually the response will be listened for in the receive loop
Example 2. Or using the akka ask pattern:
val futureResult = worker ? someWorkCould be modified to include re-tries as:
val futureResult = context.actorOf(ReTry.props(tries = 10, retryTimeOut = 1000 millis, retryInterval = 100 millis, worker)) ? SomeWork
Above will be re-tried 10 times with individual time out of 1 sec per try and with 100 milliseconds interval between tries.
Gist below:
And several test cases here...