Implementing asynchronous endpoints in REST with callbacks

(click on the image to view it in full resolution)

This is an extension of the asynchronous endpoint pattern where the consumer can register a callback URL, which is invoked when the task is completed. This only works if the consumer has the option to expose an externally hittable REST endpoint. The advantage is that the consumer does not have to keep polling the service to see if the task has completed.

Once the task completes, the service invokes the callback URL. A common approach is to send just a task id to the callback, and the consumer then invokes a call to the service to fetch the full payload. A less common approach (if sensitive info is not included in the payload) is to send the entire result-set to the callback endpoint.

On the service side, the implementation is done by reacting to the task completed event by invoking the registered callback for the completed task. There are several ways to do this. A simple approach is for the task worker itself to invoke the callback. The disadvantage there is that it’s complex to now support retries if the callback endpoint is down. With AWS, a common approach is to update the data in a DynamoDB table and for a Lambda to be triggered off that, and the Lambda would then invoke the callback. A similar approach for Azure is to have a Function triggered off Table storage or Cosmos DB, which would invoke the callback. Both AWS and Azure have mechanisms to add retry to the function/lambda, to handle scenarios where the callback URI is down.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s