This week I was working on the backend of a new application. One of the requirements was to convert addresses to coordinates. This process is also known as forward geocoding. Reverse geocoding is exactly the opposite, where you convert coordinates to addresses. Since we don’t have that geographical data ourselves, we decided to look at an external service. We choose for a competitor of Google Maps, but that does not really matter. What I want to share with you here, is the solution that we came up with for safely integrating that external service.
About integrating external services.
In my first years as a developer I would not think long about implementing the connection with this service. “How hard can it be? Just call the web service and use the response. What could go wrong?” Nowadays I tend to take a step back and consider a few things:
What happens when the external service is not available? Will this kill the process? What will the customer see? Should we just keep retrying until their web service is back? Should we cache the responses on our end?
The solution that we came up with.
When we receive a new address in our system, it needs to be forward geocoded immediately. In an event driven architecture this is easy to accomplish. We just listen to the AddressWasAdded event and dispatch a GeoCodeAddress command on the async command bus. This command gets picked up by one of the workers and is handled by the correct handler. The handler tries to connect with the external service and saves the coordinates in the database. Because the external web service is billing us per request, the GeoCoderService caches the given coordinates for about one day. So if the same address is requested, it will hit the cache and not the actual service. The real implementation is more nuanced but globally this is how it looks:
The main takeaway of this solution is the introduction of the Async Queue. This is solving most of our problems. If an error happens in the Geo Coder Service, the command is requeued and will be retried at a later time until it’s handled successfully.
Because we work with an event driven architecture and CQRS, processes are triggered via events and commands. But this solution can be implemented in any architecture. Just make sure that you application publishes a message on an async bus and let a consumer handle the rest.
I am very interested in your thoughts on this solution and if I am missing important questions about integrating external services. Please leave a comment to share your thoughts.