Handling Slow Third-Party API
Originally Published here
Recently, I had to make use of a third-Party API service that takes at least 30 seconds to respond.
The information these API services offer is not frequently requested by users but needed at some point by the user.
I can wait for the response to come back from the third-party service, but are users willing to wait for a response? A user may not be willing to wait for a reply and cancel the request before a response is returned.
If the user can’t wait for the response to complete, they won’t be able to get the info since it will always take 30 seconds or more to complete.
Every request to the API service costs money. If the user cancels the request before a response comes back, I will still pay for the request 😑. Some users will retry which means more money spent on getting nothing.
I have to come up with a way to reduce the request to the third-party API service, run the operation in the background in case the user cancels the request, and improving response time.
For this, I made use of a function that only calls the API once for a unique info_id. Other requests with the same info_id will wait for the response of the first call to complete. When the response arrives, every request waiting will get the response. The value returned is saved in a DB/Cache to speed up future requests.
This way, the user will wait for how long it takes the first request to complete. Requests made before the response of the first request will wait for the response, while requests made after it completes, get the value from the DB/Cache.
Implemetation
Using it within an HTTP handler
Test file to check if it works as expected.
Running the test file returns the image we see below. The image shows the request was completed in approximately 5 seconds, the time it takes our fake request to complete.
We can see the request was to the slow fake service was called only once, and even when the request that made the call timed out, other requests with the same info_id still got a response.
Money saved for the remaining request to the same info_id. We can now use the extra cash to buy something for ourselves.
Conclusion
I don’t know what to conclude with. I have run out of wise words to say 🤔.
Let me just say using the right algorithm can save money, time, and other things good algorithms can save.
The End.
