Back in April 2023 when I was working at a French hospitality group, some business indicators led us to believe that the performance of our APIs was degraded. As I went to investigate, I found several issues which I will cover in several articles.
Today I will address a simple point : how to parallelize independent calls to external dependencies.
Monitoring and measuring
First things first : performance must be measured. Feelings can be an interesting entry point as they may be a symptom of real issues but cannot and must not replace actual measures. There are several tools to do that and I used Application Insights and custom queries to see what stood out. I also took the time to write several k6 scripts to check if there had been any regression the past 6 months (spoiler : there was, but not the way you think).
First detected issue
As the title of this article implies, the first issue I found was that some of our dependencies ran sequentially. This is what the call graph looked like in Application Insights :
Now that’s not a problem if those calls are dependent of one another but it was not the case here. I took a look at the code and quickly saw that all these calls were mostly independent from each other.
Let’s use some sample code !
To better illustrate what was done back then and avoid compromising my client’s code, I have created a sample project here. It’s an API embedded in an Aspire project, meaning it comes with tooling (tracing, open telemetry, logs…) and a PostgreSQL database container.
The point is to visualize how sequential calls to our dependencies look like and then compare with the code making the calls in parallel. For the scope of this test, I have randomized the duration of the calls to the dependencies (see source code).
Sequential version
Here’s what the code of the sequential version looks like in the GetAvailabilities() method :
var availabilities = await availabilityProvider.GetAvailabilities(new HotelId(hotelIdAsString), startDate, endDate);
if (!availabilities.Any())
{
return NoContent();
}
var roomIds = availabilities.Select(availability => availability.RoomId).ToArray();
var roomInformation = await roomInformationProvider.GetInformation(roomIds);
var roomPictures = await roomInformationProvider.GetPictures(roomIds);
var rateIds = availabilities.Select(availability => availability.RateId);
var rateInformation = await rateInformationProvider.GetInformation(rateIds);
var result = MapToAvailableRoomWithPriceList(availabilities, roomInformation, roomPictures, rateInformation);
return Ok(new AvailableRoomsWithPrices(result));There are 4 calls to our dependencies :
- On line 1 is the main call to get the availabilities. If we cannot find any, we perform an early return which is usually good practice performance-wise (and readability-wise if you ask me).
- On line 9 is a call to get information about the available rooms.
- On line 10 is another call to get URLs to pictures of those rooms, because for some reason it’s not in the same referential.
- On line 12 is a last call to get information about the rates returned by the availabilities call.
Everything is sequential. In the case of the first call it’s normal because we can’t really act until we know which rooms are available and at what price. This code gives us the following traces in Aspire :
The sequential calls are easy to spot on this kind of chart. When you see this kind of “downwards steps”, it might be worth having a look at the code. Maybe some calls are independent from the others and we can parallelize them.
Parallel version
OK so now that we see this, we can take action and improve the code. I have done the following modifications in the GetAvailabilitiesParallel() method :
var availabilities = await availabilityProvider.GetAvailabilities(new HotelId(hotelIdAsString), startDate, endDate);
if (!availabilities.Any())
{
return NoContent();
}
var roomIds = availabilities.Select(availability => availability.RoomId).ToArray();
var roomInformationTask = roomInformationProvider.GetInformation(roomIds);
var roomPicturesTask = roomInformationProvider.GetPictures(roomIds);
var rateIds = availabilities.Select(availability => availability.RateId);
var rateInformationTask = rateInformationProvider.GetInformation(rateIds);
var (roomInformation, roomPictures, rateInformation) = (await roomInformationTask, await roomPicturesTask, await rateInformationTask);
var result = MapToAvailableRoomWithPriceList(availabilities, roomInformation, roomPictures, rateInformation);
return Ok(new AvailableRoomsWithPrices(result));See how the awaits have moved on the same line and split from the async methods invocation ? We’ll get back on this syntax later but let’s see what this code looks like in Aspire’s traces :
Now that’s more like it ! It seems like we have shaved almost 40% off of the response time ! However we will refrain from drawing conclusions over 2 samples only. Let’s see how we can gather more samples and get more reliable numbers.
Automating the calls to our API using k6
Back in 2022, I had heard about a tool named k6. I did not need it then but when the performance issues came up at work, I gave it a try.
What I did is very similar to what you can see in the k6/perfs.js script : perform N calls one after the other to each endpoint and see the numbers. Here’s what they look like for 100 calls to each endpoint :
It’s a bit verbose, so I have highlighted in red the interesting numbers for us here.
Basically, we can see the average duration improves by 26%, the median duration improves by 28% and the p(95) improves by 20%. Bear in mind that the delay is simulated with Random.Next() and follows its distribution (which is, I think, uniform but I wouldn’t bet my life on it).
Moreover, the overall duration is much shorter with the parallel version. Each gain adds up, eventually amounting to a 26% improvement, matching the average we noticed previously.
Conclusion
Obviously, parallelizing calls to external dependencies is usually a nice “quick win”. And we can spot it easily with a basic monitoring tool. Feel free to play around with the code as Aspire provides such a tool, as well as simulating the dependencies or including other infrastructure containers (e.g. Redis cache, NoSQL database, etc…).
One more thing…
I said above that we would discuss a bit more about the parallel code, especially this part :
var roomIds = availabilities.Select(availability => availability.RoomId).ToArray();
var roomInformationTask = roomInformationProvider.GetInformation(roomIds);
var roomPicturesTask = roomInformationProvider.GetPictures(roomIds);
var rateIds = availabilities.Select(availability => availability.RateId);
var rateInformationTask = rateInformationProvider.GetInformation(rateIds);
var (roomInformation, roomPictures, rateInformation) = (await roomInformationTask, await roomPicturesTask, await rateInformationTask);What happens here ? What we can see is that the async calls to the dependencies are not awaited immediately. The thing is the tasks are created and run as soon as the async methods are called. This is usually a great source of confusion among .Net developers. Most of them think a Task.WhenAll() is needed for the tasks to actually run. No, they are fired directly when the async methods are called.
On line 7, awaiting all tasks in a kind of “deconstruction” way is simply because I wanted to avoid the following code :
var roomInformation = await roomInformationTask;
var roomPictures = await roomPicturesTask;
var rateInformation = await rateInformationTask;To conclude on this topic, you can have a look at this Stack Overflow discussion. Things get a bit heated here and there but keep in mind what I just said : tasks are run as soon as the async methods are called. This is also backed up by the behaviour and charts of the API above. Then you’ll realize that the most upvoted answer advises to use Task.WhenAll() when it is, in fact, useless. And that the most upvoted comment for this same answer is also plain wrong. If, even after all this, you still don’t want to take my word for it, take Stephen Cleary‘s 😉
However, somewhere else in the discussion Stephen raises a fair point : an explicit Task.WhenAll() will remove the confusion for less-experienced developers. Hence its usefulness after all. I chose not to do it in this code specifically to discuss this point in the article. Hopefully it’s been helpful to you !
Software developer since 2000, I try to make the right things right. I usually work with .Net (C#) and Azure, mostly because the ecosystem is really friendly and helps me focus on the right things.
I try to avoid unnecessary (a.k.a. accidental) complexity and in general everything that gets in the way of solving the essential complexity induced by the business needs.
This is why I favor a test-first approach to focus on the problem space and ask questions to business experts before even coding the feature.
