Photo by Hal Gatewood on Unsplash

Shopping is now a Concurrency Problem

Now shops are limiting the number of people inside, shopping takes a lot longer than it used to. Most times I visit a store I am having to queue outside for an extended period of time. Probably longer than I spend inside. This article explores why this is the case from the eyes of a software engineer.

The Problem

Putting a limit to the number of people in a store is like having a thread pool for a server. Most new servers are moving to asynchronous processing rather than using thread pools e.g. Nginx and NodeJS. When the pool is exhausted, processes have to queue up like we do when the store is full.

You could argue that this has always been a problem in supermarkets as there are only a limited number of checkouts. So why is this so much worse now? I believe the main bottleneck now in stores is the ‘thinking’ part, this is equivalent to a thread blocking when it calls out to external services e.g. a HTTPS request. Previously an unlimited amount of threads/people were allowed to do this and the only bottleneck was the checkout where the only thinking is which payment type you are using and remembering your pin number (you should be using contactless by now!).

Autoscaling

One way to fix a bottleneck is to scale a service. You can vertically scale by increasing CPU/Memory of the underlying machine. These days it is preferable to horizontally scale, making use of ‘the cloud’ which can help keep costs under control by not having to over-provision during periods of low demand.

How can we do this for physical stores? To be honest I don’t think we can as it takes a large amount of time to set up a store. Shelves need to be installed, staff hired and stock brought in. If we removed the need for a physical store and had pop up shops from a vehicle, we could have stores on demand. These vehicles would also be able to move around if demand changed.

Why do we even need to go to a store? The ability to order online and have it delivered has been around for many years. This allows for stores to know a few days in advance what items and how many drivers are needed so they can scale appropriately. This also reduces the number of people needed to gather, potentially spreading infections.

Load balancing

Most people have a number of stores they can visit however most people will want to go to their closest store for convenience. If we understood the pool size of a store e.g. the number of customers inside, we could spread the load effectively. Not all stores are equal some are larger than others so more customers can be directed there.

To solve this we would need data, preferably a queuing system which allows customers to see the length of the queue. Customers can then self-balance themselves across stores. Customers would want to know how long they would be expecting to queue along with how far away they are from the stores. Taking into account travel time they can choose the store which would let them get their shopping done fastest.

Time Outs

Introducing a maximum time in a store is probably not going to give customers the best experience but I include this as it is a tool used in threading problems.

As mentioned above most of the ‘processing time’ is spent looking for items, and for the majority of the time, the checkouts are empty. By limiting the amount of time people can spend in the store will encourage them to quickly get their items and leave. If maps and item lists were provided outside or online, people would be able to plan ahead before entering the store. Providing this information without limiting time may help on its own as some people may use it as they want to limit their exposure.

Priority

When dealing with queues you may want to consider what you are trying to achieve. Is the order important? Is throughput more important? Are some items more important than others? You may have heard the term FIFO and FILO which are first in first out and first in last out. I can't think of a time where I have been in a physical queue which is FILO and I am not suggesting we use that at stores.

If we want to limit the exposure people have to each other then time is important. We want people to spend the least amount of time near other people. Different people are also considered to be more vulnerable than others too. Some stores are creating dedicated time for vulnerable people which is great but does not help the general public.

A separate queue could be used for vulnerable people giving them priority over others instead of having dedicated times. This would spread the load out over the day as there is more time for people to shop whilst still prioritising vulnerable people.

As time spent selecting items is the bottleneck in stores some stores could limit the number of items people can buy. This is similar to the 10 items or fewer checkouts. In larger stores, there could be two queues one which limits the number of items people can buy. This should increase throughput but may put people off who want to do a large shop.

Asynchronous processing

If we removed the time spent selecting the items from the customers we could increase throughput. All a customer would need to queue up for is to collect their times. We already have this at some stores through click and collect. Staff can collect items for multiple people at the same time and once their order is ready a notification can be sent to the customer to pick it up. Whole stores could just click and collect.

This story was a bit of fun and shows queuing and concurrency in a different light. Hopefully, stores will improve their infrastructure so we aren't queuing as much. I know I would choose a store which I would get my shopping done the fastest.

Senior Software Engineer and Observability Guild Lead at Booking.com — Transport, writing mainly about observability and micro front ends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store