Design Pattern

Container Design Pattern – Scatter/Gather Pattern

Scatter/Gather Pattern

Scatter/Gather pattern acts similar to a tree pattern. The client’s request is received by the root. Root normally contains two nodes, Distributor and Aggregator. The distributor scatters the request to different nodes that act as leaves of the tree. All these leaves process the request in parallel and send the response back to root. Aggregator receives responses and combines it to produce the result.

scatter gather

Example

A streaming platform that provides recommendations to the user for different genres. The root scatters the request based on a different genre and sends it to each node. Each node will process requests for one genre, then send the recommendation back. Now the aggregator removes the duplicates, combines the result, and sends the response back.

Advantage

  • Enhances performance by attaining parallel processing.
  • Reduces processing time.
  • Reduces contention by distributing the workload.

Summary

Even though the scatter/gather pattern is able to achieve high performance by achieving parallel processing, it comes with an overhead as well. Also, the overhead will increase every time when we scale up the node. So when we are using this pattern we have to ensure that overhead incurred is not a bottleneck for performance improvement that we achieved via parallel processing.

References

https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/45406.pdf

https://learning.oreilly.com/library/view/Designing+Distributed+Systems/9781491983638/ch07.html#fig-scatter-gather-sharded