Priority Queues in Large-Scale Distributed Systems under Heavy Workloads
DOI:
https://doi.org/10.63282/3050-9416.IJAIBDCMS-V6I4P118Keywords:
Priority Queuing, Overload Mitigation, Monitoring Query Prioritization, Catalog Update Pipelines, Fast-Lane Processing, Multi-Tenant Workload Isolation, Resource Starvation Prevention, Distributed Scheduling, Backpressure Mechanisms, High-Value Data Updates, Workload ImbalanceAbstract
Large-scale distributed systems often face periods of heavy workloads where a surge in tasks can overwhelm processing capacity. In such scenarios, critical operations risk being delayed as they compete with a backlog of routine tasks. Priority queues offer a solution by ensuring that high-priority work is serviced ahead of less critical tasks, thereby preserving system responsiveness and meeting service-level objectives. This paper examines the importance of priority queuing in distributed systems handling massive workloads, highlighting how the Priority Queue pattern can prevent important tasks (such as monitoring queries or high-value data updates) from being starved of resources. We discuss real-world scenarios—including an overloaded database where monitoring queries get stuck, and a catalog processing pipeline where urgent updates languish behind bulk jobs—and illustrate design patterns that mitigate these issues. We explore implementations ranging from cloud infrastructure (e.g., AWS SQS and workload management) to industry-scale solutions like Facebook’s FOQS distributed priority queue. Through diagrams and case studies, we show how priority-based task scheduling improves observability, data freshness, and overall system reliability under heavy load. The paper provides an academic discussion in a formal tone, supported by references to established patterns and recent industry and research insights
References
[1] Amazon Web Services, “Amazon Redshift Workload Management,” 2024. [Online]. Available: https://docs.aws.amazon.com/redshift/latest/dg/c_workload_mngmt_classification.html#:~:text=Amazon%20Redshift%20workload%20managemen t%20,well%20as%20a%20runtime%20queue
[2] Microsoft Azure, “Priority Ǫueue Pattern,” Azure Architecture Center, 2024. [Online]. Available: https://learn.microsoft.com/en- us/azure/architecture/patterns/priority-queue
[3] ByteByteGo, “How Facebook’s Distributed Priority Ǫueue Handles Trillions of Items,” Oct. 2025. [Online]. Available: https://blog.bytebytego.com/p/how-facebooks- distributed-priority
[4] J. Patil, “AWS SǪS Design Patterns – Priority Ǫueue,” 2023. [Online]. Available: https://jayendrapatil.com/tag/sqs-design patterns/#:~:text=Priority%20Ǫueue%20Pattern
[5] Rohit Pathak, “Building a Distributed Priority Ǫueue on Kafka,” Sep. 2025. [Online]. Available: https://klaviyo.tech/building-a-distributed-priority-queue-in-kafka-1b2d8063649e
[6] L. van Donkersgoed, “Implement the Priority Ǫueue Pattern with SǪS and Lambda,” 2022. [Online]. Available: https://lucvandonkersgoed.com/2022/04/25/impl ement-the-priority-queue-pattern-with-sqs-and- lambda/#:~:text=In%20the%20previous%20secti on%2C%20we,github%20source
[7] W. Velida, “The Priority Ǫueue Pattern,” DEV Community, 2024. [Online]. Available: ttps://dev.to/willvelida/the-priority-queue- pattern- 23g8#:~:text=The%20Priority%20Ǫueue%20patte rn%20is,faster%20than%20lower%20priority%20r equests
[8] Meta (Facebook) Engineering, “FOǪS Use Cases and Distributed Priority Ǫueue Behavior,” 2025. [Online]. Available: https://blog.bytebytego.com/p/how-facebooks- distributed-priority and https://engineering.fb.com/2021/02/22/productio n-engineering/foqs-scaling-a-distributed-priority- queue/
[9] Amazon Web Services, “Create and Prioritize Ǫuery Ǫueues in Amazon Redshift,” AWS re:Post, 2024. [Online]. Available: https://docs.aws.amazon.com/redshift/latest/dg/ c_workload_mngmt_classification.html#:~:text= Workload%20management%20,don%27t%20get %20stuck%20in%20queues
[10] Ratan Bajpai, Krishna Dhara, B. Chandramouli et al., “QPID: A Distributed Priority Queue with Item Locality,” 2009. [Online]. Available: https://www.researchgate.net/publication/220946085 _QPID_A_Distributed_Priority_Queue_with_Item_L ocality
[11] H. Rihani, P. Sanders, and R. Dementiev, “MultiQueues: Simpler, Faster, and Better Relaxed Concurrent Priority Queues,” arXiv:1411.1209, 2014. https://arxiv.org/abs/1411.1209
[12] L. Kleinrock, Queueing Systems, Volume 1: Theory, Wiley, 1975.
[13] A. Federgruen and H. Groenevelt, “M/G/c Queueing Systems with Multiple Customer Classes,” Columbia Business School Working Paper, 1988.
[14] D. Bertsimas and J. Niño-Mora, “Multiclass Queueing Systems in Heavy Traffic: An Asymptotic Approach,” MIT Working Paper, 1997.
[15] J. F. C. Kingman, “Queue Disciplines in Heavy Traffic,” Journal of the Royal Statistical Society B, 1982.
[16] S. Lee et al., “Analysis of a Priority Queueing System with Enhanced Priority,” Journal of Ambient Intelligence and Humanized Computing, 2024.
[17] D. Alistarh et al., “The SprayList: A Scalable Relaxed Priority Queue,” PPoPP 2015 / Microsoft Research Tech Report, 2015.
[18] H. Rihani, P. Sanders, and R. Dementiev, “MultiQueues: Simpler, Faster, and Better Relaxed Concurrent Priority Queues,” arXiv:1411.1209, 2014.
[19] M. Williams et al., “Engineering MultiQueues: Fast Relaxed Concurrent Priority Queues,” arXiv:2107.01350, 2021.
[20] S. Walzer and M. Williams, “A Simple yet Exact Analysis of the MultiQueue,” arXiv:2410.08714, 2024.
[21] M. Feldmann, M. Henzinger, et al., “Skeap & Seap: Scalable Distributed Priority Queues,” SPAA 2019.
[22] T. Zhou et al., “A Practical, Scalable, Relaxed Priority Queue (ZMSQ),” ICPP, 2019.
[23] B. S. Lin and X. Liang, “A Scalable Relaxed Distributed Priority Queue,” Stanford CS244B Project Report, 2020.
