Spring Batch Framework Patterns for Large-Scale Data Processing in Cloud-Native Microservices: A GCP Deployment Study
DOI:
https://doi.org/10.63282/3050-9416.IJAIBDCMS-V2I4P113Keywords:
Spring Batch, Cloud-Native, GCP, Microservices, Batch Processing, Distributed Computing, Java, Cloud Pub/Sub, Data Processing, Enterprise ArchitectureAbstract
Although Spring Batch is a commonly cited framework for large-scale batch data processing in enterprise Java applications, using it in conjunction with cloud-native microservices architectures on Google Cloud Platform (GCP) brings design challenges that are just as significant as those faced in on-premises deployment scenarios with regards to job partitioning, fault handling, event scales and observability. The aim of this paper is to perform a systematic investigation of the four patterns of the Spring Batch framework specifically tailored for cloud-native microservices running on GCP: job partitioning patterns to achieve distributed computing, remote chunking patterns to achieve horizontal scaling, Cloud Pub/Sub integration patterns for event-driven batch triggering, and Stackdriver-based patterns for bull batch monitoring in production. Benefits of using enterprise data processing workloads are clearly illustrated through a series of case studies that show measurable gains in throughput and operational reliability in workloads when compared to the traditional batch deployment model.
References
1. Burns, B., Grant, B., Oppenheimer, D., Brewer, E., & Wilkes, J. (2016). Borg, omega, and kubernetes. Communications of the ACM, 59(5), 50-57.
2. Indrasiri, K., & Suhothayan, S. (2021). Design patterns for cloud native applications. " O'Reilly Media, Inc.".
3. Dragoni, N., Giallorenzo, S., Lafuente, A. L., Mazzara, M., Montesi, F., Mustafin, R., & Safina, L. (2017). Microservices: yesterday, today, and tomorrow. Present and ulterior software engineering, 195-216.
4. Huberty, M. (2015). Awaiting the second big data revolution: from digital noise to value creation. Journal of Industry, Competition and Trade, 15(1), 35-47.
5. Evans, E. (2004). Domain-driven design: tackling complexity in the heart of software. Addison-Wesley Professional.
6. Kumar, M. S., & Yuvaraj, N. (2020). Building a Privacy-Aware Customer Data Foundation: A Governance-First Approach to Digital Service Systems. International Journal of Emerging Research in Engineering and Technology, 1(4), 55-68.
7. Hohpe, G., & Woolf, B. (2002, July). Enterprise integration patterns. In 9th conference on pattern language of programs (pp. 1-9).
8. Vernon, V. (2013). Implementing domain-driven design. Addison-Wesley.
9. Lui, M., Gray, M., Chan, A., & Long, J. (2011). Spring integration and spring batch. Pro Spring Integration, 561-589.
10. Turnbull, J. (2014). The Docker Book: Containerization is the new virtualization. James Turnbull.
11. Gupta, A., Goswami, P., Chaudhary, N., & Bansal, R. (2020, March). Deploying an application using google cloud platform. In 2020 2nd International Conference on Innovative Mechanisms for Industry Applications (ICIMIA) (pp. 236-239). IEEE.
12. Kaushik, P., Rao, A. M., Singh, D. P., Vashisht, S., & Gupta, S. (2021, November). Cloud computing and comparison based on service and performance between Amazon AWS, Microsoft Azure, and Google Cloud. In 2021 International Conference on Technological Advancements and Innovations (ICTAI) (pp. 268-273). IEEE.
13. Kratzke, N., & Quint, P. C. (2017). Understanding cloud-native applications after 10 years of cloud computing-a systematic mapping study. Journal of Systems and Software, 126, 1-16.
14. Cogoluegnes, A., Templier, T., & Bazoud, O. (2011). Spring batch in action. Simon and Schuster.
15. Mao, Y., Fu, Y., Gu, S., Mu, W., Cheng, L., & Liu, Q. (2020). Resource management schemes for cloud-native platforms with computing containers of docker and kubernetes. arXiv preprint arXiv:2010.10350.
16. Han, J., Park, S., & Kim, J. (2020). Dynamic OverCloud: realizing microservices-based IoT-cloud service composition over multiple clouds. Electronics, 9(6), 969.
17. Richardson, C. (2018). Microservices patterns: with examples in Java. Simon and Schuster.
18. Khan, M. G., Taheri, J., Al-Dulaimy, A., & Kassler, A. (2021). Perfsim: A performance simulator for cloud native microservice chains. IEEE Transactions on Cloud Computing, 11(2), 1395-1413.
19. Newman, S. (2021). Building microservices: designing fine-grained systems. " O'Reilly Media, Inc.".
20. Gan, Y., Liang, M., Dev, S., Lo, D., & Delimitrou, C. (2021). Sage: Using unsupervised learning for scalable performance debugging in microservices. arXiv preprint arXiv:2101.00267.
21. Bakshi, K. (2017, March). Microservices-based software architecture and approaches. In 2017 IEEE aerospace conference (pp. 1-8). IEEE.
22. Shah, J., & Dubaria, D. (2019, January). Building modern clouds: using docker, kubernetes & Google cloud platform. In 2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC) (pp. 0184-0189). IEEE.
23. Botez, R., Iurian, C. M., Ivanciu, I. A., & Dobrota, V. (2020, June). Deploying a dockerized application with Kubernetes on Google cloud platform. In 2020 13th International Conference on Communications (COMM) (pp. 471-476). IEEE.
24. Felstaine, E., & Hermoni, O. (2018). Machine Learning, Containers, Cloud Natives, and Microservices. In Artificial Intelligence for Autonomous Networks (pp. 145-164). Chapman and Hall/CRC.
25. Hunter, T., & Porter, S. (2018). Google Cloud Platform for developers: build highly scalable cloud solutions with the power of Google Cloud Platform. Packt Publishing Ltd.
