Vol. 3 No. 1 (2023): Journal of AI-Assisted Scientific Discovery
Articles

Intelligent Data Tiering in Hybrid Cloud Environments

PDF
  • Abhilash Katari,
  • Sudhir Koundinya
Abhilash Katari
Engineering Lead at Persistent Systems Inc, USA
Sudhir Koundinya
manager in persistent systems., USA
Cover

Published 22-04-2023

Keywords

  • Intelligent Data Tiering,
  • Hybrid Cloud
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

How to Cite

[1]
Abhilash Katari and Sudhir Koundinya, “Intelligent Data Tiering in Hybrid Cloud Environments ”, Journal of AI-Assisted Scientific Discovery, vol. 3, no. 1, pp. 695–715, Apr. 2023, Accessed: Dec. 28, 2024. [Online]. Available: https://scienceacadpress.com/index.php/jaasd/article/view/254

Abstract

In today's rapidly evolving digital landscape, data is growing at an unprecedented rate, and organizations face the challenge of managing this data efficiently while balancing cost and performance. Intelligent data tiering in hybrid cloud environments offers a dynamic solution to this challenge. By strategically placing data across various storage tiers—such as on-premises infrastructure, public clouds, and private clouds—companies can optimize their storage costs, improve accessibility, and maintain high-performance standards. Data that is frequently accessed, or "hot" data, can be stored on high-performance, low-latency storage, while "cold" data, which is rarely accessed, can be offloaded to more cost-effective, long-term storage solutions. Artificial intelligence (AI) and machine learning (ML) are crucial in automating this tiering process by analyzing data usage patterns and making real-time decisions on where data should reside. This automation reduces administrative burdens, minimizes human error, and ensures data is always stored in the most appropriate tier. Additionally, intelligent data tiering helps organizations adhere to regulatory requirements, providing flexibility in managing sensitive data. It also enhances data lifecycle management, as businesses can define rules and policies aligning with their goals. By combining the strengths of both cloud and on-premises infrastructure, hybrid cloud environments provide the flexibility needed to achieve these goals. The seamless integration of intelligent tiering into these environments helps organizations remain agile and scalable without compromising performance or cost efficiency. As data volumes continue to soar, intelligent data tiering offers a forward-thinking approach to storage management that empowers organizations to harness the full potential of their data while maintaining control over operational expenses and resource allocation.

PDF

Downloads

Download data is not yet available.

References

  1. George, J. (2022). Optimizing hybrid and multi-cloud architectures for real-time data streaming and analytics: Strategies for scalability and integration. World Journal of Advanced Engineering Technology and Sciences, 7(1), 10-30574.
  2. Bi, J., Zhu, Z., Tian, R., & Wang, Q. (2010, July). Dynamic provisioning modeling for virtualized multi-tier applications in cloud data center. In 2010 IEEE 3rd International Conference on Cloud Computing (pp. 370-377). IEEE.
  3. Mitton, N., Papavassiliou, S., Puliafito, A., & Trivedi, K. S. (2012). Combining Cloud and sensors in a smart city environment. EURASIP journal on Wireless Communications and Networking, 2012, 1-10.
  4. Dulloor, S. R., Roy, A., Zhao, Z., Sundaram, N., Satish, N., Sankaran, R., ... & Schwan, K. (2016, April). Data tiering in heterogeneous memory systems. In Proceedings of the Eleventh European Conference on Computer Systems (pp. 1-16).
  5. Arkian, H. R., Diyanat, A., & Pourkhalili, A. (2017). MIST: Fog-based data analytics scheme with cost-efficient resource provisioning for IoT crowdsensing applications. Journal of Network and Computer Applications, 82, 152-165.
  6. Fernandes, D. A., Soares, L. F., Gomes, J. V., Freire, M. M., & Inácio, P. R. (2014). Security issues in cloud environments: a survey. International journal of information security, 13, 113-170.
  7. Vercellis, C. (2011). Business intelligence: data mining and optimization for decision making. John Wiley & Sons.
  8. Sen, S., Joe-Wong, C., Ha, S., & Chiang, M. (2013). A survey of smart data pricing: Past proposals, current plans, and future trends. Acm computing surveys (csur), 46(2), 1-37.
  9. Xu, H., Yu, W., Griffith, D., & Golmie, N. (2018). A survey on industrial Internet of Things: A cyber-physical systems perspective. Ieee access, 6, 78238-78259.
  10. Rimal, B. P., Choi, E., & Lumb, I. (2009, August). A taxonomy and survey of cloud computing systems. In 2009 fifth international joint conference on INC, IMS and IDC (pp. 44-51). Ieee.
  11. He, W., Yan, G., & Da Xu, L. (2014). Developing vehicular data cloud services in the IoT environment. IEEE transactions on industrial informatics, 10(2), 1587-1595.
  12. Manogaran, G., Varatharajan, R., Lopez, D., Kumar, P. M., Sundarasekar, R., & Thota, C. (2018). A new architecture of Internet of Things and big data ecosystem for secured smart healthcare monitoring and alerting system. Future Generation Computer Systems, 82, 375-387.
  13. Shawish, A., & Salama, M. (2013). Cloud computing: paradigms and technologies. In Inter-cooperative collective intelligence: Techniques and applications (pp. 39-67). Berlin, Heidelberg: Springer Berlin Heidelberg.
  14. Kaur, H., Alam, M. A., Jameel, R., Mourya, A. K., & Chang, V. (2018). A proposed solution and future direction for blockchain-based heterogeneous medicare data in cloud environment. Journal of medical systems, 42, 1-11.
  15. Singh, S., Jeong, Y. S., & Park, J. H. (2016). A survey on cloud computing security: Issues, threats, and solutions. Journal of Network and Computer Applications, 75, 200-222.
  16. Katari, A., & Vangala, R. Data Privacy and Compliance in Cloud Data Management for Fintech.
  17. Katari, A., Ankam, M., & Shankar, R. Data Versioning and Time Travel In Delta Lake for Financial Services: Use Cases and Implementation.
  18. Katari, A. (2022). Performance Optimization in Delta Lake for Financial Data: Techniques and Best Practices. MZ Computing Journal, 3(2).
  19. Nookala, G., Gade, K. R., Dulam, N., & Thumburu, S. K. R. (2022). The Shift Towards Distributed Data Architectures in Cloud Environments. Innovative Computer Sciences Journal, 8(1).
  20. Nookala, G. (2022). Improving Business Intelligence through Agile Data Modeling: A Case Study. Journal of Computational Innovation, 2(1).
  21. Nookala, G., Gade, K. R., Dulam, N., & Thumburu, S. K. R. (2021). Unified Data Architectures: Blending Data Lake, Data Warehouse, and Data Mart Architectures. MZ Computing Journal, 2(2).
  22. Boda, V. V. R., & Immaneni, J. (2022). Optimizing CI/CD in Healthcare: Tried and True Techniques. Innovative Computer Sciences Journal, 8(1).
  23. Immaneni, J. (2022). End-to-End MLOps in Financial Services: Resilient Machine Learning with Kubernetes. Journal of Computational Innovation, 2(1).
  24. Boda, V. V. R., & Immaneni, J. (2021). Healthcare in the Fast Lane: How Kubernetes and Microservices Are Making It Happen. Innovative Computer Sciences Journal, 7(1).
  25. Gade, K. R. (2022). Cloud-Native Architecture: Security Challenges and Best Practices in Cloud-Native Environments. Journal of Computing and Information Technology, 2(1).
  26. Gade, K. R. (2022). Data Catalogs: The Central Hub for Data Discovery and Governance. Innovative Computer Sciences Journal, 8(1).
  27. Gade, K. R. (2022). Data Lakehouses: Combining the Best of Data Lakes and Data Warehouses. Journal of Computational Innovation, 2(1).
  28. Thumburu, S. K. R. (2022). A Framework for Seamless EDI Migrations to the Cloud: Best Practices and Challenges. Innovative Engineering Sciences Journal, 2(1).
  29. Thumburu, S. K. R. (2022). The Impact of Cloud Migration on EDI Costs and Performance. Innovative Engineering Sciences Journal, 2(1).
  30. Thumburu, S. K. R. (2022). AI-Powered EDI Migration Tools: A Review. Innovative Computer Sciences Journal, 8(1).
  31. Komandla, V. Enhancing Product Development through Continuous Feedback Integration “Vineela Komandla”.
  32. Komandla, V. Enhancing Security and Growth: Evaluating Password Vault Solutions for Fintech Companies.
  33. Komandla, V. Strategic Feature Prioritization: Maximizing Value through User-Centric Roadmaps.
  34. Muneer Ahmed Salamkar, et al. The Big Data Ecosystem: An Overview of Critical Technologies Like Hadoop, Spark, and Their Roles in Data Processing Landscapes. Journal of AI-Assisted Scientific Discovery, vol. 1, no. 2, Sept. 2021, pp. 355-77
  35. Muneer Ahmed Salamkar. Scalable Data Architectures: Key Principles for Building Systems That Efficiently Manage Growing Data Volumes and Complexity. Journal of AI-Assisted Scientific Discovery, vol. 1, no. 1, Jan. 2021, pp. 251-70
  36. Muneer Ahmed Salamkar, and Jayaram Immaneni. Automated Data Pipeline Creation: Leveraging ML Algorithms to Design and Optimize Data Pipelines. Journal of AI-Assisted Scientific Discovery, vol. 1, no. 1, June 2021, pp. 230-5
  37. Naresh Dulam. NoSQL Vs SQL: Which Database Type Is Right for Big Data?. Distributed Learning and Broad Applications in Scientific Research, vol. 1, May 2015, pp. 115-3
  38. Naresh Dulam. Data Lakes: Building Flexible Architectures for Big Data Storage. Distributed Learning and Broad Applications in Scientific Research, vol. 1, Oct. 2015, pp. 95-114
  39. Naresh Dulam. The Rise of Kubernetes: Managing Containers in Distributed Systems. Distributed Learning and Broad Applications in Scientific Research, vol. 1, July 2015, pp. 73-94
  40. Sarbaree Mishra. “A Reinforcement Learning Approach for Training Complex Decision Making Models”. Journal of AI-Assisted Scientific Discovery, vol. 2, no. 2, July 2022, pp. 329-52
  41. Sarbaree Mishra, et al. “Leveraging in-Memory Computing for Speeding up Apache Spark and Hadoop Distributed Data Processing”. Journal of AI-Assisted Scientific Discovery, vol. 2, no. 2, Sept. 2022, pp. 304-28
  42. Sarbaree Mishra. “Comparing Apache Iceberg and Databricks in Building Data Lakes and Mesh Architectures”. Journal of AI-Assisted Scientific Discovery, vol. 2, no. 2, Nov. 2022, pp. 278-03
  43. Babulal Shaik. Automating Compliance in Amazon EKS Clusters With Custom Policies . Journal of Artificial Intelligence Research and Applications, vol. 1, no. 1, Jan. 2021, pp. 587-10
  44. Babulal Shaik. Developing Predictive Autoscaling Algorithms for Variable Traffic Patterns . Journal of Bioinformatics and Artificial Intelligence, vol. 1, no. 2, July 2021, pp. 71-90

Similar Articles

You may also start an advanced similarity search for this article.