Vol. 3 No. 2 (2023): Journal of AI-Assisted Scientific Discovery
Articles

Foundation Models: The New AI Paradigm for Big Data Analytics

PDF
  • Naresh Dulam,
  • Abhilash Katari,
  • Madhu Ankam
Naresh Dulam
Vice President Sr Lead Software Engineer, JP Morgan Chase, USA
Abhilash Katari
Engineering Lead, Persistent Systems Inc, USA
Madhu Ankam
Vice President Sr Lead Software Engineer, JP Morgan Chase, USA
Cover

Published 05-10-2023

Keywords

  • Foundation models,
  • big data analytics
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

How to Cite

[1]
Naresh Dulam, Abhilash Katari, and Madhu Ankam, “Foundation Models: The New AI Paradigm for Big Data Analytics ”, Journal of AI-Assisted Scientific Discovery, vol. 3, no. 2, pp. 639–664, Oct. 2023, Accessed: Dec. 23, 2024. [Online]. Available: https://scienceacadpress.com/index.php/jaasd/article/view/231

Abstract

Foundation models are reshaping artificial intelligence, unlocking unprecedented capabilities in big data analytics. These models, pre-trained on massive, diverse datasets, can be fine-tuned for specific tasks while maintaining a versatile, general-purpose functionality. Unlike traditional AI systems designed for singular, narrowly focused applications, foundation models excel in processing complex, unstructured data and extracting meaningful insights. Their architecture, often based on transformers, enables them to capture intricate patterns and relationships across vast datasets, making them exceptionally powerful for tasks like natural language understanding, image recognition, & predictive analytics. Their impact spans diverse industries such as healthcare, finance, retail, and manufacturing, where they automate complex workflows, enhance decision-making, & uncover hidden trends. Foundation models' versatility allows organizations to harness data more effectively, turning challenges posed by information overload into opportunities for innovation and efficiency. By enabling more intelligent analytics, they support deeper contextual understanding, improve operational agility, and drive strategic outcomes. However, their immense scale & computational requirements introduce significant challenges, including accessibility, environmental impact, and concerns about bias in the underlying data. While interpretability remains an ongoing area of research, these models' ability to generalize across domains signals a paradigm shift in AI. They redefine how organizations interact with data, making it possible to extract actionable insights from the vast, complex, and fragmented information ecosystems that define the modern world. This evolution underscores the potential for foundation models to become indispensable tools for navigating an era defined by data-driven decision-making, offering a transformative approach to analytics that moves beyond traditional boundaries.

PDF

Downloads

Download data is not yet available.

References

  1. Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., ... & Liang, P. (2021). On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258.
  2. Kitchin, R. (2014). Big Data, new epistemologies and paradigm shifts. Big data & society, 1(1), 2053951714528481.
  3. Xu, Y., Liu, X., Cao, X., Huang, C., Liu, E., Qian, S., ... & Zhang, J. (2021). Artificial intelligence: A powerful paradigm for scientific research. The Innovation, 2(4).
  4. He, X., Ai, Q., Qiu, R. C., Huang, W., Piao, L., & Liu, H. (2015). A big data architecture design for smart grids based on random matrix theory. IEEE transactions on smart Grid, 8(2), 674-686.
  5. Rabah, K. (2018). Convergence of AI, IoT, big data and blockchain: a review. The lake institute Journal, 1(1), 1-18.
  6. Chang, R. M., Kauffman, R. J., & Kwon, Y. (2014). Understanding the paradigm shift to computational social science in the presence of big data. Decision support systems, 63, 67-80.
  7. Elish, M. C., & Boyd, D. (2018). Situating methods in the magic of Big Data and AI. Communication monographs, 85(1), 57-80.
  8. Pramanik, M. I., Lau, R. Y., Demirkan, H., & Azad, M. A. K. (2017). Smart health: Big data enabled health paradigm within smart cities. Expert Systems with Applications, 87, 370-383.
  9. L’heureux, A., Grolinger, K., Elyamany, H. F., & Capretz, M. A. (2017). Machine learning with big data: Challenges and approaches. Ieee Access, 5, 7776-7797.
  10. Jin, X., Wah, B. W., Cheng, X., & Wang, Y. (2015). Significance and challenges of big data research. Big data research, 2(2), 59-64.
  11. Elshawi, R., Sakr, S., Talia, D., & Trunfio, P. (2018). Big data systems meet machine learning challenges: towards big data science as a service. Big data research, 14, 1-11.
  12. Krittanawong, C., Zhang, H., Wang, Z., Aydar, M., & Kitai, T. (2017). Artificial intelligence in precision cardiovascular medicine. Journal of the American College of Cardiology, 69(21), 2657-2664.
  13. Tang, B., Chen, Z., Hefferman, G., Pei, S., Wei, T., He, H., & Yang, Q. (2017). Incorporating intelligence in fog computing for big data analysis in smart cities. IEEE Transactions on Industrial informatics, 13(5), 2140-2150.
  14. Dubey, R., Gunasekaran, A., Childe, S. J., Bryde, D. J., Giannakis, M., Foropon, C., ... & Hazen, B. T. (2020). Big data analytics and artificial intelligence pathway to operational performance under the effects of entrepreneurial orientation and environmental dynamism: A study of manufacturing organisations. International journal of production economics, 226, 107599.
  15. Zhou, Z. H., Chawla, N. V., Jin, Y., & Williams, G. J. (2014). Big data opportunities and challenges: Discussions from data analytics perspectives [discussion forum]. IEEE Computational intelligence magazine, 9(4), 62-74.
  16. Thumburu, S. K. R. (2022). The Impact of Cloud Migration on EDI Costs and Performance. Innovative Engineering Sciences Journal, 2(1).
  17. Thumburu, S. K. R. (2022). AI-Powered EDI Migration Tools: A Review. Innovative Computer Sciences Journal, 8(1).
  18. Gade, K. R. (2022). Migrations: AWS Cloud Optimization Strategies to Reduce Costs and Improve Performance. MZ Computing Journal, 3(1).
  19. Gade, K. R. (2022). Cloud-Native Architecture: Security Challenges and Best Practices in Cloud-Native Environments. Journal of Computing and Information Technology, 2(1).
  20. Katari, A., & Vangala, R. Data Privacy and Compliance in Cloud Data Management for Fintech.
  21. Katari, A., Ankam, M., & Shankar, R. Data Versioning and Time Travel In Delta Lake for Financial Services: Use Cases and Implementation.
  22. Komandla, V. Enhancing Product Development through Continuous Feedback Integration “Vineela Komandla”.
  23. Komandla, V. Enhancing Security and Growth: Evaluating Password Vault Solutions for Fintech Companies.
  24. Thumburu, S. K. R. (2021). Optimizing Data Transformation in EDI Workflows. Innovative Computer Sciences Journal, 7(1).
  25. Thumburu, S. K. R. (2021). Performance Analysis of Data Exchange Protocols in Cloud Environments. MZ Computing Journal, 2(2).
  26. Gade, K. R. (2021). Data-Driven Decision Making in a Complex World. Journal of Computational Innovation, 1(1).

Most read articles by the same author(s)

Similar Articles

You may also start an advanced similarity search for this article.