Vol. 3 No. 1 (2023): Journal of AI-Assisted Scientific Discovery
Articles

Data Integration: AI-driven approaches to streamline data integration from various sources

PDF
  • Muneer Ahmed Salamkar
Muneer Ahmed Salamkar
Senior Associate at JP Morgan Chase, USA
Cover

Published 08-03-2023

Keywords

  • Data Integration,
  • Artificial Intelligence
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

How to Cite

[1]
Muneer Ahmed Salamkar, “Data Integration: AI-driven approaches to streamline data integration from various sources”, Journal of AI-Assisted Scientific Discovery, vol. 3, no. 1, pp. 668–694, Mar. 2023, Accessed: Dec. 23, 2024. [Online]. Available: https://scienceacadpress.com/index.php/jaasd/article/view/221

Abstract

Data integration, combining data from diverse sources into a unified view, has become a cornerstone of modern data-driven decision-making. However, the proliferation of data sources, formats, and platforms poses significant challenges. AI-driven approaches are revolutionizing this domain, offering innovative solutions to streamline integration processes. By leveraging machine learning algorithms, natural language processing, and pattern recognition, AI systems can efficiently identify relationships among disparate datasets, automate schema matching, and resolve conflicts in data formats. These methods enhance scalability, accuracy, and efficiency, enabling the integration of large volumes of structured and unstructured data with minimal human intervention. AI-driven tools enable real-time integration, providing businesses with up-to-date insights critical for maintaining a competitive edge. Furthermore, AI-powered metadata analysis and anomaly detection advancements enhance data quality and governance, addressing key concerns around consistency and compliance. This paper explores the methodologies underpinning AI-driven data integration, their applications across industries, and the remaining challenges, including ethical considerations and the need for robust training datasets. By analyzing AI's transformative impact on data integration, we highlight how organizations can harness these technologies to unlock the full potential of their data ecosystems and drive innovation.

PDF

Downloads

Download data is not yet available.

References

  1. Devarasetty, N. (2022). Toward Autonomous Data Engineering: The Role of AI in Streamlining Data Integration and ETL. International Journal of Advanced Engineering Technologies and Innovations, 1(2), 133-156.
  2. Deekshith, A. (2019). Integrating AI and Data Engineering: Building Robust Pipelines for Real-Time Data Analytics. International Journal of Sustainable Development in Computing Science, 1(3), 1-35.
  3. Machireddy, J. R., Rachakatla, S. K., & Ravichandran, P. (2021). Leveraging AI and Machine Learning for Data-Driven Business Strategy: A Comprehensive Framework for Analytics Integration. African Journal of Artificial Intelligence and Sustainable Development, 1(2), 12-150.
  4. Valleru, V., & Alapati, N. K. K. (2022). Breaking Down Data Silos: Innovations in Cloud Data Integration. Advances in Computer Sciences, 5(1).
  5. Dhaliwal, N. (2019). Automating Analysis Workflows With Ai: Tools For Streamlined Data Upload And Review In Clinical Systems. Journal Of Basic Science And Engineering, 16(1).
  6. Devan, M., Shanmugam, L., & Tomar, M. (2021). AI-powered data migration strategies for cloud environments: Techniques, frameworks, and real-world applications. Australian Journal of Machine Learning Research & Applications, 1(2), 79-111.
  7. Kalusivalingam, A. K., Sharma, A., Patel, N., & Singh, V. (2012). Enhancing Patient-Specific Treatment Outcomes: Leveraging Deep Learning and Genomic Data Integration in AI-Driven Personalized Medicine. International Journal of AI and ML, 1(2).
  8. Kommera, A. R. (2015). Future of enterprise integrations and iPaaS (Integration Platform as a Service) adoption. Neuroquantology, 13(1), 176-186.
  9. Kalusivalingam, A. K., Sharma, A., Patel, N., & Singh, V. (2013). Enhancing AI-Driven Pathology Image Analysis Using Convolutional Neural Networks and Transfer Learning Techniques. International Journal of AI and ML, 2(10).
  10. Jang, M., Choe, J., & Kim, S. J. (2007). Streamline-based inversion model using simulated annealing coupled with geostatistical perturbation methods. Energy Sources, 29(14), 1285-1294.
  11. Elwood, S. (2008). Grassroots groups as stakeholders in spatial data infrastructures: challenges and opportunities for local data development and sharing. International Journal of Geographical Information Science, 22(1), 71-90.
  12. Oh, S., Byon, Y. J., Jang, K., & Yeo, H. (2015). Short-term travel-time prediction on highway: a review of the data-driven approach. Transport Reviews, 35(1), 4-32.
  13. Izza, S. (2009). Integration of industrial information systems: from syntactic to semantic integration approaches. Enterprise Information Systems, 3(1), 1-57.
  14. Sobek, M., Cleveland, L., Flood, S., Kelly Hall, P., King, M. L., Ruggles, S., & Schroeder, M. (2011). Big data: large-scale historical infrastructure from the Minnesota Population Center. Historical methods, 44(2), 61-68.
  15. Wetzel, F. T., Saarenmaa, H., Regan, E., Martin, C. S., Mergen, P., Smirnova, L., ... & Häuser, C. L. (2015). The roles and contributions of Biodiversity Observation Networks (BONs) in better tracking progress to 2020 biodiversity targets: a European case study. Biodiversity, 16(2-3), 137-149.
  16. Thumburu, S. K. R. (2022). Real-Time Data Transformation in EDI Architectures. Innovative Engineering Sciences Journal, 2(1).
  17. Thumburu, S. K. R. (2022). AI-Powered EDI Migration Tools: A Review. Innovative Computer Sciences Journal, 8(1).
  18. Gade, K. R. (2022). Data Analytics: Data Fabric Architecture and Its Benefits for Data Management. MZ Computing Journal, 3(2).
  19. Thumburu, S. K. R. (2022). Data Integration Strategies in Hybrid Cloud Environments. Innovative Computer Sciences Journal, 8(1).
  20. Gade, K. R. (2022). Cloud-Native Architecture: Security Challenges and Best Practices in Cloud-Native Environments. Journal of Computing and Information Technology, 2(1).
  21. Katari, A., Ankam, M., & Shankar, R. Data Versioning and Time Travel In Delta Lake for Financial Services: Use Cases and Implementation.
  22. Katari, A., & Vangala, R. Data Privacy and Compliance in Cloud Data Management for Fintech.
  23. Thumburu, S. K. R. (2021). Performance Analysis of Data Exchange Protocols in Cloud Environments. MZ Computing Journal, 2(2).
  24. Gade, K. R. (2021). Migrations: Cloud Migration Strategies, Data Migration Challenges, and Legacy System Modernization. Journal of Computing and Information Technology, 1(1).
  25. Thumburu, S. K. R. (2020). Leveraging APIs in EDI Migration Projects. MZ Computing Journal, 1(1).

Most read articles by the same author(s)

Similar Articles

You may also start an advanced similarity search for this article.