Vol. 2 No. 2 (2022): Journal of AI-Assisted Scientific Discovery
Articles

Data Mesh and Data Governance: Finding the Balance

PDF
  • Naresh Dulam,
  • Kishore Reddy Gade,
  • Venkataramana Gosukonda
Naresh Dulam
Vice President Sr Lead Software Engineer, JP Morgan Chase, USA
Kishore Reddy Gade
Vice President, Lead Software Engineer, JP Morgan Chase, USA
Venkataramana Gosukonda
Senior Software Engineering Manager, Wells Fargo, USA
Cover

Published 01-12-2022

Keywords

  • Data Mesh,
  • Data Governance
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

How to Cite

[1]
Naresh Dulam, Kishore Reddy Gade, and Venkataramana Gosukonda, “Data Mesh and Data Governance: Finding the Balance”, Journal of AI-Assisted Scientific Discovery, vol. 2, no. 2, pp. 226–250, Dec. 2022, Accessed: Dec. 27, 2024. [Online]. Available: https://scienceacadpress.com/index.php/jaasd/article/view/230

Abstract

Organizations face the dual challenge of empowering teams to innovate while maintaining control over data quality, compliance, and security. The Data Mesh paradigm offers a decentralized approach, enabling domain-focused teams to manage their data as a product and drive insights tailored to their needs. This shift from centralized control to domain autonomy fosters agility, scalability, and closer alignment with business goals. However, decentralization with robust governance risks consistency, compliance, & data misuse. Data Governance is vital in ensuring the organization maintains a unified framework for managing data quality, ethical use, and regulatory adherence, even in a distributed setup. Achieving a balance between Data Mesh's flexibility and governance oversight is crucial for creating a reliable and scalable data ecosystem. By aligning the principles of these two approaches, organizations can foster an environment where domain teams have the autonomy to innovate while adhering to shared governance standards. Success lies in establishing clear guardrails, promoting a culture of accountability, & implementing automated tools to manage compliance and quality across domains. Collaborative frameworks like federated computational governance allow organizations to decentralize decision-making while maintaining oversight, ensuring that teams work within predefined guidelines without stifling innovation. Moreover, transparent communication and shared goals between governance bodies & domain teams build trust and reduce friction. Technology, such as metadata management and automated policy enforcement, can further streamline this balance by embedding governance directly into workflows. Ultimately, harmonizing Data Mesh and Data Governance ensures that organizations can unlock the full value of their data assets, driving innovation and decision-making without compromising integrity or security. This balance enables a future where data ecosystems are dynamic, responsible, and equipped to adapt to the ever-evolving demands of the digital age.

PDF

Downloads

Download data is not yet available.

References

  1. Devine, K., Boman, E., Heaphy, R., Hendrickson, B., & Vaughan, C. (2002). Zoltan data management services for parallel dynamic applications. Computing in Science & Engineering, 4(2), 90-96.
  2. Siddiqa, A., Hashem, I. A. T., Yaqoob, I., Marjani, M., Shamshirband, S., Gani, A., & Nasaruddin, F. (2016). A survey of big data management: Taxonomy and state-of-the-art. Journal of Network and Computer Applications, 71, 151-166.
  3. Huang, Y., Chen, Z. X., Tao, Y. U., Huang, X. Z., & Gu, X. F. (2018). Agricultural remote sensing big data: Management and applications. Journal of Integrative Agriculture, 17(9), 1915-1931.
  4. Gharaibeh, A., Salahuddin, M. A., Hussini, S. J., Khreishah, A., Khalil, I., Guizani, M., & Al-Fuqaha, A. (2017). Smart cities: A survey on data management, security, and enabling technologies. IEEE Communications Surveys & Tutorials, 19(4), 2456-2501.
  5. Pajarola, R., & Gobbetti, E. (2007). Survey of semi-regular multiresolution models for interactive terrain rendering. The Visual Computer, 23, 583-605.
  6. Peuquet, D. J. (1984). A conceptual framework and comparison of spatial data models. Cartographica: The International Journal for Geographic Information and Geovisualization, 21(4), 66-113.
  7. Mohamed, F. A., & Koivo, H. N. (2010). System modelling and online optimal management of microgrid using mesh adaptive direct search. International Journal of Electrical Power & Energy Systems, 32(5), 398-407.
  8. Assunção, M. D., Calheiros, R. N., Bianchi, S., Netto, M. A., & Buyya, R. (2015). Big Data computing and clouds: Trends and future directions. Journal of parallel and distributed computing, 79, 3-15.
  9. Devine, K. D., Boman, E. G., Heaphy, R. T., Hendrickson, B. A., Teresco, J. D., Faik, J., ... & Gervasio, L. G. (2005). New challenges in dynamic load balancing. Applied Numerical Mathematics, 52(2-3), 133-152.
  10. Steger, J. L., & Benek, J. A. (1987). On the use of composite grid schemes in computational aerodynamics. Computer Methods in Applied Mechanics and Engineering, 64(1-3), 301-320.
  11. Szalay, A. S., Kunszt, P. Z., Thakar, A., Gray, J., Slutz, D., & Brunner, R. J. (2000). Designing and mining multi-terabyte astronomy archives: The sloan digital sky survey. ACM SIGMOD Record, 29(2), 451-462.
  12. Racke, H. (2002, November). Minimizing congestion in general networks. In The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings. (pp. 43-52). IEEE.
  13. Kirchdoerfer, T., & Ortiz, M. (2016). Data-driven computational mechanics. Computer Methods in Applied Mechanics and Engineering, 304, 81-101.
  14. Liu, L., Wang, H., Liu, X., Jin, X., He, W. B., Wang, Q. B., & Chen, Y. (2009, June). GreenCloud: a new architecture for green data center. In Proceedings of the 6th international conference industry session on Autonomic computing and communications industry session (pp. 29-38).
  15. Singh, J. P., Holt, C., Totsuka, T., Gupta, A., & Hennessy, J. (1995). Load balancing and data locality in adaptive hierarchical N-body methods: Barnes-Hut, fast multipole, and radiosity. Journal of Parallel and Distributed Computing, 27(2), 118-141.
  16. Thumburu, S. K. R. (2021). A Framework for EDI Data Governance in Supply Chain Organizations. Innovative Computer Sciences Journal, 7(1).
  17. Thumburu, S. K. R. (2021). Data Analysis Best Practices for EDI Migration Success. MZ Computing Journal, 2(1).
  18. Gade, K. R. (2021). Cloud Migration: Challenges and Best Practices for Migrating Legacy Systems to the Cloud. Innovative Engineering Sciences Journal, 1(1).
  19. Gade, K. R. (2021). Migrations: Cloud Migration Strategies, Data Migration Challenges, and Legacy System Modernization. Journal of Computing and Information Technology, 1(1).
  20. Katari, A., Muthsyala, A., & Allam, H. HYBRID CLOUD ARCHITECTURES FOR FINANCIAL DATA LAKES: DESIGN PATTERNS AND USE CASES.
  21. Komandla, V. Strategic Feature Prioritization: Maximizing Value through User-Centric Roadmaps.
  22. Komandla, V. Transforming Financial Interactions: Best Practices for Mobile Banking App Design and Functionality to Boost User Engagement and Satisfaction.
  23. Thumburu, S. K. R. (2020). Large Scale Migrations: Lessons Learned from EDI Projects. Journal of Innovative Technologies, 3(1).
  24. Thumburu, S. K. R. (2020). A Comparative Analysis of ETL Tools for Large-Scale EDI Data Integration. Journal of Innovative Technologies, 3(1).
  25. Gade, K. R. (2020). Data Mesh Architecture: A Scalable and Resilient Approach to Data Management. Innovative Computer Sciences Journal, 6(1).

Most read articles by the same author(s)

Similar Articles

You may also start an advanced similarity search for this article.