Over the past five years, we have heard many stories from data science teams about their successes and challenges when building, deploying, and monitoring models
Over the past five years, we have heard many stories from data science teams about their successes and challenges when building, deploying, and monitoring models. Unfortunately, we have also heard that many companies have internalized the model myth, or the misconception that data science should be treated like software development or data assets. This misconception is completely understandable. Data science involves code and data. Yet, people leverage data science to discover answers to previously unsolvable questions. As a result, data science work is more experimental, iterative, and exploratory than software development. Data science work involves computationally intensive algorithms that benefit from scalable compute and sometimes requires specialized hardware like GPUs. Data science work also requires data, a lot more data than typical software products require. All of these needs (and more) highlight how data science work differs from software development. These needs also highlight the vital importance of collaboration between data science and engineering, particularly for innovative model-driven companies seeking to maintain or grow their competitive advantage.
Yet, collaboration between data science and engineering is a known challenge. — https://blog.dominodatalab.com/collaboration-data-science-data-engineering-true-false/
Yet, collaboration between data science and engineering is a known challenge. — https://blog.dominodatalab.com/collaboration-data-science-data-engineering-true-false/