By Isaac Sacolick
Does it surprise you that developing user experiences, cloud applications, integrations, analytics, and machine learning models are the top priorities for IT leaders?
In IDG’s 2021 State of the CIO report, the five top initiatives driving the most IT investment include business analytics (39%), security and risk management (37%), cloud-based enterprise applications (32%), customer experiences (30%), and machine learning (25%).
That’s a tall order for CIOs and IT leaders, so it begs the question, how well are application developers, integration specialists, and data scientists collaborating?
Developers follow agile methodologies and sprint to deploy integrations and application releases, while data scientists experiment with new data sets, machine learning models, and data visualizations. Yet, despite most IT leaders claiming that they create multi-disciplinary teams, many organizations still have integration developers, app developers, and data scientists working independently.
So when organizations have a prioritized digital transformation strategy, why is it that we don’t group app, integration, and data specialists on one agile team to address integration, data, API, analytics, and user experience needs?
I believe the gap exists because of leadership, process, and technology differences. Addressing them can accelerate fulfilling transformation goals, creating a true integrated experience for users and delivering business impacts from technology, data, and analytics investments.
Identifying Common Goals Between Developers and Data Scientists
It shouldn’t be too difficult to find common goals and a shared mindset between technologists and data specialists who want to deliver new innovations.
- Both groups are using technology to deliver capabilities and demonstrate business impact.
- They require working with uncertainties around requirements, technical unknowns, changing business needs, and data complexities.
- Innovative developers and data scientists recognize that they must deliver simple solutions quickly, capture end-user feedback, and prioritize improvements.
And then, consider some of their data and technical challenges:
- They both require nimble cloud architectures and want to add and scale computing environments on-demand.
- Data scientists and developers recognize that integrations with third-party APIs and data sources are unknown inputs to their innovations.
- Both groups consume and produce data and recognize the issues of poor data quality, duplicate data, or new data that nobody loaded or defined in the data catalog.
- To maximize their time focused on customer opportunities, app and data teams implement deployment pipelines, data integrations, and other process automations.
- Keeping up with security, responding to operational issues, reducing technical debt, and maintaining systems are challenges that prevent these teams from working on business priorities.
With a common understanding of goals and a shared understanding of technical obstacles, development and data science teams are primed to work together.
Using Cloud-Native Integration Platforms to Accelerate Data Science and Application Development
To enable greater collaboration, IT leaders must consider what platforms and technologies developers, integrators, and data scientists are using to build, enhance, and support solutions.
Many teams don’t have integration solutions and develop proprietary code to connect to APIs and data sources. As integration is often a pain point for developers and data scientists, standardizing on a cloud-native integration platform that is accessible by anyone, from anywhere, and instantly connects everyone to everything fosters deeper collaboration and can provide a tenfold improvement in integration time.
IT leaders should also consider other tooling gaps. For example, data science teams may be using machine learning platforms and data visualization tools, but may not have access to a master data hub. Similarly, software developers likely have an application development stack, but having a low-code application development platform provides a faster way to develop more internal and employee workflow applications that are “on the list” but haven’t been prioritized and developed.
Selecting and using a common platform, especially on integrations and where there are tooling gaps, is the second opportunity for IT leaders to foster collaboration. It creates a common ground for application developers and data scientists to engineer solutions in the technical and data back-offices, where having a standard platform can accelerate higher quality deliverables.
Enabling Teams to Collaborate and Deliver with Agile Methodologies
With goals identified and platforms selected, a multi-skilled team of developers, integration specialists, and data scientists requires a defined way of working. Agile methodologies —especially scrum — are a strong candidate as they enable teams to prioritize, commit, deploy, and review results.
Using agile methodologies with a full integration platform as a service (iPaaS) helps IT leaders deliver new capabilities three times faster.
Let’s consider an example. A product owner or line of business lead wants to improve a field service application by incorporating weather data into the scheduling logic. Here’s how this newly created agile team might collaborate on a solution:
- Data scientists and app developers can experiment with the data and determine which sources have the required data depth, breadth, and quality for their algorithms.
- The integration specialist engineers a robust data flow for the selected data sources, and the team collaborates on a final data model. Entities for weather data are created in the master data hub, and the data catalog is updated to include the new data model.
- The data scientists incorporate the weather data into their machine learning models and deploy an API to the new version.
- App developers upgrade the end-user experience to present weather information in the field scheduling and management application.
Putting the group on one team eliminates the handoffs in orchestrating integration, analytics, and application development functions. The team collaborates on the data model and APIs, reducing the risk of rework and technical debt when there’s broad agreement on the implementation. Most important, the team is developing the product with the best engineering and data management practices by centralizing APIs and maintaining the data catalog.
When leaders provide the environment — common goals, unified platform, and agile self-organizing practices, then multi-disciplinary data science, app development, and integration teams can deliver amazing results.
The views and opinions expressed herein are those of the author and do not necessarily represent the views and opinions of Boomi.
For more StarCIO insights, read “5 Ways To Improve Developer Productivity With Low-Code Apps and Integrations.”