By Michael Bachman
What you’ll learn:
● How to define your manual processes
● The best way to categorize and prioritize manual processes for an overhaul
● How to determine your path to automate those processes, especially for spreadsheets
There’s a strong likelihood that at least one mission-critical process in your business relies on that old workhorse: the spreadsheet.
Sure, spreadsheets serve a purpose. You use them. I use them. We all use them. In fact, more than an estimated 1 billion people routinely use spreadsheets at work. Over time, they’ve woven their way into our professional lives as instrumental stores of information. The reality is spreadsheets won’t disappear anytime soon. They’re comprehensible, manipulatable, and interactive. They help us make sense of things on a personal level, even with the advances in artificial intelligence (AI) like ChatGPT, Bard, and the like.
The Case for Deprecating Spreadsheet-Based Processes
What we shouldn’t do in 2023 is depend on the data that fills those columns and rows to run our operations, drive innovation, and, ultimately, make the decisions that determine the fate of our businesses. Spreadsheets are brittle, ill-maintained, unnecessarily complex at scale, non-secure, error-prone, lack adequate version control, and pose an auditing nightmare.
In short, spreadsheets are risky business.
You might counter: “But there’s gold in them thar cells!” Yet as valuable as your golden data grid may be, those nuggets of insight might as well be pyrite if the information is not connected to business information systems or is undiscoverable. Spreadsheets are veritable data silos, and data trapped within their confines costs businesses dearly.
Now, here’s the good news. You can reduce risk and extract total value from these files for your business.
In Part One of our series, we introduced the Process Maturity Ladder:
The most challenging rung to climb is that first step: eliminating manual processes. Perhaps you or your predecessors built your company on manual processes. Maybe they were even cutting-edge for their time. Your business has operated with them for as long as you can remember – even if they are cumbersome.
But let’s not kid ourselves. Manual business processes are, at best, annoying and, at worst, an auditor’s nightmare or a competitor’s opportunity.
In 2022, I polled leaders of over one hundred manufacturing organizations. More than 75% admitted that at least half of their processes were manual! They all lamented how much they wanted to automate them but thought they couldn’t.
The goal is to convert manual processes to automated ones – digital transformation – provided the new process is a measurable improvement over its predecessor. It’s also essential that the new process does not create more risk than the previous one.
Fortunately, you can do this! Let’s dig in.
Defining Manual “Process”
Perhaps you think of process, business process, workflow, and algorithm as synonyms. But while they are very much alike, we will distinguish them a bit.
There are two types of manual processes: analog and digital. Both create the kind of inertia that leaves unchanging companies to bob helplessly in the wake of modern, AI-driven organizations’ connected and automated enterprises.
Manual Analog Processes
Manual Analog processes take a physical object as “input” and copy it (create a facsimile), archive it (store it physically), or digitize it (store it logically).
Perhaps some of the best examples of a manual analog process are found within the supply chain. Handwritten invoices, receipts, bills of lading, or template forms filled in with handwriting – even wet signatures – still exist. While these physical paper documents are not as common as they once were, they remain prevalent. Manual Analog processes are inefficient, prone to error, and increase risk, to name a few.
There are options to digitize these kinds of unwieldy, sluggish artifacts of a manual analog process. Unfortunately, digitization is neither cheap nor quick. One way is manual entry. Another is to scan the physical document and create a digital image. These types of processing require humans in the loop.
The next-level digitization is optical character recognition (OCR) software. It converts letter images to digital text. Finally, the current state-of-the-art is Intelligent Document Processing (IDP). The difference between OCR and IDP is evolutionary. OCR uses machine learning to recognize characters within an image and uses classifiers to convert them to digital text. IDP uses OCR as part of its internal processes. Depending on how evolved the IDP technology is, this solution may use modern machine learning or transformer-based AI models to prepare pipelines of scanned data for consumption by databases, third-party apps, ERPs, or a variety of Enterprise Information Systems.
Regardless of the types of manual analog processes, most companies must define, document, and maintain them for auditing purposes and retain paper document originals and their copies – the data – for the foreseeable future.
Manual Digital Processes
Unlike their analog counterparts, manual digital processes originate in software. The data that users create, and the metadata that programmers embed into the software, is digital. The canonical example of a manual digital process is – you guessed it – the spreadsheet as well as the comma-separated value (CSV) file. There are plenty of other examples, like presentation software, certain desktop software, and more. But I’m targeting the spreadsheet.
Why target spreadsheets? Because so many companies run one or more mission-critical systems – maybe even their whole business – on processes reliant upon data from these rows and columns. This reliance MUST stop if your enterprise is to remain competitive. I am not saying to stop using spreadsheets. I am saying that you should stop running business processes on them, wherever possible.
Process Collection: Take Inventory, Categorize, and Rationalize
Want to ensure the longevity of your business? Repeat this mantra: AI or Die. Say it again with vim! Also, STOP RUNNING YOUR BUSINESS ON SPREADSHEETS!
Manual processes slow down business, create risk, introduce error, and manifest many other negatives. To overcome these liabilities, automate — and automate NOW. The more manual processes that run your business today, the harder it will be to change them when all of your competitors use AI – and make no mistake: they will all use AI.
The first step is to take inventory of your manual processes so that you can automate them and infuse them with intelligence. Identify your mission-critical manual digital processes. This may not be a fun process, but it’s necessary. Ask your leaders to request an inventory of each process from their teams.
Next, categorize and prioritize your business processes into their proper bucket in our MAIO framework (refer back to the Process Maturity Ladder diagram above.) Separate manual processes into Analog and Digital categories. Work with your analysts to further categorize processes using attributes like risk, revenue, or cost. Also, apply degrees of business criticality, mission criticality, stakeholder alignment, and dependency. For dependency, map how many other processes rely upon the data derived from the process.
Finally, rationalize the processes. Rethink them as if you could assign new goals. Understand each process running in your business and then ask yourselves: Is this process what good looks like for us? You may want to invest with a consultant. Planning for the future is hard and potentially costly. But ask your stakeholders if the risk is worth the cost or, as one of my colleagues often says: “Is the juice worth the squeeze?”
Let’s consider some process examples for which your business may use spreadsheets. We will keep the examples to brief descriptions for now, but how would you inventory, categorize, prioritize, and rationalize these processes?
- Budgeting and Financial Planning: Processes used to create detailed financial models, forecast revenues and expenses, track budgetary performance, and generate financial reports.
- Inventory Management: Track inventory quantities, monitor stock levels, and generate reports on stock movement.
- Sales Forecasting: Help forecast future sales and revenues based on historical data, trends, and other factors. This guides enterprises as they plan for future growth and adjust their sales strategies accordingly.
- Project Management: Track timelines, budgets, and resources. This helps enterprises manage multiple projects simultaneously and ensures that projects are completed on time and within budget.
- Human Resource Management: Manage employee data, including contact information, salaries, benefits, and performance evaluations. This information helps in decision-making around promotions, raises, and other personnel matters.
- Customer Relationship Management: Manage customer data, including contact information, sales history, and customer feedback. The information can help businesses improve customer service, identify sales opportunities, and generate reports on customer activity.
- Data Analysis: Analyze data, generate reports, and create visualizations. This can help enterprises make informed decisions about sales, marketing, and other business activities.
- Payroll Processing: Manage employee payroll data, including hours worked, deductions, and taxes. This information can help calculate employee paychecks and generate reports on payroll expenses.
- Expense Management: Track business expenses, including travel expenses, office supplies, and equipment purchases. This information can be used to monitor expenses, track budgets, and generate reports on expense activity.
- Procurement and Supply Chain Management: Manage procurement and supply chain data, including vendor information, purchase orders, and inventory levels. This can streamline procurement processes, optimize supply chain performance, and reduce costs.
Can you think of other examples?
Maybe none of these processes is spreadsheet-dependent in your business. But there are surely spreadsheets lurking within the corridors of your enterprise. Find them. Take inventory, categorize them as manual analog or digital, weigh their priority by stakeholder, figure out how many other processes depend upon them, and see if they can be transformed or rationalized. The future of your business depends on it. Spreadsheets are weak links, and your business processes are only as strong as the weakest link.
Stellar App or Left in the Stardust
Did you think I would leave you hanging with all of this spreadsheet-bashing without a solution? Before I talk about concrete steps to transform your spreadsheet into an application, a few words about manual analog processes.
Automation is not digitization. If you have essential data derived from manual analog processes, digitize now. It might be expensive, but as 2023 progresses, multi-modal generative AI should make this process more efficient.
Now, let’s convert that spreadsheet to an application. Just know that this will not be a comprehensive how-to for conversion. This is just some scaffolding.
From Spreadsheet to Database
Your spreadsheet is a data store. The data within it, however, is not easily accessible to applications programmatically. Let’s fix that.
The first step is to get your spreadsheet into a proper database. Whether that database is relational, object, document, key:value, time-series, or something else depends on the type of data you want to store and have consumed by other applications. The critical thing to know is that data in a database can flow more easily from one place to another. The process by which that data flows can be automated with enterprise data pipelines.
There are many factors to consider when converting your spreadsheet into a database (we’ve done this in my company, so I have first-hand experience!) For now, we’ll keep it simple and just list a few of the potential items:
- Frequency of column change
- Quantity of worksheets within your workbook or spreadsheet
- What is the logic (formulas within cells – this will need to be programmed within the database)
- Logical dependencies
- Targeted degree of cardinality (unique values within the database – performance factor)
This is a simplified conversion. Many factors require careful consideration. At its simplest, converting column headers in a spreadsheet could map one-to-one to table headers in a relational database. Regardless, any implementations should involve professional services and proper consulting.
Naturally, the complexity of the spreadsheet and the corresponding degree of criticality to the business could mean a lengthy project. The pain in the near term will be worth it in the long term. With the overhaul of the spreadsheet into a database format, the silo is eliminated, and the ensuing automation that emerges could dramatically reduce the risk created by the manual version.
Automation has so many benefits. But before you automate any manual process, you must understand the as-is state, define your desired to-be state, and analyze the gaps. For example, if your manual process is redundant, the automated version could get stuck in an infinite loop, cost time and money, and exacerbate risk.
A well-reasoned process and a thoughtful conversion from manual to automated process – from spreadsheet to database – is a big step toward an AI-ready enterprise and away from a company built upon a siloed and risky file.
Then, you’ll be ready to extend your spreadsheet makeover (and other processes) into additional data services that open up through automation as you climb the ladder from AI-ready to an AI-enabled enterprise.
To find out more about how the Boomi intelligent integration and automation platform can help your business, contact us.