Recently, I was doing a deep dive on Gartner’s framework for software applications in PACE Layered Architecture. It breaks down software applications in use at enterprises into three categories: Systems of Record, Systems of Differentiation, and Systems of Innovation. Today we’re diving into Systems of Differentiation and Innovation.
In summary, Systems of Record applications manage critical master data for the core enterprise business processes that are well-established and hence have a small rate of change. ERP financial systems firmly belong in this category. Competency in this level of architecture is a must for any enterprise in the twenty-first century but by itself is not sufficient to drive the enterprise further than the competition.
Systems of Differentiation applications address needs that are unique to the company or the specific niche the company occupies in the market. Such needs tend to have a higher rate of change or shift with time. For example, a manufacturer’s special value-add product offerings may require specialized pricing, production, and sales plans. While the Systems of Record may be able to record the final plans in a database, it would not be able to meaningfully create the process or the planning logic that determines a plan that takes into account the manufacturer’s unique challenges and quirks. Competency at this level of architecture is rare. Widespread use of Excel spreadsheets is a clear sign of problems in this domain. The end result is that business differentiation is not fully operationalized and hence not fully leveraged.
Systems of Innovation up the ante on Systems of Differentiation even further. Enterprises are constantly looking to innovate in the form of new products, services and markets. Thanks largely to external market forces, the speed of change is fast and the needs evolve quickly. A static enterprise IT typically waits for the innovation to fully materialize before supporting it. Often, IT is not even aware of such innovations taking place. Competency in this level of architecture is the rarest. Only the highest skilled analysts even attempt to model the data side of the innovation – again in Excel spreadsheets – and rarely try to integrate this analysis with other data. These stand-alone, ad-hoc analyses quickly get stale. What happens as a result is that the innovators remain in the dark about the business ROI of their efforts.
Systems of Differentiation and Innovation (SDAI) offer greater flexibility
Markets are increasingly competitive and force businesses to become more complex with time. To deal with both, businesses need to constantly differentiate themselves through innovation. Innovate or die is an undisputed principle now and the only real choice is how best to manage this innovation. Innovation in turn must be supported by IT systems to succeed and be absorbed fully by the organization.
Where can systems of differentiation and innovation be helpful?
I have often come across businesses looking to adopt a professionally run S&OP (sales and operational planning) process. An unsynchronized sales and operational side of the business invariably leads to lost revenue or margin. However, synchronizing sales and operations is easier said than done. Often the business will keep the process informal and solely use inflexible tools like PowerPoint as the primary method to exchange ideas and drive final decisions. Soon it becomes clear that unless the S&OP process is data driven, it only solidifies past thinking based upon gut feel only. Businesses move to Excel next. The new tool does better but still runs into challenges for two reasons. One, the amount of data required to bring together a complete S&OP picture is staggering. Two, a fair amount of AI and data science, both impossible to do at scale in Excel, is required to construct a meaningful demand plan and make tradeoffs in a quantitative way. Going beyond Excel suddenly raises the bar. What’s required is a system that lets businesses create a S&OP process that’s unique to their company. This often requires not just S&OP but also an AI-driven demand planning and an optimization-driven tradeoff evaluation engine. In other words, what enterprises need is a sophisticated System of Differentiation.
Another example is that of the addition of a new product attribute, a key driver of innovation. The new capability could be key to the future growth of the business. However, simply tracking the new attribute in a System of Record is just the tip of the iceberg of the IT change required. The new attribute, along with data on the new related production capabilities, needs to flow into the planning applications, demand plans, and production plans – a nightmare for most rigid planning applications. What’s needed are flexible planning applications and IT staff trained to use that flexibility, together creating a robust System of Differentiation.
Three examples from the food chain where a robust system of differentiation and innovation would be beneficial
1. If a large dairy processor wishes to add aged dairy products to their product offering it will trigger a cascade of changes in their existing system. Unlike the current products, the new products would need to age as long as eight months in special rooms designed for that purpose before they can be shipped. While the majority of the current business is make-to-order, these products would need to be make-to-stock given the significant time it takes to age. This immediately creates a number of challenges for the two current point solutions in place – a demand planning solution and a production planning solution. The demand plan needs to go further out than it currently does as demand much further out now affects the current production plan. Also, the production plan, designed to be make-to-order, now needs to incorporate the constraints around the aging rooms while also dealing with a year-long planning horizon. The point solution providers make vague promises on when this type of functionality will be made available. Integrating the two point solutions is another challenge even if the functionality were to become available. The dairy processor resorts to using spreadsheets to plan such items outside the gamut of the point solutions. The disjointed planning causes sub-optimal use of the rooms and lost demand opportunities. What happened here? Enterprise IT may have done well in identifying the point solution providers and doing the initial implementation, but did not create a System of Differentiation as the point solution providers focus on meeting common requirements across all their customers and not the special needs.
2. A major starch products manufacturer with all plants located close to the seasonal agricultural production centers of the country but with demand nationwide runs a unique multi-layered production and fulfillment network to optimize their business. To keep costs down, the manufacturer uses a combination of cheaper rails and more expensive trucks as and when needed. Both production and demand is volatile but the manufacturer remains committed to high customer service levels. The manufacturer has developed practices around plant sourcing, inventory target setting, and truck usage that are key to their business. However, the manufacturer’s ERP planning tools are unable to take into account any of these unique business practices and can only give them a plain requirement plan. If enterprise IT can not even find point solutions for the unique problem, the manufacturer resorts to using Excel spreadsheets to do the best they can. For sourcing information, the transportation lane details, and a lot more, they even resort to using sticky notes, as the ERP had no place for it. This is a story repeated often. Even with an ERP system in place, the manufacturer struggles with planning. What this example illustrates is that the business differentiation is not easily supported by the traditional IT architectures centered around ERPs.
3. A large retailer, in order to differentiate itself from its competitors, sources their fresh meat products internally and vertically integrates their meat supply chain. The retailer is well-known for the quality of their meat. However, the retailer’s ERP struggles with planning such products in any meaningful way. Unlike the externally sourced products, the internally sourced supply chain needs to worry about selling all products being produced in meat processing plants at the right time. It also has to worry about live animal procurement, transportation, supply-driven promotions and, critically, export markets, as the surplus product needs to be sold at a good margin to reduce unnecessary promotions in stores. What the retailer needed in this situation is not a standard retailer planning solution but a System of Differentiation to support their distinct business.
Top Reasons why IT struggles with SDAI
Systems of Records mindset is deeply embedded in Enterprise IT
Systems of Record application providers (read, ERP companies) have long sent a message which runs counter to this layered architecture. The message is this – an enterprise must standardize their processes if they want to reduce maintenance costs and benefit from upgrades in the System of Records. This thinking has percolated so deep in enterprise IT departments that IT applies this thinking to any software application whatsoever. Resistance to any change is the direct result of this thinking. SDAI requires a mindset that values differentiation, agility, flexibility and experimentation.
Systems of Records providers’ forays into planning applications are architected incorrectly
Enterprise IT always first turns to ERP companies to solve their planning problems, as they believe that ERP companies already hold all the data required (even though this is not really true. Business holds tons of relevant planning master data outside the ERP system of record in spreadsheets, emails, and even in their minds as institutional knowledge). ERP companies have taken on the challenge of tackling planning problems but architecturally they lean the same way as they for the System of Records – the more standardization the better. This is catastrophic because the planning requires a lot more business rationalization than the System of Records. The difference between niches even within the same sector of industry are large (for example, the food industry itself has tens of sub-sectors all quite different from each other – meat, dairy, vegetables, processors… just to name a few from the B2B section of the food industry) and how they plan their businesses are different as well. A single set of standard planning applications for the entire food sector is simply not possible. The companies that do go that way realize this after spending millions of dollars with nothing to show for their hard work.
Innovation is messy
Businesses know this well – innovation is a highly non-linear and chaotic process. Missteps are common and ideas take time to mature. Enterprise IT is bombarded with many requests, large to small, on a daily basis. It’s difficult to classify these requests and find their true genesis and link to a business problem. In such an environment, most IT departments resort to only addressing those needs that are clearly laid out and have a clear business case already established. Innovative ideas are rarely such.
What should a good System of Differentiation and Innovation look like?
How an IT department puts together a SDAI can vary, but a proper SDAI must aim to meet these requirements:
- It should allow data to be ingested from System of Records and Data Warehouses in an easy and configurable way, on a one-time or on-demand or recurring basis.
- Business users should be able to easily maintain master data that’s not present in the Systems of Records within the SDAI.
- With some external help, the business users should be able to use this data to put together a pipeline of data science, wrangling and algorithmic steps to predict, optimize, and plan out their key operations
- SDAI should allow a user-interface or an application to be put together to allow all business users to interact with the plan, provide feedback, and reoptimize.
- SDAI should also allow the data going into the applications along with the plans produced by SDAI to be analyzed further to understand and improve the plans. The data flow into analytics must be bi-directional.
- The level of skills required to utilize the SDAI should not exceed that of a good data analyst as the primary job of business users is to improve their businesses and not to build applications.
- And finally, the answer is not Excel. Excel does not scale, is difficult to share, does not process complex data, does not leverage the huge developments in AI and data science, and has no easy to use modern user interface.
Summary
At their beginning, all businesses are an integrated whole as they efficiently drive a new technology or innovation into a marketplace. But as the businesses grow, silos form within them, typically out of a need to scale. The silos slow down innovation thanks to friction and lack of transparency between them. A SDAI helps reduce the dreaded friction between silos by enabling cross-silo collaboration and business optimization at low cost. Enterprises without a SDAI in place ultimately have business and IT pointing fingers at each other for failures in creating sufficient innovation. The reality is that both business and IT are right – business needs more support from IT and IT is overwhelmed in arranging it. A SDAI is a way out of this classical dysfunction as it allows the business to quickly build intelligent and interactive planning tools with only low-level help from IT. And IT can organize on a single platform a variety of planning applications integrated into the data sources and each other. IT should think of SDAI as the third leg of a modern IT strategy – System of Records (ERP), Data Warehousing/BI, and SDAI to see the company data gets leveraged in smart, margin improving planning applications that bring a smile to the faces of the business leaders.
