Predicting the unpredictable to optimise performance - InterSystems


Search the portal for all insights below:

Generic filters
Other Sectors

Predicting the unpredictable to optimise performance

As the events of the last year have shown, unexpected and potentially catastrophic changes within society and business occur more frequently than anyone tends to predict.

Whether it’s a natural disaster, a pandemic, or political unrest, events that would once have been described as unexpected are increasingly common. In part this is because technology has vastly increased the interconnectedness of today’s world. A virus quickly spreads around the globe because of air travel. A blockage in the Suez Canal causes major disruption because shipping technology means supply chains are truly global.

Technology has shaped the lives of billions of people throughout history, enabling them to eat better and live longer. And as it has developed, organisations that failed to adapt fell by the wayside – from stone axe grinding businesses to mass production of photographic film for family snaps.

Yet many of the bigger changes affecting society and business are not led by technology but by human behaviour or natural events. Within the last 20 years we have seen the dot-com bubble of 2001, the global financial crisis of 2008, the European sovereign debt crisis, and devastating natural disasters including the 2004 Indian Ocean earthquake and the 2010 Icelandic volcano eruption that caused the suspension of 100,000 trans-Atlantic flights. Currently, we face the long-term effects of the covid pandemic and the possibility of new virus outbreaks.

Such longtail events occur more frequently than most businesses foresee, causing them severe disruption and damage in some cases. This requires a major reassessment of how they can gain the agility needed to essentially predict the unpredictable and cope with the impact of these sudden changes.

Organisations need to evolve their services and better position themselves for the long-term to ensure they are more tolerant of systemic shocks. Shocks which can alter the flow of supply and demand in the same way earthquakes have permanently changed the course of rivers.

We are already seeing, for example, how successive Covid lockdowns have led to a major consumer shift to online retail from bricks-and-mortar, the extent of which seems to have taken even established retail chains by surprise.

Managing data and using AI and machine learning to prepare for what could happen

Preparation for the unpredictable requires analytics and data-driven decision making. The NHS, for example, is already a highly data-driven organisation. The clinicians within it have a good understanding of data and base their decisions on data that is plentiful, well-sourced, accurate and presented in ways that make it relevant and easy to understand. The organisation is learning how to manage data at scale so it can provide the best outcomes and concentrate resources where they will be most effective while continuing to innovate.

However, the kind of full agility that predicts the probability of trends or events and plots the optimum course of action requires the data to be fed into AI and machine learning models. They teach themselves from the historical data and establish patterns. Using current data they can spot the indicators and flag up how events are likely to develop and how organisations should handle them. Enterprises can use these capabilities to create scenarios that prepare them for major changes and shocks.

These are capabilities that could help businesses and planners adapt to anything from weather events to the growth of remote working, the downsizing of offices and the development of smart cities. Rather than using laborious manual methods and erratic data, predictive modelling would give these organisations a faster, far more comprehensive and more adaptable set of options.

In planning for the future, they would have far greater insight into how the flow of traffic, goods and people will change, allowing them to plan business locations, supply chain routes and stocking levels, fulfilment channels, transport and energy networks much more effectively. We know, for example, that electricity use will change hugely with mass use of electric cars. But energy companies and infrastructure providers will need to plan how this will be offset by the use of more efficient devices, reduced commuting, the relocation of major organisations or the provision of new or upgraded public transport.

The pivotal role of high-quality data

The key to all this lies in access to good, high-quality data. Many enterprises already collect high volumes of data from sensors and hardware as well as the manual inputs of their employees. Although they may not all be aware of it, they are on the road towards predictive capabilities, although for many there is work to be done to best manage the data available to them. A new approach, smart data fabrics, are coming to the fore as a way to overcome this issue and enable organisations to fully leverage their data.

Smart data fabrics interweave data from multiple sources and different formats, using a multi-tier approach that cleans data and employs an integration layer to make it usable. The fabric does this while leaving the data where it is, with lineage tracked for every item, enabling users to see where it has come from. This enables business to combine their data with information from relevant external sources using a data management platform which cleans and prepares it all for use in machine learning models, which are integrated within the fabric to allow for dynamic queries and data analytics, along with API management capabilities. These provide the predictive insights that organisations need to see what might be round the corner and be fully prepared for it – far earlier than competitors struggling without viable data or AI and machine learning.

As we move further forward, access to AI and machine learning-based predictive capabilities should extend to small or medium-sized businesses. We need to encourage more organisations to start harvesting data and to manage it effectively. As this happens, models and use cases suited to their scale and requirements will develop, providing them with the necessary agility for the future.

Whatever the scale of the business, if it has access to high quality data, its leaders should be thinking very seriously about using it to predict the unpredictable and become a far more adaptable and better prepared organisation.

Jon Payne

Manager of Sales Engineering and Education, InterSystems. Jon Payne is the manager for sales engineering at InterSystems and has 37 years experience building and delivering software in over 30 countries. Initially working for the NHS at the start of his career, Jon held a number of positions in both the healthcare and financial arena - including running his own consultancy - before joining InterSystems. Jon's role is to explore the opportunities for and then drive the adoption of InterSystems technologies.
Please click here to opt-in for marketing communications, newsletters, event invitations and more