Gartner’s top 10 trends will drive the future of the intelligent digital mesh. Enterprise architecture and technology innovation leaders must prepare for the impacts of these disruptive trends on people, businesses and IT departments, and determine how they can provide competitive advantage.
- An intelligent digital mesh is emerging to support the future of digital business and its underlying technology platforms and IT practices. The mesh focuses on people and the Internet of Things (IoT) endpoints, as well as the information and services that these endpoints access.
- Capabilities such as blockchains, distributed ledgers and digital twins will bring the physical and digital realms ever closer to supporting digital business initiatives.
- Artificial intelligence (AI) and machine learning are the means of injecting intelligence into new and existing apps and things to form the intelligent digital mesh.
- Relentless and ever-increasing security attacks require adaptive security architectures that emphasize security-aware solution design; AI-enabled user and entity behavior analytics; and new architecture, methods and tools to address IoT and intelligent digital mesh security.
Enterprise architecture and technology innovation leaders using enterprise architecture to master emerging and strategic trends must:
- Plan for a radical, long-term (2020 to 2025) evolution of the user experience for both customers and employees as conversational systems, augmented reality (AR), virtual reality (VR) and continuous/contextual user experiences radically change the way people interact with systems.
- Explore opportunities to create new systems that require AI and machine learning, and showcase their intelligent feature systems (such as virtual personal assistants [VPAs], robots and autonomous vehicles), and augment existing apps and things (for example, customer service, enterprise applications, consumer electronics and medical devices).
- Examine specific, targeted and high-value use cases to apply AR/VR, build digital twins to support IoT initiatives, and embrace blockchains and distributed ledgers in 2017 to 2019.
- Build a security mindset into application design and operations, while augmenting preventive controls and security monitoring with user and entity behavior analytics.
Table of Contents
- Trend No. 1: Artificial Intelligence and Advanced Machine Learning
- Trend No. 2: Intelligent Apps
- Trend No. 3: Intelligent Things
- Trend No. 4: Virtual Reality and Augmented Reality
- Trend No. 5: Digital Twins
- Trend No. 6: Blockchains and Distributed Ledgers
- Trend No. 7: Conversational Systems
- Trend No. 8: Mesh App and Service Architecture
- Trend No. 9: Digital Technology Platforms
- Trend No. 10: Adaptive Security Architecture
- Gartner Recommended Reading
- Figure 1. Top 10 Strategic Technology Trends for 2017
- Figure 2. The Characteristics of Smart Machines
- Figure 3. Artificial Intelligence and Machine Learning Will Have Obvious and Inconspicuous Uses
- Figure 4. Forecast for Sales of Head-Mounted Displays, 2015-2020
- Figure 5. Digital-Twin Models Are Digital-Entity Models for Assets
- Figure 6. Key Elements of Blockchains and Distributed Ledgers
- Figure 7. Conversational Systems Include New User Experience Design Elements
- Figure 8. The Shift to a Central Role for Event Processing in Digital Business
- Figure 9. Abstract System Model for Conversational Artificial Intelligence Platform
- Figure 10. User and Entity Behavior Analytics
Our top 10 strategic technology trends include three groupings of complementary trends (see Figure 1) that are mutually reinforcing, with amplified disruptive characteristics:
- The intelligent theme builds on the way in which data science and programming approaches are evolving to include AI and advanced machine learning. This is enabling the creation of intelligent physical and software-based systems that are programmed to learn and adapt, rather than programmed only for a finite set of prescribed actions. AI and machine-learning capabilities are seeping into virtually every technology, and represent a major battleground for technology providers over the next five years.
- The digital theme focuses on blending the digital and physical worlds to create an immersive, digitally enhanced environment. In the digitally enhanced mesh, the digital world is an increasingly detailed representation of the physical world. Rich digital services, connections and interfaces connect the two. Digital trends, along with opportunities enabled by AI and machine learning, are driving the next generation of digital business.
- The mesh theme refers to exploiting connections between an expanding set of people and businesses, as well as devices, content and services, to deliver digital business outcomes. The mesh demands new interface modalities (for example, conversational interfaces), security models, technology platforms and approaches to solution design.
Strategic technology trends have substantial disruptive potential. Our top 10 list highlights strategic trends with broad industry impact that aren’t yet widely recognized. Technologies related to the strategic trends are experiencing significant changes or reaching critical tipping points in capability or maturity. Examine the business impact of our top 10 strategic technology trends, and adjust your business and IT strategies and operational models appropriately. If you don’t, you’ll risk losing competitive advantage to those who do.
Source: Gartner (October 2016)
Trend No. 1: Artificial Intelligence and Advanced Machine Learning
Applied AI and machine learning are composed of many technologies and techniques (such as deep learning, neural networks and natural-language processing [NLP]). The more-advanced techniques move beyond traditional rule-based algorithms to create systems that appear to understand, learn, predict, adapt and potentially operate with little or no human input or guidance. This is what makes smart machines appear “intelligent.” Applied AI and machine learning enable a system to not only understand concepts in the environment, but also to learn (see Figure 2).
DNN = deep neural network
Source: Gartner (October 2016)
Through machine learning, a smart machine can change its future behavior. For example, by analyzing vast databases of medical case histories, “learning” machines can reveal insights in treatment effectiveness. They can apply such insights at the speed of data ingestion, making them useful augmenters of productivity and accuracy. In scenarios involving high precision, a smart machine using intelligent ensemble techniques can achieve a reduction in error rates of 5% to 30%, or even more, which may result in substantial cost savings or extra profits. Additionally, natural-language generation dynamically increases the volume and value of insights and context in data analytics. It automatically generates a specialized narrative for each user in context, to explain meaning or highlight key findings in data.
Evaluate a number of business scenarios in which AI and machine learning could drive specific business value, and consider experimenting with one or two high-impact scenarios. For example, you could use these technologies in a retail setting to pull together and analyze online purchase histories, and product likes and dislikes — from eye-gazing technologies in stores to sensory data from smartphones — to create propensity-to-buy models that predict which product a customer is most likely to buy. In banking, you could use AI and machine-learning techniques to model current real-time transactions, as well as predictive models of transactions based on their likelihood of being fraudulent. If you’re an early adopter or seeking to drive disruptive innovation, begin to implement predictive analytics, ensemble learning, and natural-language recognition and generation. If you’re a mainstream user or have more modest innovation goals, use third parties and packaged solutions with embedded AI and machine learning.
AI and advanced machine-learning techniques are evolving rapidly. Significant investment in skills, process and tools is needed to successfully exploit these techniques in terms of setup, integration, algorithm/approach selection, data preparation and model creation. In addition, exploiting the system’s learning capabilities, evaluating the accuracy of findings, and updating the algorithms and models to improve results can take significant effort, not only from the data scientists creating the system, but also from others who have the knowledge needed to “train” the system.
Applied AI and advanced machine learning give rise to a range of intelligent implementations. These include physical devices (such as robots, autonomous vehicles and consumer electronics), as well as apps and services (such as VPAs and smart advisors). These implementations will be delivered as a new class of obviously intelligent apps and things, and provide embedded intelligence for a wide range of mesh devices, and existing software and service solutions. The data science needed to create these systems is complex, so many organizations will consume applied AI and machine learning mainly through packaged intelligent apps and things, or through packaged “models as a service” that they can build into custom applications.
- “Cool Vendors in Smart Machines, 2016”
- “Hype Cycle for Smart Machines, 2016”
- “Cool Vendors in Data Science, 2016”
- “Machine Learning Drives Digital Business”
- “Smart Machines See Major Breakthroughs After Decades of Failure”
- “Smart Machines Primer for 2016”
Trend No. 2: Intelligent Apps
Organizations are applying AI and machine-learning techniques to create new app categories (such as VPAs) and improve traditional applications (such as worker performance analysis, sales and marketing, and security). Intelligent apps have the potential to transform the nature of work and the structure of the workplace. They could alter career structures and enhance workers’ performance, but they have challenges to overcome as they move from early-stage emerging technologies to more-robust functional products. During the next 10 years, virtually every app, application and service will incorporate some level of AI in much the same way as consumer appliances have incorporated microprocessors. Some of these apps will be obvious intelligent apps that could not exist without AI and machine learning. Others will be unobtrusive users of AI and machine learning that provide intelligence behind the scenes (see Figure 3).
Source: Gartner (October 2016)
Some intelligent apps, such as VPAs, perform some of the functions of a human assistant. VPAs make everyday tasks easier (by prioritizing emails, for example), and their users more effective (by highlighting the most important content and interactions). Other intelligent apps, such as virtual customer assistants (VCAs), are more narrow, special-purpose apps that advise in specialty areas, sales and customer service.
VPAs such as Google Now, Microsoft’s Cortana and Apple’s Siri are becoming smarter and are a rapidly maturing type of intelligent app. Some chatbots, such as Facebook Messenger chatbots, can be powered by AI (for example, Wit.ai) to deliver an intelligent app. These intelligent apps feed into the system trend to create a new intelligent intermediary layer between people and systems. If you’re an early adopter or seeking to drive disruptive innovation, begin to implement targeted VCAs and VPAs where a high-value target persona (for example, a doctor, marketing leader or high-profit customer) could achieve significant benefit. If you’re a mainstream user or have more modest innovation goals, consider more simple rule-based chatbots and exploit prepackaged assistants or simple mobile assistants based on the VPA capabilities embedded in smartphones.
Packaged app and service providers are increasingly using AI and machine-learning techniques to deliver more robust systems. For example, many user and entity behavior analytics products use these techniques to identify patterns of potentially malicious activity. For some time, many enterprise application vendors have been incorporating predictive analytics capabilities into their offerings, either directly or through partners. As the focus on AI increases, vendors such as Salesforce, Oracle and Microsoft are incorporating more advanced AI functions in their offerings. These three vendors are exploiting AI to varying degrees, but they are all focusing on sales and marketing activities as a particularly valuable area for applying AI techniques to analyze customer and third-party data. Expect AI to become the next major battleground in a wide range of software and service markets, including aspects of ERP. Much hype will surround AI, so examine how and where AI is applied and what concrete business results it can deliver. Expect app and service providers to apply AI techniques in three areas:
- Advanced analytics
- Increasingly autonomous agents
- Continuous and conversational interfaces
Expect an expanding market for models as a service. Predefined models that have been taught about a particular domain and trained to identify key patterns will be delivered as a service (often with a data feed) for incorporation into other packaged or custom applications.
During the next two to five years, we expect that B2C and B2B-to-consumer companies will adopt more smart app strategies (see “Hype Cycle for Smart Machines, 2016” ). By 2018, we expect that most of the world’s largest 200 companies will exploit intelligent apps and use the full toolkit of big data and analytical tools to refine their offerings and improve their customer experience. Discover the many different types of intelligent apps that could be created with a focus on specialization and purpose. Customers may use one or a combination of intelligent apps. For example, customers may use an intelligent app to:
- Help with health (diet, exercise or psychological well-being)
- Act as a personal shopping assistant
- Act as a financial advisor
- Help with office-specific tasks, such as calendar management, email handling and external information monitoring
Intelligent apps constitute a long-term trend that will evolve and expand the use of AI and machine learning in apps and services during the next 20 years. Establish a process to continually evaluate where your organization can apply AI today and over time. Use persona-based analysis to determine the opportunities. Compare the roadmaps for AI exploitation across your packaged app and service provider portfolio. Proceed with caution if your organization is developing applications — the underlying AI and machine-learning elements for creating intelligent apps are not ready for most application development projects at scale. Ensure such projects have a very high potential business value. Note that the competitive gaps and missed opportunity costs for laggards could be significant.
- “Gartner’s 2016 Hype Cycles Highlight Digital Business Ecosystems”
- “Smart Machines Will Be the Catalyst for One of the Most Disruptive Eras in Retail”
- “Competitive Landscape: Virtual Personal Assistants, 2016”
- “IT Strategists Must Prepare for the Rise of Virtual Personal Assistants in the Workplace”
- “Smart Agents Will Drive the Switch From Technology-Literate People, to People-Literate Technology”
- “Hype Cycle for Emerging Technologies, 2016”
- “Using Algorithmic Retailing to Drive Competitive Advantage”
Trend No. 3: Intelligent Things
Intelligent things are physical things that go beyond the execution of rigid programming models to exploit applied AI and machine learning. This enables them to deliver advanced behaviors and interact more naturally with their surroundings and with people. Like intelligent apps, new intelligent things (such as autonomous vehicles) can’t exist without AI and machine learning. Meanwhile, we can enhance existing things by embedding AI and machine learning invisibly into their normal operation. For example, we can turn a camera into a smart camera.
New intelligent things fit loosely into three broad categories:
- Autonomous vehicles
Currently, the use of autonomous vehicles in controlled settings (for example, farming, mining and warehousing) is the most mature application of intelligent things. In industrial settings, vehicles can be fully autonomous. However, in more general use, autonomous automobiles must have a person in the driver’s seat in case the technology should unexpectedly fail — several U.S. states have passed laws stipulating this. In the near term, high-technology and traditional automotive manufacturers, such as Ford, Uber, Alphabet’s Google, Volkswagen, Mercedes-Benz, Tesla, Nissan, BMW and Honda, will all be testing their autonomous vehicles. For at least the next five years, we expect that semiautonomous scenarios requiring a driver will dominate. During this time, manufacturers will test the technology more rigorously, and the nontechnology issues will be addressed, such as regulations, legal issues and cultural acceptance.
Autonomous drones and robots will undergo significant technical evolution powered by new AI and machine-learning models and algorithms. They will be used mainly in narrowly defined scenarios and controlled environments. Advances in one domain — such as more sophisticated algorithms that enable a robot to learn from its environment — will often have an application in another domain.
AI and machine learning will increasingly be embedded into everyday things, such as appliances, speakers and hospital equipment. This phenomenon is closely aligned with the emergence of conversational systems, the expansion of the IoT and the trend toward digital twins. Amazon Echo is an example of an intelligent thing — it is a simple speaker connected wirelessly to an assistant powered by AI and machine learning. As conversational interfaces are delivered through other devices with a speaker or text input option, all these objects will become intelligent things.
Other markets have similar potential for embedded intelligence. For example, today’s digital stethoscope can record and store heartbeat and respiratory sounds. Collecting a massive database of such data, relating the data to diagnostic and treatment information, and building an AI-powered doctor assistance app would enable doctors to receive diagnostic support in real time. However, in the more advanced scenarios, significant issues such as liability and patient privacy must be considered. We expect that these nontechnical issues and the complexity of creating highly specialized assistants will slow embedded intelligence in industrial IoT and other business scenarios. Organizations that can address these barriers have the potential for significant competitive advantage.
Projects such as the U.S. National Robotics Initiative are pushing automation to the next level. Planning algorithms enable robots to operate autonomously on farms. Drones operating with human scouts study solutions for farmers of specialty crops. Other intelligent systems enable the design, optimization, prototyping and field-testing of mechanized harvesting systems.
As intelligent things proliferate, we expect a shift from stand-alone intelligent things to a collaborative intelligent things model. In this model, multiple devices will work together, either independently of people or with human input. For example, if a drone examined a large field and found that it was ready for harvesting, it could dispatch an “autonomous harvester.” Researchers have demonstrated a group of drones working together to construct a rope bridge, 1 while the military is studying the use of drone swarms to attack or defend military targets. 2 In the delivery market, the most effective solution may be to use an autonomous vehicle to move packages to the target area. Robots and drones on board the vehicle could then effect final delivery of the package.
Challenge the status quo on robotics. Create business scenarios and business outcome journey maps to identify and explore the opportunities that will fulfill your organization’s strategic plans. Seek opportunities to incorporate the use of emerging intelligent things in traditional manual and semiautomated tasks. For example, examine how distribution models shift as drones become safer and more effective, helping to enhance overall business performance. Expect the indirect impacts to be just as great as the direct impacts. Industry trends, such as autonomous vehicles, usher in new business designs. These require you to create proactive and predictive risk models that provide a clear view of how your organization will create value in digital ecosystems.
- “Toolkit: How to Create Business Scenarios That Drive Digital Disruption Innovation”
- “Volvo, Autoliv Create Unique Approach for Autonomous Vehicle Software”
- “Ford Autonomous Vehicle Plan Targets Mobility as a Service”
- “Your IoT Future Is Visible in the Mining Industry Today”
- “Market Trends: Personal Assistant Robots for the Home”
- “The P&C Insurance Industry Nears a Tipping Point on Drones”
Trend No. 4: Virtual Reality and Augmented Reality
Immersive technologies, such as VR and AR, are part of a new wave of computing devices that transform the way individuals interact with one another and with software systems. Head-mounted displays (HMDs) are small displays or projection technology integrated into devices worn on the head, such as glasses and helmets. HMDs derive aspects of visual content from the digital mesh. Contextual information translates the state of the wearer and the wearer’s environment into graphically rich visual cues. Many HMDs have come to market or become available for use in pilot projects in 2016. Now that commercially viable HMDs are available, the device-mesh-based apps and services that power them represent new forms of user interaction that will enable new types of consumer and workplace behaviors.
One way of experiencing immersion is using smartphone AR. The device’s screen becomes a “magic window” that displays a virtual world. This virtual world combines digital information with the physical world around the user, as captured by the device’s camera. Smartphone AR combines digital mesh data (such as wiring schematics) with the information from the smartphone’s sensors (such as its camera). It superimposes contextual information that blends augmented data on top of real-world objects (such as hidden wiring superimposed on an image of a wall). Although this approach has significant limitations compared with more robust HMD-based approaches, it represents a widely available low-cost entry point.
Smartphones can also be an effective platform for mobile VR. Google Cardboard and Samsung Gear VR are great examples of low-cost devices that use a smartphone as their computing platform. Snap your smartphone into one of these devices, hold it to your eyes, and see and interact with compelling virtual worlds. Although these devices are considered to be at the low end of quality, they still offer the flexibility of a mobile platform. Today’s use cases are firmly centered in the consumer domain, such as watching a 360-degree immersive video or playing an immersive video game. But businesses can also use mobile VR, either for marketing (to deliver personalized product experiences), or as a tool to communicate with employees.
Dedicated HMD devices, such as Oculus Rift (VR) and Microsoft HoloLens (AR), enable more sophisticated immersive interactions. These devices allow businesses to use the power of virtual worlds and augmented spaces to integrate more effectively with the human perceptual system and have a greater impact. VR devices in this category are wired to PCs or game consoles and require advanced graphics capabilities. Businesses can use these VR systems, initially intended for the consumer market and game players, in many scenarios. Training is a great example, with the virtual world simulating equipment or situations, and the sophisticated graphics capabilities ensuring that equipment looks and behaves as though real. Using VR, employees can train for many equipment use scenarios, including ones, such as catastrophic malfunction, that don’t happen often, but that need immediate attention. Businesses are also using VR for site inspections. VisualSpection provides VR headgear that allows inspection teams in the field to improve efficiency by 30%.
AR, which vendors are also marketing as mixed reality, is the best way to blend the real and virtual worlds. Using see-through displays, an AR device can track and overlay graphics onto real-world objects. Business provides the first use cases. They include DHL’s use of wearables and AR in a warehouse toachieve a 25% improvement in the picking process. 3 Our research has found that 11% of organizations are already using AR, and 13% are piloting it. 4
The landscape of immersive consumer and business content and applications will evolve dramatically through 2021. The market for HMDs will grow and evolve significantly in 2017 and 2018. Figure 4 shows our forecast for sales of HMDs through 2020. In the near term, consumers will rapidly adopt HMDs, with video games being the first popular HMD app type. More-specialized HMDs, and VR and AR content solutions, will become available for businesses. Through 2021, HMD technology will improve drastically.
Source: Gartner (October 2016)
Integration of VR and AR with multiple mobile, wearable, IoT and sensor-rich environments and conversational systems (the mesh) will extend immersive applications beyond isolated and single-person experiences. Rooms and spaces will become active with things, and their connection through the mesh will appear and work in conjunction with immersive virtual worlds. Imagine a warehouse that can not only recognize the presence of workers, but also help them understand the state of its equipment, and can visually point out areas needing attention. Although the potential of VR and AR is impressive, there will be many challenges and roadblocks.
Identify key target personas and explore digital mesh scenarios. For example, explore the needs of, and business value for, a target user in different settings, such as at home, in a car, at work, with a customer or traveling.
- “Market Guide for Augmented Reality”
- “Forecast Snapshot: Wearable Gesture Devices, Worldwide, 2016 Update”
- “Top 10 Wearable Technologies and Capabilities in 2017 and 2018”
- “Virtual Reality: What Marketers Need to Know Now”
- “What’s Hot in Digital Commerce Marketing, 2016”
- “The First Three Steps in Evaluating the Role of Head-Mounted Displays for Field Service”
Trend No. 5: Digital Twins
A digital twin is a dynamic software model of a physical thing or system that relies on sensor data to understand the state of the thing or system, respond to changes, improve operations, and add value. Digital twins include a combination of:
- Metadata (for example, classification, composition and structure)
- Condition or state (for example, location and temperature)
- Event data (for example, time series)
- Analytics (for example, algorithms and rules)
By 2020, we estimate there will be more than 21 billion connected sensors and endpoints, and digital twins will exist for potentially billions of things. Benefits will include asset optimization and improved user experience in nearly all industries. Initially, businesses will use digital twins for more complex, high-value assets, but eventually, they will use them for lower-value assets based on the use model. They will use digital twins to:
- Repair equipment and plan for its service
- Predict equipment failure or increase operational efficiency
- Plan manufacturing processes
- Operate factories
- Perform enhanced product development (by simulating the behavior of new products based on digital-twin insight from previous products, taking into account their cost, environment and performance)
Industries with high-value assets (for example, transportation and manufacturing) and industries with mission-critical remits (for example, aerospace and defense) instrument and model complex things (for example, cars, aircraft, spacecraft, machines and pumps). However, the degree of integration between the digital model and the operation of the physical thing varies greatly. These industries can use digital twins to evolve from a traditional preventive maintenance schedule to predictive, condition-based asset maintenance.
The idea of modeling the much larger number of common things — cars, buildings and consumer products — from virtual models, with functional behavior embedded to make day-to-day decisions about the physical world, is only just emerging. Today, digital twins are used by only a few professional communities, such as product engineers and data scientists, in select industries, such as manufacturing and utilities. During the next five to 10 years, operations managers will also use them for a broader set of assets where the cost-benefit analysis of risks in operations makes the case for digital twins compelling.
If your organization has high-value assets, consider using digital twins to help increase their manageability, flexibility, reliability and efficiency. The shift from preventive to predictive (condition-based) maintenance is a particularly well-established, high-value use of digital twins. Ideally, a digital twin implements one-for-one monitoring and control for each distinct physical asset. Authorized parties can query or control the digital-twin counterpart.
If your organization has lower-value assets, consider whether you can use simpler digital twins economically to improve the reliability and user experience of those assets. Digital twins don’t have to be comprehensive. You might be able to achieve a substantial benefit by instrumenting and modeling only one critical component of a device — for example, only the high-value, critical compressor in an air conditioner. Beware of overengineering a digital twin at the risk of adding unnecessary cost (for sensors, data collection and analysis, for example) when simpler models are as effective. However, you’ll probably require more sophisticated digital models when building intelligent things.
Digital twins are proxies for a combination of skilled individuals (such as technicians) and traditional monitoring devices and controls (for example, pressure gauges and pressure valves). Organizations that perform sophisticated work, such as NASA and the military, have been building complex models of their assets for years. However, most organizations will implement simple digital-twin models or link digital services to data feeds from the physical asset at first. They will then evolve the models and services, improving their ability to collect and visualize the right data, apply the right analytics and rules, and respond more effectively to the changing condition of things. Increasingly, digital services and digital-twin models will provide the digital capabilities that the physical asset needs in order to operate (see Figure 5). This will require a culture change. Technicians, engineers and operations personnel who understand the operation and maintenance of real-world things must collaborate with data scientists and other IT professionals who use digital twins and have an expanding role in improving safety, reliability and performance.
Source: Gartner (October 2016)
Approach digital twins incrementally, concentrating on immediate business value. If your organization is on the leading edge, focused on disruptive innovation or in an industry with complex assets, be more aggressive in exploiting digital twins, despite the emerging status of the trend. If your organization is more mainstream, with more-modest innovation needs and less complex assets, take a slower approach. Create simple digital twins that monitor and control crucial aspects of things, then expand your digital twins over time to represent things more comprehensively. Seek IoT solutions — either IoT devices or IoT software — that provide digital-twin templates that you can use to create digital twins for your particular requirements and assets.
- “Use the IoT Platform Reference Model to Plan Your IoT Business Solutions”
- “Digital Business Is Transforming New Product Development Priorities”
- “Reframing Your Mindset to Match Digital-Era Reality”
- “Hype Cycle for the Internet of Things, 2016”
- “Envision the IoT-Enabled R&D Digital Laboratory of the Future”
- “Market Guide: MPM and MbM Software Enable Digital Manufacturing”
- “Industry Vision: Life Science R&D as a Digital Innovation Orchestration Engine”
Trend No. 6: Blockchains and Distributed Ledgers
A distributed ledger is an expanding list of cryptographically signed, irrevocable transactional records shared by all participants in a network. Each record contains a time stamp and reference links to the previous transactions. With this information, anyone with access rights can trace back a transactional event, at any point in its history, belonging to any participant.
A blockchain is a type of distributed ledger in which value-exchange transactions (in bitcoin or another token) are sequentially grouped into blocks (see Figure 6). Each block is chained to the previous block and immutably recorded across a peer-to-peer network, using cryptographic trust and assurance mechanisms. Depending on the implementation, transactions can include programmable behavior. The term “blockchain” is also used to refer to a loosely combined set of technologies and processes that span middleware, database, security, analytics/AI, monetary and identity management concepts. Blockchain is also becoming the common shorthand for a diverse collection of distributed-ledger products, with more than 20 offerings in the market.
Source: Gartner (October 2016)
Blockchain and distributed-ledger concepts are gaining attention, because they hold the promise to transform industry operating models. Multiple business use cases are yet to be proven, but 52% of those we surveyed believe that blockchains will affect their business. 5 Although the hype surrounding blockchains concerns their use in the financial services industry, they have many potential applications beyond financial services, including music distribution, identity verification, title registry and supply chain. Smart contracts enabled by blockchain technology will drive the programmable economy. It is likely that blockchain technology will evolve and be rapidly accepted by the manufacturing, government, healthcare and education sectors.
Today, bitcoin is the only proven blockchain. Its permissionless architecture not only supports bitcoin transactions, but also enables authoritative recording of events, immutable snippets of data and simple programmable scripts. These features are exciting, but come at a cost, including:
- Lack of scalability
- Lack of complete transparency
- Limitations concerning consumption of resources
- Operational risk from unintended centralization of resources (mining)
- Lack of alignment to, and accommodation of, existing legal and accounting frameworks
Other blockchain technologies bring further adoption challenges, including a lack of:
- Robust platforms
- Scalable distributed consensus systems
- Interoperability mechanisms
There are three types of ledgers:
- Permissionless public ledgers: Operate for any (unknown/untrusted) user. Users can access the ledger and contribute transactions or new sets of data. Examples: The bitcoin blockchain or Ethereum.
- Permissioned private ledgers: Operate exclusively within a defined community of known/trusted participants, such as financial institutions and government agencies. The community (or designated authority) controls access and contribution to the ledger. Examples: Chain, Bankchain, SETL and Domus Tower.
- Permissioned public ledgers: Operate on behalf of a community of interest. The access controls are owned/managed by rules. Example: Ripple.
A critical aspect of blockchain technology today is the unregulated, ungoverned creation and transfer of funds, exemplified by bitcoin. It is this capability that funds much of blockchain development, but also concerns regulators and governments. The debates about permissioned, permissionless, hybrid and private ecosystems and governance will force a more-robust analysis of distributed ledgers. As these analyses are completed, workable solutions will evolve.
Blockchains and distributed ledgers make transactions simpler. Using a public blockchain can potentially remove the need for central authorities in arbitrating transactions. This is because trust is built into the model through immutable records on a distributed ledger. The potential of this technology to radically transform economic interactions should raise critical questions for society, governments and enterprises, for which there are no clear answers today. Begin evaluating blockchains and distributed ledgers, even if you don’t aggressively adopt the technologies in the next few years.
Most distributed-ledger initiatives are still in the early alpha or beta testing stage. Recent versions incorporate assets, data and executable programs allowing for customized applications. These ecosystems have value, but concerns remain about, for example, the viability of the technologies, startups, security (software and hardware), scalability, legality and interoperability. It is likely that development will continue in parallel for the immediate future, and it is probable that two or more ledger models will operate together.
Develop clear language and definitions for internal discussions about the nature of the technology. Recognize that the terminology surrounding blockchains is in flux. This uncertainty masks the potential suitability of technology solutions to meet business use cases. Consequently, use extreme caution when interacting with vendors that have ill-defined/nonexistent blockchain offerings. Ensure you are clearly identifying how the term “blockchain” is being used and applied, both internally and by providers. Closely monitor distributed-ledger developments, including related initiatives, such as consensus mechanism development, sidechains and blockchains. Resources permitting, consider distributed ledger as proof-of-concept development. But, before embarking on a distributed-ledger project, ensure your team has the cryptographic skills to understand what is and isn’t possible. Identify the integration points with existing infrastructures to determine the necessary future investments, and monitor the platform evolution and maturation.
- “Hype Cycle for Blockchain Technologies and the Programmable Economy, 2016”
- “Experiment With Blockchains for Data Management Innovation”
- “The Bitcoin Blockchain: The Magic and the Myths”
- “Innovation Insight for Blockchain Security”
- “Toolkit: Overview of Blockchain Use Cases”
- “What Insurance CIOs Need to Know About Blockchain”
- “How to Evolve Your Trade Finance Strategy With Smart Assets and Blockchain”
Trend No. 7: Conversational Systems
A conversational UI is a high-level design model in which user and machine interactions occur mainly in the user’s spoken or written natural language. Interactions are typically informal and bidirectional. The interaction may be a simple request or question (such as “Stop!” or “What time is it?”) with a simple result or answer. However, the interaction can also be extremely complex (such as collecting oral testimony from crime witnesses), resulting in highly complex results (the creation of a suspect’s image based on witness testimony, for example).
NLP will rapidly replace rule-based synonym and phrase substitution approaches. Dynamic natural-language ontologies or knowledge graphs at multiple levels of specificity will be needed to support NLP capabilities, such as disambiguation, concept identification and relationship extraction.
A conversational system uses a conversational UI as its main interface mode. People and machines communicate across a wide range of mesh devices (such as sensors, appliances and IoT systems). Immersive, continuous and contextual user experience elements enable this communication using a range of input/output modalities (such as sight, sound, touch, smell, taste and radar). The “conversation” between the human and the machine uses all these modalities to create a comprehensive conversational experience.
The conversational technology from major technology providers such as Apple (Siri), Google (Google Now), Amazon (Alexa) and Microsoft (Cortana) will deliver an increasingly intelligent contextual experience. This will act as an intermediary service between users and the rapidly growing set of apps and content on their mobile devices and in the cloud.
User experiences with general-purpose VPAs are often unsatisfying, because the systems try to address a very broad set of question and action scenarios. Amazon has shown that a narrower focus increases usability. Amazon’s Echo appliance and Alexa assistant have a more narrowly targeted set of question and action domains with a focus on developing related “skills” that are simple and intuitive. VPA experiences will improve as the AI back end for VPA systems continues to evolve and providers open up their systems for developers to provide tighter links to their applications for targeted scenarios. In addition, the evolving models for delivering voice-enabled solutions will expand conversational systems well beyond speaker appliances and mobile devices.
The current conversational interface method focuses on devices with microphones and speakers, but not necessarily devices with screens. However, the device mesh — one of our top 10 strategic technology trends for 2016 — encompasses an expanding set of endpoints that people use to access applications and information, or to interact with people, social communities, governments and businesses. The device mesh moves beyond the traditional desktop computer and mobile devices (tablets and smartphones) to cover the full range of endpoints with which people might interact. We expect significant innovation in new types of devices during the next five years. User experience and app design are shifting with this expanding set of endpoints.
As the device mesh evolves, we expect that connection and interface models will expand, and greater cooperative interaction between devices will develop. This will provide an immersive and continuous conversational experience. New input/output mechanisms will emerge using audio, video, touch, taste, smell and other sensory channels, such as radar, that extend beyond human senses. This will enable people to communicate with systems, and systems to communicate with people, in rich conversations that include more than text and voice (see Figure 7).
I/O = input/output
Source: Gartner (October 2016)
Apps will target an orchestrated collection of devices being used together, rather than an individual device used in isolation. This will preserve continuity of user experience across traditional boundaries of devices, time and space. Users will be able to interact with an application in a dynamic multistep sequence that may last for an extended period. The experience will flow seamlessly across multiple devices and interaction channels. It will blend physical, virtual and electronic environments. And it will use real-time contextual information as the ambient environment changes, or as the user moves from one place to another.
The shifting user experience will create many new digital business opportunities, but will also pose significant IT security and management challenges. The realization of the continuous, immersive and conversational user experience will require a profoundly better appreciation of privacy and permission. Missteps by some organizations will probably lead to regulation that will affect everyone.
- “Smart Agents Will Drive the Switch From Technology-Literate People, to People-Literate Technology”
- “Hype Cycle for Human-Machine Interface, 2016”
- “How User Experience Can Make or Break Your Customer Experience”
- “Conversational AI to Shake Up Your Technical and Business Worlds”
- “Forecast Snapshot: VPA-Enabled Wireless Speakers, Worldwide, 2016”
- “Smartphones, Tablets and Things — It’s Time to Integrate Your Mobile and IoT Strategies”
Trend No. 8: Mesh App and Service Architecture
Exploiting the opportunities and dealing with the dynamism of the intelligent digital mesh require changes to the architecture, technologies and tools used to design, develop and deliver solutions. The mesh app and service architecture (MASA) is a multichannel solution architecture that supports multiple users in multiple roles using multiple devices and communicating over multiple networks to access application functions. The architecture encapsulates services and exposes APIs at multiple levels and across organizational boundaries. It balances the demand for agility and scalability of services with their composition and reuse. The MASA enables users to have an optimized solution for targeted endpoints (such as desktops, smartphones and automobiles), as well as a continuous experience as they shift across these different channels.
Miniservices and microservices are highly complementary service models in the MASA. Monolithic applications are refactored into shared, reusable miniservices that reduce the scope of a service down to an individual capability. Miniservices are designed to support composition and reuse. These services publish APIs that can be accessed from client apps and from other services, and they enable integration and interoperability across application systems.
Microservices reduce the scope of a service down to an individual feature or function optimized for agility and scalability at a detailed feature level. Typically, a microservice doesn’t publish its API for access outside its immediate application scope.
Teams often build a miniservice to publish an API that encapsulates a set of microservices that together implement a capability. The miniservice surfaces the capability (to mobile apps, for example), while the microservices implement the individual features within the capability. Applications themselves may expose a higher-level set of APIs that don’t expose all the underlying miniservice APIs.
Abstraction via APIs is a core MASA principle. OS containers represent an approach providing a higher level of abstraction above the virtual machine. Serverless computing is another abstraction model building on these concepts and gaining ground. In this cloud computing model, the provider fully manages the infrastructure (for example, virtual machines) to serve application requests so that the developer doesn’t have to think about the server resources. This is why it is called “serverless,” although the provider still owns and operates servers behind the scenes. MASA exploits both containers and serverless computing, in addition to APIs and events connecting services, to support a more agile, flexible and rapid-change environment.
Digital twins, IoT solutions and conversational AI platforms (such as Microsoft’s Cortana, Google Now, Apple’s Siri and Amazon Echo/Alexa) require an event-driven approach. However, most production systems are designed for web APIs and request-driven synchronous application architectures, including most service-oriented architecture (SOA) implementations and REST-based design. MASA approaches will shift to an “events first, response second” approach during the next five years. Both models are essential to modern business, but in the intelligent digital mesh, the main focus will shift toward the event-driven model (see Figure 8). For example, responding in real time to a distress signal from a home device, changing trucking itineraries in response to new road or weather information, and providing “live” purchase order support can empower customers and create a differentiating business advantage.
Source: Gartner (October 2016)
The event-driven model is particularly suitable to web-scale application design, where microservices seek to maximize autonomy and agility, and where autonomy enables parallelism for extreme scale. Event-driven architecture optimizes for agility, resiliency, lower cost for change and extension, open-ended design, and web scale. The request-driven and event-driven application design models are complementary. Both can be useful and appropriate, depending on the type of business process being implemented. However, most organizations use event processing for narrow purposes in isolated application contexts — they don’t consider it a prevailing application design model equal to the common request-driven SOA. This must change to accommodate the push to digital business and enable organizations to choose the most appropriate design model for the task at hand.
As adoption of event processing as a mainstream model of application design increases, the complementary use of service-oriented and event-driven architectures will transform the MASA into a mesh of apps, events and services.
- “Mediated APIs: An Essential Application Architecture for Digital Business”
- “Articulating the Business Value of APIs”
- “Assessing Microservices for Cloud-Native Application Architecture and Delivery”
- “Orchestrating Docker Container-Based Cloud and Microservices Applications”
- “Use Agile and DevOps to Implement Lean IT and Improve Software Delivery”
- “CTO Alert: Master Event-Driven IT to Master Digital Business”
- “Add Event-Stream Processing to Your Business Analytics Repertoire”
Trend No. 9: Digital Technology Platforms
A digital technology platform is a symbiotic collection of technology capabilities and components. These provide an interoperable set of services that can be brought together to create applications, apps and services. Digital technology platforms provide the basic building blocks for, and are a critical enabler of, digital business. The platform viewpoint gives you a technology anchor model to guide technology vision, reducing complexity and redundancy.
We have identified five major digital technology platform types to enable the new capabilities and business models of digital business:
- Information system platform — Supports the back office, operations such as ERP, core systems, and associated middleware and development capabilities to deliver solutions.
- Customer experience platform — Contains the main customer-facing elements, such as customer and citizen portals, multichannel commerce, and customer apps.
- Analytics and intelligence platform — Contains information management and analytical capabilities. Data management programs and analytical applications fuel data-driven decision making, and algorithms automate discovery and action.
- IoT platform — Connects physical assets for monitoring, optimization, control and monetization. Capabilities include connectivity, analytics, and integration with core and operational technology systems.
- Business ecosystem platform — Supports the creation of, and connection to, external ecosystems, marketplaces and communities. API management, control and security are the main elements.
The MASA highlights key platform elements for the information system and business ecosystem platforms. These include the move to modular API and event-driven services, as well as the associated tools (such as API management) to operate these next-generation systems. Two other elements are emerging to deliver customer experience, advanced analytics and intelligence, and the IoT: IoT platforms and conversational AI platforms (CAPs).
IoT platforms are a collection of technologies and standards that form a base set of capabilities for communicating, controlling, managing and securing elements of the IoT. Flexible and stable IoT platform services are needed for building IoT solutions and connecting them to business solutions. Leading organizations with multiple IoT initiatives create IoT centers of excellence to aid the cross-disciplinary collaboration required for success. Although IoT platforms are essential, they remain fragmented and immature, requiring complex integration efforts. Entrants to the IoT platform market are driving rapid change from specialized IoT platforms toward more comprehensive offerings.
CAPs are general-purpose platforms that deliver a new paradigm supporting AI-rich, pervasive, proactive and conversational applications (see Figure 9). A range of focused AI services are needed, including NLP, deep learning, sentiment analysis, personality profiling, concept-relationship extraction, and other methods for inferring intent from content and context.
Source: Gartner (October 2016)
The conversational aspect of the CAP supports the development of conversational systems, with NLP rapidly replacing rule-based synonym and phrase substitution to interpret user input. Dynamic natural-language ontologies or knowledge graphs at multiple levels of specificity will be needed to support NLP capabilities, such as disambiguation, concept identification and relationship extraction.
Tools and services to support immersive, continuous and contextual experience that goes beyond the voice-/text-powered conversational interface deliver the pervasive aspect of the CAP. The CAP’s proactive aspect offers nondisruptive simplification for the user, with the system adapting to the user rather than the user having to adapt to the system. The platform detects patterns in the user’s behavior, asks questions to clarify the user’s requests, provides unsolicited and meaningful suggestions, and autonomously takes action on the user’s behalf. CAP-enabled applications move away from fixed commands for communications between people, bots, agents, assistants, applications and other services. Many vendors are speeding to market with new CAPs that will host a broad range of solutions.
The IoT platform benefits from the CAP in many ways. The CAP’s conversational nature removes the need for the user to recall specific commands, syntax or parameters for remote-control IoT use cases. By its nature, an IoT system can consist of a myriad of protocols up and down the stack (see “IoT Communications Architecture Demystified” ). The CAP’s NLP capabilities provide an abstraction that can potentially ease cross-protocol communication issues. The CAP’s AI capabilities enable it to learn from data ingested from individual objects over time. This provides more value to the overall IoT system and can accelerate digital-twin efforts.
Similarly, the CAP benefits from the IoT platform in several ways. The IoT platform provides the underlying infrastructure that facilitates communication and action among users, objects and applications. The IoT platform enables the CAP to reach out not only to apps, but also to individual objects and systems (and their associated data and analytics). The IoT provides data from more sources (input), as well as potential actions that affect the physical world (output). This allows for a richer CAP experience for the user, with more opportunities for automation and efficiency.
Technology providers are already starting to experiment with the symbiotic relationship between the CAP and IoT-related platform services. In its August 2016 update, Microsoft’s Skype division added the “If This Then That” (IFTTT) bot to its bot directory. 6 It can interact with more than 50 different types of IoT devices, ranging from cars to wearables to connected home devices.
- “Building a Digital Business Technology Platform”
- “The Platform Architect’s Guide to Designing IoT Solutions”
- “Market Guide for IoT Platforms”
- “Cool Vendors in the Internet of Things, 2016”
- “Use the IoT Platform Reference Model to Plan Your IoT Business Solutions”
- “Conversational AI to Shake Up Your Technical and Business Worlds”
Trend No. 10: Adaptive Security Architecture
The intelligent digital mesh and related digital technology platforms and application architectures create an ever-more-complex world for security. The continuing evolution of the “hacker industry” and its use of increasingly sophisticated tools — including the same advanced technologies available to enterprises — significantly increase the threat potential. Relying on perimeter defense and rule-based security is inadequate and outdated, especially as organizations exploit more cloud-based services and open APIs for customers and partners to create business ecosystems. IT leaders must focus on detecting and responding to threats, as well as more traditional measures, such as blocking, to prevent attacks and other abuses. Organizations will need security-aware application design, application self-protection, user and entity behavior analytics, API protection, and specific tools and techniques to address IoT and intelligent app/thing vulnerabilities.
Security architecture starts with network security and access control, vulnerability management, endpoint protection and basic monitoring. However, these controls alone are insufficient. Hackers target applications and content sources, as well as individual services that have intentionally been opened to the outside world to promote the development of business ecosystems, and digital-twin models that can monitor and control physical assets. Applications, services and models are a critical element in the security equation, and a security mindset is vital when designing, developing and testing these applications.
Organizations must overcome the barriers between security teams and application teams, much as DevOps tools and processes overcome the divide between development and operations. Security teams can’t afford to wait until the end of the build-and-release pipeline to offer meaningful feedback. Security requirements must be clearly communicated and easily integrated into work processes. Security teams must work with application, solution and enterprise architects to build security into the overall DevOps process, resulting in a DevSecOps model.
User and entity behavior analytics are an important emerging category of security (see Figure 10). They profile and baseline the activity of users, peer groups and other entities, such as endpoints, applications and networks. They correlate user and other entity activities and behaviors, and detect anomalous behavior and patterns using advanced machine learning and statistical models that compare activity to profiles. User and entity behavior analytics show, for example, whether individuals are visiting sites they haven’t visited before or are downloading things they don’t normally download. Unusual behavior triggers alarms or an automated response. Much venture capital exists in this area, along with new tools and technologies.
DAP = database audit and protection; DLP = data loss prevention; SIEM = security information and event management
Source: Gartner (October 2016)
Traditional infrastructure and perimeter protection technologies can’t ensure accurate detection of application vulnerabilities and protection against application-level attacks. Moreover, they can’t protect against behind-the-perimeter insider attacks, which are as devastating as outsider attacks. Therefore, technologies are emerging that enable application security self-testing, self-diagnostics and self-protection. Still, application-layer controls that are external to the application play an important role in defending against distributed denial-of-service and automated attacks, as well as providing security capabilities on behalf of a group of protected applications.
The scale and diversity of the intelligent digital mesh poses a significant security challenge. This is driving the need for robust IoT security architecture and practices with a particular focus on endpoint devices. The IoT elements are diverse and use much non-IT hardware and many protocols, creating additional challenges. Moreover, the bridging of the divide between IT and operational technology sees a disconnection between traditional IT security technologies and practices, with engineers more familiar with, and focused on, reliability and safety practices.
Resilience and security need to be designed into digital business solutions. Business stakeholders must include privacy, safety and reliability objectives, and consider protection as well as recovery. Organizations that embrace the DevSecOps model emphasizing security-aware app/service/model design are best placed to accomplish this goal. As demonstrations of automobile hacking have shown, 7 design considerations must include levels of isolation between solution components. Different levels of security must be applied based on the risk exposure of different systems.
Digital twins consolidate massive amounts of information on individual assets and groups of assets, often providing control of those assets. As the digital-twin trend evolves, twins will communicate with one another to create “digital factory” models of multiple linked digital twins. Digital twins of assets will be linked to other digital entities for people (digital personas), processes (law enforcement) and spaces (digital cities). Understanding the links across these digital entities, isolating elements where needed and tracking interactions will be vital to support a secure digital environment.
A number of factors can help secure the IoT environment. Ensure that device hardware and software are resistant to attacks and are secure (for example, by implementing secure software updates). Secure all access and communication channels with appropriate access control, authentication or encryption, and closely monitor API access to systems, particularly where these APIs are intentionally opened for outside entities. Use established security technologies as a baseline to secure IoT platforms. Monitor user and entity behavior, particularly in IoT scenarios. Implement sufficient security monitoring and management practices for edge devices, including secure updates. However, the IoT edge is a new frontier for many IT security professionals, creating new vulnerability areas. It often requires new remediation tools and processes that must be factored into IoT platform efforts.
- “The Fast-Evolving State of Security Analytics, 2016”
- “CISOs Need to Understand the Components of Their Information Security Programs”
- “Market Guide for User and Entity Behavior Analytics”
- “Best Practices and Success Stories for User Behavior Analytics”
- “Hype Cycle for Application Security, 2016”
- “A Primer for Building Resilience and Security Into Internet of Things Solution Architecture”
- “DevSecOps: How to Seamlessly Integrate Security Into DevOps”
Some documents may not be available as part of your current Gartner subscription.
1 N. Lavars, “Drones Autonomously Build a Walkable Rope Bridge,” New Atlas, 22 September 2015.
2 L. Seligman, “How Swarming Drones Could Change the Face of Air Warfare,” DefenseNews, 17 May 2016.
3 “DHL Successfully Tests Augmented Reality Application in Warehouse,” DHL, 26 January 2015.
- Eleven percent are already using AR.
- Thirteen percent are piloting AR.
- Thirteen percent plan to implement AR in the next year.
- Seventeen percent expect to implement AR within two to three years.
6 Pradeep, “Microsoft Introduces New Bots Into the Skype Bot Directory,” MSPoweruser, 3 August 2016.
7 A. Greenberg, “The Jeep Hackers Are Back to Prove Car Hacking Can Get Much Worse,” Wired, 1 August 2016.
© 2016 Gartner, Inc. and/or its Affiliates. All Rights Reserved. Reproduction and distribution of this publication in any form without prior written permission is forbidden. The information contained herein has been obtained from sources believed to be reliable. Gartner disclaims all warranties as to the accuracy, completeness or adequacy of such information. Although Gartners research may discuss legal issues related to the information technology business, Gartner does not provide legal advice or services and its research should not be construed or used as such. Gartner shall have no liability for errors, omissions or inadequacies in the information contained herein or for interpretations thereof. The opinions expressed herein are subject to change without notice.