Real-time network visibility & analytics for operational intelligence

When I launched my network analytics practice for ACG just over a year ago, I decided that my initial research needed to focus on the value of real-time network visibility and Big Data analytics for operational intelligence. SDN, virtualization and the widespread adoption of cloud-scale technologies are enabling new techniques, including streaming telemetry, for instrumenting networks and gaining real-time visibility into traffic flows and network state. At the same time, streaming analytics allows network operators to immediately turn insights into action within seconds or minutes instead of hours or days. Big Data also supports the large-scale data sets needed to apply machine learning techniques for predictive analytics and AI-based network automation.

The ROI for real-time operational intelligence is compelling across a wide array of use cases, including: rapid root cause analysis and reduced mean time to repair (MTTR); immediate detection of security threats inside the network perimeter; real-time performance monitoring for on-the-fly traffic engineering; continuous KPI monitoring for service assurance; and the holy grail: closed-loop feedback for analytics-driven automation. The potential gains are huge and the industry is witnessing a new wave of innovation that will enable us to reinvent how networks are deployed and operated, and how services are delivered and managed.

Network operators are leveraging new real-time visibility and analytics technologies in three separate, but interconnected, domains:

  • Telecom network and communication services
  • Cloud-scale services delivered via the Internet
  • Hyperscale data center infrastructure

Therefore, my research in operational intelligence has separate tracks for covering developments in each domain, although there is overlap between tracks. For example, new telecom services are being delivered via the cloud, and SD-WANs are telecom services that use the Internet to connect users to applications in the cloud. The cloud-scale services track looks at visibility and analytics from the perspective of the network operator who is delivering or consuming services over networks that the operator doesn’t own or operate, whereas the hyperscale data center track looks at the role of visibility and analytics to manage that infrastructure, which is used for delivering cloud-scale services.

As a result, my research spans three separate, but interrelated, markets:

  • Telecom services
  • Cloud-scale services
  • Enterprise IT

While today these are three distinct markets, over the next decade I expect the lines will blur as the industry converges on delivering the majority of applications and services via public and hybrid clouds. Picture one vast market – cloud-scale services –¬†segmented by application type: consumer, enterprise IT, communications, IoT, etc. At this point, the network simply provides access and transport for user devices, machines and sensors to connect with applications running in the cloud.

As an industry, we need to solve many technical problems in order to get there, with security being the most significant challenge, but today’s breakthroughs in real-time network visibility and Big Data analytics will play a key role in realizing this vision.