A Method for Scalable Real-Time Network Performance Baselining, Anomaly Detection, and Forecasting

A Method for Scalable Real-Time Network Performance Baselining, Anomaly Detection, and Forecasting

Robert Strahan (Hewlett-Packard Corporation, USA)
Copyright: © 2012 |Pages: 21
DOI: 10.4018/jbir.2012040102
OnDemand PDF Download:
$30.00
List Price: $37.50

Abstract

Communication is the lifeblood of any business. Today, communication is predominantly facilitated by digital packets transported over the interconnected arteries of the data network infrastructure. It is imperative that this infrastructure is well managed, that unexpected behavior is quickly identified and explained, and that problems are predicted and preempted. Therefore, network performance management systems should be able to detect unusual or anomalous behavior as it happens, and quickly trigger automatic analysis or alert a human operator. Growth trends in network traffic must also be identified so that future problems may be anticipated and prevented. To meet these challenges, this paper proposes an integrated, scalable method to perform baselining, anomaly detection, and forecasting on time series network metrics. The method is based on the popular Holt-Winters triple exponential smoothing technique – a technique that compares favorably to other more complex and costly approaches.
Article Preview

Introduction

The scale of today’s enterprise and service provider data network infrastructure presents unprecedented management challenges. This is particularly true in the area of network performance monitoring, which involves the real-time analysis of numerous variables collected continually from the potentially vast number of instrumented elements that make up the computing network infrastructure.

At the same time, businesses depend more than ever on the sustained reliability and performance of the computing network infrastructures, which host virtually every business function. Effective management of these assets is a business imperative.

Functionality versus cost is, of course, a key dynamic of the challenge. Poorly managed networks will prove unreliable, costing the business dearly. Well managed networks will reliably support the needs of the business, but can represent significant expense. The goal for network management application providers is to convince prospective customers that their application suite (or service) represents the best in class balance – delivering features that provide the most effective performance management capabilities, at the lowest total cost of ownership.

The network performance management solution must process potentially vast quantities of incoming data from the instrumented elements of the network, and from that data identify accurate, relevant, actionable information that allows network support staff and/or automation applications to address problems quickly and to preemptively address looming problems before they negatively affect service. This is a Business Intelligence (BI) challenge – to derive insights that inform business decisions, from the oceans of available data.

Three highly desirable features for an effective Performance Management application include:

  • Baselining - establishing performance patterns that should be considered normal, or expected, at any point in time

  • Anomaly detection - identifying traffic patterns (in real time) that are abnormal or unexpected.

  • Forecasting / Trend projection – extrapolating observed past performance into the future in order to make predictions for capacity planning and problem prevention.

These three areas are especially challenging to implement. They can be complex to design and describe, difficult and/or expensive to achieve good performance and high scale, and can produce confusing or inaccurate results.

Nevertheless, when done well, the appeal is undeniable; to effectively address these three areas is to produce concise, relevant and powerful insights that enable network operations to monitor and predict network performance and to maintain desired service levels.

Baselining

Historical trends and seasonal patterns must be analyzed to determine a baseline for the normal ranges of behavior. Application usage and network traffic patterns are closely tied to business activities, and so can be expected to vary with business cycles. For example, network usage at 4AM may not be the same as at 10AM, and Saturday may not have the same traffic patterns as Monday, and so baselining algorithms must be able to characterize normal behavior at each point in a business cycle.

The baseline of normal behavior for the network can be used to provide insight for optimization and planning activities. Perhaps scheduled workloads, such as backups, can be adjusted to make more efficient use of capacity during typically quiet periods.

Baselining also provides input to the performance management areas of anomaly detection and forecasting.

Complete Article List

Search this Journal:
Reset
Open Access Articles: Forthcoming
Volume 8: 2 Issues (2017): 1 Released, 1 Forthcoming
Volume 7: 2 Issues (2016)
Volume 6: 2 Issues (2015)
Volume 5: 4 Issues (2014)
Volume 4: 4 Issues (2013)
Volume 3: 4 Issues (2012)
Volume 2: 4 Issues (2011)
Volume 1: 4 Issues (2010)
View Complete Journal Contents Listing