Humans have been collecting data on variables sequenced in time since they began charting weather patterns and population figures.
Yet, analyzing metrics data, also known as time-series data, has never been more important than it is today — across a wide variety of industries.
The Rise of Big Data
A metric is a sequence of values of any defined variable, measured successively over time, often in evenly spaced intervals.
No more than one data point is gathered for each time interval. This includes everything from monthly unemployment figures to the CPU load measured continuously on a given machine.
The emergence of high-frequency trading created massive amounts of data, and it became valuable to analyze both short-term patterns and long-term histories.
Now, an ever-growing range of industries produce voluminous amounts of data, particularly with the rapid growth of the “Internet of Things.” Sensors, consumer devices, and machines are reporting thousands of measurements per second.
From “smart” thermostats, to medical devices, to manufacturing equipment, distributed information sources are yielding data collections with billions of points. In addition, virtualization, micro-services, and containerization are making today’s technology systems increasingly complex.
From Logs to Metrics
For decades, companies have collected and analyzed computerized data in the form of log files. These are text files that record events taking place in an operating system or other software, or messages between users, in order to monitor performance and identify problems.
As companies have moved to more distributed, agile, and software-dependent infrastructures, reliance on log files has become problematic.
With millions of devices, it becomes too expensive and slow to store and analyze log-based data. Companies are losing visibility on their data while the systems they’re overseeing are becoming infinitely more complex.
Meanwhile, there’s real pressure to provide uninterrupted service, and solve problems immediately. As a result, companies need a platform that enables them to identify, diagnose and solve problems in real-time.
They need a single system that can be used across teams, rather than in silos.
Metrics analysis has a number advantages over log analysis, up-down monitoring, and other traditional monitoring platforms. These include:
- Robust to-version changes
- Directly computable, without taking time to translate a log file into a metric before querying
- Less resource intensive: No parsing required, unlike logs which are made primarily of strings
- Can scale up to human understanding much broader than logs
- Can leverage for predictive purposes, such as capacity planning and correlation
- Can scale with you as you grow—resource and cost-wise
- Lower latency (ingest to query); can alert for real issues
- Applies to a wide range of applications and use cases, from managing data-center infrastructure to point-of-sale devices to software services
As companies move toward metrics as their primary data format, storing and analyzing these massive data sets has posed an enormous challenge. Products producing a billion events per day must be monitored, analyzed and displayed to users in an understandable way. Many companies have no way to retain and easily query this stream of data.
Yet the value of being able to analyze this data efficiently is enormous. It allows:
- Detection of unusual data values or previously unknown critical states before failures occur
- Correlation of events or conditions that lead to failures, allowing reduction of future risks
- Prediction of fatigue or failure
- Classification of long-term patterns and trends
- Back-testing of new models
Wavefront offers a metrics analytics solution that can handle millions of data points per second, and provides a powerful query language for capitalizing on the real-time data stream. Wavefront’s dashboards and charts are easy to use and share across teams.
Large software and Internet of Things companies have been some of the earliest to realize the power of metrics analysis, and others are catching up. Wavefront’s platform delivers for a wide variety of industry use cases, including:
- Online Services (access to applications and services 24×7)
- Energy (Internet of Things)
- Financial Services (insight into trading, payment, claims processing)
- Health Care (optimizing patient care systems)
- Manufacturing (correlating across IT and industrial)
- Retail (operational visibility across sales channels)
To learn more, visit our homepage at www.wavefront.com!