Առնակ Պողոսյանի հրապարակումներ (Full profile)

Distributed tracing is cutting-edge technology used for monitoring, managing, and troubleshooting native cloud applications. It offers a more comprehensive and continuous observability, surpassing traditional logging methods, and is indispensable for navigating modern complex software architectures. However, the sheer volume of generated traces is staggering in distributed applications, and the direct storage and utilization of every trace is impractical due to associated operational costs. This entails a sampling strategy to select which traces warrant storage and analysis. Historically, sampling methods have included a rate-based approach, often relying heavily on a manual configuration. There is a need for a more intelligent approach, and we propose a hierarchical sampling methodology to address multiple requirements concurrently. Initial rate-based sampling mitigates the overwhelming volume of traces, as no further analysis can be performed on this level. In the next stage, more nuanced analysis is facilitated based on the previous foundation, incorporating information regarding trace properties and ensuring the preservation of vital process details even under extreme conditions. This comprehensive approach not only aids in the visualization and conceptualization of applications but also enables more targeted analysis in later stages. As we delve deeper into the sampling hierarchy, the technique becomes tailored to specific purposes, such as the simplification of application troubleshooting. In this context, the sampling strategy prioritizes the retention of erroneous traces from dominant processes, thus facilitating the identification and resolution of underlying issues. The focus of this paper is to reveal the impact of sampling on troubleshooting efficiency. Leveraging intelligent and explainable artificial intelligence solutions enables the detection of malfunctioning microservices and provides transparent insights into root causes. We advocate for using rule-induction systems, which offer explainability and efficacy in decision-making processes. By integrating advanced sampling techniques with machine-learning-driven intelligence, we empower organizations to navigate the complexities of large-scale distributed cloud environments effectively.

In a vast majority of cases, remediation of IT issues encoded into domain-specific or user-defined alerts occurring in cloud environments and customer ecosystems suffers from accurate recommendations, which could be supplied in a timely manner for recovery of performance degradations. This is hard to realize by furnishing those abnormality definitions with appropriate expert knowledge, which varies from one environment to another. At the same time, in many support cases, the reported problems under Global Support Services (GSS) or Site Reliability Engineering (SRE) treatment ultimately go down to the product teams, making them waste costly development hours on investigating self-monitoring metrics of our solutions. Therefore, the lack of a systematic approach to adopting AI Ops significantly impacts the mean-time-to-resolution (MTTR) rates of problems/alerts. This would imply building, maintaining, and continuously improving/annotating a data store of insights on which ML models are trained and generalized across the whole customer base and corporate cloud services. Our ongoing study aligns with this vision and validates an approach that learns the alert resolution patterns in such a global setting and explains them using interpretable AI methodologies. The knowledge store of causative rules is then applied to predicting potential sources of the application degradation reflected in an active alert instance. In this communication, we share our experiences with a prototype solution and up-to-date analysis demonstrating how root conditions are discovered accurately for a specific type of problem. It is validated against the historical data of resolutions performed by heavy manual development efforts. We also offer experts a Dempster–Shafer theory-based rule verification framework as a what-if analysis tool to test their hypotheses about the underlying environment.

We investigate the convergence of the quasi-periodic approximations in different frameworks and reveal exact asymptotic estimates of the corresponding errors. The estimates facilitate a fair comparison of the quasi-periodic approximations to other classical well-known approaches. We consider a special realization of the approximations by the inverse of the Vandermonde matrix, which makes it possible to prove the existence of the corresponding implementations, derive explicit formulas and explore convergence properties. We also show the application of polynomial corrections for the convergence acceleration of the quasi-periodic approximations. Numerical experiments reveal the auto-correction phenomenon related to the polynomial corrections so that utilization of approximate derivatives surprisingly results in better convergence compared to the expansions with the exact ones.

The main purpose of an application performance monitoring/management (APM) soft- ware is to ensure the highest availability, efficiency and security of applications. An APM software accomplishes the main goals through automation, measurements, analysis and diagnostics. Gartner specifies the three crucial capabilities of APM softwares. The first is an end-user experience moni- toring for revealing the interactions of users with application and infrastructure components. The second is application discovery, diagnostics and tracing. The third key component is machine learn- ing (ML) and artificial intelligence (AI) powered data analytics for predictions, anomaly detection, event correlations and root cause analysis. Time series metrics, logs and traces are the three pillars of observability and the valuable source of information for IT operations. Accurate, scalable and robust time series forecasting and anomaly detection are the requested capabilities of the analytics. Ap- proaches based on neural networks (NN) and deep learning gain an increasing popularity due to their flexibility and ability to tackle complex nonlinear problems. However, some of the disadvantages of NN-based models for distributed cloud applications mitigate expectations and require specific approaches. We demonstrate how NN-models, pretrained on a global time series database, can be applied to customer specific data using transfer learning. In general, NN-models adequately operate only on stationary time series. Application to nonstationary time series requires multilayer data processing including hypothesis testing for data categorization, category specific transformations into stationary data, forecasting and backward transformations. We present the mathematical back- ground of this approach and discuss experimental results based on implementation for Wavefront by VMware (an APM software) while monitoring real customer cloud environments.

We address several problems in intelligent log management of distributed cloud computing applications and their machine learning solutions. Those problems concern various tasks on characterizing data center states from logs, as well as from related or other quantitative metrics (time series), such as anomaly and change detection, identification of baseline models, impact quantification of abnormalities, and classification of incidents. These are highly required jobs to be per-formed by today’s enterprise-grade cloud management solutions. We describe several approaches and algorithms that are validated to be effective in an automated log analytics combined with analytics from time series perspectives. The paper introduces novel concepts, approaches, and algorithms for feasible log-plus-metric-based management of data center applications in the context of integration of relevant technology products in the market.

We consider convergence acceleration of the modified Fourier expansions by rational trigonometric corrections which lead to modified-trigonometric-rational approximations. The rational corrections contain some unknown parameters and determination of their optimal values for improved pointwise convergence is the main goal of this paper. The goal was accomplished by deriving the exact constants of the asymptotic errors of the approximations with further elimination of the corresponding main terms by appropriate selection of those parameters. Numerical experiments outline the convergence improvement of the optimal rational approximations compared to the expansions by the modified Fourier basis.

The paper investigates the pointwise convergence of the quasi-periodic trigonometric interpolation and derives exact constant of the main term of asymptotic error for smooth functions.

We consider the convergence acceleration of the Krylov-Lanczos interpolation by rational correction functions and investigate convergence of the resultant parametric rational-trigonometric-polynomial interpolation. Exact constants of asymptotic errors are obtained in the regions away from discontinuities, and fast convergence of the rational-trigonometric-polynomial interpolation compared to the Krylov-Lanczos interpolation is observed. Results of numerical experiments confirm theoretical estimates and show how the parameters of the interpolations can be determined in practice.

A data-agnostic approach to automated problem (fault or change) root cause determination in complex IT systems based on monitoring and anomaly event data correlation is developed. It also includes a framework for identification of bottlenecks and black swan type issues in IT infrastructures. The relevant anomaly event correlation engine (AECE) is designed, prototyped, and successfully applied to data from real environments. This approach is based on a historical analysis of abnormality events (obtained by application of an IT management analytics) with their probabilistic correlations. The general theory that applies information measures between the random variables embodying those events is described, and a particular scenario is discussed. The method is capable of detecting origins of problems and generating real-time recommendations for their locations in a hierarchical system. Moreover, it allows localizing the bottleneck IT resources which are persisting causes of historical problems. The black swan type events are shown to be retrospectively identified and used in their risk estimation on-line. The technology and algorithms are based on statistical processing of virtual directed graphs produced from historical anomaly events.

We investigate convergence of the rational-trigonometric-polynomial interpolation that performs convergence acceleration of the classical trigonometric interpolation by sequential application of polynomial and rational correction functions. Unknown parameters of the rational corrections are determined along the ideas of the Fourier-Pade approximations. The resultant interpolation we call as Fourier-Pade interpolation and investigate its convergence in the regions away from singularities. Comparison with other rational-trigonometric-polynomial interpolations outlines the convergence properties of the Fourier-Pade interpolation.

We consider trigonometric interpolations with shifted equidistant nodes and investigate their accuracies depending on the shift parameter. Two different types of interpolations are in the focus of our attention: the Krylov-Lanczos and the rational-trigonometric-polynomial interpolations. The Krylov-Lanczos interpolation performs convergence acceleration of the classical trigonometric interpolation by polynomial corrections. Additional acceleration is achieved by application of rational corrections which contain some extra parameters. In both cases, we derive the exact constants of the asymptotic errors and, based on these estimates, we find the optimal shifts that provide with the best accuracy. Optimizations are performed for the pointwise convergence in the regions away from the endpoints. Asymptotic estimates allow optimal selection of the extra parameters in the rational corrections which provides with additional accuracy. Results of numerical experiments clarify theoretical investigations.

We investigate a convergence of the L2-optimal rational approximation in the regions away from singularities where the approximated function is smooth. Theoretical estimates show that the rate of convergence is greater than for the classical KrylovLanczos method by the order which equals to the order of denominator of the rational approximants.

We consider convergence acceleration of the truncated Fourier series by sequential application of polynomial and rational corrections. Polynomial corrections are performed along the ideas of the Krylov-Lanczos approximation. Rational corrections contain unknown parameters which determination is a crucial problem for realization of the rational approximations. We consider approach connected with the Fourier-Pade approximations. This rational-trigonometric-polynomial approximation we continue calling the Fourier-Pade approximation. We investigate its convergence for smooth functions in different frameworks and derive the exact constants of asymptotic errors. Detailed analysis and comparisons of different rational-trigonometric-polynomial approximations are performed and the convergence properties of the Fourier-Pade approximation are outlined. In particular, fast convergence of the Fourier-Pade approximation is observed in the regions away from the endpoints.

A sequence of Hermite trigonometric interpolation polynomials with equidistant interpolation nodes and uniform multiplicities is investigated. We derive relatively compact formula that gives the interpolants as functions of the coefficients in the DFTs of the derivative values. The coefficients can be calculated by the FFT algorithm. Corresponding quadrature formulae are derived and explored. Convergence acceleration based on the Krylov-Lanczos method for accelerating both the convergence of interpolation and quadrature is considered. Exact constants of the asymptotic errors are obtained and some numerical illustrations are presented.

A sequence of Hermite trigonometric interpolation polynomials with equidistant interpolation nodes and uniform multiplicities is investigated. We derive relatively compact formula that gives the interpolants as functions of the coefficients in the DFTs of the derivative values. The coefficients can be calculated by the FFT algorithm. Corresponding quadrature formulae are derived and explored. Convergence acceleration based on the Krylov-Lanczos approach for accelerating both the convergence of interpolation and quadrature is considered. Exact constants of the asymptotic errors are obtained and some numerical illustrations are presented.

A sequence of Hermite trigonometric interpolation polynomials with equidistant interpolation nodes and uniform multiplicities is investigated. We derive relatively compact formula that gives the interpolants as functions of the coefficients in the DFTs of the derivative values. The coefficients can be calculated by the FFT algorithm. Convergence acceleration based on the Eckhoff method is considered. Approximation of jumps is explored and the corresponding exact constant of the asymptotic error is obtained.

Hereby, the convergence acceleration of two-dimensional trigonometric interpolation for a smooth functions on a uniform mesh is considered. Together with theoretical estimates some numerical results are presented and discussed that reveal the potential of this method for application in image processing. Experiments show that suggested algorithm allows acceleration of conventional Fourier interpolation even for sparse meshes that can lead to an efficient image compression/decompression algorithms and also to applications in image zooming procedures.

The current paper considers the problem of recovering a function using a limited number of its Fourier coefficients. Specifically, a method based on Bernoulli-like polynomials suggested and developed by Krylov, Lanczos, Gottlieb and Eckhoff is examined. Asymptotic behavior of approximate calculation of the so-called “jumps” is studied and asymptotic L 2 constants of the rate of convergence of the method are computed.

A nonlinear method of accelerating both the convergence of Fourier series and trigonometric interpolation is investigated. Asymptotic estimates of errors are derived for smooth functions. Numerical results are represented and discussed.

The Fourier series of a smooth function on a compact interval usually has slow convergence due to the Gibbs phenomena. A class of Fourier-Pade approximations is introduced and studied for performing a boundary correction. Additional acceleration is achieved by applying Fourier–Bernoulli scheme.

Automated computational methods and systems to classify and troubleshoot problems in information technology (“IT”) systems or services provided by a distributed computing system are described. Each IT system of the distribution computing system or IT service provided by the distributed computing system has an associated key performance indicator (“KPI”) used to monitor performance of the IT system or service. When real-time KPI data violates a KPI threshold, a real-time event-type distribution is computed from event messages generated by event sources associated with the IT system or service following the threshold violation. The real-time event-type distribution is compared with historical event-type distributions recorded for the KPI data in order to identify the problem and execute remedial action to resolve the problem.

Automated processes and systems for detecting abnormally behaving objects of a distributed computing system are described. Processes and systems obtain metrics that are generated in a historical time window and are associated with an object of the distributed computing system. Processes and system use the metrics to compute a time dependent system indicator over the historical time window. Each value of the system indicator corresponds to a point in time of the historical time window when the object was in a normal or an abnormal state. Processes and systems use the normal and abnormal states of the system indicator in the historical time window to train a state classifier that is used to detect run - time abnormal behavior of the object. When the state classifier identifies abnormal behavior of the object, an alert is generated, indicating the abnormal behavior of the object.

Automated methods and systems to determine a baseline event - type distribution of an event source and use the baseline event type distribution to detect changes in the behavior of the event source are described. In one implementation, blocks of event messages generated by the event source are collected and an event - type distribution is computed for each of block of event messages. Candidate baseline event - type distributions are determined from the event - type distributions. The candidate baseline event - type distribution has the largest entropy of the event - type distributions. A normal discrepancy radius of the event - type distributions is computed from the baseline event - type distribution and the event - type distributions. A block of run time event messages generated by the event source is collected. A run - time event - type distribution is computed from the block of run - time event messages. When the run - time event - type distribution is outside the normal discrepancy radius, an alert is generated indicating abnormal behavior of the event source.

The current document is directed to methods and systems that collect metric data within computing facilities, including large data centers and cloud-computing facilities. In a described implementation, lower and higher metric-data-value thresholds are used to partition collected metric data into outlying metric data and inlying metric data. The inlying metric data is quantized to compress the inlying metric data and adjacent data points having the same quantized metric-data values are eliminated, to further compress the inlying metric data. The resulting compressed data includes original metric-data representations for outlier data points and compressed metric-data representations for inlier data points, providing accurate restored metric-data values for significant data points when compressed metric data is decompressed.

The current document is directed to methods and systems for detecting the occurrences of abnormal events and operational behaviors within the distributed computer system. The currently described methods and systems continuously collect metric data from various metric-data sources, generate a sequence of metric-data observations, each metric-data observation comprising a set of temporally aligned metric data, and employ principle-component analysis to transform the metric-data observations to facilitate reduction of the dimensionality of the metric-data observations. The currently described methods and systems then employ clustering methods to identify outlying transformed-metric-data observations, accordingly label the transformed metric-data observations to generate a training dataset, and then apply one or more of various types of machine-learning techniques to the training dataset in order to generate an abnormal-observation detector that can be used to detect, in real time, abnormal metric-data observations as they are generated within the distributed computing system.

The current document is directed to methods and subsystems within computing systems, including distributed computing systems, that collect, store, process, and analyze population metrics for types and classes of system components, including components of distributed applications executing within containers, virtual machines, and other execution environments. In a described implementation, a graph-like representation of the configuration and state of a computer system included aggregation nodes that collect metric data for a set of multiple object nodes and that collect metric data that represents the members of the set over a monitoring time interval. Population metrics are monitored, in certain implementations, to detect outlier members of an aggregation.

Methods and systems that estimate a degree of abnormality of a complex system based on historical time-series data representative of the complex system's past behavior and using the historical degree of abnormality to determine whether or not a degree of abnormality determined from current time-series data representative of the same complex system's current behavior is worthy of attention. The time-series data may be metric data that represents behavior of a complex system as a result of successive measurements of the complex system made over time or in a time interval. A degree of abnormality represents the amount by which the time-series data violates a threshold. The larger the degree of abnormality of the current time-series data is from the historical degree of abnormality, the larger the violation of the thresholds and the greater the probability the violation in the current time-series data is worthy of attention.

Methods and systems of automatic confidence - controlled sampling to analyze , detect anomalies and problems in monitoring data and event messages generated by sources of a distributed computing system are described. A source can be virtual or physical object of the distributed computing system , a resource of the distributed computing system, or an event source running in the distributed computing . Monitoring data includes metric data generated by resources and data that represents meta - data properties of event sources . Confidence - controlled sampling is used to determine characteristics of the monitoring data , identify periodic patterns in the behavior of a source , detect changes in behavior of a source , and compare the behavior of two sources . Confidence - controlled sampling speeds up characterization the data sets , determination of behavior patterns , and detection and reporting of anomalies and problems of the resources and event sources of the distributed computing system.

The current document is directed to methods and systems that collect metric data within computing facilities, including large data centers and cloud - computing facilities . In a described implementation, two or more metric - data sets are combined to generate a multidimensional metric - data set . The multidimensional metric - data set is compressed for efficient storage by clustering the multidimensional data points within the multidimensional metric - data set to produce a covering subset of multidimensional data points and by then representing the multidimensional - data - point members of each cluster by a cluster identifier rather than by a set of floating - point values , integer values, or other types of data representations . The covering set is constructed to ensure that the compression does not result in greater than a specified level of distortion of the original data.

A problem in a cloud infrastructure may be identified when a server computer deviates from a normal level of operation based on anomaly scores, which generates an alert and an alert time that indicates when the alert is generated. Methods then determine which virtual machine (“VM”) and other IT objects/resources or their pools contribute to the problem within a time window surrounding the estimated problem start time and calculate which objects show similar, related anomalous behavior. Methods also generate ranked remediation recommendations on an object level and server computer-to-object level. The methods generate results that enable a system administrator to identify the start time of the problem and identify the objects that are responsible for the problem.

This disclosure is directed to data-agnostic computational methods and systems for adjusting hard thresholds based on user feedback. Hard thresholds are used to monitor time-series data generated by a data-generating entity. The time-series data may be metric data that represents usage of the data-generating entity over time. The data is compared with a hard threshold associated with usage of the resource or process and when the data violates the threshold, an alert is typically generated and presented to a user. Methods and systems collect user feedback after a number of alerts to determine the quality and significance of the alerts. Based on the user feedback, methods and systems automatically adjust the hard thresholds to better represent how the user perceives the alerts.

Methods and systems are directed to detecting and classifying changes in a distributed computing system. Divergence value are computed from distributions of different types of event messages generated in time intervals of a sliding time window. Each divergence value is a measure of change in types of events generated in each time interval. When a divergence value, or a rate of change in divergence values, exceeds a threshold, the time interval associated with the threshold violation is used to determine a change point in the operation of the distributed computing system. Based on the change point, a start time of the change is determined. The change is classified based on various previously classified change points in the disturbed computing system. A recommendation may be generated to address the change based on the classification of the change.

Methods determine a capacity-forecast model based on historical capacity metric data and historical business metric data. The capacity-forecast model may be to estimate capacity requirements with respect to changes in demand for the data center customer's application program. The capacity-forecast model provides an analytical “what-if” approach to reallocating data center resources in order to satisfy projected business level expectations of a data center customer and calculate estimated capacities for different business scenarios.

Methods and systems to evaluate data center performance and prioritize data center objects and anomalies for remedial actions are described. Methods rank data center objects and determine object performance trends. Methods calculate an object rank of each object of the data center over a period of time and calculate an object trend of each object of the data center based on relative frequencies of alerts at different times. The objects may be prioritized for remedial actions based on the object ranks and object trends.

This disclosure is directed to automated methods and systems for calculating hard thresholds used to monitor time-series data generated by data-generating entity. The methods are based on determining a cumulative distribution that characterizes the probability that data values of time-series data generated by the data-generating entity violate a hard threshold. The hard threshold is calculated as an inverse of the cumulative distribution based on a user defined risk confidence level. The hard threshold may then be used to generate alerts when time-series data generated later by the data-generating entity violate the hard threshold.

This disclosure is directed to data-agnostic computational methods and systems for adjusting hard thresholds based on user feedback. Hard thresholds are used to monitor time-series data generated by a data-generating entity. The time-series data may be metric data that represents usage of the data-generating entity over time. The data is compared with a hard threshold associated with usage of the resource or process and when the data violates the threshold, an alert is typically generated and presented to a user. Methods and systems collect user feedback after a number of alerts to determine the quality and significance of the alerts. Based on the user feedback, methods and systems automatically adjust the hard thresholds to better represent how the user perceives the alerts.

This disclosure is directed to computational, closed-loop user feedback systems and methods for ranking or updating beliefs for a user based on user feedback. The systems and methods are based on a data-agnostic user feedback formulation that uses user feedback to automatically rank beliefs for a user or update the beliefs. The methods and systems are based on a general statistical inference model, which, in turn, is based on an assumption of convergence in user opinion. The closed-loop user feedback methods and systems may be used to rank or update beliefs prior to inputting the beliefs to a recommender engine. As a result, the recommender engine is expected to be more responsive to customer environments and efficient at deployment and reducing the level of unnecessary user recommendations.

This disclosure presents systems and methods for run-time analysis of streams of log data for abnormalities using a statistical structure of meta-data associated with the log data. The systems and methods convert a log data stream into meta-data and perform statistical analysis in order to reveal a dominant statistical pattern within the meta-data. The meta-data is represented as a graph with nodes that represent each of the different event types, which are detected in the stream along with event sources associated with the events. The systems and methods use real-time analysis to compare a portion of a current log data stream collected in an operational window with historically collected meta-data represented by a graph in order to determine the degree of abnormality of the current log data stream collected in the operational window.

This paper introduces an innovative approach to regression analysis by incorporating the Dempster-Shafer theory to enhance the interpretability and accuracy of regression models. Our method, Embedded Interpretable Regression (EI Regression), segments continuous output variables into discrete interpretable classes and applies targeted regression models to these classes. We demonstrate the potential and effectiveness of our approach through initial experimental validation, showing that our model achieves competitive accuracy compared to traditional regression methods while significantly improving the model’s interpretability. This work contributes to interpretable machine learning and offers a practical framework for applying Dempster-Shafer’s theory in predictive modeling.

Identifying waste resources in modern-scale cloud infrastructures is a critical sustainability issue since it helps free up additional capacities for extra tasks and improves the performance of the systems while optimizing their costs. It is already well-recognized as a challenging task for human operators regarding manual and massive action efforts. At the same time, the problem is quite complicated – a complete and satisfactory solution is yet to be achieved. The paper proposes a novel and AI-driven approach to the problem. Applying rule induction learning across the history of service deployment instances to the log event data of the underlying entities, we extract conditions that lead to specific patterns, such as Resource Termination, thus providing a predictive mechanism for detecting objects subject to such actions in a real-time fashion. This explainable recommender system (called Cloud Sweeper) serves as an AI operations assistant for cloud users and Site Reliability Engineers (SRE) in their administrative duties.

Identification of a problem fingerprint or incident in a data center is of crucial importance for the system administrators. Automated discovery of such important patterns in cloud environments is recently gaining a lot of popularity for effective and efficient root cause analysis of business-critical is-sues. A problem incident is a group of alerts with sufficient historical evidence in reoccurrence and similarity. Presumably, all known incidents should be stored in user’s knowledge base together with available annotations regarding the problem description and its possible resolutions. The knowledge base of incidents with enough coverage of system’s possible failures is an invaluable asset for any system administrator. It will help to detect and isolate a problem, accelerate its remediation. In many cases it will also help to anticipate upcoming performance degradations before they impact the system. This classical approach totally relies on authentic alert definitions which mostly require expert knowledge regarding the environment peculiarities which makes impossible automation of the root cause analysis of unknown and very complex systems. In this paper, we consider essentially different approach that bypasses the alert definition and management process. Application of rule-learning algorithms to system indicators defines appropriate incidents in terms of rules with enough statistical evidence. We consider different possibilities both with labeled and unlabeled metric spaces. The labels can be derived from users’ feedbacks on system failures or performance degradations. In case of unlabeled metric space, outlier detection procedures can be applied for data labeling.

Cloud management technologies increasingly automate different aspects of data center administration, where the final goal is to make self-driving solutions. Learning fingerprints of KPI- or SLO-impacting performance problems in IT infrastructures is a relevant task towards such a vision. Instead of defining problem types for data center components (resources/objects of various kinds) using domain knowledge, which is hard to obtain and unreliable because of complexities and sophistication of modern cloud systems, we propose a ML framework to detect those issue categories. Then alerting engines can run on top of those patterns to notify the users on conditions that are impacting system’s KPIs thus providing explainability for troubleshooting and long-term performance optimization of the infrastructure. We consider several scenarios for learning problem definitions in terms of constructs by vRealize Operations – one of the leading solutions in the cloud management market. Using association rules mining concepts we can recommend problem patterns (fingerprints) in form of minimum size attribute combinations that constitute core structures highly associated with degradation of the KPI or SLO loss. We demonstrate experimental insights on virtualized environments applying our prototype algorithm.

Cloud management solutions provide full real-time visibility into mod- ern software-defined data centers (SDDC) of high complexity and sophistication through measuring millions of indicators with increasingly high sampling rate. This high frequency monitoring of metrics allows capturing the expected ever- growing dynamism of business-critical applications resulting in huge bases of time series data to be stored for analysis, pattern detection, and training predic- tive/forecasting models. That causes high analytics overhead and product perfor- mance issues. Therefore, identifying optimal sampling rates of time series data subject to preserving their main information content could mitigate this issue. A particular use case is tuning the sampling rates to be efficient for training ML models accurate enough in analytics tasks, such as anomaly detection. In this paper, we analyze a large collection of cloud application metrics and show that the sampling rate can be substantially reduced with a small information diver- gence. Moreover, we show that those anomaly detection modules perform suffi- ciently/tolerably accurate for the reduced data sets.

Leveraging cloud management products to effectively control performance of IT applications and infrastructures inevitably leads to the issue of automatically identifying baseline structures (typical behavioral patterns) of measured data sets including log sources. Those structures can be utilized for a variety of purposes from anomaly and change detection, to characterization of the application or infrastructure state in large (for instance, high/low stress levels, sickness, overprovisioning, security threats, etc.). Particularly, VMware vRealize Operations Manager performs such an analysis for any time series metric of an IT object through its basic dynamic thresholding analytics, while building a similar capability for log analytics is challenging - the very high volume of log data makes machine learning extremely expensive. To overcome the learning complexity, we propose random sampling techniques. Our method allows for controlling the confidence of the learned model by tuning the sampling rate. In this paper, we focus on learning the baseline model of log sources in terms of the distribution of log event types generated by vRealize Log Insight. Moreover, our algorithms identify the expected normal discrepancy from such a baseline that the log source exhibits. We demonstrate the proposed approach by applying our prototype algorithms to different data sets.

Reliable management of modern cloud computing infrastructures is unrealizable without monitoring and analysis of a huge number of system indicators (metrics) as time series data stored in big databases. Efficient storage and processing of collected historical data from all "objects" of those infrastructures are technology challenges for this Big Data application. We propose a data compression framework for databases of time series that applies correlation content of the data set. Specifically, the fundamental statistical concepts of independent component analysis (ICA) and principal component analysis (PCA) are employed to demonstrate the viability of the approach. We experimentally show significant compression rates for real data sets from IT systems.

Today's IT management faces the problem of “virtualized big environments” with hundreds of thousands of objects/resources as virtual machines, hosts, clusters, etc., evolving into cloud services. Admins of those infrastructures heavily rely on smart data-agnostic approaches to get reliable and accurate information regarding any current or upcoming health deterioration, increasingly requesting more proactive solutions. We architected a multi-layer enterprise analytics that employs statistical and machine learning methods to maximally automate the data center operations for an optimal performance management. We share several experience stories on application of the developed approaches and address noise and complexity reduction requirements to increase the operational efficiency of the analytics.

We demonstrate an enterprise Dynamic Thresholding System for data-agnostic management of monitoring flows. The dynamic thresholding based on data historical behavior enables adaptive and more accurate control of business environments compared to static thresholding. We manifest the main blocks of a complex analytical engine that is implemented in VMware vCenter Operations Manager as a principal foundation of the company’s data-driven anomaly detection.

Data-agnostic management of today's virtualized and cloud IT infrastructures motivates statistical inference from unstructured or semi-structured data. We introduce a universal approach to the determination of statistically relevant patterns in unstructured data, and then showcase its application to log data of a Virtual Center (VMware's virtualization management software). The premise of this study is that the unstructured data can be converted into events, where an event is defined by time, source, and a series of attributes. Every event can have any number of attributes but all must have a time stamp and optionally a source of origination (be it a server, a location, a business process, etc.) The statistical relevance of the data can then be made clear via determining the joint and prior probabilities of events using a discrete probability computation. From this we construct a Directed Virtual Graph with nodes representing events and the branches representing the conditional probabilities between two events. Employing information-theoretic measures the graphs are reduced to a subset of relevant nodes and connections. Moreover, the information contained in the unstructured data set is extracted from these graphs by detecting particular patterns of interest.

We discuss the problem of recovering a function by its discrete Fourier coefficients and consider acceleration of convergence of the classical trigonometric interpolation.

In this paper we continue investigations started in [6] and consider parametric interpolation of smooth but non periodic function defined on the finite interval. We analyze L2−convergence of such interpolations and obtain exact formulae for the principal term of L2−error. An optimization problem is solved. Numerical results are presented.