Home > Blogs > VMware vFabric Blog


Spring Insight “split-agent” Architecture

Spring Insight is a technology that gives visibility into an application’s runtime performance and behavior. The original design called for the deployment of an agent application at each monitored server instance. Furthermore, the agent used only an in-memory mechanism to process the collected data that was assumed to originate from Java code. Recently, a “split-agent” mechanism has been implemented that allows the separation of the collection stage from the analysis one. In other words, the data is (still) collected by the same instrumentation plugins, but can be sent via various mechanisms to an analyzer application that performs the analysis and generates the same data as before – i.e., endpoints, metrics, external resources, etc. There are several advantages to such architecture – the most important ones being

  • Decreasing the impact on the monitored application by transferring the analysis functions to a separate process
  • Simplify installation and deployment of the Spring Insight framework by minimizing the number of required artifacts
  • Enabling other monitoring frameworks to use the Spring Insight analysis and display capabilities – including non-Java ones e.g., .NET, Ruby, PHP, Python, etc.

Technical background

Spring Insight uses AOP in order to instrument “APIs of interest” via a plugin(s) mechanism. Each such plugin basically consists of one or more aspects that collect the information from the API calls that interest the user and package them inside a standard “container”. This container (a.k.a. Frame, Operation, Trace) is then forwarded to an engine that performs analysis on it in order to extract execution flow “tags” (a.k.a. Endpoints) and metrics. The analysis can further be enhanced using the same plugins framework so that the user can analyze/extract the endpoints and/or generate metrics according to the specific monitored API characteristics.

Originally, the collected information was forwarded for analysis via a direct API call to the analysis engine – thus mandating its deployment to each and every server instance. For various reasons, the two functions (collection and analysis) have been separated and are using a dispatch transport in order to communicate between them. The analysis result is then forwarded using the “legacy” mechanism to the Insight dashboard for storage and display.

The dispatch transports

The collection agents and the analyzer use a dispatch transport to carry the information back and forth. The information is further encoded in order to match specific transport requirements – e.g., some transports may be able to carry only UTF-8 payloads. Currently there are 2 transports and 3 encodings built into the existing implementation5:

Transports

  • HTTP – uses HTTP POSTs to send the collected data to the analyzer. Every time such data is posted, the analyzer may optionally reply with a set of commands it needs to pass back to the agent. In other words, each HTTP session is actually an exchange of messages. This transport is suitable for “split” agent topology where several agents report to a single analyzer.
  • JMX – uses direct API calls to exchange messages. This is suitable for “integrated” topology where the collection and analysis should occur within the same application server – e.g., development “mode”.

Encoding(s)

  • As-is (a.k.a. “noop”) – used only for in-memory exchange of messages7
  • Java serialization
  • JSON

In addition to being responsible for encoding/decoding the messages via its configured encoder, each transport must provide the identity of the encoder it has used. This enables the analyzer to communicate with multiple agents via various transports and using various encoders. It is worth mentioning that both the transport and encoding are “pluggable” – i.e., one can easily add transports/ encodings and then select each via the available Spring Insight configuration mechanism.

Security

The exchanged messages can be secured either by using a secure transport (e.g., https) or by employing an encryption transformer on the message. Currently, there are several builtin encryption transformers that rely on the javax.crypto package.

Connecting non-Java agents

Because of the distinction between transport and encoding, it is possible to set up a system where nonJava agents collect data and send it to the Spring Insight analyzer – as long as the exchanged messages conform to the protocol. Specifically, one can already use the outofthebox HTTP + JSON combination to do so – in addition to supplying other transport/encoding mechanisms 8. The reason for this is that HTTP and JSON are readily available packages for most used targets besides Java – e.g., Ruby, Python, .NET. Example of such use is already under way for the .NET instrumentation being developed
for the Spring Insight project.

It is worth noting that the open nature of the “split-agent” architecture is not limited only to the collection of API execution traces. It can also be used to “inject” analysis results into the system – thus enabling the user to use the same environment as the monitored target (Ruby, Python, .NET) for analyzing the traces and not have to write Java code. Thus, one can ensure quick development as well as a quick learning curve and leverage of existing code in whatever language/architecture is more “natural” to the user.

About the Author:  Lyor Goldstein is a Java veteran with over 23 years of experience in software development. Lyor has been a member of the Spring Insight team since May 2010 when he joined VMware. Lyor obtained his M.Sc. (summa cum laude) in Computer Science from the Technion – Haifa Institute of Technology.

4 thoughts on “Spring Insight “split-agent” Architecture

  1. Pingback: Understanding the Difference between Spring Insight Developer & Insight Operations | VMware vFabric Blog - VMware Blogs

  2. Pingback: How We Improved APM’s Monitoring Capacity by 500% with Gemfire, RabbitMQ, vPostgres, and a Few Scalability Design Patterns | VMware vFabric Blog - VMware Blogs

  3. Pingback: VMware vFabric Blog: 3 Steps on Using Spring Insight Developer to Analyze Code | Strategic HR

  4. Pingback: 3 Steps on Using Spring Insight Developer to Analyze Code | VMware vFabric Blog - VMware Blogs

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>