This blog post is co-authored by Nico Guerrera, Sr. Technical Account Manager and Chris McClanahan, Sr. Technical Product Line Marketing Manager.
We’ve previously detailed how to forward events from vRealize Log Insight and Logstash to our new Log Intelligence SaaS log aggregator. Today, we are going to take a sneak peak at an upcoming version of Log Intelligence, and how you can use its built-in event-forwarding capability to forward to Splunk and other SIEM services.
In the new version of Log Intelligence, our event forwarder appliance will act as a proxy, forwarding events to both Log Intelligence and any specified 3rd party endpoint. This will allow customers who use products like Splunk for SIEM to continue using existing security reporting capabilities while leveraging Log Intelligence for log aggregation and root cause analysis, by only forwarding necessary security events.
Because the remote data collector in your datacenter will act as the proxy, pulling log messages from Log Intelligence and forwarding to the third-party system (i.e. Splunk), this design doesn’t require anything in the DMZ.
Log Forwarding from Log Intelligence Data Flow
1. Incoming Events
In this exercise, it is assumed you already have logs and events flowing into Log Intelligence from either an on-premises data collector or from HTTP API ingestion via an API key. If you need help setting this up, please see our other blog posts on forwarding via the data collector or the API.
We executed a search in Log Intelligence for events containing our local vCenter administrator. In the screenshot below you can see logins and logouts, exactly what security administrators want to keep track of in SIEM.
2. Setting Up Forwarding in Log Intelligence
Log Intelligence now includes a new menu option under the ‘Manage’ dropdown in the Log Intelligence dashboard, called ‘Log Forwarding’.
Selecting this option will take you to the Log Forwarding dashboard, where you can see existing forwarding rules, and set up new rules by selecting ‘New Configuration’.
Log Intelligence will then ask you to provide the following basic information:
- Name: The display name of your rule.
- Data collector: The Log Intelligence data collector that events are coming from. This collector will also serve as the proxy, which will pull the logs back from Log Intelligence to forward to the on-premise Splunk.
- Endpoint Type: Splunk or Default. As we are forwarding to Splunk in this example, that’s what we have selected.
- Endpoint URL: The endpoint ingestion URL for your other logging product. Refer to your other logging product’s documentation to find this.
- Tags: Add any desired tags for events you’re forwarding here.
- Headers: For Splunk ingestion via HTTP we will need a special API key in the header of our events for basic authorization. We can generate this key in Splunk under ‘Settings -> Data Input -> HTTP Event Collector’. Without this key, events will be dropped once they hit the Splunk URL.
Once your configuration is set, you need to select the events to be forwarded. As we want to forward local Administrator vCenter events, we’ve just written a basic query looking for the vc_username tag ‘VSPHERE.LAB\\Administrator’. A second backslash was added between the default SSO domain and the username during transport to Log Intelligence, possibly as an escape, so this needs to be looked out for.
Now that we see events in our query test window we can save our configuration, and events should start forwarding to Splunk. If you see any dropped events, then there is an issue somewhere between your Log Intelligence data collector and Splunk that needs to be fixed.
3. Checking Splunk for our Forwarded Events
Now your events are forwarding, you can log into Splunk and run a search for your Administrator. If you run a basic search for your Administrator user, the logon and logoff events from Log Intelligence will display in your Splunk search window. With this information your security teams can create alerts and dashboards to monitor when the vSphere Administrator takes any action, like logging in to vCenter, modifying a virtual machine’s settings, or logging out.
Conclusion
You can now forward specific security-based events from Log Intelligence to a SIEM service such as Splunk. This method allows you to save on Splunk ingestion costs by only forwarding events necessary for security and event management, while keeping your VMware product logs in Log Intelligence. If you are collecting your private cloud’s underlying network, server and storage logs, this will also help you to conduct more effective root cause analyses between VMware products, across both the public cloud and on-premises environments.
Want to learn more? Visit the Log Intelligence website.
