Updated: 12/02/2024
Editorial Note: this is a living blog featuring important service updates for VMware Cloud on AWS logs. Please check back frequently for the latest updates and additional answered FAQs. As this feature is in active development, all information in this document is subject to change.
Important Updates
In August 2024, VMware Cloud on AWS customers were notified that Aria Operations for Logs SaaS would be turned off on 6 December 2024. Over the past few months, multiple email reminders were sent to customers notifying them of the termination date.
Today, we are updating the announcement to clarify that active VMware Cloud on AWS (VMC) customers will have their time on Aria Operations for Logs SaaS extended and their access will not be terminated on 6 December.
As a result, no action is required from VMC customers before the Aria Operations for Logs SaaS 6 December 2024 deadline. All VMC customers will have their time extended to leverage Aria Operations for Logs SaaS. This extension only applies to Aria Operations for Logs SaaS. Further details about the service will be provided in future communication.
Original blog details below.
What is Changing?
Today, VMware Cloud on AWS (VMC) leverages VMware Aria Operations for Logs SaaS for customers to monitor logs and troubleshoot their SDDC deployments. These logs are accessible via the Aria Operations for Logs SaaS Console and can subsequently be forwarded to other logging solutions for further analysis and archiving.
With the End of Availability (EOA) of the Aria SaaS portfolio of products including VMware Aria Operations for Logs SaaS, the VMC team is ending the existing integration of Aria Operations for Logs SaaS with VMC. To enable continued customer access to SDDC logs, VMC will provide customers with access to their SDDC logs from Amazon Simple Storage Service (Amazon S3), and customers must provide their own logging tool for collecting and processing those logs.
What is the Go Forward Plan?
Going forward, VMware Cloud on AWS SDDC logs will be deposited into an Amazon S3 bucket dedicated to each customer’s VMC organization. SDDC logs will be delivered to the Amazon S3 bucket automatically, and read-only access to this S3 bucket will be granted to the native AWS account(s) linked to their VMC organization through the Account Linking (CloudFormation) process. With read-only access to their SDDC logs from the S3 bucket, customers can configure a logging tool/solution of their choice that can collect logs from Amazon S3. This tool must be acquired independently from VMware Cloud on AWS for customers to collect, process/store, and analyze SDDC logs.
Once this feature is available, SDDC logs will remain available in the S3 bucket for 48 hours before they expire. It is essential that customers retrieve their logs from the S3 bucket at least every 48 hours as the logs will be permanently removed beyond that time window and will not persist.
Frequently Asked Questions
1. What is changing with log consumption for VMware Cloud on AWS?
To ensure uninterrupted access to SDDC logs, the VMware Cloud on AWS team is ending the existing integration with Aria Operations for Logs SaaS and providing customers with read-only access to an Amazon S3 bucket for retrieving their SDDC logs. Customers will then need to provide their own solution to collect and process/store those logs.
2. How will customers access their SDDC logs without Aria Operations for Logs SaaS?
The VMware Cloud on AWS service will provide customers with access to SDDC logs via a read-only Amazon S3 bucket, unique for each customer organization and for each region they have an SDDC deployed within their organization. SDDCs deployed in the same region within an organization will share the same S3 bucket, and each SDDC will have its own directory within the regional S3 bucket. VMC will employ a standardized naming convention for buckets and directories to simplify management for customers, described later in this document.
3. Will VMware Cloud on AWS provide a replacement tool to store and analyze SDDC logs?
No, VMC will only provide access to SDDC logs via an Amazon S3 bucket. Customers must provide their own tool to subsequently collect and process/store those SDDC logs from the Amazon S3 bucket. Any logging tool capable of ingesting from Amazon S3 can be used.
4. How long will logs be retained in the Amazon S3 bucket provided by the VMware Cloud on AWS service?
The S3 bucket will include SDDC logs from the last 48 hours. After 48 hours, older logs will be removed. Customers will need to collect and process/store their SDDC logs from the Amazon S3 bucket by leveraging common commercially available solutions.
5. What specific logs will be delivered to the S3 bucket?
At feature launch, SDDC audit logs generated by ESXi, vCenter, and NSX will be available via the S3 bucket. VMC Activity and Notification logs will be available at a later date.
6. Will existing log data and/or configurations be migrated from Aria Operations for Logs SaaS?
No, the Amazon S3 bucket provided by VMC will include only the most recent SDDC logs from the last 48 hours. For questions related to migrating from Aria Operations for Logs SaaS, please contact your Broadcom team. VMC customers will need to provide their own solution to collect and process/store logs from the Amazon S3 bucket and are responsible for migrating any logs required from Aria Operations for Logs SaaS to that solution, if supported.
7. Will customers manage and control the Amazon S3 bucket for their SDDC logs?
No. Although the Amazon S3 bucket is dedicated per VMC organization, it is managed by VMC and customers are provided with read-only access. The S3 bucket is created in the VMC organization’s managed AWS account and subsequently shared with the customer’s externally linked AWS account(s) for accessing SDDC logs.
8. How will customers get and control user access to the Amazon S3 bucket?
Read-only access to the Amazon S3 bucket will be automatically granted to the native AWS account(s) linked to the customer’s VMC organization through the Account Linking (CloudFormation) process. Customers will be able to control access to the S3 bucket through their AWS IAM policies in their connected account(s) and must further grant access to authenticated IAM users in their connected account(s). Details for managing access for cross-account S3 buckets can be found in AWS public documentation.
9. Can customers extend the log retention time or request additional features?
No, logs will be retained in the S3 bucket for 48 hours to provide VMC customers with enough time to collect and store them through their own logging solution. After 48 hours, older logs will be removed and replaced with the most recent logs. We recommend customers use commercially available solutions to collect, process, and store their SDDC logs from the Amazon S3 bucket. The S3 bucket is provided as is, and VMC will not offer different storage tiers or replication. It is intended only for time-limited access to logs so that customers can collect for consumption via their own tooling.
10. When are log files created and at what intervals?
Log files are created in real-time once they are generated by the SDDC management appliances. As soon as logs are received by the VMC service, they are compressed and pushed to the S3 bucket. For small to medium-sized deployments, logs are pushed as often as every few seconds. The delay from log file generation by the SDDC management appliances and their subsequent push to S3 should be no longer than a few minutes.
11. What is the naming convention and format of the S3 buckets and the log files?
The Amazon S3 bucket will be regional and will follow the naming convention vmw.logs.<<org_id>>.<<region>>. Logs will be provided in JSON format as compressed files. The logs will be contained in SDDC-specific folders, under the directory logs/<<sddc_id>>/<<current_date>>/<<uuid.uuid4()>>_log.ndjson.gz. The <<uuid.uuid4()>> is a random UUID (example 4f54c40d-0699-45d2-9049-2083ae884dae).
The format for <<current_date>> is “yyy/mn/dd” and SDDC logs for the day will fall under this directory. Logs will be pushed to their directory whenever they are generated by the SDDC components for that day. The <<uuid.uuid4()>>_log.ndjson.gz will include logs for every batch of logs sent at a time for that day. Whenever logs are pushed from the SDDC components, a new <<uuid.uuid4()>>_log.ndjson.gz will be created for that batch. This information is subject to change, and any updates will be published prior to the release of the S3 bucket solution.
12. What permissions are granted for the S3 bucket to the linked AWS account?
Permissions granted to the customer’s externally linked AWS account will follow this policy structure:
{
“Effect”: “Allow”,
“Principal”: {
“AWS”: “arn:aws:iam::<linkedacc>:root”
},
“Action”: [
“s3:List*”,
“s3:Get*”
],
“Resource”: [
“arn:aws:s3:::<shadow acc bucket>”,
“arn:aws:s3:::<shadow acc bucket>/*”
]
}
13. What if customers have additional questions?
For additional questions related to VMware Cloud on AWS and the change to SDDC logs, please reach out directly to your Broadcom team. For questions specific to Aria Operations for Logs SaaS, please reach out to [email protected].
