Best practices for Cloud Audit Logs | Cloud Logging

This document recommends a sequence of audit logging tasks to help yourorganization maintain security and minimize risk.

This document isn't an exhaustive list of recommendations. Instead, its goal isto help you understand the scope of audit logging activities and planaccordingly.

Each section provides key actions and includes links for further reading.

Understand Cloud Audit Logs

Audit logs are available for most Google Cloud services.Cloud Audit Logs provides the following types of audit logs for eachGoogle Cloud project, folder, and organization:

Audit log type	Configurable	Chargeable
Admin Activity audit logs	No; always written	No
Data Access audit logs	Yes	Yes
Policy Denied audit logs	Yes; you can exclude these logs from being written to log buckets	Yes
System Event audit logs	No; always written	No

Data Access audit logs—except for BigQuery—are disabled bydefault. If you want Data Access audit logs to be written for Google Cloudservices, then you must explicitly enable them; for details, seeConfigure Data Access audit logs on thispage.

For information about the overall landscape for audit logging withGoogle Cloud, see Cloud Audit Logs overview.

Control access to logs

Due to the sensitivity of audit logging data, it is especially important toconfigure the appropriate access controls for your organization's users.

Depending on your compliance and usage requirements, set these access controlsas follows:

Set IAM permissions
Configure log views
Set log entry field-level access controls

Set IAM permissions

IAM permissions androles determine users' ability to access audit logsdata in the Logging API, theLogs Explorer, and theGoogle Cloud CLI. UseIAM to grant granular access to specificGoogle Cloud buckets and prevent unwanted access to other resources.

The permission-based roles that you grant to your users depend on theirauditing-related functions within your organization. For example, you mightgrant your CTO broad administrative permissions whereas your developer teammembers might require logs-viewing permissions. For guidance on which roles togrant to your organization's users, seeconfiguring roles for audit logging.

When setting IAM permissions, apply the security principle ofleast privilege, so you grant users only the necessary access to your resources:

Remove all nonessential users.
Grant essential users the correct and minimal permissions.

For instructions on setting IAM permissions, seeManage access to projects, folders, and organizations.

Configure log views

All logs, including audit logs, received by Logging are writteninto storage containers calledlog buckets.Log views let you control who hasaccess to the logs within your log buckets.

Because log buckets can contain logs from multiple Google Cloud projects,you might need to control which Google Cloud projects different users canview logs from. Create custom log views, which give you more granular accesscontrol for those buckets.

For instructions on creating and managing log views, seeConfigure log views on a log bucket.

Set log field-level access controls

Field-level access controls let you hide individual LogEntry fields from usersof a Google Cloud project, providing you a more granular way to control the logsdata a user can access. Compared to logs views, which hide theentire LogEntry, field-level access controls hide individual fields of theLogEntry. For example, you might want to redact external user PII, such as anemail address contained in the log entry payload, from the majority of yourorganization's users.

For instructions on configuring field-level access controls, seeConfigure field-level access.

Configure Data Access audit logs

When enabling new Google Cloud services, evaluate whether or not to enableData Access audit logs.

Data Access audit logs help Google Support troubleshoot issues with youraccount. Therefore, we recommend enabling Data Access audit logs when possible.

To enable all audit logs for all services, follow theinstructions to update the Identity and Access Management (IAM) policywith the configuration listed in theaudit policy.

After you define your organization-level data access policy and enable DataAccess audit logs, use a test Google Cloud project to validate theconfiguration of your audit logs collection before creating developer andproduction Google Cloud projects in the organization.

For instructions on enabling Data Access audit logs, seeEnable Data Access audit logs.

Control how your logs are stored

You can configure aspects of your organization's buckets and also createuser-defined buckets to centralize or subdivide your log storage. Depending onyour compliance and usage requirements, you might want to customize your logsstorage as follows:

Choose where your logs are stored.
Define the data retention period.
Protect your logs with customer-managed encryption keys (CMEK).

Choose where your logs are stored

In Logging buckets are regional resources: the infrastructurethat stores, indexes, and searches your logs is located in a specificgeographical location.

Your organization might be required to store its logs data in specific regions.The primary factors in selecting the region where your logs are stored includemeeting your organization's latency, availability, or compliance requirements.

To automatically apply a particular storage region to the new_Default and _Required buckets created in your organization, you canconfigure a default resource location.

For instructions on configuring default resource locations, seeConfigure default settings for organizations.

Define data retention periods

Cloud Logging retains logs according to retention rules applying to the logbucket type where the logs are held.

To meet your compliance needs, configure Cloud Logging to retain logs between1 day and3650 days. Custom retention rules apply to all thelogs in a bucket, regardless of the log type or whether that log has beencopied from another location.

For instructions on setting retention rules for a log bucket, seeConfigure custom retention.

Protect your audit logs with customer-managed encryption keys

By default, Cloud Logging encrypts customer content stored at rest. Yourorganization might have advanced encryption requirements that the defaultencryption at rest doesn't provide. To meet your organization's requirements,instead of Google managing the key encryption keys that protect your data,configure customer-managed encryption keys (CMEK) to control and manage your ownencryption.

For instructions on configuring CMEK, seeConfigure CMEK for logs storage.

Pricing

Cloud Logging doesn't charge to route logs to asupported destination; however, the destination might apply charges.With the exception of the _Required log bucket,Cloud Logging charges to stream logs into log buckets andfor storage longer than the default retention period of the log bucket.

Cloud Logging doesn't charge for copying logs,for defining log scopes,or for queries issued through theLogs Explorer or Log Analytics pages.

Query and view audit logs

If you need to troubleshoot, being able to quickly look at logs is arequirement. In the Google Cloud console, use the Logs Explorerto retrieve your audit log entries for your organization:

In the Google Cloud console, go to the Logs Explorer page:
Go to Logs Explorer
If you use the search bar to find this page, then select the result whose subheading isLogging.
Select your organization.
In the Query pane, do the following:
- In Resource type, select the Google Cloud resource whoseaudit logs you want to see.
- In Log name, select the audit log type that you want to see:
  - For Admin Activity audit logs, select activity.
  - For Data Access audit logs, select data_access.
  - For System Event audit logs, select system_event.
  - For Policy Denied audit logs, select policy.
  If you don't see these options, then there aren't any audit logs of thattype available in the organization.
- In the query editor, further specify the audit log entries that you wantto see. For examples of common queries, seeSample queries using the Logs Explorer.
Click Run query.

For more information about querying by using the Logs Explorer, seeBuild queries in the Logs Explorer.

Monitor your audit logs

You can use Cloud Monitoring to notify you when conditions you describeoccur. To provide Cloud Monitoring with data from your logs,Logging lets you create log-based alerting policies,which notify you anytime that a specific event appears in a log.

Configure alerting policies to distinguish between events that require immediateinvestigation versus low-priority events. For example, if you want to know whenan audit log records a particular data-access message, you can create alog-based alerting policy that matches the message and notifies you when themessage appears.

For instructions about configuring log-based alerting policies, seeManaging log-based alerting policies.

Route logs to supported destinations

Your organization may face requirements for creating and preserving auditlogs. Using sinks, you can route some orall of your logs to these supported destinations:

Cloud Storage
Pub/Sub,including third parties such asSplunk
BigQuery
Another Cloud Logging bucket

Determine whether you need folder-level or organization-level sinks, androute logs from all the Google Cloud projects inside the organization orfolder using aggregated sinks. Forexample, you might consider these routing use cases:

Organization-level sink: If your organization uses a SIEM to manage multiple audit logs, you might want to route all of your organization's audit logs. Thus, an organization-level sink makes sense.
Folder-level sink: Sometimes, you might want to only route departmental audit logs. For example, if you have a "Finance" folder and an "IT" folder, you might find value in only routing the audit logs belonging to the "Finance" folder, or the other way around.
For more information on folders and organizations, seeResource hierarchy.

Apply the same access policies to the Google Cloud destination that youuse to route logs as you applied to the Logs Explorer.

For instructions on creating and managing aggregated sinks, seeCollate and route organization-level logs to supported destinations.

Understand data format in sink destinations

When routing audit logs to destinations outside of Cloud Logging,understand the format of the data that has been sent.

For example, if routing logs to BigQuery, Cloud Loggingapplies rules to shorten BigQuery schema field names foraudit logs and for certainstructured payload fields.

To understand and find log entries that you routed fromCloud Logging to supported destinations, seeView logs in sink destinations.

Copy log entries

Depending on your organization's compliance needs, you might need to shareaudit log entries with auditors outside of Logging. If you needto share log entries that are already stored in Cloud Logging buckets, youcan manually copy them to Cloud Storage buckets.

When you copy log entries to Cloud Storage, the log entries also remainin the log bucket they were copied from.

Note that copy operations don't replacesinks, which automatically send all incoming log entries to a pre-selectedsupported storage destination, includingCloud Storage.

For instructions on routing logs to Cloud Storage retroactively,see Copy log entries.

Best practices for Cloud Audit Logs | Cloud Logging | Google Cloud (2024)