Skip to main content

AWS CloudTrail

Overview

The AWS CloudTrail source supports both Full Refresh and Incremental syncs. You can choose if this connector will copy only the new or updated data, or all rows in the tables and columns you set up for replication, every time a sync is run.

This Source Connector is based on a Boto3 CloudTrail.

Output schema

This Source is capable of syncing the following core Streams:

Insight events are not supported right now. Only Management events are available.

Data type mapping

Integration TypeAirbyte TypeNotes
stringstring
numberinteger
arrayarray
objectobject

Features

FeatureSupported?(Yes/No)Notes
Full Refresh SyncYes
Incremental SyncYes
NamespacesNo

Performance considerations

The rate of lookup requests for events stream is limited to two per second, per account, per region. This connector gracefully retries when encountering a throttling error. However if the errors continue repeatedly after multiple retries (for example if you setup many instances of this connector using the same account and region), the connector sync will fail.

Getting started

Requirements

  • AWS Access key ID
  • AWS Secret access key
  • AWS region name

Setup guide

Please, follow this steps to get your AWS access key and secret.

Reference

Config fields reference

Field
Type
Property name
string
aws_key_id
string
aws_secret_key
string
aws_region_name
string
start_date

Changelog

Expand to review
VersionDatePull RequestSubject
0.1.112024-06-0639246[autopull] Upgrade base image to v1.2.2
0.1.102024-06-0338911Replace AirbyteLogger with logging.Logger
0.1.92024-06-0338911Replace AirbyteLogger with logging.Logger
0.1.82024-05-2038448[autopull] base image + poetry + up_to_date
0.1.72024-04-1537122Base image migration: remove Dockerfile and use the python-connector-base image
0.1.62024-04-1237122schema descriptions
0.1.52023-02-1523083Specified date formatting in specification
0.1.42022-04-1111763Upgrade to Python 3.9
0.1.32021-12-238434Update fields in source-connectors specifications
0.1.22021-08-045152Fix connector spec.json
0.1.12021-07-064539Add AIRBYTE_ENTRYPOINT for Kubernetes support
0.1.02021-06-234122Initial release supporting the LookupEvent API