Cloud Composer: A Managed Apache Airflow Service

01 February 2021

Cloud Composer is a fully managed workflow orchestration service which is built on Airflow. It is a hosted solution for Airflow, which is an open-source platform to programmatically author, schedule and monitor workflows.

Architecture

Cloud Composer distributes the environment’s resources between a Google-managed tenant project and a customer project, as shown in the following diagram:

Tenant project resources:

For unified Identity and Access Management access control and an additional layer of data security, Cloud Composer deploys Cloud SQL and App Engine in the tenant project.

Customer project resources:

Cloud Composer deploys Cloud Storage, Google Kubernetes Engine, Cloud Logging, and Cloud Monitoring in the customer project.

Benefits and drawbacks of Google Cloud Composer

• Setting up the environment is quick and simple.
• Easy to integrate any required Python libraries.
• Deployment is straightforward with drag-and-drop or programmatic features.
• Workflows are streamlined for users of all experience levels.
• Is built on an open source orchestration tool that allows for frequent updates and upgrades.

Pricing

It charges customers based on consumption, ensuring they pay for what they use. The pricing model is broken down as follows:

• Web core hours – $0.074 per vCPU hour
• Database core hours – $0.125 per vCPU hour
• Web and database storage – $0.273 per GB/month
• Network egress – $0.156 per GB

Refer doc for more information.

🔊 Listen