.png)
dbt Cloud Integration Guide
Connect dbt Cloud jobs, run status, metadata, webhooks, and execution artifacts with enterprise applications through REST, GraphQL, and workflow-based integrations.
dbt Cloud integration options at a glance
dbt Cloud provides REST APIs for accounts, projects, environments, jobs, runs, and artifacts, while its Discovery API offers GraphQL access to selected metadata and lineage-oriented information. Selected job and run lifecycle events can be delivered through webhook-style notifications. Job execution is asynchronous: an integration starts a job, captures its run identifier, and monitors the result through polling or supported notifications. dbt Cloud also produces artifacts such as manifest.json and run_results.json. Martini can consume these APIs, expose endpoints for webhooks, schedule reconciliation workflows, transform metadata, and route successful or failed runs to downstream systems. Warehouse data should normally be accessed through the warehouse directly.
| Integration point | Supported by dbt Cloud? | Common use cases | How Martini supports it |
|---|---|---|---|
| REST APIs | Yes | Manage or retrieve accounts, projects, environments, jobs, runs, and related resources; trigger jobs and retrieve run details or artifacts. | Martini can consume dbt Cloud REST APIs from workflows, map responses, expose normalized APIs, and orchestrate follow-up actions. |
| GraphQL APIs | Limited | The Discovery API provides metadata, lineage-oriented information, models, sources, exposures, metrics, and semantic information where available. | Martini can consume the Discovery GraphQL API for targeted metadata queries and transform the response into internal or downstream models. |
| Webhooks / outbound callbacks | Limited | Selected job and run lifecycle events can notify external endpoints; webhook coverage does not extend to every dbt Cloud object or change. | Martini can expose a REST endpoint, validate and normalize notifications, deduplicate deliveries, and retrieve authoritative run state when necessary. |
| Asynchronous job execution | Limited | A job execution request creates a run that must be monitored separately through polling or supported webhook events. | Martini can capture the run ID, schedule bounded status checks, process terminal states, and route success or failure outcomes. |
| File / artifact retrieval | Limited | Completed executions can produce manifest.json, run_results.json, catalog.json, and documentation-related artifacts for metadata or audit synchronization. | Martini can retrieve available artifacts over HTTP, parse JSON, map selected fields, and write them to catalogs, dashboards, or audit services. |
| Authentication | Yes | API tokens, service tokens, and applicable OAuth patterns authenticate requests subject to account, project, and resource permissions. | Martini can store credentials as environment-managed secrets and apply the authorization configuration required by the selected dbt Cloud API. |
| Database / analytics access | Not confirmed | dbt Cloud runs transformations against external warehouses but is not a general-purpose warehouse query layer. | Martini can connect directly to a relevant warehouse when row-level or analytical data is required, rather than routing those queries through dbt Cloud. |
How dbt Cloud exposes data and business events
dbt Cloud REST APIs
dbt Cloud REST APIs cover administration and execution resources such as accounts, projects, environments, jobs, runs, and artifacts. They support both operational lookup and job execution use cases, subject to API version, regional host, account configuration, and permissions.
Martini implementation pattern
Martini implementation pattern: a workflow authenticates with an environment-managed token, calls the appropriate dbt Cloud endpoint, validates the response, maps the vendor object into an internal model, and applies retry, pagination, and error rules before continuing.
Implementation sequence
dbt Cloud Discovery GraphQL API
The Discovery API provides GraphQL access to selected dbt project and warehouse-backed metadata, including models, sources, exposures, metrics, semantic information, and lineage-oriented data where available. It is a metadata interface rather than a complete replacement for REST administration APIs.
Martini implementation pattern
Martini implementation pattern: a workflow sends a narrowly scoped GraphQL query, validates the returned data and errors, maps metadata into a catalog or governance model, and accounts for plan-specific schema and access differences.
Implementation sequence
dbt Cloud Webhooks
dbt Cloud supports webhook-style notifications for selected job and run lifecycle events. Coverage is not universal, so notifications should be combined with REST reconciliation when an event is missing, incomplete, duplicated, or out of order.
Martini implementation pattern
Martini implementation pattern: Martini exposes a REST endpoint, validates the incoming request according to the configured security requirements, normalizes the event, deduplicates it using account, job, run, and event identifiers, and retrieves authoritative run state when required.
Implementation sequence
Asynchronous Job Execution
Starting a dbt Cloud job creates an asynchronous run rather than immediately returning transformation results. The resulting run identifier is used to monitor progress and retrieve details or artifacts after completion.
Martini implementation pattern
Martini implementation pattern: a Martini API or workflow validates the requested job, submits the execution request, stores the returned run ID, and uses polling or supported notifications to process terminal states with bounded retries and timeouts.
Implementation sequence
dbt Cloud Artifacts
dbt Cloud executions can produce artifacts such as manifest.json, run_results.json, catalog.json, and documentation-related output. These files support metadata, lineage, data-quality, and deployment-audit integrations but are not a general-purpose attachment API.
Martini implementation pattern
Martini implementation pattern: after a successful or otherwise relevant run, Martini retrieves the applicable artifact, parses its JSON structure, selects stable fields, transforms them into the destination model, and records the source run for traceability.
Implementation sequence
Common dbt Cloud integration patterns
Pattern 1: Trigger dbt Cloud jobs from an operational system
When to use this pattern
Use this pattern when an application, deployment process, ingestion platform, or data operations tool needs to start a permitted dbt Cloud job and receive a durable reference to the resulting execution.
Integration direction
Example Mapping
| dbt Cloud Field | Canonical Field | Target Field |
|---|---|---|
| applicationProcessId | processReference | job selection context |
| requestedEnvironment | executionEnvironment | dbt Cloud environment |
| jobId | dbtJobId | dbt Cloud Job ID |
| runId | executionReference | dbt Cloud Run ID |
Martini implementation pattern
A Martini API receives the request, validates the caller and allowed job mapping, selects the dbt Cloud job, submits the asynchronous execution request, and returns or persists the run ID. Business rules restrict permitted jobs and parameters, while error handling covers authentication failures, invalid job selection, transient responses, and duplicate requests.
Martini capabilities used
- APIs
- workflows
- API consumption
- data mapping
- business rules
- error handling
Pattern 2: Synchronize dbt Cloud run status and failures
When to use this pattern
Use this pattern when operations, incident management, or audit systems need a normalized view of dbt Cloud executions, including successful, failed, cancelled, and skipped terminal states.
Integration direction
Example Mapping
| dbt Cloud Field | Canonical Field | Target Field |
|---|---|---|
| id | executionReference | run_id |
| status | executionStatus | status |
| job_definition_id | jobReference | job_id |
| finished_at | completionTime | completed_at |
Martini implementation pattern
A scheduled Martini workflow retrieves recent runs, follows pagination, compares them with stored checkpoints, and maps status and failure details into an operational model. It uses the run ID and terminal state for idempotency, applies bounded retries for transient failures, and creates notifications or incidents only when the state has not already been processed.
Martini capabilities used
- scheduled workflows
- REST API consumption
- pagination handling
- data mapping
- idempotency rules
- error handling
Pattern 3: Receive dbt Cloud run notifications
When to use this pattern
Use this pattern when supported job or run lifecycle events should initiate near-real-time downstream processing without relying solely on frequent polling.
Integration direction
Example Mapping
| dbt Cloud Field | Canonical Field | Target Field |
|---|---|---|
| eventType | executionEventType | event_type |
| runId | executionReference | dbt_run_id |
| jobId | jobReference | dbt_job_id |
| status | executionStatus | incident_state |
Martini implementation pattern
Martini exposes a REST endpoint to receive supported webhook events, validates and normalizes the notification, and performs a follow-up REST lookup when the event is incomplete. A durable idempotency key prevents duplicate incidents, while reconciliation covers missed or out-of-order notifications.
Martini capabilities used
- REST APIs
- webhook consumption
- workflows
- data mapping
- business rules
- duplicate handling
Pattern 4: Synchronize dbt metadata and artifacts
When to use this pattern
Use this pattern when a data catalog, governance service, quality dashboard, or audit platform needs dbt project metadata, lineage-oriented information, model details, or execution artifacts.
Integration direction
Example Mapping
| dbt Cloud Field | Canonical Field | Target Field |
|---|---|---|
| unique_id | modelIdentifier | asset_id |
| name | modelName | asset_name |
| depends_on | dependencyReferences | upstream_assets |
| status | testOrRunStatus | quality_status |
Martini implementation pattern
A scheduled or event-triggered workflow queries the Discovery API or retrieves REST artifacts after a relevant run. Martini parses JSON or GraphQL responses, maps only required fields, tolerates unknown fields, versions mappings when schemas change, and writes the resulting model or lineage data with the source account, project, and run identifiers.
Martini capabilities used
- scheduled workflows
- GraphQL API consumption
- REST API consumption
- JSON handling
- data mapping
- schema-aware transformations
Applications commonly integrated with dbt Cloud
dbt Cloud is commonly positioned alongside data warehouses, ingestion platforms, orchestration tools, and analytics applications. Martini can coordinate these systems with dbt Cloud by invoking jobs, receiving supported run events, reconciling execution state, and transforming metadata or artifacts.
| Application | Scenario | Direction | Martini Pattern |
|---|---|---|---|
| Snowflake | Coordinate dbt transformations with Snowflake data operations and publish run, model, or governance metadata alongside warehouse workflows. | Snowflake → Martini → dbt Cloud | Use a Martini workflow or API to validate a warehouse-related request, invoke the selected dbt Cloud job, store the returned run ID, and reconcile completion before continuing downstream processing. |
| Google BigQuery | Coordinate dbt transformations running against BigQuery and synchronize execution status or model metadata with BigQuery-oriented operations. | Google BigQuery → Martini → dbt Cloud | Trigger dbt Cloud through its REST API, monitor the asynchronous run with bounded polling or supported webhooks, and map terminal states into the surrounding BigQuery workflow. |
| Databricks | Orchestrate dbt Cloud jobs with lakehouse workflows and exchange execution status between dbt Cloud and Databricks operations. | Databricks → Martini → dbt Cloud | Expose a Martini API for orchestration requests, invoke the appropriate dbt Cloud job, and route successful or failed run outcomes back to Databricks-related processes. |
| Amazon Redshift | Start dbt transformations executed against Redshift and route completion or failure information into Redshift-oriented data operations. | Amazon Redshift → Martini → dbt Cloud | Use REST API consumption to submit and monitor dbt Cloud jobs, then transform run details into an operational status model or downstream warehouse audit record. |
| Fivetran | Coordinate ingestion completion with downstream dbt transformations after source data has landed. | Fivetran → Martini → dbt Cloud | Receive or retrieve the relevant ingestion outcome, apply a job-selection rule, invoke dbt Cloud, and persist the run ID so duplicate triggers do not start unintended executions. |
| Apache Airflow | Include dbt Cloud execution in broader orchestration and synchronize job state between orchestration environments. | Apache Airflow → Martini → dbt Cloud | Use Martini as an API-mediated orchestration layer that accepts a request from Airflow, starts or checks a dbt Cloud run, and returns normalized status with timeout and retry handling. |
| Tableau | Coordinate analytics operations after dbt Cloud transformations complete and use normalized run status to control downstream processing. | dbt Cloud → Martini → Tableau | Consume supported dbt Cloud run notifications or reconcile the run through REST, apply terminal-state rules, and invoke the required Tableau operation through its separately configured API. |
| Looker | Synchronize dbt Cloud deployment or run completion with Looker data-model and analytics workflows. | dbt Cloud → Martini → Looker | Retrieve authoritative run details, map relevant project or model metadata, and call the required Looker endpoint only after business rules confirm that the dbt Cloud run is complete. |
How to build a dbt Cloud integration in Martini
Objective
Establish the dbt Cloud API connection with the correct regional host, API version, account scope, and least-privilege identity.
Instructions in Martini
- Store API tokens, service tokens, or applicable OAuth configuration in Martini environment-managed secrets.
- Confirm account, project, job, environment, and Discovery API permissions before implementation.
- Configure the authorization header and endpoint settings for the selected dbt Cloud API.
Objective
Select the event, API request, or schedule that should initiate the integration workflow.
Instructions in Martini
- Use a Martini API for operational requests to start or inspect jobs.
- Receive supported dbt Cloud webhook events through an exposed Martini endpoint.
- Use a scheduler for reconciliation, pagination, artifact retrieval, or event-coverage gaps.
Objective
Call dbt Cloud and obtain the current resource, run, metadata response, or artifact required by the process.
Instructions in Martini
- Submit REST or Discovery GraphQL requests through a Martini workflow.
- Capture run identifiers returned by asynchronous job execution.
- Retrieve authoritative run state when webhook payloads are incomplete or out of order.
- Continue through paginated list responses and persist checkpoints where practical.
Objective
Coordinate API calls, asynchronous execution, notifications, and downstream operations as one maintainable integration flow.
Instructions in Martini
- Separate job submission from run completion and downstream processing.
- Route success, failure, cancellation, and skipped states to explicit branches.
- Use bounded polling, timeouts, retry-after guidance, and exponential backoff for transient conditions.
Objective
Convert dbt Cloud objects, GraphQL metadata, and JSON artifacts into stable internal or downstream models.
Instructions in Martini
- Map account, project, job, run, and artifact identifiers into canonical fields.
- Parse manifest.json, run_results.json, catalog.json, or relevant GraphQL responses where authorized.
- Tolerate unknown fields and version mappings when API or artifact structures change.
Objective
Control which jobs, environments, events, and terminal states are allowed to produce downstream effects.
Instructions in Martini
- Restrict job selection to approved job and environment mappings.
- Use account ID, job ID, run ID, and event type as idempotency context.
- Avoid creating duplicate alerts or records when scheduled reconciliation overlaps with webhook processing.
Common dbt Cloud data objects used in integrations
| Object | Typical Use | Common target systems | Martini handling |
|---|---|---|---|
| Accounts | Represent top-level dbt Cloud customer or organizational containers and provide the scope for API access and permissions. | Identity services, governance platforms, operational databases, data catalogs | Martini retrieves account identifiers and permissions context through REST workflows and uses them as part of scoped mappings and deduplication keys. |
| Projects | Represent dbt projects configured within an account and provide project-level context for jobs, environments, and metadata. | Data catalogs, governance platforms, deployment services, operational databases | Martini lists or retrieves projects, maps required identifiers and attributes, and stores checkpoints when synchronizing project inventories. |
| Environments | Define where and how dbt commands run within a project. | Release management tools, governance services, operational databases | Martini can retrieve environment configuration data and apply business rules before selecting an environment for an orchestration request. |
| Jobs | Define scheduled or manually invoked dbt execution behavior, including the job selected for orchestration. | Orchestration platforms, incident systems, operational databases | Martini maps application-level process names to dbt Cloud job IDs, invokes permitted jobs, and records the selected job context. |
| Runs | Represent individual job executions, including status, timing, command information, and terminal outcome. | Incident management, notification platforms, audit stores, dashboards | Martini captures run IDs, polls or reconciles status, deduplicates events, and routes success, error, cancelled, or skipped outcomes. |
| Artifacts | Provide manifests, run results, catalogs, and documentation-related output from dbt executions. | Data catalogs, lineage indexes, data-quality dashboards, audit services | Martini retrieves applicable artifacts, parses JSON structures, maps selected metadata, tolerates unknown fields, and versions mappings when formats change. |
Authentication and security considerations
Credential options
dbt Cloud supports API tokens, service tokens, and applicable OAuth patterns, with access constrained by account membership and permissions for projects, environments, jobs, runs, artifacts, and Discovery API resources.
Martini security pattern
Store tokens, OAuth secrets, regional hosts, and account configuration in Martini environment-managed secrets. Do not embed credentials in workflows, mappings, or payloads.
- Use a dedicated service identity for machine-to-machine workflows.
- Apply least-privilege access to the required dbt Cloud account and resources.
- Protect exposed Martini webhook and API endpoints with the applicable authentication and authorization controls.
- Record correlation identifiers without exposing credential values in logs.
Operational considerations for dbt Cloud integrations
Execution and delivery
Job submission is asynchronous. Store the returned run ID and distinguish submission from completion. Handle success, error, cancelled, and skipped states explicitly.
Reliability controls
- Handle HTTP 429 responses, transient 5xx errors, timeouts, and retry-after guidance with bounded exponential backoff.
- Use webhooks where the required event is supported and scheduled reconciliation for missed or uncovered events.
- Process paginated list responses and persist checkpoints for larger synchronizations.
- Use account ID, job ID, run ID, and event type to prevent duplicate processing.
- Map only required fields and tolerate unknown fields as API, GraphQL, or artifact schemas evolve.
Warehouse separation
dbt Cloud controls transformations and exposes execution metadata; it is not a general-purpose warehouse query proxy. Connect Martini directly to Snowflake, BigQuery, Databricks, Amazon Redshift, or another relevant warehouse when row-level data is required.
Why use Martini instead of scripts or point-to-point integrations?
Orchestrate more than one API call
Martini coordinates dbt Cloud job submission, asynchronous status monitoring, webhook handling, artifact retrieval, metadata transformation, and downstream actions in maintainable workflows.
Reduce point-to-point coupling
A Martini API can provide a controlled façade for internal applications while workflows centralize authentication, job-selection rules, mappings, retries, idempotency, and error handling.
Support operational reliability
- Use scheduled reconciliation alongside event-driven processing.
- Persist run identifiers and checkpoints for traceability.
- Normalize dbt Cloud responses and artifacts for multiple target systems.
- Reuse integration logic across warehouses, operations platforms, catalogs, and analytics applications.
Frequently asked questions
dbt Cloud can be integrated through its REST APIs for accounts, projects, environments, jobs, runs, and artifacts; its Discovery GraphQL API for selected metadata and lineage use cases; and webhook-style notifications for selected job and run lifecycle events. Job execution is asynchronous, so integrations should capture the run ID and monitor or reconcile the resulting run.
Yes. Martini can consume dbt Cloud REST APIs, query the Discovery GraphQL API where appropriate, expose an endpoint for supported dbt Cloud webhook events, and orchestrate job submission, status monitoring, metadata synchronization, and artifact processing.
No dedicated dbt Cloud connector is required. Martini can use dbt Cloud's native REST APIs, Discovery GraphQL API, supported webhook events, authentication methods, and artifact endpoints through workflows and exposed APIs.
Lonti does not charge an additional per-connector or per-vendor fee to integrate dbt Cloud. The integration is subject to the provisioned capacity of the Martini environment. Separate costs may apply from dbt, a connected warehouse, cloud infrastructure, or other third-party services based on subscription, usage, and deployment model.
Use the REST APIs for administration, job execution, runs, environments, projects, and artifacts. Use the Discovery GraphQL API when the requirement is metadata, discovery, or lineage-oriented querying. GraphQL is not a complete replacement for the REST APIs, and availability depends on plan, account configuration, and permissions.
dbt Cloud supports webhook-style notifications for selected job and run lifecycle events. Coverage is not universal for every object or platform change, so a resilient integration combines supported notifications with REST API reconciliation and handles duplicate or out-of-order delivery.
A Martini workflow can trigger a job, store the returned run ID, and monitor the run through polling or supported webhook events. Scheduled workflows can reconcile recent runs, process paginated results, retrieve artifacts, and use durable checkpoints and terminal-state keys to avoid duplicate downstream processing.
Yes. Martini can expose a controlled REST API that hides dbt Cloud endpoint details, validates job and environment selections, applies authorization and business rules, invokes dbt Cloud workflows, and returns normalized run or metadata responses to internal applications.
Related Martini documentation
Connect dbt Cloud with Martini
Use Martini to orchestrate dbt Cloud jobs, process supported run notifications, synchronize metadata and artifacts, and connect execution outcomes with the rest of your enterprise architecture.