Ellipse Gradient for Header

IBM Cloud Object Storage Integration Guide

Integrate IBM Cloud Object Storage with enterprise systems through its S3-compatible HTTPS API, object events, IAM, HMAC authentication, and Martini workflows.

IBM Cloud Object Storage integration options at a glance

IBM Cloud Object Storage provides an S3-compatible HTTPS API for creating buckets, listing and transferring objects, managing metadata, copying and deleting files, handling versions, and performing multipart uploads. Selected bucket and object events can be published through configured notification destinations, although event coverage is not universal. IBM supports IBM Cloud IAM bearer tokens and HMAC credentials using Signature Version 4 conventions. Martini can consume these APIs through REST-oriented workflows, receive configured event notifications, process JSON, XML, CSV, Excel, and binary files, and orchestrate scheduled polling, transformation, validation, downstream API calls, checkpoints, retries, and audit handling.

Integration pointSupported by IBM Cloud Object Storage?Common use casesHow Martini supports it
S3-compatible REST APIsYesCreate and list buckets; list, upload, download, copy, delete, and inspect objects; manage versions, lifecycle rules, policies, and multipart uploads.Martini can consume the HTTPS API through REST-oriented workflows and APIs, map request and response data, and orchestrate multi-step object processing.
File and attachment APIsYesStore and retrieve JSON, XML, CSV, Excel, images, archives, and other binary content as objects. Object metadata can include content type, length, timestamps, and custom values.Martini can download, validate, transform, upload, copy, and route files while preserving relevant metadata and processing status.
Webhooks / outbound callbacksLimitedSelected bucket and object events can be published through configured notification destinations. Notifications are not universal callbacks for every storage operation.Martini can expose an API or webhook workflow to receive configured notifications, validate payloads, apply idempotency, and retrieve the referenced object.
Multipart uploadsYesUpload large objects in independently managed parts, complete or abort transfers, and support resumable large-file processing.Martini can orchestrate multipart request sequences, track upload identifiers and part numbers, and route failed or incomplete transfers for cleanup.
Bulk and multi-object operationsYesDelete multiple objects and process paginated object listings. These capabilities support controlled batch movement but are not a general asynchronous business-job API.Martini can paginate through listings, apply filters and checkpoints, and coordinate bounded batches with retry and reconciliation logic.
Incremental synchronizationYesDetect changes using event notifications, object modification timestamps, object versions, prefixes, and persisted processing checkpoints.Martini can implement scheduled polling or event-driven synchronization and store deterministic checkpoints based on bucket, key, version, timestamp, or event identifier.
IBM Cloud IAM authenticationYesAuthenticate with IAM-issued bearer tokens for IBM Cloud service APIs using API keys, service IDs, or other supported IAM identities.Martini can keep environment-specific credentials in secure configuration and attach bearer tokens to outbound requests, with refresh handling modeled in the workflow where required.
HMAC / Signature Version 4 authenticationYesAuthenticate S3-compatible object operations using access-key and secret-key credentials with AWS Signature Version 4 request signing.Martini can use configured HTTP authentication capabilities or custom request-signing logic when dynamic signing of the host, path, headers, query, and payload is required.

How IBM Cloud Object Storage exposes data and business events

IBM Cloud Object Storage REST APIs

IBM Cloud Object Storage exposes an S3-compatible HTTPS API for bucket, object, metadata, versioning, lifecycle, policy, and multipart operations. Requests use regional or cross-region endpoints and may require IAM bearer tokens or HMAC Signature Version 4 signing.

Martini implementation pattern

Martini implementation pattern: Martini workflows construct authenticated HTTP requests, manage endpoint and bucket configuration, handle pagination and continuation tokens, map responses into internal models, and route object content to downstream applications or storage locations.

Implementation sequence

Resolve the environment-specific COS endpoint and bucket
Load IAM or HMAC credentials from secure configuration
Construct and authenticate the S3-compatible request
Handle pagination, response metadata, and provider errors
Map the result into the canonical workflow model
Write the output or object and record processing state

IBM Cloud Object Storage Event Notifications

COS supports notifications for selected bucket and object events through supported destinations. Coverage is event-specific, and a notification may contain metadata or an object reference rather than the complete object body.

Martini implementation pattern

Martini implementation pattern: Martini exposes an API or webhook workflow for the configured notification route, validates the event, derives an idempotency key, retrieves the referenced object when needed, and sends the content through downstream processing.

Implementation sequence

Receive the configured COS event notification
Validate the event structure and destination security
Build an idempotency key from bucket, key, version, and event data
Retrieve the current object or referenced version
Transform and route the object to the target system
Persist success, duplicate, or failure status

IBM Cloud Object Storage Multipart Uploads

Multipart uploads divide large objects into independently uploaded parts and support completion or abortion of the staged upload. They are designed for large-object transfer rather than general asynchronous business processing.

Martini implementation pattern

Martini implementation pattern: Martini orchestrates initiation, part upload, completion, and cleanup as a workflow, using bounded retries and preserving the upload identifier so interrupted transfers can be reconciled or aborted.

Implementation sequence

Initiate the multipart upload
Split or receive the object into bounded parts
Upload each part with its part number and integrity information
Retry transient part failures with bounded backoff
Complete the multipart upload
Abort and record incomplete uploads when processing fails

Scheduled Object Synchronization

COS does not provide a standard modified-business-record synchronization API. Incremental processing is implemented with paginated listings, prefixes, modification timestamps, object versions, event notifications, and persisted checkpoints.

Martini implementation pattern

Martini implementation pattern: A scheduler-triggered workflow scans a stable prefix, follows continuation tokens, compares objects with a persisted checkpoint, processes eligible files, and advances the checkpoint only after downstream success.

Implementation sequence

Start the workflow on a configured schedule
List objects using a stable prefix and continuation token
Compare keys, timestamps, versions, and processing state
Retrieve and validate newly eligible objects
Commit downstream results and advance the checkpoint
Repeat or reconcile objects affected by ordering and clock precision

Common IBM Cloud Object Storage integration patterns

Pattern 1: Ingest and transform files

When to use this pattern

Use this pattern when supplier files, financial statements, claims documents, product catalogs, or operational exports arrive in an inbound bucket and must be validated and sent to another application or bucket.

Integration direction
IBM Cloud Object Storage
Martini
IBM Cloud DataStage
Example Mapping
IBM Cloud Object Storage FieldCanonical FieldTarget Field
object.keysourceFileNameinputFileName
object.contentTypecontentTypefileFormat
object.lastModifiedsourceModifiedAtsourceTimestamp
object.bodyvalidatedPayloadpipelineInput
Martini implementation pattern

A scheduled or event-triggered Martini workflow lists or receives the object, checks its key, metadata, size, and content structure, transforms CSV, JSON, XML, Excel, or binary content as required, and sends the result onward. It records a deterministic processing key and routes validation or transport failures to controlled retry handling.

Martini capabilities used
  • workflows
  • API consumption
  • scheduling
  • data mapping
  • file processing
  • business rules
  • error handling

Pattern 2: Process object-created events

When to use this pattern

Use this pattern when selected COS object events should initiate near-real-time processing without repeatedly scanning an entire bucket.

Integration direction
IBM Cloud Object Storage
Martini
IBM watsonx
Example Mapping
IBM Cloud Object Storage FieldCanonical FieldTarget Field
bucket.namesourceBucketstorageLocation
object.keysourceObjectKeyinputReference
object.versionIdsourceVersionpayloadVersion
event.ideventIdcorrelationId
Martini implementation pattern

COS sends a configured notification to a Martini-exposed API. The workflow validates the notification, rejects or records duplicates, retrieves the referenced object or version, applies content and security validation, and invokes the downstream API. If the event lacks the complete body, Martini performs a separate authenticated object retrieval.

Martini capabilities used
  • APIs
  • webhook consumption
  • workflows
  • idempotency rules
  • data validation
  • API consumption
  • error handling

Pattern 3: Archive and retain objects

When to use this pattern

Use this pattern when operational objects should be copied to an archive bucket or storage location after age, prefix, metadata, or processing-status rules are met.

Integration direction
IBM Cloud Object Storage
Martini
IBM Cloud Object Storage
Example Mapping
IBM Cloud Object Storage FieldCanonical FieldTarget Field
object.keyarchiveObjectKeyarchiveKey
object.lastModifiedobjectAgeReferenceretentionDecisionDate
object.versionIdsourceVersionarchivedVersion
object.metadataretentionMetadataarchiveMetadata
Martini implementation pattern

A scheduled Martini workflow lists eligible objects, evaluates age and retention rules, copies each object, verifies the result, and only then marks or deletes the source when policy permits. Versioning, legal holds, retention restrictions, and delete-marker behavior are treated as explicit business rules rather than assumed defaults.

Martini capabilities used
  • scheduling
  • workflows
  • API consumption
  • business rules
  • data validation
  • audit handling
  • error handling

Pattern 4: Exchange data with enterprise applications

When to use this pattern

Use this pattern when an application exports data to COS and Martini must parse the file, map it into target API requests, and publish results or rejected records back to storage.

Integration direction
IBM Db2
Martini
IBM Cloud Object Storage
Example Mapping
IBM Cloud Object Storage FieldCanonical FieldTarget Field
customer_idcustomerIdentifiercustomerId
transaction_datetransactionDateeffectiveDate
amounttransactionAmountamount
processing_statusresultStatusstatus
Martini implementation pattern

Martini detects or polls for a completed object, validates the file format and required fields, maps rows into target API payloads, applies duplicate and business validation rules, and writes a result file or status object. Failed rows or files are separated for controlled reprocessing while the original source remains unchanged.

Martini capabilities used
  • scheduled workflows
  • file processing
  • mapping
  • JSON handling
  • XML handling
  • business rules
  • retries

Applications commonly integrated with IBM Cloud Object Storage

IBM Cloud Object Storage can serve as a durable landing zone, exchange layer, or source for data and analytics workflows. The following named products represent practical integration targets based on the supplied IBM architecture and common enterprise deployment patterns.

Application Scenario Direction Martini Pattern
IBM watsonx Store documents, datasets, model inputs, and generated artifacts for AI and analytics workflows. IBM Cloud Object Storage → Martini → IBM watsonx Martini can detect or poll for new objects, validate and transform their contents, then invoke the relevant watsonx APIs or write prepared artifacts to the required bucket. Processing state and failures can be recorded for replay.
IBM Cloud DataStage Use IBM Cloud Object Storage as a landing zone and source or target for batch data pipelines. IBM Cloud Object Storage → Martini → IBM Cloud DataStage A scheduled Martini workflow can paginate through bucket listings, apply prefix and timestamp filters, validate files, and route them into DataStage-oriented ingestion flows while retaining checkpoints and error details.
IBM Db2 Exchange extracts, staged files, backups, or analytical data between Db2 workloads and object storage. IBM Db2 → Martini → IBM Cloud Object Storage Martini can receive or schedule Db2 extracts, serialize them into agreed file formats, upload them to COS, and separately process COS files into Db2 using database workflows where the Db2 deployment is accessible.
IBM Cloud Pak for Data Provide object storage for data assets, analytics inputs, and platform-managed datasets. IBM Cloud Object Storage → Martini → IBM Cloud Pak for Data Martini can coordinate object arrival, validate metadata and payload structure, and invoke Cloud Pak for Data endpoints or prepare objects for platform consumption. Environment-specific endpoints and credentials remain configurable.
Apache Spark Read large datasets from COS for distributed processing and write transformed datasets back to object storage. IBM Cloud Object Storage → Apache Spark Martini can manage pre-processing, manifest creation, validation, and post-processing around Spark jobs or datasets. It can also move completed objects and publish processing status through an API or output bucket.
Snowflake Load files from COS into Snowflake stages or unload analytical results to COS. IBM Cloud Object Storage → Martini → Snowflake Martini can detect completed files, validate and classify them, then coordinate Snowflake ingestion or export workflows through the available application APIs and object operations. It can maintain a deterministic file-processing checkpoint.
Tableau Make datasets prepared in COS available to reporting workflows through an intermediate analytics or database layer. IBM Cloud Object Storage → Martini → Tableau Martini can transform source objects into curated outputs, publish them to an analytics or database layer, and call Tableau-related APIs where required for refresh or publication. COS is treated as the file source rather than a direct reporting engine.
IBM Event Streams Route selected COS object-event notifications into Kafka-compatible event processing. IBM Cloud Object Storage → IBM Event Streams → Martini Where the configured COS notification destination supports the event route, Martini can consume the resulting notification, validate event identity, retrieve the referenced object when necessary, and coordinate downstream processing or reconciliation.

How to build a IBM Cloud Object Storage integration in Martini

Objective

Configure the COS endpoint, bucket, region, and environment-specific credentials without embedding secrets in workflow definitions.

Instructions in Martini

  • Select IAM bearer-token or HMAC Signature Version 4 authentication for the required operations
  • Store API keys, access keys, secret keys, and tokens in Martini secrets or secure environment configuration
  • Keep development, test, and production endpoints and buckets separate
  • Apply least-privilege IAM and bucket permissions

Objective

Select event-driven, scheduled, or API-led execution according to the reliability and latency requirements of the object flow.

Instructions in Martini

  • Receive selected COS notifications through a Martini-exposed API when event coverage is sufficient
  • Use a scheduler for reconciliation and incremental polling
  • Expose a Martini API when another system should request object retrieval or processing
  • Define a stable prefix and checkpoint strategy for scans

Objective

List, page through, download, or upload COS objects while preserving the identifiers and metadata needed for processing.

Instructions in Martini

  • Follow continuation tokens until the relevant listing is complete
  • Treat object keys as opaque identifiers and encode request paths correctly
  • Retrieve a specific object version when processing must be tied to an immutable payload
  • Use multipart operations for large transfers where required

Objective

Coordinate validation, transformation, downstream calls, object writes, and status recording as one maintainable integration flow.

Instructions in Martini

  • Create a workflow that separates event intake, object retrieval, transformation, and delivery
  • Use correlation identifiers based on bucket, key, version, and event data
  • Keep source objects unchanged until downstream processing is confirmed
  • Use reusable services or workflows for common object and error operations

Objective

Convert file content and COS metadata into the canonical model expected by downstream systems.

Instructions in Martini

  • Validate content type, size, file structure, required fields, and untrusted input
  • Map CSV, JSON, XML, Excel, and binary payloads to target models
  • Preserve relevant content type, encoding, checksum-related values, and custom metadata
  • Apply schema-version and prefix rules when producers change payloads

Objective

Use object age, prefix, metadata, version, processing state, and retention requirements to determine the correct action.

Instructions in Martini

  • Filter objects by stable prefixes, timestamps, suffixes, or metadata
  • Make duplicate processing decisions using deterministic idempotency keys
  • Distinguish current versions, prior versions, delete markers, and permanently deleted objects
  • Require successful copy or downstream delivery before deletion or archival marking

Common IBM Cloud Object Storage data objects used in integrations

ObjectTypical UseCommon target systemsMartini handling
BucketContainer associated with a service instance and storage location for objects, policies, lifecycle rules, and notifications.IBM Cloud Object Storage, IBM watsonx, IBM Cloud DataStage, SnowflakeMartini stores configurable bucket and endpoint values, lists or validates buckets, and applies environment-specific routing and access rules.
ObjectFile or binary payload stored under an object key, including data exchange files, documents, images, archives, and analytics inputs.IBM Cloud Object Storage, IBM Db2, IBM Cloud DataStage, Snowflake, Apache SparkMartini retrieves metadata and content, validates and transforms the payload, then uploads, copies, routes, or archives the resulting object.
Object keyOpaque name or path-like identifier used to address an object within a bucket.IBM Cloud Object Storage, downstream file-processing workflowsMartini treats keys as identifiers, applies prefix and suffix filters where appropriate, and URL-encodes path segments correctly.
Object versionHistorical version of an object when bucket versioning is enabled, supporting precise processing and retention decisions.IBM Cloud Object Storage, audit stores, archival workflowsMartini can include version identifiers in idempotency keys, checkpoints, retrieval requests, and reconciliation records.
Multipart uploadStaged large-object transfer composed of independently uploaded parts before completion or abortion.IBM Cloud Object Storage, file exchange workflows, analytics pipelinesMartini coordinates upload IDs and part numbers, handles retries and cleanup, and verifies the completed transfer where applicable.
Bucket lifecycle configurationRules that transition or expire objects according to age, storage class, or other conditions.IBM Cloud Object Storage, archival and retention processesMartini can read or update lifecycle-related configuration through the API and coordinate business-level archival decisions separately from provider retention rules.

Authentication and security considerations

Authentication options

IBM Cloud Object Storage supports IBM Cloud IAM bearer tokens and HMAC access-key and secret-key credentials using S3-compatible Signature Version 4 signing. The appropriate method depends on the API operation and the configured Martini HTTP authentication approach.

Credential protection

Store IAM API keys, service credentials, HMAC secrets, and tokens in environment-specific Martini secrets rather than workflow definitions. Separate development, test, and production credentials and use least-privilege IAM roles, service IDs, bucket policies, and object permissions.

Object security

  • Treat public bucket access as an explicit exception.
  • Restrict read, write, copy, and delete permissions by workflow responsibility.
  • Validate downloaded objects as untrusted input before processing.
  • Use TLS validation and private network controls where supported by the deployment.

Operational considerations for IBM Cloud Object Storage integrations

Pagination and checkpoints

Bucket and object listings can require continuation tokens. Workflows should process pages consistently and persist a checkpoint based on object key, timestamp, version, or event identifier.

Idempotency and ordering

Notifications may be delayed, duplicated, or delivered out of order. Include bucket, object key, version when available, and event identity in the processing key, and make downstream writes idempotent where possible.

Large objects

Use multipart uploads for large or resumable transfers. Track upload identifiers and part numbers, retry transient failures with bounded backoff, and abort incomplete uploads when a workflow fails.

Versions and retention

Versioning, delete markers, retention policies, legal holds, and immutable storage settings can change deletion behavior. Do not delete or overwrite an object until the downstream operation has been confirmed and policy checks have passed.

Testing and change management

Test endpoint, region, signing, pagination, notification, schema, and failure scenarios in isolated buckets. Preserve rejected source objects and validation details so producers can be corrected and files can be replayed deterministically.

Why use Martini instead of scripts or point-to-point integrations?

Orchestrate more than object transfers

Scripts can move files, but enterprise integrations also need validation, mapping, business rules, downstream API calls, checkpoints, retries, and audit outcomes. Martini represents this behavior as maintainable workflows and APIs.

Separate configuration from implementation

Martini keeps endpoints, buckets, credentials, and environment-specific values configurable while reusable integration logic handles common COS operations across environments.

Support event and scheduled models

Martini can receive selected COS notifications, expose controlled APIs, and run scheduled reconciliation workflows when event coverage is incomplete or a full scan is required.

Handle operational complexity

Pagination, multipart transfers, duplicate events, object versions, transient failures, and schema changes can be implemented as explicit workflow behavior rather than scattered across point-to-point scripts.

Frequently asked questions

How can IBM Cloud Object Storage be integrated with enterprise systems?

IBM Cloud Object Storage integrates through its S3-compatible HTTPS API for bucket and object operations, multipart uploads, versioning, lifecycle configuration, and metadata. Selected object and bucket events can be published through configured notification destinations. IAM bearer tokens and HMAC Signature Version 4 credentials provide authentication, while scheduled listings and checkpoints support incremental synchronization.

Can Martini integrate with IBM Cloud Object Storage?

Yes. Martini can consume the IBM Cloud Object Storage S3-compatible REST API, authenticate with IBM IAM or HMAC credentials, receive selected event notifications through an exposed API, and orchestrate object movement, transformation, validation, downstream API calls, retries, and audit handling.

Do I need a connector to integrate IBM Cloud Object Storage with Martini?

No. A dedicated IBM Cloud Object Storage connector is not required or documented in the supplied Martini context. Martini can integrate using the product's native S3-compatible REST API, configured event notifications, IAM or HMAC authentication, and file-oriented workflows.

Is there any extra Lonti cost to integrate IBM Cloud Object Storage with Martini?

Lonti does not charge an additional per-connector or per-vendor fee to integrate IBM Cloud Object Storage. The integration is subject to the provisioned capacity of the Martini environment. Separate costs may apply from IBM, cloud infrastructure, or other third-party systems based on subscription, storage, network, usage, and deployment model.

Which IBM Cloud Object Storage integration methods should be used?

Use the S3-compatible REST API for current bucket, object, metadata, version, lifecycle, and multipart operations. Use event notifications for selected object events when near-real-time processing is appropriate, and scheduled paginated listings with checkpoints for reconciliation or event coverage gaps. IBM IAM bearer tokens and HMAC Signature Version 4 are the documented authentication choices.

Are IBM Cloud Object Storage events or webhooks available?

COS supports notifications for selected bucket and object events through supported destinations, but they are not universal callbacks for every storage operation. Martini can receive configured HTTP notifications through an exposed API or webhook workflow, validate them, apply idempotency, and retrieve the object separately when the event contains metadata rather than the complete body.

How does synchronization with IBM Cloud Object Storage work?

Synchronization typically combines event notifications, paginated object listings, stable prefixes, modification timestamps, object versions, and a Martini-managed checkpoint. Because objects can arrive out of order and notifications can be delayed or duplicated, workflows should use deterministic identifiers and reconciliation scans rather than relying only on timestamps.

How are mapping, errors, retries, and duplicates handled?

Martini can validate and transform JSON, XML, CSV, Excel, and binary payloads before calling target systems or writing output objects. Workflows can use bounded exponential backoff for transient failures, preserve request identifiers and provider responses, and route permanent failures for investigation. Idempotency keys based on bucket, object key, version, and event identifier help prevent duplicate processing.

Related Martini documentation

APIs

Workflows

Data Processing

Security

Operations

Integrate IBM Cloud Object Storage with Martini

Use Martini to build secure, observable workflows for IBM Cloud Object Storage object movement, event processing, file transformation, and enterprise data exchange.

EXPLORE MARTINI
BOOK A DEMO