Ellipse Gradient for Header

Elasticsearch Integration Guide

Connect Elasticsearch with enterprise applications through REST APIs, bulk indexing, scheduled workflows, controlled search APIs, and selected Watcher webhook actions.

Elasticsearch integration options at a glance

Elasticsearch is primarily integrated through HTTP REST APIs for indexing, document updates, searches, aggregations, index administration, aliases, mappings, and cluster operations. The _bulk API supports batch indexing, updates, and deletes, while asynchronous search supports longer-running queries. Elasticsearch also provides SQL and ES|QL interfaces for query and analytics use cases. Watcher can invoke outbound webhook actions for selected configured conditions, but it is not a universal event stream. Martini can consume these endpoints in workflows, map source data into Elasticsearch documents, expose controlled search APIs, and apply scheduling, validation, retries, and secure credential management.

Integration pointSupported by Elasticsearch?Common use casesHow Martini supports it
REST APIsYesUse Elasticsearch HTTP APIs for document indexing, retrieval, updates, deletion, searches, aggregations, index administration, aliases, mappings, cluster operations, and security administration.Martini can consume REST endpoints from workflows, map request and response payloads, apply business rules, and expose APIs that mediate controlled access to Elasticsearch.
Bulk / async / batch APIsYesThe _bulk API supports multiple index, update, and delete operations in one newline-delimited JSON request. Asynchronous search supports long-running queries that can be polled by identifier.Martini can construct bulk payloads, submit them in bounded batches, inspect every per-item result, and orchestrate polling for asynchronous searches.
Webhooks / outbound callbacksLimitedElastic Watcher can invoke a webhook action when a configured watch, schedule, query, and condition are satisfied. This is not a general notification stream for every document or cluster event.Martini can expose a REST API or receive the outbound HTTP call through a webhook-consuming workflow, then enrich, route, and acknowledge the resulting alert.
Database / analytics accessYesElasticsearch provides SQL and ES|QL interfaces, plus documented JDBC and ODBC options, for query and analytics-oriented use cases rather than conventional transactional database behavior.Martini can call the SQL REST API through HTTP workflows and transform query results for downstream systems. Direct JDBC or ODBC use depends on the deployment and implementation requirements.
File / attachment APIsLimitedElasticsearch can process binary content through supported ingest processors such as the attachment processor, but it is not a general-purpose file or attachment repository.Martini can receive or retrieve file content from an upstream system, transform the payload, and submit supported content for Elasticsearch ingest processing while leaving original-file storage elsewhere.
AuthenticationYesSupported deployment configurations include API keys, Basic Authentication, bearer or service-account tokens, TLS client authentication, and configured federated identity options.Martini can store endpoint credentials and secrets in secure environment configuration and send the required authorization headers over TLS.
SDKs and client librariesYesElastic documents clients for Java, JavaScript, Python, Go, .NET, PHP, and Ruby. These may be useful when custom application logic is required outside direct HTTP calls.Martini integrations can normally use the REST APIs directly; custom JVM-compatible logic can be considered when the HTTP interface alone is insufficient.

How Elasticsearch exposes data and business events

Elasticsearch REST APIs

Elasticsearch exposes HTTP REST APIs for document operations, search, aggregations, index and mapping administration, aliases, cluster operations, and security administration. These APIs are the primary integration mechanism for application and enterprise workflows.

Martini implementation pattern

Martini implementation pattern: a workflow receives or retrieves source data, constructs the appropriate Elasticsearch request, authenticates using secure environment configuration, maps the response, and routes HTTP and application-level failures according to business rules.

Implementation sequence

Receive source data or a search request
Validate the payload and permitted Elasticsearch operation
Construct the REST request and authorization headers
Call the Elasticsearch endpoint
Map the response to the target application model
Record the result and route failures appropriately

Elasticsearch Bulk and Async APIs

The _bulk API accepts multiple index, update, and delete operations in newline-delimited JSON. Asynchronous search returns an identifier that can be polled for results when a query may take longer to complete.

Martini implementation pattern

Martini implementation pattern: a workflow groups documents into bounded batches, formats the newline-delimited payload, submits the request, inspects both the HTTP response and each item result, and separately orchestrates polling for asynchronous searches.

Implementation sequence

Collect and validate the source documents
Assign deterministic document identifiers
Build the newline-delimited bulk payload
Submit a bounded bulk request
Inspect every bulk item result
Retry only eligible failures and record rejected items

Elasticsearch Watcher Webhooks

Watcher can evaluate scheduled watches and invoke an outbound webhook action when a configured query and condition are satisfied. Coverage is selective and depends on the watch definition rather than representing a universal Elasticsearch event stream.

Martini implementation pattern

Martini implementation pattern: Martini exposes a controlled API or webhook-consuming workflow for the Watcher call, authenticates and validates the incoming alert, enriches it with related data, and routes it to an incident or notification system.

Implementation sequence

Receive the configured Watcher webhook action
Authenticate and validate the alert payload
Retrieve related Elasticsearch or application context
Apply severity and routing rules
Create or update the operational target
Record the alert outcome and prevent duplicate handling

Elasticsearch Search and Pagination

Elasticsearch supports search_after, point-in-time searches, scroll searches for suitable batch workloads, and time-based filters for large or time-series result sets. Large from and size values are not appropriate for deep pagination.

Martini implementation pattern

Martini implementation pattern: a scheduled or API-triggered workflow stores the retrieval checkpoint, uses a stable sort and appropriate pagination method, processes each page, and advances the checkpoint only after successful downstream handling.

Implementation sequence

Establish the time, identifier, or point-in-time boundary
Request a bounded page of results
Transform and deliver the returned documents
Capture the next search_after or continuation state
Repeat until the result set is complete
Persist the checkpoint after successful processing

Elasticsearch SQL and ES|QL

Elasticsearch provides SQL and ES|QL interfaces for query and analytics use cases, with JDBC and ODBC options documented for compatible clients. These interfaces are query-oriented and do not replace the document APIs for normal indexing workflows.

Martini implementation pattern

Martini implementation pattern: Martini receives an approved analytical request, validates its query scope and parameters, calls the SQL or ES|QL endpoint, and transforms tabular or query results for a reporting or business application.

Implementation sequence

Receive an approved analytical request
Validate query scope, fields, and result limits
Call the SQL or ES|QL endpoint
Normalize the returned result structure
Apply downstream business rules
Deliver or store the analytical response

Common Elasticsearch integration patterns

Pattern 1: Index operational data for unified search

When to use this pattern

Use this pattern when data from applications such as ServiceNow, Jira, or Salesforce must be searchable alongside operational documents in Elasticsearch. It is suitable for customer-service lookup, operational dashboards, and cross-system search.

Integration direction
ServiceNow or Jira
Martini
Elasticsearch
Kibana
Example Mapping
Elasticsearch FieldCanonical FieldTarget Field
source.iddocumentId_id
source.titletitletitle
source.updatedAtlastModifiedAtlast_modified_at
source.statuslifecycleStatusstatus
Martini implementation pattern

A Martini workflow retrieves or receives source objects, normalizes identifiers and timestamps, validates required fields against the target mapping, and sends documents to a configured index or data stream. Deterministic IDs make retries idempotent, while rejected documents are routed for review rather than repeatedly submitted.

Martini capabilities used
  • workflows
  • API consumption
  • data mapping
  • JSON handling
  • business rules
  • error handling

Pattern 2: Run scheduled incremental synchronization

When to use this pattern

Use this pattern when Elasticsearch must be refreshed from a source application that exposes update timestamps, sequence values, or another change marker. It supports scheduled indexing of tickets, knowledge articles, customer data, or product information.

Integration direction
Source application
Martini
Elasticsearch
Example Mapping
Elasticsearch FieldCanonical FieldTarget Field
source.updatedAtchangeCheckpointquery filter or checkpoint
source.keydocumentId_id
source.attributesdocumentFieldsdocument fields
source.deleteddeletionStatedelete operation
Martini implementation pattern

A scheduled Martini workflow reads the last successful checkpoint, retrieves source pages, transforms changed objects, and submits bounded _bulk requests. It advances the checkpoint only after successful processing, handles source deletions through an explicit strategy, and separates retryable failures from mapping or authorization errors.

Martini capabilities used
  • scheduled workflows
  • API consumption
  • pagination orchestration
  • data mapping
  • checkpoint management
  • error handling and retry

Pattern 3: Expose a controlled Elasticsearch search API

When to use this pattern

Use this pattern when applications need search access without direct Elasticsearch exposure. It is appropriate for enforcing authorization, limiting fields and page sizes, standardizing query parameters, or combining search results with another system.

Integration direction
Client application
Martini
Elasticsearch
Martini
Example Mapping
Elasticsearch FieldCanonical FieldTarget Field
request.queryapprovedSearchCriteriaquery DSL
request.pageSizeboundedPageSizesize
request.sortapprovedSortsort
_source fieldspermittedResponseFieldsresponse projection
Martini implementation pattern

A Martini API accepts a constrained request model, validates fields, operators, sorting, aggregations, and maximum page sizes, then constructs the Elasticsearch query. The workflow maps the result to a stable response contract and rejects unrestricted query DSL or unauthorized fields.

Martini capabilities used
  • API exposure
  • API consumption
  • validation
  • business rules
  • data transformation
  • security controls

Pattern 4: Route Watcher alerts to incident management

When to use this pattern

Use this pattern when configured Elasticsearch Watcher conditions should initiate an incident or notification workflow. It is suitable for threshold alerts, error-rate detection, infrastructure conditions, and selected data-quality checks.

Integration direction
Elasticsearch Watcher
Martini
ServiceNow or Jira
Example Mapping
Elasticsearch FieldCanonical FieldTarget Field
watch.idalertSourcesource
watch.triggered_timealertTimeopenedAt
watch.payloadalertContextdescription and context
watch.conditionseverityRulepriority
Martini implementation pattern

Watcher calls a Martini API when a configured condition is met. Martini authenticates and validates the request, derives a stable alert key for deduplication, enriches the alert if required, and creates or updates the target incident. Because Watcher is selective rather than a universal event feed, reconciliation can be added for critical conditions.

Martini capabilities used
  • API exposure
  • webhook receiving
  • workflow orchestration
  • business rules
  • deduplication
  • error handling

Applications commonly integrated with Elasticsearch

Elasticsearch can be connected with Elastic products, event platforms, and enterprise applications to support search, observability, alerting, and operational workflows. Martini can coordinate these integrations when data needs normalization, enrichment, controlled access, or delivery to multiple destinations.

Application Scenario Direction Martini Pattern
Kibana Provide search, dashboards, visualizations, and analytics over Elasticsearch documents, data streams, and aggregations. Elasticsearch → Kibana Martini workflows can normalize and index application or operational data into the indices and data streams consumed by Kibana, while controlled Martini APIs can expose curated search results to downstream applications.
Logstash Collect, parse, enrich, and route event data into Elasticsearch indices or data streams. Logstash → Martini → Elasticsearch Martini can receive normalized events from Logstash or another source, apply validation and business rules, and send single-document or _bulk requests to Elasticsearch with per-item failure handling.
Elastic Agent Ingest logs, metrics, endpoint data, and telemetry into the Elastic platform for operational analysis. Elastic Agent → Elasticsearch Martini can complement agent ingestion by enriching selected telemetry, routing application metadata, or indexing business context into compatible Elasticsearch data streams.
APM Server Store application performance telemetry for analysis alongside application and operational data. APM Server → Elasticsearch Martini can coordinate supplementary application metadata or derived operational events with APM-related Elasticsearch data, using controlled mappings and scheduled or event-driven workflows.
Apache Kafka Stream application and event data toward Elasticsearch and coordinate event-driven ingestion architectures. Apache Kafka → Martini → Elasticsearch Martini can consume or receive prepared event payloads, transform them into Elasticsearch document structures, batch them into newline-delimited _bulk requests, and route rejected items for retry or review.
ServiceNow Index incidents, knowledge articles, configuration data, or operational events for unified search and alert-driven workflows. ServiceNow → Martini → Elasticsearch A scheduled or API-triggered Martini workflow can retrieve changed ServiceNow objects, map stable identifiers and timestamps into Elasticsearch documents, and use deterministic IDs to support safe retries and updates.
Jira Make issues, comments, and project information searchable across operational and business contexts. Jira → Martini → Elasticsearch Martini can retrieve changed Jira data, normalize issue and project fields, enrich documents where required, and index them individually or through the _bulk API while tracking pagination and checkpoints.
Splunk Migrate, replicate, or correlate selected operational and log data across observability platforms. Splunk → Martini → Elasticsearch Martini can mediate selected exports or API responses, transform event structures, and write them to Elasticsearch with bounded concurrency, timestamp-based retrieval, and classified error handling.

How to build a Elasticsearch integration in Martini

Objective

Establish the Elasticsearch endpoint and authentication configuration without embedding credentials in workflow definitions.

Instructions in Martini

  • Choose the target Elastic Cloud or self-managed endpoint and confirm its version.
  • Use an appropriately scoped API key, service token, Basic Authentication credential, or configured certificate.
  • Store credentials and endpoint settings in Martini secure environment configuration.
  • Confirm TLS and index or data-stream privileges for the workflow.

Objective

Select the execution model that matches the integration requirement and the availability of source changes.

Instructions in Martini

  • Use a REST-triggered workflow for synchronous indexing or search requests.
  • Use a scheduled workflow for source APIs that expose timestamps, sequence values, or reconciliation endpoints.
  • Use a Martini API for selected Elasticsearch Watcher webhook actions.
  • Do not treat Watcher as a universal document-change event stream.

Objective

Acquire source objects, search requests, or alert payloads with bounded volume and explicit continuation state.

Instructions in Martini

  • Call source REST APIs or receive the configured inbound request.
  • Use source pagination and persist a checkpoint where incremental retrieval is required.
  • Use search_after with a point-in-time search or an appropriate scroll strategy for large Elasticsearch result sets.
  • Validate incoming Watcher payloads before enrichment or routing.

Objective

Coordinate Elasticsearch calls, transformations, business rules, and target-system actions as a maintainable Martini workflow.

Instructions in Martini

  • Separate retrieval, normalization, Elasticsearch interaction, and downstream delivery into clear workflow stages.
  • Use bounded concurrency and payload sizes to protect cluster capacity.
  • Poll asynchronous searches through a controlled workflow when a long-running query returns an asynchronous identifier.
  • Keep index, alias, data-stream, and environment settings configurable.

Objective

Convert source objects and search responses into structures that match Elasticsearch mappings and downstream contracts.

Instructions in Martini

  • Map stable source identifiers to deterministic Elasticsearch document IDs.
  • Transform timestamps, nested fields, status values, and source-specific names into the target document model.
  • Construct valid newline-delimited JSON for _bulk requests, including the required final newline.
  • Normalize search, SQL, or alert responses before sending them to consumers.

Objective

Protect data quality and Elasticsearch access by validating operations and applying integration-specific decisions.

Instructions in Martini

  • Validate required fields and compatible mapping types before indexing.
  • Choose index, create, update, or delete semantics based on source state and idempotency requirements.
  • Constrain search fields, operators, aggregations, sorting, and page sizes for exposed APIs.
  • Classify authentication, mapping, query, timeout, and per-item bulk failures separately.

Common Elasticsearch data objects used in integrations

ObjectTypical UseCommon target systemsMartini handling
DocumentsJSON objects indexed, updated, deleted, and searched in Elasticsearch. They may include metadata such as _id, _index, and routing information.Kibana, Logstash, Elastic Agent, APM Server, ServiceNow, Jira, and operational applicationsMartini maps source payloads into document structures, applies deterministic IDs and validation, and sends individual or bulk document operations.
IndicesLogical collections of documents with mappings and settings for application, operational, or analytical workloads.Kibana, search APIs, reporting applications, and data synchronization workflowsMartini can target configured indices, create or update documents, and apply routing rules while keeping index and endpoint configuration outside workflow logic.
Data streamsAppend-oriented collections for logs, metrics, traces, and other time-series data using backing indices behind a stable stream name.Kibana, Elastic Agent, APM Server, observability applications, and operational dashboardsMartini can write timestamped documents to configured data-stream names and apply source validation, batching, and time-based synchronization rules.
AliasesAlternate names for indices or data streams that support filtered access, write routing, and controlled zero-downtime replacement.Search APIs, applications, migration workflows, and KibanaMartini can call alias APIs during migrations or controlled cutovers and can keep clients pointed at stable names while index versions change.
MappingsDefinitions of document fields, data types, analyzers, and indexing behavior.Indices, data streams, indexing workflows, and search applicationsMartini validates and transforms source fields to match the target mapping and can route incompatible payloads to operational review rather than repeatedly retrying them.
Search requests and aggregationsRequest structures for filtering, sorting, retrieving documents, calculating metrics, and grouping results.Kibana, client applications, reporting systems, and Martini APIsMartini can construct constrained queries, validate allowed fields and limits, call search or SQL endpoints, and return a normalized response.

Authentication and security considerations

Use scoped credentials

Elasticsearch supports API keys, Basic Authentication, bearer or service-account tokens, TLS client authentication, and configured federated identity options. For machine integrations, use the least-privileged API key or service credential required for the target indices and operations.

Protect credentials and transport

Store Elasticsearch endpoints and credentials in Martini secure environment configuration rather than workflow mappings. Use TLS and restrict access to only the required indices, aliases, data streams, and API operations.

Control exposed search access

If Martini exposes a search API, validate fields, operators, sorting, aggregations, and page sizes before constructing Elasticsearch requests. Do not pass unrestricted user-provided query DSL directly to Elasticsearch.

Operational considerations for Elasticsearch integrations

Capacity and request volume

Cluster capacity, shard count, thread pools, request size, and deployment configuration affect safe throughput. Use bounded concurrency, controlled payload sizes, throttling, and exponential backoff rather than assuming a fixed client rate limit.

Bulk responses and retries

Inspect both the HTTP response and every item returned by the _bulk API. Classify authentication failures, mapping errors, rejected requests, timeouts, and transient cluster pressure separately, and retry only eligible failures.

Pagination and checkpoints

For large result sets, prefer point-in-time searches with search_after or an appropriate scroll strategy. Persist checkpoints only after successful downstream processing, and define how source deletions and synchronization gaps are reconciled.

Mappings and refresh behavior

Mapping changes can require versioned indices and reindexing rather than in-place modification. Newly indexed documents may not be immediately searchable, so workflows should account for refresh timing without forcing refreshes unnecessarily.

Version and test compatibility

Confirm the Elasticsearch version, deployment model, endpoint paths, authentication scheme, mapping behavior, and client compatibility before deployment. Test malformed queries, per-item bulk failures, retries, duplicates, timeouts, and schema changes.

Why use Martini instead of scripts or point-to-point integrations?

Orchestration beyond a script

Martini provides a maintainable workflow layer for retrieving source data, transforming documents, applying business rules, calling Elasticsearch, and delivering results to other systems. This keeps integration behavior visible and reusable as requirements change.

Controlled APIs and access

Martini can expose a constrained search or indexing API instead of giving every client direct Elasticsearch access. Validation, authorization, response shaping, and query limits can be applied consistently at the integration boundary.

Reliable synchronization

Scheduled workflows can manage pagination, checkpoints, deterministic document IDs, bounded bulk requests, per-item error handling, and retry decisions. This is more resilient than an isolated script that treats every response as success or failure at the request level.

Reusable enterprise integration assets

Martini separates secure configuration, API calls, mappings, workflow orchestration, and error handling so the same integration patterns can be reused across indices, data streams, source applications, and deployment environments.

Frequently asked questions

How can Elasticsearch be integrated with enterprise systems?

Elasticsearch is primarily integrated through HTTP REST APIs for document indexing, updates, deletion, searches, aggregations, index management, and cluster operations. The _bulk API supports batch operations, SQL and ES|QL support query-oriented access, and Watcher can invoke webhook actions for selected configured conditions.

Can Martini integrate with Elasticsearch?

Yes. Martini can consume Elasticsearch REST APIs through workflows, map source data into documents, submit individual or _bulk operations, run scheduled searches, expose controlled search APIs, and receive selected outbound Watcher webhook actions through a Martini API.

Do I need a connector to integrate Elasticsearch with Martini?

No. A dedicated Elasticsearch connector is not required. Martini can use Elasticsearch's confirmed native REST APIs, bulk and asynchronous operations, SQL interfaces, selected Watcher webhook actions, and supported authentication methods.

Is there any extra Lonti cost to integrate Elasticsearch with Martini?

Lonti does not charge an additional per-connector or per-vendor fee to integrate Elasticsearch with Martini. Integrations are subject to the provisioned capacity of the Martini environment. Separate costs may apply from Elastic, cloud infrastructure, or other third-party systems based on subscriptions, usage, and deployment model.

Which Elasticsearch APIs should an integration use?

Use document REST APIs for individual indexing, retrieval, updates, and deletes, and use the _bulk API for batch workloads. Search and aggregation APIs support retrieval, while SQL or ES|QL are appropriate for query and analytics use cases. Asynchronous search can support longer-running queries.

Are Elasticsearch events or webhooks available?

Elasticsearch does not provide a general webhook for every document, index, or cluster event. Watcher can invoke outbound webhook actions when configured watches detect selected conditions. Martini can receive those calls, validate them, enrich alerts, and route them to operational systems.

How does synchronization with Elasticsearch work?

Synchronization is commonly implemented with scheduled REST calls, source timestamps or sequence markers, time-based filters, explicit checkpoints, and deterministic document IDs. Elasticsearch does not provide a universal change-data-capture stream for every document mutation, so source deletions and reconciliation require an explicit strategy.

How does Martini handle Elasticsearch mapping, failures, and retries?

Martini can transform source payloads to match Elasticsearch mappings, validate fields, classify HTTP and application-level errors, and retry only appropriate failures. For _bulk requests, the workflow must inspect every item because a successful HTTP response can contain individual rejected operations. Stable document IDs help make retries idempotent.

Related Martini documentation

APIs

Workflows

Data

Security

Operations

Connect Elasticsearch with Martini

Use Martini to build secure, reliable Elasticsearch integrations for indexing, search, synchronization, alert routing, and enterprise API orchestration.

EXPLORE MARTINI
BOOK A DEMO