---
Title: Manage sources
URL Source: https://company-skill.com/p/opensearch/opensearch-manage-sources
Language: en
Description: You want to connect external data sources—such as Alibaba Cloud OSS, MaxCompute tables, or HTTP-based API endpoints—to OpenSearch for indexing and search. This involves configuring authentication,…
---

# Manage sources

Part of **OpenSearch**. Route queries via `POST https://company-skill.com/api/route`.

## What You Want to Do

You want to connect external data sources—such as Alibaba Cloud OSS, MaxCompute tables, or HTTP-based API endpoints—to OpenSearch for indexing and search. This involves configuring authentication, paths, schemas, and validation settings.

**Typical User Questions**:
- How to connect OSS as a data source?
- Can I ingest data from an API endpoint?

## Decision Tree

Pick the best path for your situation:

- **If** you are configuring OSS, MaxCompute, or an API data source for the first time and prefer using a graphical interface → Use (go to *opensearch/opensearch-document*)
- **If** you need to programmatically create or manage data sources (e.g., in CI/CD pipelines, multi-environment deployments, or system integrations) → Use API (go to *opensearch/opensearch-instance*)
- **If** you lack programming experience or only need a one-time setup → Use (go to *opensearch/opensearch-document*)
- **Otherwise (default)** → Start with ****, as it provides guided workflows for common sources like OSS, MaxCompute, and API Data Source without requiring code.

## Path Comparison

| Path | Best For | Complexity | Code Required | Automation | Key Fact | Detail Skill |
|------|----------|------------|---------------|------------|----------|-------------|
| OSSMaxComputeAPI | low | No | No | Not automation-friendly as it requires manual UI interactions | `opensearch/guide/opensearch-document` |
| API | medium | Yes | Yes | Requires proper API key management with `Authorization: Bearer $DASHSCOPE_API_KEY` | `opensearch/api/opensearch-instance` |

## Path Details

### Path 1: Console / Dashboard
**Best For**: OSSMaxComputeAPI

**Brief Description**: This approach uses OpenSearch’s web console to configure data sources through guided UI workflows. You interact with features like **Add Data Source**, **Data Source Configuration**, and **Configuration Center** to define connections to **MaxCompute**, **API Data Source**, or **OSS Path**, followed by **Validate** and **Verify** steps to ensure correctness.

**Key technical facts**:
- Billing: Document Split: billed per request (0.0005 per request); Multimodal Pipeline Services: billed per 1,000 tokens; OpenSearch Retrieval Engine Edition: billed per instance hour

**When to Use**:
- User prefers graphical interface over coding
- Configuring OSS, MaxCompute or API data sources for the first time
- User lacks programming experience
- One-time setup of data ingestion pipelines

**When NOT to Use**:
- Need programmatic or automated data source management
- Building CI/CD pipelines for data ingestion
- Managing multiple data sources at scale
- Requiring integration with external systems programmatically

**Known Limitations**:
- Data Source Name cannot be changed after creation for MaxCompute data sources
- OSS Path must contain 'opensearch' and cannot include '=', '&', or '?'
- Deletion of API data source fails if referenced by an index table
- MaxCompute table fields must use only STRING, BOOLEAN, DOUBLE, BIGINT, and DATETIME data types
- Not automation-friendly as it requires manual UI interactions

### Path 2: API

**Brief Description**: This method uses REST APIs and SDKs to programmatically manage OpenSearch data ingestion resources. Key operations include **CreateAppGroup**, **CreateFunctionInstance**, **CreateFunctionTask**, and **PushDocuments**, with resource inspection via **ListFunctionInstances** and **GetFunctionInstance**, and lifecycle control through **UpdateFunctionInstance** and **DeleteFunctionInstance**.

**Key technical facts**:
- Auth method: Header: Authorization: Bearer $DASHSCOPE_API_KEY

**When to Use**:
- Need programmatic creation and management of data sources
- Building automated CI/CD pipelines
- Managing multiple OpenSearch resources at scale
- Integrating OpenSearch with other systems programmatically
- Requiring consistent, repeatable deployments

**When NOT to Use**:
- User prefers graphical interface over coding
- One-time simple data source setup
- Lack of programming experience
- No need for automation or integration with external systems

**Known Limitations**:
- Requires programming knowledge and SDK integration
- Authentication requires proper API key management
- Error handling needed for various HTTP status codes (400, 401, 403, 404, 500)
- Rate limits apply (e.g., 100 QPS per account for CreateAppGroup)
- Complex parameter structures required for function instances

## FAQ

Q: Which path should I start with?
A: Start with **** if you're new to OpenSearch or setting up a single data pipeline. It guides you through **Add Data Source**, **OSS Path** rules, and **MaxCompute** permissions without writing code.

Q: What if I need to deploy the same MaxCompute sync job across dev, staging, and prod—but used the console?
A: You’ll face inconsistent configurations and manual rework, since the console doesn’t support export/import or versioning of data source definitions.

Q: What if I try to delete an API data source created via console that’s linked to an index table?
A: The deletion will fail—a known limitation. You must first remove the reference from the index table before deleting the **API Data Source**.

Q: Can I use the API path without knowing about appGroupIdentity or functionName?
A: No—these are required parameters in calls like **CreateFunctionInstance** and **CreateAppGroup**. Missing them causes 400 errors, so review the API detail skill first.

Q: Does the console support all MaxCompute data types?
A: No—it only supports STRING, BOOLEAN, DOUBLE, BIGINT, and DATETIME. If your table uses other types (e.g., ARRAY or MAP), ingestion will fail during **Validate**.

Q: What happens if I exceed the API rate limit when calling CreateAppGroup?
A: You’ll receive a 429 or 500 error. The API path enforces ~100 QPS per account, so implement retry logic with backoff in your automation scripts.

## Related queries

connect OSS to OpenSearch, ingest from MaxCompute, sync MaxCompute data, pull data from API endpoint, configure external data source, add data source in OpenSearch, manage data ingestion, set up OSS data pipeline, how to connect OSS, how to sync MaxCompute, can I use API as source, what data sources

---
Part of [OpenSearch](https://company-skill.com/p/opensearch.md) · https://company-skill.com/llms.txt