Ingestion

Run Ingestion

post

Run the ingestion pipeline by delegating to the Temporal wrapper via AgentService.

This endpoint validates the request, delegates to the ingestion wrapper, which calls gRPC AgentService to trigger a Temporal workflow.

Body

application/json

Model for the ingestion API request body.

Responses

200

Successful Response

application/json

422

Validation Error

application/json

post

/ingestion/run

HTTPcURLJavaScriptPython

POST /ingestion/run HTTP/1.1
Content-Type: application/json
Accept: */*
Content-Length: 597

{
  "config": {
    "source_path": "",
    "recursive": true,
    "pages_to_read": 3,
    "temp_dir": "text",
    "reader_type": "text",
    "api_base_url": "text",
    "storage_provider": "aws",
    "storage_credentials": {
      "access_key": "text",
      "secret_key": "text",
      "bucket": "text",
      "region": "us-east-1",
      "account_id": "text",
      "project_id": "text",
      "bucket_name": "text",
      "credentials_json": "text",
      "application_key_id": "text",
      "application_key": "text"
    },
    "workspace_id": "text",
    "project_id": "text",
    "task_id": "text",
    "user_id": "text"
  },
  "catalogs": [
    {
      "id": "text",
      "instruction": "text",
      "fetch_all_metadata": false,
      "metadata_scan": {
        "ANY_ADDITIONAL_PROPERTY": "anything"
      }
    }
  ]
}

{
  "success": true,
  "message": "text",
  "task_id": "text",
  "status": "text",
  "error": "text",
  "total_folders": 0,
  "total_files": 0,
  "successful": 0,
  "failed": 0,
  "success_rate": "0.00%",
  "execution_time": "0.00s",
  "folders": [
    {
      "folder_path": "text",
      "successful_count": 1,
      "failed_count": 1,
      "files": [
        {
          "source_path": "text",
          "status": "text",
          "classified_catalog_id": "text",
          "error_message": "text",
          "metadata": {
            "ANY_ADDITIONAL_PROPERTY": "anything"
          }
        }
      ]
    }
  ],
  "errors": [
    "text"
  ]
}

Health Check

get

Health check endpoint for the ingestion API.

Responses

200

Successful Response

application/json

Responseany

get

/ingestion/health

HTTPcURLJavaScriptPython

GET /ingestion/health HTTP/1.1
Accept: */*

200

Successful Response

No content

List Prefixes

post

List available prefixes/folders in an S3 bucket.

This endpoint allows clients to browse available folders/prefixes in their storage bucket before triggering ingestion.

Body

application/json

Model for list-prefixes request.

storage_providerstringOptional

Storage provider type

Default: aws

prefixstringOptional

Optional prefix path

Default: ""

Responses

200

Successful Response

application/json

422

Validation Error

application/json

post

/ingestion/list-prefixes

HTTPcURLJavaScriptPython

POST /ingestion/list-prefixes HTTP/1.1
Content-Type: application/json
Accept: */*
Content-Length: 279

{
  "storage_provider": "aws",
  "storage_credentials": {
    "access_key": "text",
    "secret_key": "text",
    "bucket": "text",
    "region": "us-east-1",
    "account_id": "text",
    "project_id": "text",
    "bucket_name": "text",
    "credentials_json": "text",
    "application_key_id": "text",
    "application_key": "text"
  },
  "prefix": ""
}

{
  "success": true,
  "prefixes": [
    "text"
  ],
  "errors": [
    "text"
  ]
}