Data Service

Response data storage, processing, bulk operations, and export pipeline

Data Service (Port 8002)

The data-service manages survey response data, attachments, bulk operations, and exports. It serves as the primary repository for all collected field data.

Database

Database: data_db

TablePurpose
responsesSurvey response data (JSONB answers)
attachmentsFile attachments linked to specific question IDs

API Endpoints

MethodPathPurpose
GET/api/v1/responsesList responses (paginated)
POST/api/v1/responsesSubmit a new response
GET/api/v1/responses/{id}Get full response and history
PATCH/api/v1/responses/{id}Update specific response values
DELETE/api/v1/responses/{id}Permanently delete a response
POST/api/v1/responses/{id}/approveApprove a response (workflow)
GET/api/v1/responses/{id}/attachmentsList all files for this response
POST/api/v1/bulkBatch status updates or deletions

Attachment Management

The data-service coordinates with the media-service for file handling:

  1. When a survey includes media questions (Image, Signature, etc.), the client uploads the file to media-service.
  2. The resulting asset_id is stored in the responses table.
  3. The attachments table links these assets to the specific response for easy retrieval.

Export Pipeline

The service supports large-scale data exports in multiple formats:

  • Formats: CSV, XLSX, JSON.
  • Async Execution: Exports are processed via Celery to prevent API timeouts for large datasets.
  • Workflow:
    1. Client triggers export via POST /api/v1/export.
    2. Service initiates a task and returns a task ID.
    3. Client polls for completion.
    4. Finished file is served via a temporary download URL.

Events Published

Events are published to the Redis webhook:events channel:

  • response.submitted
  • response.approved
  • response.deleted
# Event payload structure
{
    "event_type": "response.submitted",
    "tenant_id": "uuid",
    "data": {
        "response_id": "uuid",
        "survey_id": "uuid"
    }
}