# Form Service

# Form Service (Port 8010) {#form-service-port-8010}


The **form-service** provides the engine for web-based data collection. It renders surveys created in the `survey-service` as interactive web forms, evaluates skip logic and constraints in real time, and manages the submission pipeline.


## Database {#database}


**Database:** `mquad_forms_db`

| Table | Purpose |
|-------|---------|
| `form_sessions` | Active respondent sessions for resuming drafts |
| `form_submissions` | Temporary staging for incoming submissions |


## API Endpoints {#api-endpoints}


| Method | Path | Purpose |
|--------|------|---------|
| GET | `/api/v1/forms/{survey_id}/render` | Get rendered form structure (JSON) |
| POST | `/api/v1/forms/submit` | Submit finalized form data |
| POST | `/api/v1/forms/draft` | Save partial progress |
| GET | `/api/v1/forms/resume/{token}` | Resume a saved draft |
| POST | `/api/v1/forms/validate` | Validate form data without submitting |
| POST | `/api/v1/validation/field` | Validate a single field constraint |
| POST | `/api/v1/validation/evaluate` | Evaluate an XPath expression with form data |
| GET | `/api/v1/validation/functions` | List all available XPath functions |
| POST | `/api/v1/validation/test-constraint` | Quick test of a constraint expression |


## Expression Engine {#expression-engine}


The form-service includes a complete XPath 1.0 expression evaluator with ODK/XLSForm extensions. The same engine powers skip logic (relevance), constraints, and calculations.


### Architecture {#architecture}


The evaluator has four layers:

1. **Tokenizer** (`XPathTokenizer`) — Converts expression strings into tokens: numbers, strings, operators, function names, field references (`${name}`), parentheses.
2. **Parser** (`XPathParser`) — Recursive-descent parser that builds an AST (Abstract Syntax Tree) from tokens. Supports operator precedence: `or` (lowest) → `and` → comparison (`=`, `!=`, `<`, `>`, `<=`, `>=`) → arithmetic (`+`, `-`) → multiplicative (`*`, `div`, `mod`) → unary (`-`, `not`).
3. **Evaluator** (`XPathEvaluator`) — Walks the AST against form data. Resolves `${field}` references to answer values, dispatches function calls to the function registry.
4. **Function Registry** (`ODKFunctionRegistry`) — 40+ registered ODK/XLSForm functions, all case-insensitive.


### Supported Functions {#supported-functions}



#### String Functions {#string-functions}
| Function | Description |
|----------|-------------|
| `string-length(value)` | Length of string |
| `substring(value, start, [length])` | Substring (1-based start) |
| `contains(value, substr)` | Check if string contains |
| `starts-with(value, prefix)` | Check string prefix |
| `normalize-space(value)` | Trim and collapse whitespace |
| `concat(a, b, ...)` | Concatenate strings |
| `translate(value, from, to)` | Character translation |
| `regex(value, pattern)` | Regex match |


#### Number Functions {#number-functions}
| Function | Description |
|----------|-------------|
| `number(value)` | Convert to number |
| `int(value)` | Convert to integer |
| `round(value, [precision])` | Round to precision |
| `floor(value)` | Floor |
| `ceiling(value)` | Ceiling |
| `abs(value)` | Absolute value |


#### Date/Time Functions {#datetime-functions}
| Function | Description |
|----------|-------------|
| `today()` | Current date |
| `now()` | Current datetime |
| `date(value)` | Parse string to date |
| `format-date(value, format)` | Format date using strftime |
| `decimal-date-time(value)` | Days since epoch |
| `days-between(a, b)` | Days between two dates |
| `today-offset(days)` | Date offset from today |


#### ODK-Specific Functions {#odk-specific-functions}
| Function | Description |
|----------|-------------|
| `selected(value, choice)` | True if choice selected (multi-select) |
| `count-selected(value)` | Count selected options |
| `selected-at(value, index)` | Choice at position |
| `count(a, b, ...)` | Count non-empty args |
| `sum(a, b, ...)` | Sum numeric args |
| `min(a, b, ...)` | Minimum value |
| `max(a, b, ...)` | Maximum value |
| `join(sep, a, b, ...)` | Join with separator |
| `coalesce(a, b, ...)` | First non-empty value |


#### Conditional Functions {#conditional-functions}
| Function | Description |
|----------|-------------|
| `if(cond, then, else)` | Conditional evaluation |
| `boolean(value)` | Convert to boolean |


#### Utility Functions {#utility-functions}
| Function | Description |
|----------|-------------|
| `random()` | Random 0-1 |
| `uuid()` | Generate UUID |
| `position()` | Current repeat position |
| `last()` | Last repeat position |
| `empty(value)` | Check if empty |


### Constraint Validation {#constraint-validation}


Constraints are evaluated at two points:

**Client-side (during form fill):** The `ConstraintValidator` evaluates each constraint expression against current form data using the `XPathEvaluator`. Empty values skip constraint evaluation (non-required fields pass validation when unanswered).

**Server-side (on submit):** `FormSubmissionService._validate_form_data()` runs all constraints server-side before persisting. If validation fails and the submission is marked complete, the submission is rejected with validation errors.


### Date Arithmetic {#date-arithmetic}


Date expressions use `days-between()` and `today-offset()` for reliable cross-platform date arithmetic:

```xpath
days-between(today(), ${dob}) >= 6570    -- Age >= 18 years
. <= today-offset(-10)                     -- Within last 10 days
. >= today-offset(10)                      -- Within next 10 days
```

Direct date subtraction (`today() - ${dob}`) is converted to days-since-epoch automatically. The `_to_numeric()` method in the evaluator handles `date` objects by converting to days since January 1, 1970 before performing arithmetic.


### Multi-Select Functions {#multi-select-functions}


Multi-select questions store selections as space-separated strings in ODK format. The `selected()`, `count-selected()`, and `selected-at()` functions parse this format:

```xpath
selected(${colors}, 'red')          -- Is 'red' selected?
count-selected(${colors}) > 1       -- More than one selected?
selected-at(${colors}, 0) = 'red'   -- First selection is 'red'?
```


### Cross-Platform Parity {#cross-platform-parity}


The same expression engine runs on:

- **Web** — form-service (Python FastAPI)
- **Mobile** — Flutter field app (Dart)

Both implement identical tokenizer, parser, function registry, and evaluation logic. Test suites verify deterministic output across both platforms for all 40+ functions.


## Form Runtime Logic {#form-runtime-logic}


The service implements XLSForm-compatible logic evaluation:

- **Skip Logic (Relevance)** — Dynamically hides/shows questions based on expressions. The survey designer's visual builder generates XPath relevance expressions. During form fill, the engine evaluates relevance on every answer change and updates visible questions.
- **Constraints** — Real-time validation of answers against range, regex, or custom expressions. Constraints run on every answer change. Violations show the constraint message immediately.
- **Calculations** — Auto-computation of hidden fields using expression evaluation.
- **Repeat Groups** — Management of nested, multi-instance question blocks.


### Choice Constraints (Per-Choice Visibility) {#choice-constraints-per-choice-visibility}


Each choice in a select list can have a `constraint` expression defined in the choices sheet of the XLSForm. These per-choice constraints control whether individual options are visible to the respondent.

**Evaluation flow at render time:**
1. The full choice list (including `constraint` fields) is included in the rendered form schema.
2. The frontend (web) and mobile app evaluate per-choice constraints during form fill.
3. Both platforms use the same expression evaluator for deterministic behavior.

**Constraint evaluation order:**
1. **Question-level `choice_filter`** — Filters choices based on extra column values matching question answers.
2. **Per-choice constraint** — Each remaining choice's constraint expression is evaluated against form data. Choices whose constraint returns `false` are removed.
3. **Lookup data filter** — For referee questions with lookup assignments, choices are further restricted.

**Expression context during evaluation:**
- The dot `.` resolves to the choice's `name`.
- The `label` variable resolves to the choice's `label`.
- All other question answers are available by name.
- The choice's extra columns (from the choices sheet) are also available as variables.


## Responsive Design {#responsive-design}


The form rendering engine is optimized for both mobile and desktop browsers:
- **Paging** — Support for single-page and multi-page form layouts.
- **Media Support** — Direct integration with the `media-service` for file uploads.
- **Offline Resilience** — Local caching of form structures to prevent data loss.


## Submission Workflow {#submission-workflow}


1. Respondent fills out the form. Skip logic updates dynamically on every answer.
2. Constraint validation runs on every answer change — violations are shown immediately.
3. On submit, the service validates all constraints server-side.
4. If valid, data is pushed to the `data-service` for permanent storage.
5. `form.submitted` event is published to Redis.