Skip to content

Developers

Building AnselIQ

How AnselIQ is put together, how to build, run and test it from source, and the HTTP API its web app is built on.

Stack

Code layout

PathContents
backend/The API, background workers and scheduler, as one application.
common/Shared models: rules and their transforms, parser configurations, API payloads.
frontend/The web app: pages, components, routing and the API client.
k8s/Kustomize manifests: base, plus overlays/local (Docker Desktop) and overlays/eks (the customer install template).
website/This site.

The backend follows a hexagonal (ports and adapters) architecture under backend/src/main/scala/anseliq:

domain/model
Accounts, movements, reconciliations, specs, file feeds, exceptions, jobs and events.
domain/service
The business logic: Ingestor and the file parsers, Bookkeeper (posting movements), InMemoryReconciler, ManualReconciler (manual matches, unmatch, write-offs), ExceptionService, MatchSuggestionService, JobManager and ReportingService.
domain/port
Interfaces the domain needs from the outside world: repositories, file sources, the match suggester.
adapter/driving
The HTTP routes, health server, and mapping between API payloads and domain models.
adapter/driven
Implementations of the ports: PostgreSQL repositories, the S3 file source, the local LLM client, migrations.
app
Startup: which services this instance runs (HTTP, scheduler, each worker), wired from configuration.

How data flows

  1. Every minute the scheduler queues a CreateFileInventory job, which lists each feed's S3 folder, records new files in the file inventory and queues an Ingestion job for each.
  2. Ingestion parses the file in batches and posts each row as a movement through the movement spec whose routing filter claims it. It checkpoints the last committed line, so a retried job resumes rather than starting over, and raises feed exceptions for unrouted rows or too many parse errors.
  3. Ingestion queues a Reconciliation job for each Recon account it touched. The reconciler streams unreconciled movements from both sides ordered by match key, groups equal keys, and commits reconciliations in batches, also checkpointed. Each alternative of the rule is its own pass over what's left.
  4. If AI matching is on, reconciliation queues a SuggestMatches job for what it left unmatched.
  5. Every minute the scheduler also queues GenerateReconStats, which takes each ledger's daily snapshot once its cutoff hour has passed. Reports read those snapshots.

Jobs live in a PostgreSQL table. A worker claims a job under a lease that a heartbeat renews, so a crashed worker's job is requeued when its lease lapses. Failed attempts are retried with exponential backoff. Unique indexes keep at most one queued and one running job per job type and target, which is what makes it safe to run any number of workers.

The ledger is append-only: corrections are new movements (adjustments and reversals), and every significant change is also recorded as an event in the event store.

Running it locally

Prerequisites

First run

# JWT secret, admin user, LocalStack settings and (on a Mac) JAVA_HOME in .env,
# created the first time and loaded into this shell
eval "$(scripts/dev-env.sh)"

localstack start -d          # S3
make create-db               # an empty recon_db, migrated

# The API on :8080 (health on :8081, workers and scheduler) in the background,
# then the web app on :9876, proxying /api to :8080
sbt "backend/bgRun; frontend/devServer"

Sign in at http://localhost:9876 as the ADMIN_EMAIL from .env. Ctrl+C stops the web app, and sbt backend/bgStopAll the API. In sbt's own prompt, the menu's up and x do the same. For a ledger with ten days of history already in it, stop the backend and run make seed-demo, or make seed-demo-llm to include AI suggestions from a local Ollama.

The backend run from sbt reads .env itself each time it starts, so an edited .env only needs a backend restart. Every sbt in the project connects to one server, which keeps running between commands with the environment it started with; after changing a variable the tests read, run sbt shutdown. The server also runs one command at a time, so while the web app's dev server runs, other sbt commands wait; that's why the API starts in the background first.

Tests

sbt test        # tests that failed or whose code changed since the last run
sbt testFull    # every test

Backend tests use munit against the local PostgreSQL, each rolling back its own transaction. They share the development database, so stop a running backend first: its workers would claim the tests' jobs.

A local database on macOS sorts text bytewise, while PostgreSQL on Linux (CI, RDS) sorts linguistically by default. Anything that depends on sort order must say which it wants (match keys use byte order explicitly), or tests can pass locally and fail in CI.

Database migrations

The schema is managed by Flyway, with migrations in backend/src/main/resources/db/migration. Add a new V<n>__<description>.sql to change it, and never edit one that has been applied anywhere. The backend applies pending migrations at start (DB_MIGRATE_ON_START); sbt "backend/runMain anseliq.app.Main migrate" migrates without starting the app, which is what the Kubernetes init container runs.

Local Kubernetes

k8s/overlays/local runs the full install on Docker Desktop's Kubernetes (Settings → Kubernetes → Enable Kubernetes), with its own PostgreSQL in the cluster and LocalStack on the host for S3:

cp k8s/overlays/local/secrets.env.example k8s/overlays/local/secrets.env   # then fill it in
sbt "backend/Docker/publishLocal; frontend/Docker/publishLocal"
kubectl apply -k k8s/overlays/local

The app is then at http://localhost:8088. After rebuilding the images, kubectl -n anseliq rollout restart deployment picks them up, and kubectl delete namespace anseliq removes everything, the database included.

CI and releases

CI runs the tests on every push to main and on pull requests, against a fresh PostgreSQL and LocalStack. Pushing a version tag such as v1.2.3 on a commit whose CI passed builds and publishes anseliq-backend:1.2.3 and anseliq-frontend:1.2.3. Versions come from git: any other build is versioned like 1.2.3-4-abcd1234 and also tagged dev, which the local overlay deploys.

HTTP API

The web app is built entirely on a JSON API under /api/v1, served on the same origin as the app. One name differs from the app's: a reconciliation rule is a recon spec.

Authentication

Sign in to get a token, then send it as a bearer token on every other request:

curl -s https://anseliq.example.com/api/v1/login \
  -H 'Content-Type: application/json' \
  -d '{"email": "you@example.com", "password": "..."}'
# {"accessToken": "eyJ...", "refreshToken": "..."}

curl -s https://anseliq.example.com/api/v1/ledgers \
  -H 'Authorization: Bearer eyJ...'

A missing or invalid token gets 401, and an Admin-only action by anyone else gets 403.

Endpoints

AreaEndpoints
LedgersGET /ledgers, POST /ledger, PUT /ledger/:id, GET /ledger/:id/summary
AccountsGET /ledgers/:id/accounts, GET /accounts, GET|PUT /account/:id, POST /account
ReconcilingPOST /account/:id/reconcile, GET /account/:id/reconciliations, GET /account/:id/reconciliation-counts
Movement specsGET /ledgers/:id/movement-specs, POST /movement-spec, PUT|DELETE /movement-spec/:id
File feedsGET /file-feeds, POST /file-feed, PUT|DELETE /file-feed/:id, POST /file-feed/:id/test-parse, GET /file-inventory
MovementsPOST /movements/search, GET /movement/:id, GET /movements/:movementSpecId/unreconciled, POST /movements/bad-file-preview, POST /movements/bad-file-fix
RulesGET /account/:id/recon-specs, POST /recon-spec, PUT|DELETE /recon-spec/:id, GET /recon-spec/:id/rule-proposals
ReconciliationsGET /reconciliation/:id, GET /recon-spec/:id/reconciliations, POST /recon-spec/:id/manual-match, POST /reconciliation/:id/unmatch, POST /reconciliation/:id/write-off, POST /reconciliation/:id/reverse-write-off (Admin)
AI suggestionsGET /recon-spec/:id/match-suggestions, POST /match-suggestion/:id/accept, POST /match-suggestion/:id/reject
ExceptionsGET /exceptions, POST /exception, GET /exception/:id, POST /exception/:id/comments, POST /exception/:id/assign
ReportsGET /reports/summary, GET /reports/trend, GET /reports/unreconciled-movements
ActivityGET /jobs/:jobType, GET /events/:eventType
UsersGET /users/assignable; Admin only: GET /users, POST /users, PUT /users/:id, POST /users/:id/reset-password

Request and response bodies are the payload types in common/src/main/scala/anseliq/common/model/Payload.scala.