← Alle Tools

NSAdaSync

Version 26.1 (build 214)

Diese Dokumentation gibt es noch nicht auf Deutsch: Englisch wird gezeigt.

NSAdaSync
Inhalt

What is NSAdaSync

nsadasync (NSAdaSync) is Nacarsoft’s Adabas replication: it keeps Adabas files (Open Systems) in sync with Oracle, PostgreSQL, MySQL, MariaDB, SQLite, MSSQL, MongoDB, Elasticsearch and Kafka without an Event Changes leave the nucleus through a user exit (ADAUEX_4), an in-process supervisor reads them from the FIFO pipe and publishes them to the embedded nsmq broker (in-process, no TCP), and consumers apply each subscription (file → table/collection/topic/index) at the destination, with full initial loads, checkpoints, DLQ replay and auditing.

Adabas nucleus --(UEX4 exit, NSQ2 frames)--> FIFO pipe --(built-in supervisor)--> embedded broker
                                                                                         |
                                         +---------------+---------------+---------------+
                                         |               |               |               |
                                      Oracle/PG/       MongoDB /      Kafka (topics   SQLite
                                      MySQL/MSSQL    Elasticsearch    + tombstones)   (WAL)

NSAdaSync subscriptions: source files streaming into destination tables

Commercial deck: PDF (ES) · PPTX (ES) · PDF (EN) · PPTX (EN)

Database, file, table and destination names in the examples are fictitious.

NSAdaSync fan-out: one Adabas source replicated to Oracle, PostgreSQL, MySQL, MariaDB, SQL Server, SQLite, MongoDB, Elasticsearch and Kafka

NSAdaSync destinations panel: the nine engines with their state, kind and number of subscriptions

Key features

  • Minimal capture in the nucleus: a minimal UEX4 exit (one non-blocking writev per record; never blocks, retries or touches the network) + FIFO + one supervised pipe reader per origin + a persistent fsync-per-batch broker.
  • Nine destinations: Oracle, PostgreSQL, MySQL, MariaDB, SQLite, MSSQL, MongoDB, Elasticsearch and Kafka, with ping/test, table/row inspection and generated DDL. No native clients: built-in drivers.
  • File → table subscriptions: DDM field mapping (JSON), MU/PE as JSON or child tables, ISN identity and upsert with a _seq recency guard (late events never overwrite newer ones).
  • Computed columns: timestamp functions (created_at on insert, updated_at on insert/update) filled automatically at apply time, plus hand-added Adabas fields with their destination data; Align to destination adds the columns without touching data.
  • Initial load + streaming: merge/empty/drop modes, per-destination capacity (max_loads), checkpoints (high_water_seq) and backlog resume once the table exists. Loads first check destination reachability and origin liveness (409 while either is down).
  • Destination and origin panels: reachability, latency, subscriptions and queued records per endpoint, with detail tooltips.
  • Down destination = queued, not failed: unreachable targets park changes on the broker and resume on their own when back, with no DLQ spam.
  • On-demand, cancellable source ↔ target compare by ISN (counters + per-field diffs); origin and destination rows share one viewer.
  • DLQ with replay: failures never lose events; they are inspected and retried, and replay refuses unmapped fields instead of corrupting (deletes replay by ISN, with no image to decode).
  • Load history with per-entry or bulk delete and per-run age (20h:30m ago).
  • Kafka as a streaming sink: one topic per table, ISN key, tombstones on delete (converges under compaction), idempotence and SASL/TLS.
  • Web console in Spanish and English: Dashboard, Origins, Subscriptions, Destinations, Users, Roles, Logs and Config.
  • Machine tokens (nstm_...) with per-subscription scope, expiry and CIDR allowlists (machine API currently disabled; internal traffic uses no tokens).
  • RBAC (15 permissions, Admin/SyncOperators/Viewers roles), every action audited.

Use cases

  • Analytics and BI: replicate to Oracle, PostgreSQL, MySQL or MariaDB and feed reporting live, with no nightly loads and no intermediate files.
  • Event streaming: publish to Kafka with one topic per table, keyed by ISN and a tombstone on delete: it converges with compaction and is idempotent.
  • Search and text: keep one index per table in Elasticsearch, created during the initial load and fed live.
  • Modern applications: MongoDB and SQL Server receive documents and rows that feed APIs and applications without touching the nucleus.
  • Several targets at once: the same subscription is duplicated onto each target (fan-out), or one subscription per file is created.
  • Consolidation: Adabas Open Systems becomes the source and SQL, NoSQL or the streaming platforms become consumers, not the other way round.

Requirements

  • Linux (RHEL 8/9) with prebuilt binaries (no build dependencies).
  • Adabas Open Systems with LOGGING (CLOG) and LOGRB, USEREXITS=4 and ADAUEX_4=<path>/libnsadasync.so (built with exits/build.sh). Important: the LOGGING=(CB) must be effective — a later blank LOGGING = line in the INI disables it silently and the exit receives no changes (symptom: loads replicate but streaming never advances).
  • One dictionary per origin: nsddm JSON (same semantics as nsadabrowse); up to 8 MiB per file.
  • adashow/adarep on the PATH for origin probing (the server only adds /opt/softwareag/Adabas{,/bin} itself).
  • Reverse proxy with HTTPS in production (console :11035; the broker is embedded and listens on no TCP port).

Startup

# Initialise (config.yaml 0600, 0600 db, admin user) and start
./nsadasync init ./data
./nsadasync start ./data
./nsadasync status ./data
./nsadasync stop ./data

Web console at http://host:11035/ (login admin; change the password after the first sign-in). Handy commands: nsadasync passwd [dir] <user> [password] (recover access) and nsadasync license (status, kind, expiry and fingerprint).

Architecture

NSAdaSync architecture: Adabas nucleus, UEX4 exit, FIFO pipe, embedded nsmq broker and one worker per subscription

Three stages between the nucleus and the target, each observable:

  1. Nucleus exit (exits/libnsadasync.so, ADAUEX_4). Per record it emits one binary frame (76 B NSQ2 header + raw CL/FB/RB; legacy NSQ1 still accepted) with a single non-blocking writev. It captures everything except the FNRs in NSADASYNC_EXIT_NOFILES; subscription filtering happens downstream (unsubscribed data is dropped before publishing, never touches disk). Without an explicit pipe it stays inert (fail-closed). On outages it spills to the outage journal (<pipe>.journal, 1 GB cap) which the supervisor replays. The exit version (from exits/VERSION + exits/BUILD) is printed in the init trace. Deletes travel imageless (fb empty): they apply by ISN with no decoding.
  2. Forward supervisor (inside the server; no separate process or token). It keeps one FIFO reader per origin with a pipe_path (reconciles every 30 s), with per-origin spool and journal drain after outages. It publishes to the broker on numeric topics adasync.<dbid>.<FNR>: the pipe/journal rendezvous comes from the Origins row.
  3. Broker + consumers (on the server). The nsmq broker persists one segmented log per topic under data_dir with fsync per batch and enforces retention (retention_bytes 10 GB / retention_hours 72 h). The consumer manager keeps one worker per active subscription (reconciles every 30 s and after every UI change), applies with the destination adapter and advances the high_water_seq checkpoint. When the destination is unreachable the worker parks on the broker with backoff (no checkpoint advance, no DLQ) and resumes when back; when the table is missing it parks the same way until the table exists. Every subscription owns its log (logs/subs/<id>.log, 10 MB rotation) and error counter; debug logs each apply.

Configuration

Origins (source Adabas)

Each origin lives on the Origins page (SQLite), not in config.yaml: name (must be the numeric DBID, e.g. 50: topics are always adasync.<dbid>.<FNR>), dbid, config_file (nsddm JSON), connect, adabas_host/adabas_port (live inspection; empty = localhost, port 0 = adashow TCP port) and pipe_path/journal_path (FIFO + outage-journal rendezvous; an empty journal derives to <pipe>.journal; the pipe must be an absolute path). An origin without a pipe has no streaming (loads and inspection only).

The console validates the dictionary on attach (POST /api/adabas/:db/dictionary, up to 8 MiB, stored at data/ddm/<db>.json), reloads it (POST .../reload), browses files (GET /api/adabas/:db/files), flattened long-name fields (GET .../files/:fnr, with dotted DV.DW paths for PE/MU), the source DDM (GET .../raw?ddm=NAME), flags files without a subscription (GET .../missing-subscriptions, with live counts) and probes the nucleus (GET .../probe, POST .../validate, via adaprobe + adashow/adarep).

Deleting an origin (DELETE, config.manage permission) cascades: its subscriptions (and destination tables when the destination has drop_on_delete), its DLQ, its load history and its broker topics.

Nucleus exit

# On the nucleus host
./build.sh                      # in exits/: builds libnsadasync.so
ADAUEX_4=<path>/libnsadasync.so # + USEREXITS=4, effective LOGGING=(CB), LOGRB

Exit variables: NSADASYNC_EXIT_PIPE (mandatory), NSADASYNC_EXIT_NOFILES="(1,2,3)" (FNRs to skip), NSADASYNC_EXIT_JOURNAL (default <pipe>.journal) and NSADASYNC_EXIT_DEBUG=1|2 + NSADASYNC_EXIT_DEBUG_FILE (0600 trace capped at 64 MB; level 2 dumps truncated FB/RB; the boot marker shows the effective values). Without a pipe the exit stays inert and says so once in the trace.

Destinations

KindConnectionNotes
sqlitepath to the file (created in WAL mode on add)Full DDL + inspection; max_loads 1 by default (serialises writers)
postgres/postgresqlDSN postgres://user:pass@host:5432/dbNative upsert + _seq guard; NULs are dropped in text only (kept in BYTEA)
mysql/mariadbDSN user:pass@tcp(host:3306)/dbNative upsert + _seq guard
mssql/sqlserverDSN sqlserver://user:pass@host:1433?database=dbupsert + _seq guard
oracleDSN oracle://user:pass@host:1521/SIDNo native client (go-ora)
mongodb/mongoDSN mongodb://user:pass@host:27017/db (db mandatory)Documents with natively nested groups; collections are created by the load
elasticsearchDSN http(s)://host:9200[/prefix]One index per table (created by the load); per-field types from the mapping
kafkaDSN kafka://h1:9092,h2:9092[/prefix][?sasl=…&user=…&pass=…&tls=1]Streaming-only: no initial loads or reads; one topic per table ensured on activation; tombstone on delete; delete keeps the topic

Common to all: pe_mode (json|child, default json) for the periodic groups, mu_mode (json|flat|child, default json) for the multiple-value fields, isn_named (ISN_<TABLE> instead of isn), groups_as_json (collapses PE/MU groups into one JSON column instead of flattening), skip_supersub (default true: hides superdescriptors and their components), drop_on_delete (deleting the subscription drops parent + children), max_loads (concurrent loads per destination), max_records (0 = uncapped, caps loads), options (free JSON). GET /api/destinations/drivers reports which driver this build ships; use ping (destinations.view) and test (destinations.manage) before replicating, and browse tables and rows (GET .../tables[/:table/rows]; typed columns on SQL and documents; up to 2048 columns listed on wide collections).

NSAdaSync destinations: all nine kinds with ping and inspection

Console (/config)

A sectioned panel (Server, Mail, License, MQ, Exec…) persisted to config.yaml (0600); saving reports which areas need a restart. Test Email button. Secrets also via environment: NSADASYNC_SMTP_PASSWORD and NSADASYNC_TOKEN_PEPPER (rotating the pepper invalidates every token). Interactive reference at /api/docs + /api/openapi.yaml (apiDocs.access: public|private|off, default private).

SectionMain keys (defaults)
serverhost 0.0.0.0, port 11035, trustedProxies [127.0.0.1, ::1]
authjwtSecret (min 32 chars), tokenPepper, jwtExpiryMin 15, cookieName session, auto secureCookies
mqEmbedded, no TCP; data_dir ./mq-data, retention_bytes 10 GB, retention_hours 72, batch_size 200, batch_timeout_ms 100
sync.defaultsmu_pe json, conflict upsert
correomailer sendmail|smtp, host/port/security/user/password/from, sendmailPath, baseUrl (public URL for links)
execworkDir, 60/300 s timeouts, maxConcurrent 10, 1 MB caps, ratePerMin 600, queueTimeoutSec 20, callRetentionDays 30 (daily audit/DLQ prune; 0 = keep)
logging/license/updates10 MB × 10 rotation, 30 days; trialDays 30; updates URL

Usage

NSAdaSync dashboard: origins, destinations, subscriptions and chain health

The Dashboard shows running loads, destinations and their state (reachability, latency, subscriptions and queued records, with detail tooltips) and origins and their state (nucleus activity and forwarder heartbeat) side by side, plus per-subscription states, I/U/D totals, the change chart, queued records per subscription, chain health and consistency.

Subscriptions

One subscription = one file (origin, FNR, DDM) → (destination, table) with a field mapping, pe_mode, mu_mode, mu_flat_max, ddl_override, is_active and debug. Names are unique (suffixed per destination) and a (destination, table) pair cannot be claimed twice. filter_expr is stored with the subscription (reserved). Empty pe_mode and mu_mode inherit the destination values; mu_flat_max is mandatory when mu_mode=flat.

The mapping is a column JSON [{adabas, column, mode?, type?, length?, precision?, index?, func?}] where adabas is the short DDM name; without an entry the dictionary type and default column apply (sanitised long name, short fallback; on mongo/ES/kafka, the lowercase path). index: false un-indexes even when the DDM marks a descriptor. Recommended flow: Prepare (POST .../prepare, validates the FDT against the dictionary) → Preview DDL (POST .../preview-ddl, no execution) → Ensure table (POST .../ensure-table, creates parent + children + indexes and stores the latest DDL error on the row) → Load → streaming.

Custom fields (+ button in the fields table) open a modal to add an Adabas field (pick from the not-yet-added list with level, short, format and length; set column, type, length, precision, mode and index) or a computed column (created_at: stamps the insert moment; updated_at: insert and update). Computed columns always render last and exist in DDL, loads, streaming and replays. After editing, Align to destination (POST .../align-columns) adds any missing columns without touching data, counters or checkpoint (409 while a load runs or the table does not exist yet).

Duplicate (POST .../duplicate) clones the subscription onto another destination (fan-out, inherits filter and modes); Compare (POST/GET .../compare, cancellable, 409 while one runs) walks by ISN and returns counters (read/total/outdated/missing/extra) + per-field diffs; origin rows render in the same panel as destination rows (table/form, ISN search, paging); debug makes the worker log each apply (errors always) to logs/subs/<id>.log; reset-counters zeroes the counters; the per-subscription log (GET .../log?lines=200, max 1000; DELETE clears it).

StateMeaning
okActive, table exists, origin/destination counts match
diferenteInformational only: the target transiently lags; content is verified with compare
sin_tablaActive without a table: the consumer parks on the broker without DLQ spam; creating the table resumes the backlog (on mongo/ES the load creates it; on kafka the topic is ensured on activation)
cargando / en_colaAn initial load owns the tables (or waits for capacity); live/replay park
errorConsumer error or pending DLQ (see reason)
inactivais_active=false (ON/OFF switch)
desconocidoOrigin unreachable for counting or destination not verifiable

NSAdaSync subscriptions panel: the state of each subscription, the initial load, the I/U/D counters and the control SEQ

Initial loads, journal and DLQ

Loads (POST /api/loads {subscription_id, mode}, loads.run; POST /api/loads/batch for several; GET /api/loads[/:id], POST .../cancel, DELETE .../:id and DELETE ...?subscription_id= to delete finished history — 409 while a load is active): modes merge (default), empty and drop (the latter two require a SQL/mongo/ES destination; kafka refuses the load with 501: it is streaming-only, activating is enough). Before queueing, a load requires a reachable destination and an active origin (409 while either is down) and ready tables (409 instead of dying mid-run). Capacity follows per-destination max_loads (the rest queue); on restart, interrupted loads park as cancelled and queued ones resume on their own (Resume). Each run shows its age (20h:30m ago).

Journal (GET /api/journal/status, journal.read): per-subscription high_water_seq checkpoints. System/Health (GET /api/system, /api/health, plus the aggregate GET /api/health/endpoints with destinations and origins, subscriptions and backlog): global health with destination reachability. MQ (GET /api/mq/status): adasync.<dbid>.<FNR> topics with offsets and checkpoints; inspection (GET .../topics/:topic/messages), purge (POST .../purge {mode: all|to_offset, offset}, mq.manage) and delete (DELETE, mq.manage). Broker retention plus the daily audit/DLQ prune (callRetentionDays) bound disk usage.

DLQ (GET /api/dlq, dlq.view): failed entries; replay (POST /api/dlq/:id/replay) and delete (DELETE, dlq.replay). Replay refuses unmapped fields (unmapped fields… check catalog) instead of corrupting the target; deletes replay by ISN with no image to decode.

Schema changes: no initial load needed

The initial load is only the backfill: once the subscription is streaming, the schema can evolve without loading again. The three usual scenarios:

ScenarioWhat to doInitial load?
Adabas changes (new field in the file, updated DDM)Reload the dictionary (POST /api/adabas/:db/reload), add the field to the subscription mapping and press Apply to destination (POST .../align-columns): it adds the column to the existing table without touching data, counters or checkpointNo
The target changes (new column, corrected type)The upsert respects the column; if the type does not fit, fix it with Apply to destination or with manual DDLNo
The target table is droppedThe subscription goes sin_tabla and the worker parks in the broker (not an error, and it does not pollute the DLQ); recreate it (Prepare → Create table) and the backlog resumesNo

If a field arrives that is neither in the dictionary nor in the mapping, the apply is rejected with unmapped fields… check catalog and the entry goes to the DLQ: fix the mapping and replay it — no reload either.

NSAdaSync subscription editor: field validation, DDL preview, table creation and apply to destination

Note: align-columns only applies to SQL targets (501 on mongo, ES and kafka: there the documents, indices and topics materialise on write).

PE groups and MU multiple fields

The treatment of groups and of multiple fields is chosen apart, per destination and per subscription (the subscription wins over the destination; empty = inherit):

SelectorValuesApplies to
Groups (PE)json | childperiodic groups and groups with subfields
Multiple fields (MU)json | flat | childfields with multiple occurrences (MESES, DES-BENEFICIARIO…)
Max occurrences1…999 (subscription only)mandatory when the MU mode is flat
  • Groups json: the whole periodic group travels as one JSON document (native arrays on MongoDB/ES/Kafka). child: normalized child tables, resolved the same way in the DDL, the loads and the applies.
  • MU fields json: every occurrence in one JSON column. child: one row per occurrence in a child table (VALUE column). flat: the occurrences are expanded into numbered columns of the same table: if MESES has 8 occurrences and the mode is flat with a maximum of 8, the table carries MESES_1, MESES_2 … MESES_8.
-- SQL target with an MU field of 8 occurrences in flat mode (max 8)
CREATE TABLE "CLIENTES" (
  "ISN_CLIENTES" INTEGER PRIMARY KEY,
  "MESES_1" TEXT, "MESES_2" TEXT, "MESES_3" TEXT, "MESES_4" TEXT,
  "MESES_5" TEXT, "MESES_6" TEXT, "MESES_7" TEXT, "MESES_8" TEXT,
  ...
);

Details of the flat mode of an MU field:

  • The number of occurrences is set per subscription (mu_flat_max, 1…999) and is mandatory when the mode is flat; without it the API answers 400.
  • When a record is applied, the 1st occurrence goes to MESES_1, the 2nd to MESES_2, and so on; the missing ones stay NULL.
  • If a record carries more occurrences than the maximum, the first ones are kept and the rest are dropped with a warning in the subscription log (logs/subs/<id>.log): replication keeps running.
  • It works the same in SQL and in documents (MongoDB, Elasticsearch, Kafka): there the field is stored with the same numbered names.
  • The DDL is generated with the numbered names, and Apply to destination (align-columns) adds the missing columns without touching data, counters or checkpoint.
  • The origin ↔ target compare checks every occurrence against its column.

The explicit mode of a field in the mapping (the “Mode” column) wins over the selector: an MU field can be marked flat even when the destination uses json. With groups_as_json the destination also collapses the plain groups into one column.

The identity is isn (or ISN_<TABLE> with isn_named) and the conflict is resolved with upsert + the _seq guard (a late event never overwrites a newer one).

mu_pe: json stores the whole periodic group as one JSON document (native arrays on MongoDB/ES/kafka); child builds normalized child tables (resolved identically in DDL, loads and applies). With groups_as_json the destination collapses groups into one column. Identity is isn (or ISN_<TABLE> with isn_named) and conflicts resolve with upsert + the _seq guard (a late event never overwrites a newer one).

Command line

nsadasync init ./prod                        # project (0600 config, db, admin)
nsadasync token ./prod --name poller --scope VENTAS --days 365 --cidr 10.0.0.0/8
nsadasync token ./prod --list                # prefix, name, scope, expiry, revoked, last use
nsadasync token ./prod --revoke nstm_a1b2c3d4
nsadasync passwd ./prod admin [newpass]      # emergency (min 8 chars)
nsadasync start|stop|status ./prod [--port N]
nsadasync license                            # status, kind, expiry, fingerprint

Email notifications

NSAdaSync can notify by email when something changes in the replication. The engine evaluates the state (destinations, origins, subscriptions, DLQ, loads and licence) and notifies only when something changes, not on every check; while the problem lasts it reminds every reminder (4 h by default) and the recovery notice goes to the same recipients as the original one.

Enable and tune it in the console (Configuration → Notifications) or in config.yaml (the notify section):

notify:
  enabled: true
  methods: email
  recipients:
    critical: ["dba@empresa.com"]
    warning: ["data-team@empresa.com"]
    info: []
  intervalSec: 60
  reminder: 4h
  quiet: { enabled: false, from: "22:00", to: "07:00" }
  rules:
    destination_down: { enabled: true, severity: critical }
    lag_high: { enabled: true, severity: warning, threshold: 1000 }
EventSeverityWhen it fires
destination_downcriticala destination does not answer (changes queue up)
origin_stoppedcriticalan origin is stopped with active subscriptions
dlq_pendingcriticalthere are changes that could not be applied
load_failedcriticalan initial load ends in error
subscription_errorwarninga subscription stays in error
table_missingwarningthe target table is missing (backlog waiting)
lag_highwarningmore queued changes than the threshold
destination_slowwarningtarget latency above the threshold
broker_diskwarningthe queue broker is near its disk limit
license_expiringwarningthe licence expires in a few days
login_failureswarningseveral failed sign-in attempts in a row
destination_recoveredinfoa destination comes back (the backlog is applied)
load_okinfoan initial load finishes successfully
subscription_changeinfoa subscription is created or changed

From the CLI: nsadasync notify test [address] sends a test notice and nsadasync notify status shows the rules, the open incidents and the last notices sent. The history is stored and available in the console and at GET /api/notifications.

Programming

NSAdaSync brings its own scheduling engine (it does not use the system cron): every 30 s it evaluates the rules and fires the ones that are due. In the console it lives under Programming (side menu) and the Panel shows how many rules are programmed and when the next one runs.

Maintenance window: pauses the replication of an origin, a destination or all of them between two times (daily, weekly, monthly or one-off recurrence). During the window the changes queue in the broker and are applied when it ends. When the window opens, the previous state of every endpoint is stored, so:

  • what was already stopped by hand is left alone;
  • when it closes, only what the window stopped is resumed;
  • the console tells a manual stop from a scheduled pause apart.

Scheduled tasks: an initial load (merge/empty/drop) or an origin-vs-destination compare of a subscription, with a time and a recurrence. If a load is already running, the task is skipped and an e-mail notice is sent; with catch_up it runs even if the time passed while the server was down.

Bulky data is not stored in the database: the detail of a compare (up to 200 differences with values) is written to a JSON file in the data directory (compares/*.json); the database keeps the summary (counters + an excerpt of 10 differences + the file path) and the execution history, bounded.

Every e-mail notice (window opened and closed, task skipped, error, finished load and compare result) carries the date and time of the event.

GET  /api/schedules · /api/schedules/summary · /api/schedules/runs
GET  /api/schedules/compares   (summaries of the compares)
POST /api/schedules · POST /api/schedules/:id/run
PUT/DELETE /api/schedules/:id

The history is bounded and pruned with the usual retention (30 days by default): the notices, the executions (the last 20 per rule), the compare summaries and the compares/*.json detail files.

From the CLI: nsadasync schedule list and nsadasync schedule rm <id>.

Users, groups and permissions

Single-tenant RBAC: system roles Admin (everything), SyncOperators (operate subscriptions, destinations, journal, loads and DLQ: dashboard.view, subscriptions.view, destinations.view, journal.read, loads.run, dlq.view, dlq.replay, calls.view, logs.view) and Viewers (read-only); groups Administrators, Operators, Readers. The catalog refreshes on every boot. Users (GET/POST /api/users, PUT/DELETE /api/users/:id, PUT .../active, PUT .../groups, all under users.manage); roles and groups are listed (GET /api/roles, /api/groups). Email recovery (/forgot-password, POST /api/auth/forgot-password, POST .../reset-password) when mail is configured; otherwise nsadasync passwd.

PermissionCovers
dashboard.viewDashboard and states
subscriptions view|manageView / create, edit, delete, ensure-table, align-columns, duplicate, debug
destinations view|manageView, ping / create, edit, delete, test
journal.readJournal, system, health (+endpoints), MQ (read)
mq.manageTopic purge and delete
loads.runStart, cancel and delete loads
dlq view|replayView / replay and delete entries
calls.view, logs.viewCalls and audit/logs
tokens view|manageView / create, revoke and delete tokens
users.manageUsers, roles and groups
config.manageConfiguration, mail test, origins and dictionaries

Machine tokens (nstm_..., shown only once) carry a subscription or all scope, expiry (--days, 0 = never) and source CIDR/IP allowlists; they are stored as HMAC-SHA256 with the pepper (rotating it invalidates them all). The machine API (/api/ext) is disabled in this version: all traffic is local console or loopback.

POST /api/auth/login · /api/auth/refresh · POST /api/auth/logout
POST /api/auth/forgot-password · POST /api/auth/reset-password
GET  /api/auth/me · POST /api/auth/{theme,language}
GET  /api/version · /api/license · /api/docs · /api/openapi.yaml
GET  /api/subscriptions · /api/subscriptions/states · /api/subscriptions/check-name
POST /api/subscriptions · PUT/DELETE /api/subscriptions/:id
GET  /api/subscriptions/:id/log · DELETE .../log
POST /api/subscriptions/:id/{debug,reset-counters,ensure-table,align-columns,prepare,duplicate}
POST /api/subscriptions/{preview-ddl,validate-fields}
POST /api/subscriptions/:id/compare · GET .../compare · POST .../compare/cancel
GET  /api/subscriptions/:id/origin-rows
GET  /api/adabas/databases · GET /api/adabas/:db/files[/:fnr[/raw]]
GET  /api/adabas/:db/missing-subscriptions · GET /api/mapping-template
PUT/DELETE /api/adabas/databases/:db · POST /api/adabas/:db/{dictionary,reload}
GET  /api/adabas/:db/probe · POST /api/adabas/:db/validate
GET  /api/destinations · /api/destinations/drivers
POST /api/destinations · PUT/DELETE /api/destinations/:id
POST /api/destinations/:id/ping · POST /api/destinations/test
GET  /api/destinations/:id/tables[/:table/rows]
POST /api/loads[/batch] · GET /api/loads[/:id] · POST /api/loads/:id/cancel
DELETE /api/loads/:id · DELETE /api/loads?subscription_id=
GET  /api/journal/status · /api/system · /api/health · /api/health/endpoints
GET  /api/mq/status · GET /api/mq/topics/:topic/messages
POST /api/mq/topics/:topic/purge · DELETE /api/mq/topics/:topic
GET  /api/dlq · POST /api/dlq/:id/replay · DELETE /api/dlq/:id
GET  /api/users · /api/roles · /api/groups · /api/tokens · /api/audit
GET  /api/config · PUT /api/config · GET /api/config/disk · POST /api/config/test-email

Everything lands in the audit log (LOGIN/LOGOUT/LOGIN_FAILED, SUBSCRIPTION_*, DESTINATION_*, ORIGIN_DELETE, TABLE_CREATE/TABLE_DROP*/TABLE_ALIGN, LOAD_*/LOADS_DELETE, PREPARE, MQ_TOPIC_*, DLQ_*, TOKEN_*, USER_*, CONFIG_SAVE, EMAIL_TEST, PASSWORD_*, with client IP). Sessions last jwtExpiryMin (default 15 min); on expiry the console returns to login.

Security

  • HS256 JWT with configurable expiry (default 15 min; no silent refresh: expiry returns to login), brute-force protection on login/reset, bcrypt passwords (cost 12) and single-use reset tokens.
  • config.yaml (0600) and DB (0700); secrets redacted from GET /api/config and never logged; machine tokens as HMAC hash + visible prefix, with CIDR allowlists and expiry.
  • server.trustedProxies (loopback by default) so audit/rate-limit IPs cannot be spoofed; auto Secure cookies (TLS/X-Forwarded-Proto); Content-Security-Policy, X-Content-Type-Options, Referrer-Policy and Permissions-Policy headers.
  • Names of subscriptions/tables, DDL, DSNs and paths (config_file, path, pipe/journal) are validated and sanitised; topics are always numeric and there is no remote agent here (everything is local).

Installation state and control

  • Start and stop: nsadasync init|start|stop|status <dir>; the process starts the forward supervisor, the embedded broker and the workers of the active subscriptions, and stops them cleanly.
  • State: nsadasync status <dir> and the console panel (origins, destinations, subscriptions, queued records, DLQ, chain health).
  • Resumption: on restart, interrupted loads are parked as cancelled and queued ones resume on their own; the broker backlog is applied in order and every subscription continues from its checkpoint.
  • CLI operations: nsadasync passwd <dir> <user> [password] (recover access), nsadasync token <dir> --name … --scope … --days … (machine tokens) and nsadasync license (installation status).
  • Auditing: every action is recorded in the console (LOGIN, SUBSCRIPTION_*, DESTINATION_*, LOAD_*, DLQ_*, TOKEN_*, USER_*, CONFIG_SAVE…) with user and client IP, and in the log of each subscription (logs/subs/<id>.log).

Troubleshooting

  • Exit emits nothing (loads replicate but streaming stalls): besides ADAUEX_4, USEREXITS=4 and CLOG+LOGRB, check that LOGGING=(CB) is effective: a later blank LOGGING = line in the INI disables it silently (it does not show in PARSET) and the exit receives no changes. With NSADASYNC_EXIT_DEBUG_FILE you get the init trace and every pipe/journal event.
  • adaopr db=<id> disp=hi aborts with Signal 8: a Software AG bug (divide-by-zero in dis_hwm of libadautl.so while formatting high-water marks), not the exit: the nucleus stays alive. Avoid that display.
  • Forwarder publishes nothing: check the Origins row pipe_path / journal_path, the per-origin spool and that the supervisor is reading the FIFO ([FORWARD] log); with no reader the exit spills to the journal and the supervisor replays it on return.
  • Down destination: changes queue on the broker (see the per-point backlog on the destinations panel) and resume on their own; nothing goes to the DLQ. Check with ping/test, GET /api/health/endpoints and GET /api/destinations/drivers (the build may lack that driver). Initial loads refuse with 409 until the target answers.
  • Stopped origin: the initial load refuses with 409; streaming simply receives no more changes (the backlog is kept).
  • sin_tabla subscription: parking, not an error — create the table (ensure-table) and the broker backlog resumes (on kafka, just activate: the topic is ensured alone).
  • Permanent diferente: normal while live (transient lag); if content diverged, run an ISN compare.
  • DLQ with unmapped fields: the DDM does not cover those fields — extend the dictionary or the mapping, then replay.
  • Load stuck in cargando/en_cola: see GET /api/loads/:id (per-destination max_loads capacity); cancel and relaunch; on restart, interrupted loads park themselves. History deletes per entry or in bulk (DELETE /api/loads[/:id]).
  • Edited the mapping and columns are missing: use Align to destination (only adds columns, resets nothing).
  • Compare 409: one compare per subscription at a time; cancel it or wait.
  • Rejected token: check scope, CIDR and expiry; if you rotated the pepper, reissue every token. The machine API is disabled in this version.
  • No files on the origin: check config_file (valid nsddm JSON, ≤ 8 MiB) and connect; use the origin probe/validate.
  • Expired session: back to login (no silent refresh); tune jwtExpiryMin for long sessions.
  • Port in use: change server.port (11035) in config.yaml.

Auch verfügbar in: es