# Blueprint for Production-Grade eFTI Platform Setup

This document is a deployment blueprint and AI-generation prompt for building a
production-grade eFTI platform aligned with EU eFTI requirements and Fintraffic
integration needs. However, user have to always check, is the application ready
for production use and make decissions.

When asking Ai tool to run this blueprint, you will have code for application,
guide to deploy that to cloud and guide to run application in your own computer.

## Part 1: System Requirements & AI Software Generation Prompt

### 1. Objective & Regulatory Framework

Act as a Lead Enterprise Architect and EU eFTI Security Expert. Generate a
secure, production-grade, full-stack eFTI (electronic Freight Transport
Information) platform enabling logistics operators to manage datasets and
support standardized, pull-based data exchange with competent authorities via
Fintraffic's national eFTI Gate.

The architecture, data models, and logic MUST align with the following
authoritative sources:

- Regulation (EU) 2020/1056 (core eFTI framework)
- Delegated Regulation (EU) 2024/2024 (eFTI Common Data Set and eFTI data
  subsets)
- Implementing Regulation (EU) 2025/2243 (technical specifications and Gate
  access)
- GEFeg eFTI Common Data Set publication, especially DS1 as the authoritative
  semantic source for the Common Data Set:
  https://svn.gefeg.com/svn/efti-publication/Draft/CDS/ds1.htm
- GEFeg subset pages as the authoritative source for subset composition
- Fintraffic API documentation and examples:
  - API overview: https://efti.fintraffic.fi/en/api/
  - Consignment identifier:
    https://efti.fintraffic.fi/en/consignment-identifier/
  - Problem: https://efti.fintraffic.fi/en/problem/
  - Consignment common: https://efti.fintraffic.fi/en/consignment-common/
  - Follow-up: https://efti.fintraffic.fi/en/follow-up/

Source precedence rule for integration implementation:

- When behavior or payload examples differ between generic eFTI/GEFeg examples
  and Fintraffic Gate interface behavior, follow Fintraffic documentation and
  schema validation outcomes for Fintraffic-facing endpoints.

### 2. Full-Stack Production Architecture

Separate the generated codebase into clean, independent directories:

- `/frontend`: a modern, responsive single-page application (SPA) built with
  React, Vue, or another robust framework. It must support dynamic data entry
  forms that can adapt to the selected eFTI subset.
- `/backend`: a secure REST API backend (FastAPI, .NET, or Node.js/Express) that
  handles business logic, Fintraffic API integrations, subset validation, and
  mTLS communication.
- `/database`: PostgreSQL schemas for eFTI datasets, logs, code lists, and
  retention handling.
- `/auth`: dedicated Identity and Access Management (IAM) with role-based access
  control (RBAC).

### 3. Authoritative Data Model Mapping Principles

The platform MUST treat the GEFeg Common Data Set pages as the authoritative
source for semantic data model mapping.

#### 3.1 Mapping source of truth

- Use the GEFeg DS1 Common Data Set pages as the source of truth for eFTI data
  structures, including ABIE, ASBIE, and BBIE entities and their semantic
  definitions.
- Preserve traceability from internal fields to eFTI element identifiers
  wherever possible.
- Store code list values according to the referenced code lists.
- Do not invent fields, cardinalities, or semantics that are not present in the
  authoritative eFTI sources.

#### 3.2 Internal persistence model

The application may implement the internal persistence model freely, provided
that it remains traceable to the authoritative eFTI source model. The
implementation may use:

- relational tables
- JSON/JSONB structures
- hybrid normalized + document approaches

A recommended pattern is:

- a canonical dataset entity for stored transport information
- subset metadata describing which authoritative subset is used
- traceability metadata linking stored fields to eFTI element identifiers

#### 3.3 Subsets and subset handling

- The authoritative definition of subsets comes from the GEFeg subset
  publications and the applicable delegated regulation.
- Subset selection logic is an application-internal responsibility.
- The user may choose which authoritative subset is used for the dataset.
- The application must validate that the selected subset exists and that the
  required fields for that subset are present before exposing the dataset
  externally.
- The application must not redefine subset content; it must only implement
  handling and validation of authoritative subsets.

### 4. API Contract, Integration Behaviour and Data Examples

This section adds concrete implementation anchors based on the published OpenAPI
and schema references.

#### 4.1 API design principles

- Treat the Fintraffic/eFTI API pages as integration examples and contract
  guidance.
- Keep a strict separation between:
  - internal persistence model
  - external API contract
  - XML/XSD payloads used in gate/platform exchanges
- Use request correlation headers such as `X-Request-ID` for authority-driven
  retrieval and follow-up workflows.
- Represent dataset identity separately from searchable identifiers.
- For Fintraffic-facing exchange endpoints, treat Fintraffic schema
  acceptance/rejection responses as authoritative implementation feedback.

#### 4.2 Required capabilities reflected in the published API examples

The implementation should support these externally visible capabilities:

1. **Register identifiers for a dataset**
2. **Find datasets by identifier**
3. **Retrieve a dataset by UIL / dataset identity with one or more subset
   filters**
4. **Send follow-up messages linked to an earlier dataset request**
5. **Return clear error objects for invalid requests and integration failures**

#### 4.3 Contract anchors from the published examples

The published OpenAPI example contains the following contract anchors:

- `POST /identifiers/{datasetId}` for platform-side registration of consignment
  identifiers using XML content
- `GET /identifiers/{identifier}` for authority-side identifier lookup with
  optional query parameters such as `modeCode`, `identifierTypes`,
  `registrationCountryCode` and `dangerousGoodsIndicator`
- `GET /dataset/{gateId}/{platformId}/{datasetId}` for dataset retrieval using
  one or more `subsetId` query parameters and an `X-Request-ID` header
- `POST /follow-up/{gateId}/{platformId}/{datasetId}/{datasetRequestId}` for
  sending a follow-up message related to a previous dataset query

#### 4.4 Data formats explicitly referenced by the published examples

- Identifier registration request body is XML under the consignment identifier
  namespace.
- Dataset retrieval uses dataset identity values (`gateId`, `platformId`,
  `datasetId`) plus one or more `subsetId` values.
- Identifier query responses contain dataset discovery metadata including a
  `uil` object with `datasetId`, `gateId`, and `platformId`.
- The published schema references point to:
  - consignment identifier XSD
  - consignment common XSD
  - example XML documents for both identifier and common dataset forms

#### 4.5 Suggested application-side endpoint mapping

Implement an internal application API that mirrors the external contract while
keeping internal storage technology independent:

- `POST /api/v1/datasets`
- `GET /api/v1/datasets/{datasetId}`
- `POST /api/v1/identifiers/{datasetId}`
- `GET /api/v1/identifiers/search`
- `POST /api/v1/follow-up/{datasetId}/{datasetRequestId}`

#### 4.6 Example: identifier registration payload model

The published documentation states that identifier registration uses XML under
the identifier namespace and refers to the identifier XSD. A minimal
implementation skeleton can therefore treat identifier registration as an XML
payload that is traceable to the identifier subset schema.

Example placeholder XML skeleton (implementation example, not authoritative
source text):

```xml
<consignment xmlns="http://efti.eu/v1/consignment/identifier">
  <!-- identifier subset payload aligned to the published identifier XSD -->
</consignment>
```

#### 4.7 Example: identifier discovery response shape

The published OpenAPI example shows that identifier lookup returns an array of
JSON objects that can include:

- `carrierAcceptanceDateTime`
- `deliveryEvent.actualOccurrenceDateTime`
- `identifierCountryOfOrigin`
- `mainCarriageTransportMovement`
- `usedTransportEquipment`
- `uil.datasetId`
- `uil.gateId`
- `uil.platformId`

Example implementation-oriented JSON skeleton derived from the published field
names:

```json
[
  {
    "uil": {
      "datasetId": "11111111-1111-1111-1111-111111111111",
      "gateId": "fi-gate-1",
      "platformId": "demo-platform"
    },
    "identifierCountryOfOrigin": "FI",
    "mainCarriageTransportMovement": [
      {
        "modeCode": "3",
        "dangerousGoodsIndicator": false,
        "usedTransportMeans": {
          "id": { "value": "ABC-123" },
          "registrationCountry": { "code": "FI" }
        }
      }
    ]
  }
]
```

#### 4.8 Example: dataset retrieval request shape

The published OpenAPI example shows dataset retrieval using path parameters and
one or more `subsetId` query parameters plus `X-Request-ID`.

Implementation-oriented HTTP example:

```http
GET /dataset/fi-gate-1/demo-platform/11111111-1111-1111-1111-111111111111?subsetId=FI01
X-Request-ID: req-12345
```

#### 4.9 Example: follow-up request shape

The published OpenAPI example shows a follow-up endpoint using `gateId`,
`platformId`, `datasetId`, `datasetRequestId`, and an `X-Request-ID` header. The
request body content type is JSON.

Implementation-oriented JSON example:

```json
{
  "message": "Please clarify the transport means registration data in the previously returned dataset.",
  "referenceRequestId": "req-12345"
}
```

#### 4.10 Problem responses

If the chosen implementation exposes JSON error payloads, prefer a consistent
problem response model. At minimum, define and document the problem fields your
API returns. If the Fintraffic `/problem/` page is used as source guidance
later, update the exact contract accordingly.

Suggested implementation shape:

```json
{
  "type": "about:blank",
  "title": "Invalid request",
  "status": 400,
  "detail": "subsetId is required",
  "instance": "/api/v1/datasets/11111111-1111-1111-1111-111111111111"
}
```

#### 4.11 Source anchors for Section 4

- Fintraffic API pages: https://efti.fintraffic.fi/en/api/,
  https://efti.fintraffic.fi/en/consignment-identifier/,
  https://efti.fintraffic.fi/en/problem/,
  https://efti.fintraffic.fi/en/consignment-common/,
  https://efti.fintraffic.fi/en/follow-up/
- Published sandbox OpenAPI: https://eu-ee32.eftisandbox.eu/v1/openapi
- Reference implementation XSD README:
  https://github.com/EFTI4EU/reference-implementation/blob/main/schema/xsd/README.md
- Fintraffic model visualizer:
  https://model.fintraffic-efti-dev.aws.fintraffic.cloud/
- Delegated regulation for subset content:
  https://eur-lex.europa.eu/eli/reg_del/2024/2024

### 5. Core Functional & Compliance Features

#### 5.1 Identifiers

- Automated generation of Unique Identification Links (UIL) and UUIDs mapped to
  `eftiDataSetId`.
- Support searchable identifier registration and retrieval consistent with the
  relevant Fintraffic/eFTI identifier mechanisms.

#### 5.2 Retention policy

- Implement automated data lifecycle handling where transport datasets and
  access logs are retained for six full calendar years plus the current year.
- Retention handling must be implemented structurally in database archiving or
  partitioning logic.

#### 5.3 Cryptographic module

- Provide a backend utility or command-line script to generate a private key and
  Certificate Signing Request (CSR).
- Support mTLS onboarding and certificate-based communication with the
  Fintraffic eFTI Gate.

#### 5.3a Bidirectional mTLS Requirement (Gate Integration)

For Fintraffic Gate integration, mTLS must be enforced in both directions:

- **Outbound (platform -> Gate):** backend must present platform client
  certificate and private key when calling Gate APIs.
- **Inbound (Gate -> platform):** platform must validate Gate client certificate
  before allowing external dataset retrieval processing.

Minimum implementation expectations:

- Keep certificate material in a secrets manager, never hardcoded in source.
- Enforce certificate verification at ingress (ALB mutual-auth trust store or
  equivalent mTLS-terminating proxy).
- Propagate trusted verification state to backend only from controlled ingress
  components.
- Reject unauthenticated Gate-originated retrieval calls when mTLS verification
  is missing or invalid.

#### 5.3b XML Import Capability

Implement XML file import functionality for bulk or batch eFTI dataset creation:

**Backend Service (`backend/app/services/xml_import.py`):**

- `parse_xml_file(xml_content: bytes)` → (subset_id, extracted_data_dict)
  - Detects subset type from XML root element and namespaces
  - Extracts eFTI field values using namespace-aware XPath queries
  - Supports both FI01 (Consignment) and ID01 (Identifier) formats
  - Maps XML elements to internal dataset payload fields
- `validate_extracted_data(subset_id, data)` → (is_valid, errors_list)
  - Validates extracted data against subset requirements
  - Checks for required fields per subset
  - Returns validation errors or success status
- XPath-based field extraction with fallback paths for compatibility
- Namespace handling with support for GEFeg namespaces

**API Endpoint (`POST /api/v1/datasets/import/xml`):**

- Accepts multipart form with XML file upload
- Returns: `{status, message, subset_id, extracted_data, field_count, preview}`
- Preview includes key fields: shipperName, recipientName, origin, destination,
  transportModeCode, goodsDescription
- All extracted data returned for UI preview before dataset creation
- HTTP 400 for XML parsing errors or validation failures
- HTTP 422 for data validation errors with detailed field-level error messages

**Frontend UI (`frontend/src/App.jsx`, `frontend/src/validation.js`):**

- New "📥 Import XML" tab in main navigation
- File upload input with drag-and-drop support
- Real-time loading indicator during XML processing
- Preview table showing key extracted fields
- Scrollable list of all extracted fields with values (max 300px height)
- "Create Dataset from XML" button to populate form and continue with normal
  flow
- Cancel button to discard import and reset
- Extracted data automatically mapped to transportation, identifier, and custom
  fields
- Full form validation applied before final dataset creation

**Data Flow:**

1. User uploads XML file via Import tab
2. Frontend sends file to backend via multipart form
3. Backend parses XML, detects subset, extracts fields using XPath
4. Backend validates extracted data against subset requirements
5. Backend returns extracted data + preview summary
6. Frontend displays preview with all fields
7. User confirms import → form populated with data
8. User completes dataset creation with standard flow (validation, submission)
9. Dataset created with XML-derived data

**Error Handling:**

- Invalid XML: descriptive parsing error message
- Unknown format: error listing root element
- Missing required fields: validation errors per subset
- Empty file: descriptive error

**Integration Points:**

- Uses same validation rules as manual dataset creation
- Integrates with field validation framework (60+ fields)
- Respects subset definitions and required fields
- Does not bypass any programmatic validation

#### 5.4 Field Validation Framework

Implement a comprehensive field validation system that ensures all eFTI data
fields conform to their specified formats, types, and constraints:

**Backend Validation (`backend/app/services/field_validation.py`):**

- Define validation rules for 60+ eFTI fields covering:
  - **Type validation:** STRING, EMAIL, DATE, DATETIME, NUMERIC, COUNTRY_CODE,
    LANGUAGE_CODE, PHONE, URL, CODE_LIST
  - **Format constraints:** regex patterns, length limits, min/max values,
    allowed code list values
  - **Field examples:**
    - contactEmail (EMAIL type, an..35)
    - estimatedArrival (DATE, formatId 102-205)
    - weight (NUMERIC, n..16,6, unit: GRM/KGM/TNE per CL-031)
    - transportMode (CODE_LIST per CL-037: 1=Maritime, 2=Rail, 3=Road, 4=Air,
      8=Inland Water)
    - undgCode (CODE_LIST per CL-008: 0004-9999, UN Dangerous Goods number)
    - technicalName (TEXT, an..128, mandatory per ADR/RID/ADN if dangerous goods
      present)
    - hazardClassificationID (IDENTIFIER, an..7, per CL-016)
- Integration points:
  - `POST /api/v1/datasets` - validate all payload fields before dataset
    creation
  - `PATCH /api/v1/datasets/{id}` - validate all updated payload fields before
    modification
  - Returns HTTP 422 with detailed field-specific error messages on validation
    failure
- Source of truth: Field rules derived from authoritative eFTI sources (GEFeg
  Common Data Set, Fintraffic specifications)

**Frontend Validation (`frontend/src/validation.js`):**

- Client-side mirror of backend validation rules for real-time user feedback
- Functions: `validateField(fieldName, value)` for single-field validation,
  `validatePayload(payload)` for full-form validation
- Real-time error display as users type with visual feedback (red border, error
  messages)
- Form submission prevention until all validation passes
- Improved UX with inline error messages below invalid fields

**UX Integration (`frontend/src/App.jsx`, `frontend/src/styles.css`):**

- Real-time validation on field change with error state tracking in
  `fieldValidationErrors`
- `.field-error` CSS class: red border + light red background for invalid fields
- `.field-error-message` class: small red text showing specific validation error
  below each field
- Updated field handlers to validate and display errors immediately across:
  - Transportation information fields (shipperName, recipientName, origin,
    destination, transportModeCode, vehicleRegistration,
    transportDocumentNumber, goodsDescription)
  - Dangerous goods conditional fields (undgCode, technicalName,
    hazardClassificationID, packagingDangerLevelCode)
  - Waste conditional fields (wasteMaterialID, wasteMaterialWeight,
    wasteMaterialWeightUnit)
  - Custom eFTI fields (dynamically added via field suggestions)

**Validation Scope:**

- Applied to all dataset create and update operations
- Covers standard transportation info and custom fields added via field
  suggestion mechanism
- Does NOT override programmatic subset validation (which checks required fields
  per subset)
- Complements subset validation by ensuring field values conform to eFTI data
  type requirements

#### 5.5 Dataset Creation Form Structure

Implement a streamlined, user-friendly dataset creation form with automatic ID
assignment and flexible transport type selection:

**Form Components:**

1. **Dataset Identification (Auto-assigned)**
   - eFTI Dataset ID automatically generated on form creation (format:
     `EFTI-{timestamp}-{random}`)
   - Displayed as read-only text to user, no manual input required
   - Ensures globally unique dataset identifiers without user effort

2. **Transport Type Selection (Checkboxes - Independent)**
   - Checkbox: "Contains Dangerous Goods"
   - Checkbox: "Contains Waste"
   - Both checkboxes can be selected independently and simultaneously
   - Allows a single dataset to indicate both dangerous goods AND waste
     transport
   - No mutually exclusive restrictions

3. **Transportation Information Fieldset (Core consignment details)**

   **Party Information — Fintraffic consignment-common.xsd structure:**

   - **Shipper / Consignor Party** (Fintraffic: `<consignor>` element)
     - Simple display: "Shipper Name" (Text, an..35, required) — maps to
       Fintraffic `<consignor><name>`
     - Advanced mapping (internal storage for Gate export):
       - Name (eFTI51 - Trade Party Name Text)
       - ID / Business Identifier (eFTI49 - Trade Party Identification
         Identifier, an..17)
       - Postal Address: street, city, postcode, country code (eFTI54-eFTI59)
       - Contact Person: given name, family name (eFTI1152, eFTI1154)
       - Contact Details: telephone (eFTI52), email (eFTI53)
       - Tax Registration ID (eFTI859) if applicable
     - Internal JSON storage: `consignor_full_record` captures all above fields

   - **Recipient / Consignee Party** (Fintraffic: `<consignee>` element)
     - Simple display: "Recipient Name" (Text, an..35, required) — maps to
       Fintraffic `<consignee><name>`
     - Advanced mapping (internal storage for Gate export):
       - Name (eFTI68 - Trade Party Name Text)
       - ID / Business Identifier (eFTI66 - Trade Party Identification
         Identifier, an..17)
       - Postal Address: street, city, postcode, country code (eFTI72-eFTI77)
       - Contact Person: given name, family name (eFTI1156, eFTI1158)
       - Contact Details: telephone (eFTI71), email (eFTI1457)
       - Tax Registration ID (eFTI1458) if applicable
     - Internal JSON storage: `consignee_full_record` captures all above fields

   **Location Information:**

   - **Origin / Carrier Acceptance Location** (Text, an..35, required) —
     Fintraffic: `<carrierAcceptanceLocation><name>`
     - Format: City/Port name per GEFeg (eFTI138)
     - Optional: UN/LOCODE (eFTI136) stored as location ID if available

   - **Destination / Consignee Receipt Location** (Text, an..35, required) —
     Fintraffic: `<consigneeReceiptLocation><name>`
     - Format: City/Port name per GEFeg (eFTI154)
     - Optional: UN/LOCODE (eFTI152) stored as location ID if available

   **Main Carriage Transport Movement:**

   - **Transport Mode Code** (Code, n1, required) — Fintraffic:
     `<mainCarriageTransportMovement><modeCode>`
     - GEFeg CL-037 values: 1=Maritime, 2=Rail, 3=Road, 4=Air, 8=Inland Water
       (eFTI581)
     - Stored in internal record as `mode_code`

   **Transport Equipment & Documents:**

   - **Vehicle/Transport Equipment Registration** (Text, an..17) — Fintraffic:
     `<utilizedLogisticsTransportEquipment><id>`
     - Identifier of transport equipment used (eFTI374)
     - Optional category code (eFTI378 - e.g., "AE" for Body trailer, "SM" for
       Semi-trailer)

   - **Transport Document Reference Number** (Text, an..17) — Fintraffic:
     `<transportDocument><id>`
     - Reference to transport document such as air waybill, bill of lading, or
       road consignment note (eFTI170)

   **Goods Information:**

   - **Consignment Goods Description** (Text, an..512) — Fintraffic:
     `<natureIdentificationCargo><identification>`
     - Free-text or coded description of cargo nature (eFTI699)
     - Supports both narrative and code-based representation

   **Internal Data Model:**
   - All party and location information stored with full traceability to
     Fintraffic schema elements
   - JSON metadata maintained for XML export to ensure lossless mapping to
     `consignment-common.xsd` structure
   - Form UI simplified for user entry; backend expands to full Fintraffic XML
     structure on Gate export

4. **Conditional Dangerous Goods Fieldset** (shows when checkbox selected)
   - **UN Dangerous Goods Number / UNDG Code** (Code, n4, required if dangerous
     goods checked) — GEFeg: Transport Dangerous Goods UNDG Identification Code
     (eFTI232)
   - **Technical Name** (Text, an..128, mandatory field per regulations) —
     GEFeg: Transport Dangerous Goods Technical Name Text (eFTI235)
   - **Hazard Classification Identifier** (Identifier, an..7, conditional) —
     GEFeg: Transport Dangerous Goods Hazard Classification Identifier (eFTI242)
   - **Packing Danger Level Code** (Code, an..3, conditional) — GEFeg: Transport
     Dangerous Goods Packaging Danger Level Code (eFTI236)

5. **Conditional Waste Fieldset** (shows when checkbox selected)
   - **Waste Material Identification** (Identifier, an..17, required if waste
     checked) — GEFeg: Transportation Waste Material Identification (eFTI1496)
   - **Waste Material Weight** (Measure, n..16,3) — GEFeg: Transportation Waste
     Material Weight (eFTI1745)
   - **Waste Material Weight Unit Code** (Code, an..3) — GEFeg: Measure Unit
     Code (eFTI1497) - Valid values: GRM/KGM/TNE
   - Note: Waste handling aligns with GEFeg Supply Chain Consignment Notified
     Waste Material structure (ASBIE1666)

6. **Custom eFTI Fields Section**
   - Dynamic field suggestion and addition via autocomplete
   - Search interface with real-time field suggestion from 70+ eFTI Common Data
     Set fields
   - Added fields appear in scrollable list
   - Remove button for each custom field
   - Do NOT display transportation info fields as custom fields (avoid
     duplication)

7. **Traceability (JSON)**
   - Optional JSON metadata for audit and compliance tracking

**Form UX Features:**

- Real-time validation with inline error messages (red borders, error text below
  fields)
- Conditional section visibility based on checkbox state
- Form submission only when all validation passes
- Automatic field extraction from XML import (if XML source used)
- Consistent styling and responsive layout

**Removed Elements:**

- Subset selection dropdown (always uses FI01 - Consignment)
- Separate eFTI Identifiers section (merged into Transportation Information)
- Invoice Number field (consolidated into main transport info)
- Dangerous Goods as single boolean checkbox (replaced with independent DG/Waste
  checkboxes)

#### 5.6 Fintraffic Gate XML Export: Internal-to-Fintraffic Mapping

**Objective:** Internal dataset JSON is stored simplified for UX; export to
Fintraffic Gate API requires full transformation to `consignment-common.xsd` XML
structure.

**Mapping Layer (`backend/app/services/fintraffic_export.py`):**

**Consignor (Shipper) Expansion:**

```
Internal field: "shipper_name" (string) + optional consignor_full_record (JSON)
↓
Fintraffic XML:
<consignor>
  <name>...</name>                           <!-- from shipper_name or consignor_full_record.name -->
  <id schemeAgencyId="87">...</id>           <!-- from consignor_full_record.id if present, else generated UUID -->
  <postalAddress>
    <streetName>...</streetName>             <!-- from consignor_full_record.street -->
    <cityName>...</cityName>                 <!-- from consignor_full_record.city -->
    <postcode>...</postcode>                 <!-- from consignor_full_record.postcode -->
    <countryCode>...</countryCode>           <!-- from consignor_full_record.country_code (2-char ISO) -->
  </postalAddress>
  <definedContactDetails>                    <!-- optional, if contact data present -->
    <personName>...</personName>
    <telephone><completeNumber>...</completeNumber></telephone>
    <emailAddress><completeNumber>...</completeNumber></emailAddress>
  </definedContactDetails>
  <specifiedContactPerson>                   <!-- optional -->
    <givenName>...</givenName>
    <familyName>...</familyName>
  </specifiedContactPerson>
  <taxRegistration>                          <!-- optional -->
    <id>...</id>
  </taxRegistration>
</consignor>
```

**Source:** eFTI fields eFTI51 (name), eFTI49 (ID), eFTI54-59 (address),
eFTI52-53 (contact), eFTI1152/1154 (person), eFTI859 (tax)

**Consignee (Recipient) Expansion:**

- Identical structure to consignor, using recipient-related data
- Maps to `<consignee>` element in Fintraffic XML
- Source fields: eFTI68, eFTI66, eFTI72-77, eFTI69-71, eFTI1156/1158, eFTI1458

**Location Expansion:**

**Carrier Acceptance Location:**

```
Internal: "origin" (city name, Text)
↓
Fintraffic XML:
<carrierAcceptanceLocation>
  <id schemeAgencyId="6">...</id>            <!-- UN/LOCODE if available in consignor_full_record -->
  <name>...</name>                           <!-- from origin field -->
  <postalAddress>
    <cityName>...</cityName>                 <!-- parsed from origin if structured, else origin itself -->
    <countryCode>FI</countryCode>            <!-- platform-specific or inferred from dataset context -->
  </postalAddress>
</carrierAcceptanceLocation>
```

**Source:** eFTI138 (location name), eFTI136 (ID/UN/LOCODE)

**Consignee Receipt Location:**

```
Internal: "destination" (city name, Text)
↓
Fintraffic XML:
<consigneeReceiptLocation>
  <id schemeAgencyId="6">...</id>            <!-- UN/LOCODE if available -->
  <name>...</name>                           <!-- from destination field -->
  <postalAddress>
    <cityName>...</cityName>
    <countryCode>...</countryCode>
  </postalAddress>
</consigneeReceiptLocation>
```

**Source:** eFTI154 (location name), eFTI152 (ID/UN/LOCODE)

**Transport Movement:**

```
Internal: "transport_mode" (Code, "3" for Road, "2" for Rail, etc.)
↓
Fintraffic XML:
<mainCarriageTransportMovement>
  <modeCode>...</modeCode>                   <!-- from transport_mode, per CL-037 -->
  <usedTransportMeans>
    <id>...</id>                             <!-- from vehicle_registration field -->
  </usedTransportMeans>
</mainCarriageTransportMovement>
```

**Source:** eFTI581 (mode code), eFTI618 (means ID)

**Transport Equipment:**

```
Internal: "vehicle_registration" (Text, an..17)
↓
Fintraffic XML:
<utilizedLogisticsTransportEquipment>
  <id>...</id>                               <!-- from vehicle_registration -->
  <categoryCode>...</categoryCode>           <!-- optional, if type known (e.g., "AE", "SM") -->
</utilizedLogisticsTransportEquipment>
```

**Source:** eFTI374 (equipment ID), eFTI378 (category)

**Dangerous Goods Mapping:**

```
Internal:
  "contains_dangerous_goods": true
  "undg_code": "1800"
  "technical_name": "Phosphorus, white"
  "hazard_classification": "4.2"
  "packing_danger_level": "I"
↓
Fintraffic XML:
<dangerousGoods>
  <uNDGID>1800</uNDGID>                      <!-- from undg_code -->
  <technicalName>Phosphorus, white</technicalName>  <!-- from technical_name, mandatory per ADR/RID/ADN -->
  <hazardClassificationID>4.2</hazardClassificationID>  <!-- from hazard_classification -->
  <packagingDangerLevelCode>I</packagingDangerLevelCode>  <!-- from packing_danger_level -->
</dangerousGoods>
```

**Source:** eFTI232 (UNDG), eFTI235 (technical name), eFTI242 (hazard), eFTI236
(packing level)

**Goods Information:**

```
Internal: "goods_description" (Text, an..512)
↓
Fintraffic XML:
<natureIdentificationCargo>
  <identification>...</identification>       <!-- from goods_description -->
</natureIdentificationCargo>
```

**Source:** eFTI699 (cargo identification)

**Export Service Methods:**

- `to_fintraffic_xml(dataset_json) → XML string` — Transforms internal dataset
  to Fintraffic `consignment-common.xsd` XML
- `validate_fintraffic_xml(xml_string) → (is_valid, errors)` — Validates
  generated XML against published Fintraffic schema
- `send_to_gate_api(xml_string, mTLS_cert) → (success, gate_response)` —
  Authenticates with mTLS and sends to Fintraffic Gate identifier registration
  endpoint

**Integration Points:**

- Called by `PUT /api/v1/datasets/{datasetId}/export-to-gate` endpoint
- Triggered manually by user before final dataset submission to Gate
- Preview mode available to show generated XML before actual submission
- Logs all mapping transformations for audit trail

**Dataset Lifecycle:**

- ACTIVE: newly created datasets can be edited and have fields added/modified
- COMPLETED: datasets become immutable after marking complete, no further edits
  allowed
- Status transition is one-way (ACTIVE → COMPLETED)
- Edit view only available for ACTIVE datasets; view-only for COMPLETED

**Form State Persistence:**

- Dataset selection auto-populates form with existing data on edit
- Detects dangerous goods / waste from existing payload and checks appropriate
  boxes
- All fields maintain validation errors across user interactions

#### 5.7 Authentication and User Management (Lightweight but Usable)

Implement a lightweight, production-oriented authentication baseline with
role-based access control.

**Backend (`backend/app`):**

- Add a `users` entity with at minimum: `id`, `username`, `password_hash`,
  `role`, `is_active`, `full_name`, `created_at`.
- Enforce Bearer JWT authorization in protected endpoints (replace
  development-only role headers).
- Store and validate passwords using salted PBKDF2-SHA256 hashing.
- Roles: `operator`, `authority`, `admin`.
- Provide JWT configuration through environment variables:
  - `JWT_SECRET`
  - `JWT_ALGORITHM` (default `HS256`)
  - `JWT_ACCESS_TOKEN_MINUTES`
  - optional `JWT_ISSUER`, `JWT_AUDIENCE`

**Auth API (`/api/v1/auth`):**

- `GET /bootstrap-status`: indicate whether first-user bootstrap is required.
- `POST /register`: create user (bootstrap allowed for first admin user,
  otherwise admin-only).
- `POST /login`: username/password login returning access token and user
  profile.
- `GET /me`: return current authenticated user.
- `GET /users`: admin-only user list.
- `PATCH /users/{user_id}`: admin-only updates (role, active state, full name).

**Frontend (`frontend/src`):**

- Token-aware API client using `Authorization: Bearer <token>`.
- Initial bootstrap screen for first admin user creation.
- Login screen, session restore, and logout.
- Admin user management view with create user, activate/deactivate, and role
  change controls.

#### 5.8 Gate Send UX and Dataset Transmission Status

Implement explicit Gate transmission UX and persistence indicators.

**Behavior:**

- On dataset creation, prompt user to optionally send the dataset to Gate
  immediately.
- In dataset edit/view, provide manual "Send to Fintraffic" action.
- Persist transmission outcome per dataset with:
  - `sent_to_gate` (boolean)
  - `sent_to_gate_at` (timestamp)
- Display "sent / not sent" status badges in list and detail views.

**Fintraffic payload practice:**

- Build consignment XML according to Fintraffic-accepted schema shapes.
- For rejected payload fields or structures, adjust to Fintraffic acceptance
  criteria and examples.
- Keep request correlation support (`X-Request-ID`) in Fintraffic communication
  paths.

#### 5.9 Subset Exposure and Filtering Behavior

Keep subset handling compliant while simplifying UI data exposure.

- Public dataset response payload may omit subset identifier fields in
  UI-focused responses.
- Support subset filtering where required by authority/gate retrieval use cases:
  - dataset retrieval with one or more `subsetId` query parameters
  - identifier search endpoints supporting subset-based filtering
- Internal logic must continue validating authoritative subset requirements even
  when subset metadata is hidden from selected UI responses.

### 6. Non-Functional Requirements

- Security-by-design for all external interfaces
- Strong audit logging for access events and integrations
- Versioning of data model mappings and subset definitions
- Explicit separation of authoritative eFTI source data and application-defined
  operational logic
- Support for future updates when GEFeg or EU specifications evolve
- Data quality assurance through comprehensive field validation at both backend
  and frontend

## Part 2: Manual Operations & Cloud Deployment Guide (For Human Operators)

### 1. Prerequisites Checklist

Before deployment, ensure the following are available:

- A cloud provider account (AWS)
- A registered public domain name
- Administrative access to the domain DNS management panel
- Secure storage for certificates and private keys

### 2. Infrastructure Deployment Steps

- **Database setup:** initialize PostgreSQL in a private network segment and
  deploy schemas for datasets, subset metadata, code lists, and logs.
- **Containerization:** build all application components with Dockerfiles
  generated by the implementation.
- **Secrets handling:** store private keys, certificates, and API credentials in
  a secrets manager or secure equivalent.
- **TLS & mTLS configuration:** configure HTTPS for public endpoints and mutual
  TLS for Gate communication.
- **Bidirectional mTLS:** verify both outbound and inbound mTLS paths during
  deployment validation, not only outbound Gate connectivity checks.
- **Observability:** enable logging, metrics, and audit retention for
  operational and compliance monitoring.

Operational deployment verification guidance:

- After ECS updates, verify not only service steady state but also running task
  image digests and target health.
- Do not rely solely on "primary task definition" metadata; confirm that active
  running tasks and load balancer targets use the intended revision/image.
- Validate secret access paths end-to-end when using Secrets Manager field
  references (for example `secretArn:KEY::`).
- Ensure ECS task execution role permissions include every referenced secret ARN
  after secret rotation or naming changes.

### 3. Manual Onboarding Responsibilities

Human operators remain responsible for:

- domain configuration
- DNS setup
- certificate submission and onboarding procedures
- operational approval of external connectivity
- production hardening and go-live governance

## Part 3: Implementation Guidance for the Generated Software

The generated platform should implement the following behavior:

1. Allow the user to create a dataset against the authoritative eFTI model.
2. Support subset handling according to authoritative definitions; subset
   selection may be fixed or configurable per deployment profile.
3. Validate the dataset against the authoritative subset requirements before
   publication or retrieval.
4. Preserve traceability between internal persistence fields and authoritative
   eFTI elements.
5. Support identifier-based discovery and UIL-based retrieval.
6. Ensure that internal application logic never replaces or overrides the
   authoritative subset definitions.
7. Keep authored API documentation synchronized with the published Fintraffic
   contract examples and XSD/schema references.
8. Require authenticated access to operational APIs with role-based
   authorization.
9. Provide lightweight user management suitable for controlled operational use.

## Part 5: Implementation Learnings and Accepted Practices (2026-07-02)

### A. Fintraffic Connectivity and Payload Compatibility

- Fintraffic Gate connectivity must be validated with real endpoint checks and
  mTLS-enabled runtime configuration.
- Treat bidirectional mTLS as mandatory for production profile: outbound
  client-auth to Gate and inbound Gate client-certificate verification.
- If a payload is rejected by Fintraffic schema validation, prioritize
  Fintraffic-compatible structure over generic examples.

### B. AWS ECS and Secrets Operations

- Successful deployment requires verification at three levels:
  1. ECS service stability
  2. Running task/image digest correctness
  3. Functional API checks through ALB
- Secrets Manager values and JSON/key structure must match container secret
  references exactly.
- Execution role permissions are a common failure point after introducing new or
  renamed secret ARNs.

### C. Product Behavior Baseline

- Dataset lifecycle and Gate send are now coupled with user-visible transmission
  status.
- UI includes optional immediate Gate send on dataset creation and explicit
  manual send in edit flow.
- Authentication baseline (JWT + users + RBAC) is mandatory for production
  profile.

## Part 4: Source Anchoring

The implementation and generated documentation must explicitly acknowledge the
following source roles:

- GEFeg DS1 pages: authoritative semantic source for Common Data Set structure
- GEFeg subset pages: authoritative source for subset composition
- EU delegated regulation: authoritative legal basis for the Common Data Set and
  subsets
- Fintraffic API documentation and interoperability material: authoritative
  source for integration behavior and example payload shapes
- XSD and model visualization references published from the reference
  implementation and sandbox API: implementation guidance for request and
  response payload design