AOS Cloud — Deployable Items Lifecycle

Overview

AOS Cloud supports deploying next kinds of items:

• Services — OCI-based application containers managed by Service Managers on devices.
• Layers — Layers for OCI-based application.
• Components — Firmware/OS components (FOTA) managed by Update Managers on devices.

General info

Inside AosCloud, all types of deployable items are unified in processing, storing, and deploying them in to units.

Any object has unique identity to simplify storing, querying, and referencing. Detailed information: object indentification.

All deployable items have version. Item versions identified by version field in SemVer format, and can have different instances for each CPU architecture. E.g. service hello-world: can have version 1.0.0 for amd64 and arm64 architectures.

All are versioned, cryptographically signed, and delivered through the same pipeline: bundle upload → processing → verification → campaign → fleet rollout.

┌─────────────────────────────────────────────────────────────────────────────┐
│  Service / Component version                                                │
│       │                                                                     │
│       ▼                                                                     │
│  Deployment Bundle  (upload & extract)                                      │
│       │                                                                     │
│       ▼                                                                     │
│  Verification Batch  (dependency resolution + per-architecture approval)    │
│       │  ◄── gate: manual operator approval                                 │
│       ▼                                                                     │
│  Vefication → Verificication Unit-Sets  →  Remaining Units  →  SUCCESS      │
└─────────────────────────────────────────────────────────────────────────────┘

---

A service belongs to a Service Provider. A component belongs to an OEM and represents a firmware or OS artifact (FOTA/SOTA). A layer belongs to an OEM and represents a common layer (code or data) that can be shared by multiple services.

State	Meaning
uploaded	File received; processing not started
building	Container image being assembled
processing	Image encryption / signing in progress
ready	Image ready for deployment
build_failed	Build or signing failed

Dependencies

A component version may declare version constraints on other components inside its metadata_info. The platform resolves these before allowing deployment:

{
  "identity": {
    "codename": "base-firmware"
  },
  "versions": "==2.1.0"
}

While any constraint is unsatisfied the component version stays in Waiting dependencies and the associated Verification Batch stays in STATE_WAITING_DEPENDENCIES. Deployment is blocked until all dependencies are uploaded and accepted.

---

2. Deployment Bundles

2.1 Concept

A deployment bundle is an archive (.tar.gz) that packages one or more service or component versions together. Uploading a bundle is the single entry point for introducing new deployable content to the cloud.

2.2 Bundle States

State	Meaning
uploaded	File stored; processing not yet started
building	Background task extracting and validating contents
done	All items extracted; Verification Batch created
error	Processing failed; build_info contains the error message

2.3 Upload and Processing Pipeline

1. Operator uploads bundle
        POST /deployment-bundles/upload/
        Body: multipart file

2. Cloud stores file and creates UpdateBundle (state=uploaded)

3. Background task fires (task_update_bundle_build)
        state → building

4. UpdateBundleProcessor.process()
   • Extracts each service/component version from the archive
   • Creates or updates UpdateItemVersion records
     – Stores architectures[], dependencies JSONB, index_digest
   • Validates dependency graph
   • Creates a VerificationBatch for the bundle's items

5. On success
        state → done
        items JSONB populated with extracted content list

6. On error (up to 3 retries with exponential backoff)
        state → error
        build_info contains error message (max 1 000 chars)

2.4 Key Endpoints

Method	Path	Description
GET	/deployment-bundles/	List bundles
POST	/deployment-bundles/upload/	Upload bundle
DELETE	/deployment-bundles/{id}/	Delete bundle (only when state=error)

---

3. Verification Batches

3.1 Concept

A verification batch is the approval gate between an uploaded bundle and a full fleet rollout. It groups the items extracted from a bundle by target CPU architecture and tracks per-architecture operator approval.

3.2 Batch States

STATE_WAITING_DEPENDENCIES
    │  [all dependency constraints satisfied]
    ▼
STATE_WAITING_VALIDATION
    │  [operator approves every required architecture]
    ▼
STATE_VALID  ←── campaign may now proceed
    
STATE_PARTIALLY_VALIDATED  (some architectures approved, others pending)
    
STATE_INVALID  ←── campaign is blocked; at least one architecture rejected

3.3 Approval Flow

Approval is per CPU architecture (e.g. x86_64, arm64). Each architecture must be explicitly approved by an operator before the batch reaches STATE_VALID.

PATCH /verification-batch/{id}/
Body:
[
  { "architecture": "x86_64", "is_approved": true  },
  { "architecture": "arm64",  "is_approved": false }
]

Result when all approved  → STATE_VALID
Result when any rejected  → STATE_INVALID
Result when partial       → STATE_PARTIALLY_VALIDATED

The approval record stores is_approved, updated_at, and user_id so the decision is auditable.

3.4 Deployment to Validation Unit-Sets

Once a batch enters STATE_WAITING_VALIDATION the platform immediately deploys the items to all units that belong to validation unit-sets (UnitSet.is_validation_set = true). This gives operators real-world evidence before they approve:

1. VerificationBatchProcessor.process() collects units in validation sets.
2. task_send_desired_status_message_to_units sends desired-state messages to those units.
3. Units download, install, and report back — visible in unit dashboards.
4. Operator reviews results and then approves or rejects per architecture.

3.5 Key Endpoints

Method	Path	Description
GET	/verification-batch/	List batches (filter by state, fleet, service)
GET	/verification-batch/{id}/	Batch details with items
PATCH	/verification-batch/{id}/	Approve / reject per architecture

---

4. Campaigns

4.1 Concept

A campaign orchestrates rolling out approved items to the rest of the fleet. It references a VerificationBatch as its approval gate and targets one or more unit-sets through a deployment strategy.

4.2 Campaign States

WAIT_APPROVAL
    │  [verification_batch.state = STATE_VALID]
    ▼
WAIT_START
    │  [start_date reached OR operator manually starts]
    ▼
STARTED
    │  [all phases complete successfully]
    ▼
SUCCESS

STARTED  →  PAUSED   (operator pauses)
PAUSED   →  STARTED  (operator resumes)
STARTED  →  STOPED   (operator stops early)
Any      →  FAIL     (stop_rules trigger on error threshold)

4.3 Deployment Strategies

Strategy	Behaviour
CANARY	Units split into SMALL (e.g. 30 %) and BIG (remaining 70 %). Validation-set units always go to SMALL. Campaign deploys SMALL first, then BIG after SMALL phase passes.
BLUE_GREEN	Units split 50 / 50 into BLUE and GREEN sets. Validation-set units always go to BLUE. BLUE deploys first; GREEN follows.
ROLLOUT	Operator pre-selects ordered unit-sets. Campaign deploys to each set sequentially.

4.4 Campaign Lifecycle

1. Create campaign
        POST /campaigns/
        Body: validation_batch_id, strategy_type, unit_sets, start_date (optional)

2. Verification gate
   Campaign stays in WAIT_APPROVAL until the linked verification batch reaches
   STATE_VALID. If the batch is rejected the campaign cannot advance.

3. Campaign starts
   • At start_date OR manual trigger via PATCH /campaigns/{id}/ (state=STARTED)
   • task_split_unit_to_unit_sets_in_campaign assigns units to deployment phases
   • task_trigger_campaign_start fires for Phase 1

4. Phase execution
   For each unit-set phase (sorted by strategy order):
   a. Desired-status messages sent to online units in this phase
   b. UnitUpdateItem.pending_* fields populated for each unit
   c. Units download → install → report back status
   d. Campaign monitors progress against stop_rules (e.g. error_rate threshold)
   e. Phase marked passed (CampaignUnitSets.passed = true)

5. Advance to next phase
   Repeat step 4 for each remaining unit-set.

6. Completion
   All phases pass → Campaign = SUCCESS
   stop_rules trigger   → Campaign = FAIL

4.5 Key Endpoints

Method	Path	Description
GET	/campaigns/	List campaigns
POST	/campaigns/	Create campaign
GET	/campaigns/{id}/	Campaign details with unit-set status
PATCH	/campaigns/{id}/	Change campaign state (approve, start, pause, stop)
GET	/campaigns/{id}/unit-sets/	Per-unit-set pass status

---

5. Unit-Level Deployment Tracking

Each service or component pending on a specific unit is tracked by a UnitUpdateItem record.

5.1 Pending Item Statuses

Status	Meaning
TO_BE_INSTALLED	Campaign started; item queued
DOWNLOADING	Unit is downloading the item
DOWNLOADED	Download complete
INSTALLING	Installation in progress
INSTALLED	Successfully installed
WAITING_VALIDATION	Installed on validation set; awaiting batch approval
PENDING	Waiting for a dependency or resource
ERROR	Installation failed
REJECTED	Item rejected by unit

5.2 Message Protocol

The desired-state message is sent via:

Protocol	Transport
V7	WebSocket (with transaction ID)

The unit reads the desired state, resolves what needs to change, and performs downloads and installs autonomously. Status updates flow back to the cloud and update UnitUpdateItem.

---

6. End-to-End Flow Summary

 ┌─────────────┐
 │ Upload      │  POST /deployment-bundles/upload/
 │ bundle      │
 └──────┬──────┘
        │ background task
        ▼
 ┌─────────────┐
 │ Process     │  Extract services/components
 │ bundle      │  Resolve dependency graph
 │             │  Create VerificationBatch
 └──────┬──────┘
        │ state = WAITING_DEPENDENCIES  (if deps unsatisfied)
        │ state = WAITING_VALIDATION    (if deps OK)
        ▼
 ┌──────────────────┐
 │ Deploy to        │  Units in validation unit-sets receive
 │ validation sets  │  desired-status messages
 │                  │  Operators monitor install results
 └──────┬───────────┘
        │ PATCH /verification-batch/{id}/  (approve per architecture)
        │ state → STATE_VALID
        ▼
 ┌─────────────┐
 │ Campaign    │  state: WAIT_APPROVAL → WAIT_START → STARTED
 │ starts      │  Units split by strategy (CANARY / BLUE_GREEN / ROLLOUT)
 └──────┬──────┘
        │ Phase 1 (e.g. SMALL / BLUE / first unit-set)
        ▼
 ┌─────────────┐
 │ Phase       │  Desired-status → units download → install → report
 │ execution   │  Campaign monitors stop_rules
 └──────┬──────┘
        │ phase passes
        ▼
 ┌─────────────┐
 │ Next phase  │  (e.g. BIG / GREEN / second unit-set)
 │ …           │  Repeat until all phases complete
 └──────┬──────┘
        │ all phases passed
        ▼
 ┌─────────────┐
 │ Campaign    │  state = SUCCESS
 │ SUCCESS     │
 └─────────────┘

---

7. Key Relationships

Service / Component
  └── Version  (ServiceVersion / ComponentVersion)
       └── UpdateItemVersion  (per OEM + architecture)
            └── dependencies JSONB

UpdateBundle
  └── VerificationBatch
       ├── VerificationBatchItem (1 per version)
       └── Campaign  (1-to-many, via verification_batch_id)

Campaign
  ├── VerificationBatch  (approval gate)
  ├── CampaignUnitSets   (phase targets, with passed flag)
  └── UnitUpdateItem     (per-unit deployment state)
       ├── pending_version
       ├── pending_item_status
       ├── pending_campaign_id
       └── pending_verification_batch_id