Architecture Map
Detailing the modular structure that drives Skyops operations.
Last updated
Detailing the modular structure that drives Skyops operations.
Last updated
Architecture Map serves as the structural backbone of the ecosystem, enabling efficient orchestration of decentralized resources, real-time task allocation and seamless interaction between all system components. The architecture is modular, scalable and designed to maximize performance while ensuring fault tolerance and security.
The Node Layer consists of all computational resources GPU Workers and Broker Nodes that contribute to the network. These nodes interact with the system via APIs and act as the primary compute engines.
Example Workflow for GPU Workers:
⬡ A user registers a node with an RTX 3060 GPU and 12 GB RAM.
⬡ Tasks requiring moderate memory are automatically routed to the GPU.
⬡ The node executes computations and sends results back to the system.
Code for Node Registration:
The Orchestration Layer is the heart of Skyops architecture, responsible for distributing workloads across the network. It consists of:
⬡ Task Scheduler: Determines how tasks are divided and assigned to nodes based on their specifications.
⬡ Resource Manager: Continuously monitors the availability and capacity of all nodes.
⬡ Fault Handler: Detects and manages failures in the system by reallocating tasks dynamically.
Code for Task Submission:
The Communication Layer ensures seamless interaction between nodes and the orchestration system.
⬡ Inter-Node Communication: Facilitates data transfer and synchronization between nodes.
⬡ API Gateway: Provides external access points for users to interact with the network.
Example Communication Flow:
⬡ A Broker Node receives a task submission request via API.
⬡ The Broker queries GPU Workers for availability using gRPC.
⬡ Task data is transmitted from the Broker to selected GPU Workers for processing.
This layer ensures secure and efficient handling of data, including task input/output and system logs.
⬡ Data Storage: Stores task-related data, models and logs.
⬡ Data Encryption: Ensures that sensitive information is transmitted securely between components.
Code for Secure Data Upload:
End-to-End Task Execution
⬣ Task Submission:
⬡ A user submits a training task for a ResNet model via the CLI.
⬡ Input: Model type, dataset, budget and required compute power.
⬣ Task Scheduling:
⬡ The Orchestration Layer evaluates GPU Worker availability and allocates tasks accordingly.
⬡ High-memory tasks are sent to A100 GPUs, while smaller tasks are routed to RTX 3060 GPUs.
⬣ Task Execution:
⬡ GPU Workers execute their assigned portions of the task.
⬡ Results are periodically sent back to the Orchestration Layer.
⬣ Task Validation:
⬡ Results are validated using checksum and redundancy mechanisms.
⬡ Any discrepancies are flagged and recomputed.
⬣ Task Completion:
⬡ Final outputs are encrypted and delivered to the user.
⬡ Node contributors are rewarded with SKYOPS tokens.
Below is an example of how the Skyops architecture operates:
Code Example: Full Workflow
Scenario: A user submits a training task and the architecture processes it end-to-end.
