Using the API

In this section you will learn how to create a simple computation task and retrieve the results.

It is assumed that you obtained a username and password for a vantage6 server and that the organization to which you belong participates at least in one collaboration (which could be a collaboration consisting only of your organization). It is also assumed that the algorithm you want to execute is already available (and allowed in this collaboration). If you want to add your own algorithm please see Create New Algorithm.

The following steps are required:

1. Authenticate

2. Request new computation

3. Obtain the results

Login / Authentication

Before you can do any computations you need to authenticate to the server to obtain a JWT-token. This JWT-token can then be used to create new computation tasks and obtain the results when ready.

url = "https://api.distributedlearning.ai"
api_path = ""


username = "john-doe"
password = "super-secret-password!"

Token

GET /token/user

Obtain a user JWT-token using your username and password:

Request Body

Name	Type	Description
username	string
password	string

200

Ok, authenticated

400

No or wrong body specified, or username/password combination unknown

To authenticate you can use the following REST endpoint (todo: insert link to REST documentation) <server_url><api_path>/token/user (e.g. https://api.distributedlearning.ai/token/user).

import requests
import json

json_data = {
  "username": username,
  "password": password
}

# make an http-call to obtain a JWT-token
response = requests.post(
  f"{url}{api_path}/token/user", 
  json=json_data
)

# extract JWT access token
data = response.json()
access_token = data["access_token"]
refresh_token = data["refresh_token"]

Once the Bearer/JWT token (access_token) is obtained it is possible to create tasks (or in more general access other API endpoints). Tasks are always assigned to collaborations (it is possible to assign tasks to specific organizations within a collaboration though). It is mandatory that you, as a researcher, belong to an organization that is part of the collaboration to which you are posting the computation task.

Permissions

You only can post tasks to collaborations in which the organization you are part participates in this collaboration.

Task

POST /task

Create a new task for a specific collaboration.

Request Body

Name	Type	Description
organization_ids	string	Organizations (that are within the collaboration) that should execute this task. By default all organizations will execute the task
description	string	Human readable description
name	string	Human readable name
image	string	Docker image to use
collaboration_id	string	collaboration to which the task should be posted

200

An example would be:

headers = {'Authorization': 'Bearer ' + access_token}

task_spec = {
  "description": "human readable description, not used by the code",
  "image": "docker-registry.distributedlearning.ai/docker-image-name",
  "input": "some input",
  "name": "human readable",
  "collaboration_id":2
}

task = requests.post(
  f"{url}{api_path}/task", 
  headers=headers,
  json=task_spec
)

When the computation task is created, the task is distributed to all the nodes (if not specified by the organization_ids) that belong to this collaboration.

Organization and Nodes

Each organization can have multiple collaborations. For each collaboration a separate node instance is created. As a researcher you only have to specify the collaboration (optionally you can specify specific organizations within the collaboration) to which you want to post the task. So do not worry about nodes to much!

All the nodes from the organization execute the tasks (if online, otherwise they will start the task when they come online). Once they completed the task the results are returned to the central server where they can be collected.

Obtain results

There are two ways in order to know when the results are available. The first method is polling for results at the <server_url><api_path>/task/{id}/result and the other method is listening to a websocket channel. The latter is the preferred way, although polling is easier to implement. In both cases you need to make a API call to obtain the results.

Polling

The polling method checks if the Boolean field complete is set to true, after which the results can be obtained. Make sure to pause between each poll to not overload the server with requests.

from time import sleep

task_id = task.get("id")
task = requests.get(
  f"{url}{api_path}/task/{task_id}", 
  headers=headers
)

while not task.get("complete"):
	task = requests.get(
      f"{url}{api_path}/task/{task_id}",
  	  headers=headers
    )
  sleep(1)

Websocket

The websocket method subscribes to the collaboration channel to which status updates are posted.

from socketIO_client import SocketIO, SocketIONamespace

class TasksNamespace(SocketIONamespace):
	"""Socket namespace"""
  
    def on_status_update(self, result_id):
        results = client.get_results(id=result_id)
        for result in results:
            node_id = result.get("node")
            print("-"*80)
            print(f"Results from node = {node_id}")
            print(result.get("result"))
    
socket = SocketIO(url, headers={
    "Authorization": f"Bearer {access_token}"
})


taskNamespace = socket.define(TasksNamespace, "/tasks")
taskNamespace.emit("join_room", f"collaboration_{collaboration_id}")
socket.wait()