Sedna/examples/lifelong_learning/cityscapes

• Using Lifelong Learning in Campus Robot Delivery Scenario
• • Introduction
  • • Install Sedna
    • Configurations
    • • Prepare Images
      • Prepare user nodes
      • Prepare Dataset
    • Create Lifelong Learning Job
    • • Create Initial Dataset Resource
      • Create Robot Dog Delivery Lifelong Learning Job
    • Check Lifelong Learning Status
    • Effect Display
    • • Evaluation results of two round lifelong learning
      • Segmentation inference display

Using Lifelong Learning in Campus Robot Delivery Scenario

Introduction

This example introduces how to use Sedna lifelong learning to implement the lifelong learning delivery task of the robot in a campus. Based on open source project RFNet as base model, we realize intelligent perception of environment with Sedna lifelong learning in this example.

The demo mainly shows:

1. Lifelong learning unseen task recognition algorithm (prediction) can identify and collect unseen task samples.
2. Lifelong learning unseen task recognition algorithm (detection) can trigger alarms to remind robot admin that emergency response is required.
3. Lifelong learning unseen task training algorithm improves the inference accuracy of known categories and the ability to identify new categories.

Install Sedna

Follow the Sedna installation document to install Sedna.

Configurations

Prepare Images

This example uses the following images:

• Training worker: kubeedge/sedna-example-lifelong-learning-cityscapes-segmentation:v0.6.0
• Evaluation worker: kubeedge/sedna-example-lifelong-learning-cityscapes-segmentation:v0.6.0
• Inference worker: kubeedge/sedna-example-lifelong-learning-cityscapes-segmentation:v0.6.0

These images are generated by the script build_images.sh.

Users can also generate customized images for different workers and config them in yaml which will be presented in the following steps.

Prepare user nodes

WORKER_NODE=sedna-mini-control-plane

DATA_NODE=$WORKER_NODE 
TRAIN_NODE=$WORKER_NODE 
EVAL_NODE=$WORKER_NODE 
INFER_NODE=$WORKER_NODE 

Particularly, data node, train node, eval node and infer node are custom codes which can be specified by users for actual running. Here, for simplicity, we use the same node to demonstrate.

Prepare Dataset

Step 1: Users can use semantic segmentation datasets from CITYSCAPES. While we also provide a re-organized dataset segmentation_data.zip of CITYSCAPES as an example for training and evaluation.

Download and unzip segmentation_data.zip. Put it into /data of $DATA_NODE. Or you can use docker exec to get into $DATA_NODE and then execute the following commands.

mkdir /data
cd /data
wget https://kubeedge.obs.cn-north-1.myhuaweicloud.com/examples/robo_dog_delivery/segmentation_data.zip
unzip segmentation_data.zip

Step 2: download test data which is for demonstration at the inference stage, unzip test_data.zip and put it into /data of $INFER_NODE. Or you can use docker exec to get into $INFER_NODE and then execute the following commands.

mkdir /data
cd /data
wget https://kubeedge.obs.cn-north-1.myhuaweicloud.com/examples/robo_dog_delivery/test_data.zip
unzip test_data.zip

After finishing the above preparations, execute the following commands to config.

local_prefix=/data
cloud_image=kubeedge/sedna-example-lifelong-learning-cityscapes-segmentation:v0.6.0
edge_image=kubeedge/sedna-example-lifelong-learning-cityscapes-segmentation:v0.6.0
data_url=$local_prefix/segmentation_data/data.txt
OUTPUT=$local_prefix/lifelonglearningjob/output
job_name=robo-demo

Create Lifelong Learning Job

Create Initial Dataset Resource

kubectl create -f - <<EOF
apiVersion: sedna.io/v1alpha1
kind: Dataset
metadata:
  name: lifelong-robo-dataset
spec:
  url: "$data_url"
  format: "txt"
  nodeName: "$DATA_NODE"
EOF

Create Robot Dog Delivery Lifelong Learning Job

kubectl create -f - <<EOF
apiVersion: sedna.io/v1alpha1
kind: LifelongLearningJob
metadata:
  name: $job_name
spec:
  dataset:
    name: "lifelong-robo-dataset"
    trainProb: 0.8
  trainSpec:
    template:
      spec:
        nodeName: $TRAIN_NODE
        dnsPolicy: ClusterFirstWithHostNet
        containers:
          - image: $cloud_image
            name:  train-worker
            imagePullPolicy: IfNotPresent       
            args: ["train.py"]
            env:
              - name: "num_class"
                value: "24"
              - name: "epoches"
                value: "1"
              - name: "attribute"
                value: "real, sim"
              - name: "city"
                value: "bremen"
              - name: "BACKEND_TYPE"
                value: "PYTORCH"
            resources:
              limits:
                cpu: 6
                memory: 12Gi
              requests:
                cpu: 4
                memory: 12Gi
            volumeMounts:
            - mountPath: /dev/shm
              name: cache-volume
        volumes:
        - emptyDir:
            medium: Memory
            sizeLimit: 256Mi
          name: cache-volume
    trigger:
      checkPeriodSeconds: 30
      timer:
        start: 00:00
        end: 24:00
      condition:
        operator: ">"
        threshold: 100
        metric: num_of_samples
  evalSpec:
    template:
      spec:
        nodeName: $EVAL_NODE
        dnsPolicy: ClusterFirstWithHostNet
        containers:
          - image: $cloud_image
            name:  eval-worker
            imagePullPolicy: IfNotPresent
            args: ["evaluate.py"]
            env:
              - name: "operator"
                value: "<"
              - name: "model_threshold"
                value: "0"
              - name: "num_class"
                value: "24"
              - name: "BACKEND_TYPE"
                value: "PYTORCH"
            resources:
              limits:
                cpu: 6
                memory: 12Gi
              requests:
                cpu: 4
                memory: 12Gi
  deploySpec:
    template:
      spec:
        nodeName: $INFER_NODE
        dnsPolicy: ClusterFirstWithHostNet
        hostNetwork: true
        containers:
        - image: $edge_image
          name:  infer-worker
          imagePullPolicy: IfNotPresent
          args: ["predict.py"]
          env:
            - name: "test_data"
              value: "/data/test_data"
            - name: "num_class"
              value: "24"
            - name: "unseen_save_url"
              value: "/data/unseen_samples"
            - name: "INFERENCE_RESULT_DIR"
              value: "/data/infer_results"
            - name: "BACKEND_TYPE"
              value: "PYTORCH"
          volumeMounts:
          - name: unseenurl
            mountPath: /data/unseen_samples
          - name: inferdata
            mountPath: /data/infer_results
          - name: testdata
            mountPath: /data/test_data
          resources:
            limits:
              cpu: 6
              memory: 12Gi
            requests:
              cpu: 4
              memory: 12Gi
        volumes:
          - name: unseenurl
            hostPath:
              path: /data/unseen_samples
              type: DirectoryOrCreate
          - name: inferdata
            hostPath:
              path: /data/infer_results
              type: DirectoryOrCreate
          - name: testdata
            hostPath:
              path: /data/test_data
              type: DirectoryOrCreate
  outputDir: $OUTPUT/$job_name
EOF

Check Lifelong Learning Status

kubectl get lifelonglearningjob

Effect Display

Evaluation results of two round lifelong learning

Rounds	Metrics
	Pixel Accuracy Class (CPA)	Mean Intersection over Union (mIoU)	Frequency Weighted Intersection over Union (FWIoU)
Round 1	0.385	0.319	0.648
Round 2	0.402	0.339	0.659

Segmentation inference display