PARL/examples/NeurIPS2019-Learn-to-Move-Challenge

• Dependencies
• Part1: Final submitted model
• • Test
• Part2: Curriculum learning
• • • 1. Run as fast as possible -> run at 3.0 m/s -> walk at 2.0 m/s -> walk slowly at 1.3 m/s
    • 2. difficulty=1
    • 3. difficulty=2
    • 4. difficulty=3, first target
    • 5. difficulty=3

The PARL team gets the first place in NeurIPS reinforcement learning competition, again! This folder contains our final submitted model and the code relative to the training process.

Dependencies

• python3.6
• parl==1.2.1
• paddlepaddle==1.5.1
• parl>=1.2.1
• osim-rl==3.0.11

Part1: Final submitted model

Test

• How to Run

  1. Enter the sub-folder final_submit
  2. Download the model file from online storage service: Baidu Pan (password: b5ck) or Google Drive
  3. Unpack the file:
     tar zxvf saved_models.tar.gz
  4. Launch the test script:
     python test.py

Part2: Curriculum learning

1. Run as fast as possible -> run at 3.0 m/s -> walk at 2.0 m/s -> walk slowly at 1.3 m/s

The curriculum learning pipeline to get a walking slowly model is the same pipeline in our winning solution in NeurIPS 2018: AI for Prosthetics Challenge. You can get a walking slowly model by following the guide.

We also provide a pre-trained model that walk naturally at ~1.3m/s. You can download the model file (naming low_speed_model) from online storage service: Baidu Pan (password: q9vj) or Google Drive.

2. difficulty=1

We built our distributed training agent based on PARL cluster. To start a PARL cluster, we can execute the following two xparl commands:

# starts a master node to manage computation resources and adds the local CPUs to the cluster.
xparl start --port 8010 

# if necessary, adds more CPUs (computation resources) in other machine to the cluster.
xparl connect --address [CLUSTER_IP]:8010 

For more information of xparl, please visit the documentation.

In this example, we can start a local cluster with 300 CPUs by running:

xparl start --port 8010 --cpu_num 300

Then, we can start the distributed training by running:

# NOTE: You need provide a self-trained model, or download the `low_speed_model` as mentioned above.
sh scripts/train_difficulty1.sh ./low_speed_model

Optionally, you can start the distributed evaluating by running:

sh scripts/eval_difficulty1.sh

3. difficulty=2

sh scripts/train_difficulty2.sh [TRAINED DIFFICULTY=1 MODEL]

4. difficulty=3, first target

sh scripts/train_difficulty3_first_target.sh [TRAINED DIFFICULTY=2 MODEL]

5. difficulty=3

sh scripts/train_difficulty3.sh [TRAINED DIFFICULTY=3 FIRST TARGET MODEL]