PARL/examples/GA3C

• Reproduce GA3C with PARL
• • Atari games introduction
  • Benchmark result
• How to use
• • Dependencies
  • Distributed Training
  • • Learner
    • Simulators (Suggest: 24 simulators in 12+ CPUs)
  • Reference

Reproduce GA3C with PARL

Based on PARL, the GA3C algorithm of deep reinforcement learning has been reproduced, reaching the same level of indicators as the paper in Atari benchmarks.

Original paper: GA3C: GPU-based A3C for Deep Reinforcement Learning

A hybrid CPU/GPU version of the Asynchronous Advantage Actor-Critic (A3C) algorithm.

Atari games introduction

Please see here to know more about Atari games.

Benchmark result

Results with one learner (in a P40 GPU) and 24 simulators (in 12 CPU) in 10 million sample steps.

How to use

Dependencies

• paddlepaddle>=1.5.1
• parl
• gym
• atari-py

Distributed Training

Learner

python train.py 

Simulators (Suggest: 24 simulators in 12+ CPUs)

for i in $(seq 1 24); do
    python simulator.py &
done;
wait

You can change training settings (e.g. env_name, server_ip) in ga3c_config.py.
Training result will be saved in log_dir/train/result.csv.

[Tips] The performance can be influenced dramatically in a slower computational environment, especially when training with low-speed CPUs. It may be caused by the policy-lag problem.

Reference

• tensorpack