1 Star 0 Fork 0

陈狗翔 / POMDP-ac-cartpole

Create your Gitee Account
Explore and code with more than 6 million developers,Free private repositories !:)
Sign up
This repository doesn't specify license. Without author's permission, this code is only for learning and cannot be used for other purposes.
Clone or download
Cancel
Notice: Creating folder will generate an empty file .keep, because not support in Git
Loading...
README.md

POMDP-ac-cartpole

solving POMDP by LSTM in gym.cartpole environment, in pytorch

Requirements

  • tensorflow (for tensorboard logging)
  • pytorch (>=1.0, 1.0.1 used in my experiment)
  • gym

POMDP setting

the ideal of convert Cartpole-v0 into a POMDP task comes from HaiyinPiao

and the full observation of cartpole in gym is in 4 dimensions :

  1. cart position (-4.8, 4.8)
  2. cart velocity (-inf, inf)
  3. pole angle (-24°, 24°)
  4. pole velocity at tip (-inf, inf)

and we can delete one or more dimensions of the standard states and make the task become a partial observed markov decision process(POMDP).

Delete the cart velocity

LSTM no LSTM
LSTM without LSTM

Delete the cart velocity and pole velocity

LSTM no LSTM
LSTM without LSTM

Conclusion

When the partial observability becomes more severe, LSTM would significantly improving the performance of RL agent.

Comments ( 0 )

Sign in for post a comment

About

solving POMDP by LSTM in gym.cartpole environment, in pytorch spread retract
Cancel

Releases

No release

Contributors

All

Activities

load more
can not load any more
Python
1
https://gitee.com/ChenGouXiang/POMDP-ac-cartpole.git
git@gitee.com:ChenGouXiang/POMDP-ac-cartpole.git
ChenGouXiang
POMDP-ac-cartpole
POMDP-ac-cartpole
master

Search