By default, returns False if the initial state is an empty dict (or None).
By default, RLlib assumes that the module is non-recurrent, if the initial
state is an empty dict and recurrent otherwise.
This behavior can be customized by overriding this method.