velora.gym¶

Documentation

User Guide - Tutorials: Gymnasium

Dedicated Gymnasium [] utility methods to simplify working with the Gymnasium API.

`EnvResult` `dataclass` ¶

Container for environment search results.

Parameters:

Name	Type	Description	Default
`name`	`str`	the name of the environment	required
`type`	`str`	the environment type, either `discrete` or `continuous`	required

Source code in velora/gym/search.py

Python
@dataclass
class EnvResult:
    """
    Container for environment search results.

    Parameters:
        name (str): the name of the environment
        type (str): the environment type, either `discrete` or `continuous`
    """

    name: str
    type: str

`EnvSearch` ¶

A utility class for searching for Gymnasium environments.

Source code in velora/gym/search.py

Python
class EnvSearch:
    """
    A utility class for searching for [Gymnasium](https://gymnasium.farama.org/)
    environments.
    """

    _result_cache: List[EnvResult] = []
    _helper = SearchHelper()

    @classmethod
    def _build_name_cache(cls) -> None:
        """
        Builds the environment name cache storing the latest
        [Gymnasium](https://gymnasium.farama.org/) environment names.
        """
        if cls._result_cache:
            return

        names = cls._helper.get_latest_env_names()
        print(f"Caching {len(names)} environments...", end="")
        results = []

        for env in names:
            env_type = cls._helper.get_env_type(env)
            results.append(EnvResult(name=env, type=env_type))

        cls._result_cache = results
        print("Complete.")

    @classmethod
    def find(cls, query: str) -> List[EnvResult]:
        """
        Find a [Gymnasium](https://gymnasium.farama.org/) environment that contains `query`.

        Parameters:
            query (str): partial or complete name of an environment
                (e.g., `Lunar` or `Pendulum`)

        Returns:
            result (List[EnvResult]): a list of environment results matching the query.
        """
        cls._build_name_cache()

        matches = [env for env in cls._result_cache if query in env.name]
        print(f"{len(matches)} environments found.")
        return matches

    @classmethod
    def discrete(cls) -> List[EnvResult]:
        """
        Get all available discrete [Gymnasium](https://gymnasium.farama.org/) environments.

        Returns:
            names (List[EnvResult]): a list of available discrete environments.
        """
        cls._build_name_cache()

        matches = [env for env in cls._result_cache if env.type == "discrete"]
        print(f"{len(matches)} environments found.")
        return matches

    @classmethod
    def continuous(cls) -> List[EnvResult]:
        """
        Get all available continuous [Gymnasium](https://gymnasium.farama.org/) environments.

        Returns:
            names (List[EnvResult]): a list of available continuous environments.
        """
        cls._build_name_cache()

        matches = [env for env in cls._result_cache if env.type == "continuous"]
        print(f"{len(matches)} environments found.")
        return matches

`continuous()` `classmethod` ¶

Get all available continuous Gymnasium environments.

Returns:

Name	Type	Description
`names`	`List[EnvResult]`	a list of available continuous environments.

Source code in velora/gym/search.py

Python
@classmethod
def continuous(cls) -> List[EnvResult]:
    """
    Get all available continuous [Gymnasium](https://gymnasium.farama.org/) environments.

    Returns:
        names (List[EnvResult]): a list of available continuous environments.
    """
    cls._build_name_cache()

    matches = [env for env in cls._result_cache if env.type == "continuous"]
    print(f"{len(matches)} environments found.")
    return matches

`discrete()` `classmethod` ¶

Get all available discrete Gymnasium environments.

Returns:

Name	Type	Description
`names`	`List[EnvResult]`	a list of available discrete environments.

Source code in velora/gym/search.py

Python
@classmethod
def discrete(cls) -> List[EnvResult]:
    """
    Get all available discrete [Gymnasium](https://gymnasium.farama.org/) environments.

    Returns:
        names (List[EnvResult]): a list of available discrete environments.
    """
    cls._build_name_cache()

    matches = [env for env in cls._result_cache if env.type == "discrete"]
    print(f"{len(matches)} environments found.")
    return matches

`find(query)` `classmethod` ¶

Find a Gymnasium environment that contains query.

Parameters:

Name	Type	Description	Default
`query`	`str`	partial or complete name of an environment (e.g., `Lunar` or `Pendulum`)	required

Returns:

Name	Type	Description
`result`	`List[EnvResult]`	a list of environment results matching the query.

Source code in velora/gym/search.py

Python
@classmethod
def find(cls, query: str) -> List[EnvResult]:
    """
    Find a [Gymnasium](https://gymnasium.farama.org/) environment that contains `query`.

    Parameters:
        query (str): partial or complete name of an environment
            (e.g., `Lunar` or `Pendulum`)

    Returns:
        result (List[EnvResult]): a list of environment results matching the query.
    """
    cls._build_name_cache()

    matches = [env for env in cls._result_cache if query in env.name]
    print(f"{len(matches)} environments found.")
    return matches

`SearchHelper` ¶

A helper class containing methods for creating the EnvSearch cache.

Source code in velora/gym/search.py

Python
class SearchHelper:
    """
    A helper class containing methods for creating the `EnvSearch` cache.
    """

    @staticmethod
    def get_env_type(env_name: str) -> Literal["discrete", "continuous"] | None:
        """
        Determines if an environment has a discrete or continuous action space.

        Parameters:
            env_name (str): name of the environment

        Returns:
            env_type (str | None): one of three values -

            - `discrete` for Discrete action space.
            - `continuous` for Box action space.
            - `None`, otherwise.
        """
        try:
            env = gym.make(env_name)

            if isinstance(env.action_space, gym.spaces.Discrete):
                return "discrete"
            elif isinstance(env.action_space, gym.spaces.Box):
                return "continuous"
            return None

        except Exception as e:
            raise e
        finally:
            if "env" in locals():
                env.close()

    @staticmethod
    def get_latest_env_names() -> List[str]:
        """
        Builds a list of the latest [Gymnasium](https://gymnasium.farama.org/) environment names.

        Returns:
            names (List[str]): a list of names for all latest env versions.
        """
        env_dict: Dict[str, int] = {}

        for env_name in gym.envs.registry.keys():
            # Skip envs with paths (JAX-based) or old Gym versions
            if "/" in env_name or env_name.startswith("GymV"):
                continue

            match = re.match(r"(.+)-v(\d+)$", env_name)

            if match:
                base_name, version = match.groups()
                version = int(version)

            # Keep only the latest version
            if base_name not in env_dict or version > env_dict[base_name]:
                env_dict[base_name] = version

        return [f"{name}-v{version}" for name, version in env_dict.items()]

`get_env_type(env_name)` `staticmethod` ¶

Determines if an environment has a discrete or continuous action space.

Parameters:

Name	Type	Description	Default
`env_name`	`str`	name of the environment	required

Returns:

Name	Type	Description
`env_type`	`str \| None`	one of three values -
	`Literal['discrete', 'continuous'] \| None`	`discrete` for Discrete action space.
	`Literal['discrete', 'continuous'] \| None`	`continuous` for Box action space.
	`Literal['discrete', 'continuous'] \| None`	`None`, otherwise.

Source code in velora/gym/search.py

Python
@staticmethod
def get_env_type(env_name: str) -> Literal["discrete", "continuous"] | None:
    """
    Determines if an environment has a discrete or continuous action space.

    Parameters:
        env_name (str): name of the environment

    Returns:
        env_type (str | None): one of three values -

        - `discrete` for Discrete action space.
        - `continuous` for Box action space.
        - `None`, otherwise.
    """
    try:
        env = gym.make(env_name)

        if isinstance(env.action_space, gym.spaces.Discrete):
            return "discrete"
        elif isinstance(env.action_space, gym.spaces.Box):
            return "continuous"
        return None

    except Exception as e:
        raise e
    finally:
        if "env" in locals():
            env.close()

`get_latest_env_names()` `staticmethod` ¶

Builds a list of the latest Gymnasium environment names.

Returns:

Name	Type	Description
`names`	`List[str]`	a list of names for all latest env versions.

Source code in velora/gym/search.py

Python
@staticmethod
def get_latest_env_names() -> List[str]:
    """
    Builds a list of the latest [Gymnasium](https://gymnasium.farama.org/) environment names.

    Returns:
        names (List[str]): a list of names for all latest env versions.
    """
    env_dict: Dict[str, int] = {}

    for env_name in gym.envs.registry.keys():
        # Skip envs with paths (JAX-based) or old Gym versions
        if "/" in env_name or env_name.startswith("GymV"):
            continue

        match = re.match(r"(.+)-v(\d+)$", env_name)

        if match:
            base_name, version = match.groups()
            version = int(version)

        # Keep only the latest version
        if base_name not in env_dict or version > env_dict[base_name]:
            env_dict[base_name] = version

    return [f"{name}-v{version}" for name, version in env_dict.items()]

`add_core_env_wrappers(env, device)` ¶

Wraps a Gymnasium environment with the following (in order) if not already applied:

RecordEpisodeStatistics - for easily retrieving episode statistics.
NumpyToTorch - for turning environment feedback into PyTorch tensors.

Used in all pre-built algorithms.

Parameters:

Name	Type	Description	Default
`env`	`gym.Env`	the Gymnasium environment	required
`device`	`torch.device`	the PyTorch device to perform computations on	required

Returns:

Name	Type	Description
`env`	`gym.Env`	an updated Gymnasium environment

Source code in velora/gym/wrap.py

Python
def add_core_env_wrappers(env: gym.Env, device: torch.device) -> gym.Env:
    """
    Wraps a [Gymnasium](https://gymnasium.farama.org/) environment with the following (in order) if not already applied:

    - [RecordEpisodeStatistics](https://gymnasium.farama.org/api/wrappers/misc_wrappers/#gymnasium.wrappers.RecordEpisodeStatistics) - for easily retrieving episode statistics.
    - [NumpyToTorch](https://gymnasium.farama.org/api/wrappers/misc_wrappers/#gymnasium.wrappers.NumpyToTorch) - for turning environment feedback into `PyTorch` tensors.

    Used in all pre-built algorithms.

    Parameters:
        env (gym.Env): the Gymnasium environment
        device (torch.device): the PyTorch device to perform computations on

    Returns:
        env (gym.Env): an updated Gymnasium environment
    """
    if isinstance(env, gym.Env):
        from gymnasium.wrappers import RecordEpisodeStatistics
        from gymnasium.wrappers.numpy_to_torch import NumpyToTorch
    else:
        raise ValueError(f"{type(env)}' is not supported.")

    has_stats = False
    has_torch = False

    current_env = env
    while hasattr(current_env, "env"):
        if isinstance(current_env, RecordEpisodeStatistics):
            has_stats = True
        if isinstance(current_env, NumpyToTorch):
            has_torch = True

        current_env = current_env.env

    if not has_stats:
        env = RecordEpisodeStatistics(env)

    if not has_torch:
        env = NumpyToTorch(env, device=device)

    return env

`wrap_gym_env(env, wrappers)` ¶

Creates a new Gymnasium environment with one or more gymnasium.Wrappers applied.

Parameters:

Name	Type	Description	Default
`env`	`gymnasium.Env \| str`	a name of a Gymnasium environment or the environment itself to wrap	required
`wrappers`	`List[gym.Wrapper \| gym.vector.VectorWrapper \| functools.partial]`	a list of wrapper classes or partially applied wrapper functions	required

Examples:

A Gymnasium environment with normalization and reward clipping:

Python
from functools import partial

from gymnasium.wrappers import (
    NormalizeObservation,
    NormalizeReward,
    ClipReward,
)

env = wrap_gym_env("InvertedPendulum-v5", [
    partial(NormalizeObservation, epsilon=1e-8),
    partial(NormalizeReward, gamma=0.99, epsilon=1e-8),
    partial(ClipReward, max_reward=10.0)
])

Returns:

Name	Type	Description
`env`	`gymnasium.Env`	The wrapped environment

Source code in velora/gym/wrap.py

Python
def wrap_gym_env(
    env: gym.Env | str | DiscreteGymNames | ContinuousGymNames,
    wrappers: List[gym.Wrapper | gym.vector.VectorWrapper | Callable],
) -> gym.Env:
    """
    Creates a new [Gymnasium](https://gymnasium.farama.org/) environment with
    one or more [gymnasium.Wrappers](https://gymnasium.farama.org/api/wrappers/table/) applied.

    Parameters:
        env (gymnasium.Env | str): a name of a Gymnasium environment or the
            environment itself to wrap
        wrappers (List[gym.Wrapper | gym.vector.VectorWrapper | functools.partial]): a list of wrapper classes or partially applied wrapper functions

    Examples:
        A Gymnasium environment with normalization and reward clipping:
        ```python
        from functools import partial

        from gymnasium.wrappers import (
            NormalizeObservation,
            NormalizeReward,
            ClipReward,
        )

        env = wrap_gym_env("InvertedPendulum-v5", [
            partial(NormalizeObservation, epsilon=1e-8),
            partial(NormalizeReward, gamma=0.99, epsilon=1e-8),
            partial(ClipReward, max_reward=10.0)
        ])
        ```

    Returns:
        env (gymnasium.Env): The wrapped environment
    """
    if isinstance(env, str):
        env = gym.make(env, render_mode="rgb_array")

    def apply_wrapper(env: gym.Env, wrapper: WrapperType) -> gym.Env:
        return wrapper(env)

    return reduce(apply_wrapper, wrappers, env)

velora.gym¶

EnvResult dataclass ¶

EnvSearch ¶

continuous() classmethod ¶

discrete() classmethod ¶

find(query) classmethod ¶

SearchHelper ¶

get_env_type(env_name) staticmethod ¶

get_latest_env_names() staticmethod ¶

add_core_env_wrappers(env, device) ¶

wrap_gym_env(env, wrappers) ¶

`EnvResult` `dataclass` ¶

`EnvSearch` ¶

`continuous()` `classmethod` ¶

`discrete()` `classmethod` ¶

`find(query)` `classmethod` ¶

`SearchHelper` ¶

`get_env_type(env_name)` `staticmethod` ¶

`get_latest_env_names()` `staticmethod` ¶

`add_core_env_wrappers(env, device)` ¶

`wrap_gym_env(env, wrappers)` ¶