Skip to content

velora.metrics

Documentation

User Guide - Tutorials: Training Metrics

Classes and methods dedicated to offline metric storage.

Episode

Bases: SQLModel

Episode-level metrics tracking reward, length, and agent performance.

Attributes:

Name Type Description
id int

a unique identifier for the episode
experiment_id int

the experiment ID associated to the episode
episode_num int

the episode index
reward float

the episodic reward (return)
length int

the number of timesteps performed to terminate the episode
reward_moving_avg float

the episodes reward moving average based on a window size
reward_moving_std float

the episodes reward moving standard deviation based on a window size
actor_loss float

the average Actor loss for the episode
critic_loss float

the average Critic loss for the episode
entropy_loss float

the average Entropy loss for the episode
created_at datetime

the date and time when the the entry was created
Source code in velora/metrics/models.py
Python
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
class Episode(SQLModel, table=True):
    """
    Episode-level metrics tracking reward, length, and agent performance.

    Attributes:
        id (int): a unique identifier for the episode
        experiment_id (int): the experiment ID associated to the episode
        episode_num (int): the episode index
        reward (float): the episodic reward (return)
        length (int): the number of timesteps performed to terminate the episode
        reward_moving_avg (float): the episodes reward moving average based on a window size
        reward_moving_std (float): the episodes reward moving standard deviation based on a
            window size
        actor_loss (float): the average Actor loss for the episode
        critic_loss (float): the average Critic loss for the episode
        entropy_loss (float): the average Entropy loss for the episode
        created_at (datetime): the date and time when the the entry was created
    """

    id: int | None = Field(default=None, primary_key=True)
    experiment_id: int = Field(foreign_key="experiment.id", index=True)
    episode_num: int = Field(index=True)

    # Core metrics
    reward: float
    length: int

    # Statistical metrics
    reward_moving_avg: float
    reward_moving_std: float

    # Loss metrics
    actor_loss: float
    critic_loss: float
    entropy_loss: float = Field(default=0.0)

    # Timestamps
    created_at: datetime = Field(default_factory=datetime.now)

    # Relationships
    experiment: Experiment = Relationship(back_populates="episodes")

Experiment

Bases: SQLModel

Experiment information tracking agent, environment, and metadata.

Attributes:

Name Type Description
id int

unique identifier for the experiment
agent str

the name of the agent used in the experiment
env str

the name of the environment used in the experiment
config str

a JSON string containing the agent's configuration details
created_at datetime

the date and time the experiment was created
Source code in velora/metrics/models.py
Python
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
class Experiment(SQLModel, table=True):
    """
    Experiment information tracking agent, environment, and metadata.

    Attributes:
        id (int): unique identifier for the experiment
        agent (str): the name of the agent used in the experiment
        env (str): the name of the environment used in the experiment
        config (str): a JSON string containing the agent's configuration details
        created_at (datetime): the date and time the experiment was created
    """

    id: int | None = Field(default=None, primary_key=True)
    agent: str = Field(index=True)
    env: str = Field(index=True)
    config: str  # JSON object
    created_at: datetime = Field(default_factory=datetime.now)

    # Relationships
    episodes: List["Episode"] = Relationship(back_populates="experiment")

get_current_episode(session, experiment_id, current_ep)

Queries a database session to retrieve the current episode for an experiment.

Parameters:

Name Type Description Default
session sqlmodel.Session

a metric database session

 required
experiment_id int

the current experiment's unique ID

 required
current_ep int

the current episode index

 required

Returns:

Name Type Description
results ScalarResult[Episode]

a iterable set of matching episodes.
Source code in velora/metrics/db.py
Python
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
def get_current_episode(
    session: Session,
    experiment_id: int,
    current_ep: int,
) -> ScalarResult[Episode]:
    """
    Queries a database session to retrieve the current episode for an experiment.

    Parameters:
        session (sqlmodel.Session): a metric database session
        experiment_id (int): the current experiment's unique ID
        current_ep (int): the current episode index

    Returns:
        results (ScalarResult[Episode]): a iterable set of matching episodes.
    """
    statement = select(Episode).where(
        Episode.experiment_id == experiment_id,
        Episode.episode_num == current_ep,
    )
    return session.exec(statement)

get_db_engine()

Starts the metric database engine and returns it.

Returns:

Name Type Description
engine sqlalchemy.Engine

a database engine instance
Source code in velora/metrics/db.py
Python
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
def get_db_engine() -> Engine:
    """
    Starts the metric database engine and returns it.

    Returns:
        engine (sqlalchemy.Engine): a database engine instance
    """
    from velora.metrics.models import Episode, Experiment  # pragma: no cover

    if os.getenv("VELORA_TEST_MODE", "").lower() in ("true", "1"):
        # Use in-memory database for testing
        engine = create_engine("sqlite:///:memory:")
    else:
        # Regular file-based database
        engine = create_engine("sqlite:///metrics.db")

    SQLModel.metadata.create_all(engine)
    return engine