API Reference
Module containing components for L2 Hilbert spaces and related operations.
A class representing the L2-space of random variables defined on a given probability space.
Suppose given a probability space \((\Omega, \mathcal{F}, P)\), where \(\Omega\) is a sample space, \(\mathcal{F}\) is a \(\sigma\)-algebra on \(\Omega\), and \(P\) is a probability measure on \(\mathcal{F}\). In the mathematical literature, the space \(L^2(\Omega, \mathcal{F}, P)\) is defined as the set of all (equivalence classes of) random variables \(X\) defined on \(\Omega\) that are \(\mathcal{F}\)-measurable and satisfy \(E(X^2) < \infty\).
The equivalence relation is defined as follows: two random variables \(X\) and \(Y\) are considered equivalent if they are equal almost surely, i.e., if \(P(X \neq Y) = 0\). In other words, in the mathematical definition of an \(L^2\)-space, two random variables that differ only on a set of probability zero are identified as the same element of \(L^2\).
The class L2 is SigAlg's model of these \(L^2\)-spaces of random variables.
Mimicking the mathematical definition of an \(L^2\)-space described above, an instance H of L2 is initialized with a SampleSpace, a SigmaAlgebra on that sample space, and a ProbabilityMeasure.
Since all sample spaces in SigAlg are finite, the condition \(E(X^2) < \infty\) is automatically satisfied for all random variables defined on the sample space. Therefore, the only condition for an instance X of RandomVariable to be in an instance H of L2 is that X is measurable with respect to the SigmaAlgebra attribute of H. This can be checked via the in operator by writing X in H.
Note that the in operator acts on a random variable itself, not on the equivalence class of the random variable described above. Indeed, these equivalence classes are not explicitly modeled in SigAlg.
Besides providing the in operator, an instance of L2 also provides several Hilbert space methods, including inner for computing the inner product of two random variables, norm for computing the norm of a random variable, and metric for computing the (norm induced) distance between two random variables.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
sample_space
|
SampleSpace
|
The sample space on which the L2-space is defined. |
required |
sigma_algebra
|
SigmaAlgebra | None
|
The sigma algebra on which the L2-space is defined. If |
None
|
probability_measure
|
ProbabilityMeasure | None
|
The probability measure on which the L2-space is defined. If |
None
|
name
|
Hashable | None
|
The name of the L2-space. |
"H"
|
Raises:
| Type | Description |
|---|---|
TypeError
|
If |
Examples:
>>> from sigalg.core import ProbabilityMeasure, RandomVariable, SampleSpace, SigmaAlgebra
>>> from sigalg.l2 import L2
>>> Omega = SampleSpace().from_sequence(size=4)
>>> atom_ids = {
... 0: 0,
... 1: 0,
... 2: 1,
... 3: 1,
... }
>>> F = SigmaAlgebra(sample_space=Omega).from_dict(atom_ids)
>>> probabilities = {
... 0: 0.2,
... 1: 0.1,
... 2: 0.4,
... 3: 0.3,
... }
>>> P = ProbabilityMeasure(sample_space=Omega).from_dict(probabilities)
>>> H = L2(sample_space=Omega, sigma_algebra=F, probability_measure=P)
>>> outputs_X = {
... 0: 3,
... 1: 3,
... 2: 5,
... 3: 5,
... }
>>> X = RandomVariable(domain=Omega).from_dict(outputs_X)
>>> outputs_Y = {
... 0: 1,
... 1: 3,
... 2: 4,
... 3: 2,
... }
>>> Y = RandomVariable(domain=Omega, name="Y").from_dict(outputs_Y)
>>> # X is measurable with respect to F, so it is in H
>>> print(X in H)
True
>>> # Y is not measurable with respect to F, so it is not in H
>>> print(Y in H)
False
property
basis
Return a vector space basis of the L2-space.
Consider the space \(H = L^2(\Omega, \mathcal{F}, P)\). In SigAlg, we restrict ourselves to finite sample spaces \(\Omega\), so the \(\sigma\)-algebra \(\mathcal{F}\) is completely determined by its atoms, i.e., the minimal nonempty events in \(\mathcal{F}\). The atoms form a partition of \(\Omega\).
The indicator functions \(I_A\), as \(A\) ranges over the atoms of \(\mathcal{F}\), form an orthogonal basis of the \(L^2\)-space. Their squared norms are computed as
since \(I_A^2 = I_A\). Therefore, a convenient choice of orthonormal basis of \(H\) is given by all normalized indicator functions \(I_A / \sqrt{P(A)}\), for which \(P(A) > 0\). Note that if \(P(A) = 0\), then the indicator function \(I_A\) is the zero vector in \(H\) (since it is equal to the zero vector almost surely), so it does not contribute to the basis.
The basis attribute of an instance H of L2 returns a dictionary mapping the atom ID to the corresponding normalized indicator function of that atom, for all atoms that have positive probability.
Returns:
| Name | Type | Description |
|---|---|---|
basis |
dict[str, RandomVariable]
|
A dictionary mapping the atom ID to the corresponding basis vector of the L2-space. |
Examples:
>>> from sigalg.core import ProbabilityMeasure, SampleSpace, SigmaAlgebra
>>> from sigalg.l2 import L2
>>> Omega = SampleSpace().from_sequence(size=3)
>>> F = SigmaAlgebra(sample_space=Omega).from_dict({0: 0, 1: 0, 2: 1})
>>> # A probability measure assigning nonzero probability to all atoms
>>> P = ProbabilityMeasure(sample_space=Omega).from_dict({0: 0.2, 1: 0.5, 2: 0.3})
>>> H = L2(
... sample_space=Omega,
... sigma_algebra=F,
... probability_measure=P,
... )
>>> e_0, e_1 = H.basis.values()
>>> e_0
Random variable '0':
0
sample
0 1.195229
1 1.195229
2 0.000000
>>> # A probability measure assigning zero probability to one atom
>>> Q = ProbabilityMeasure(sample_space=Omega).from_dict({0:0.2, 1:0.8, 2:0})
>>> G = L2(
... sample_space=Omega,
... sigma_algebra=F,
... probability_measure=Q,
... name="G"
... )
>>> G.basis
{np.int64(0): Random variable '0':
0
sample
0 1.0
1 1.0
2 0.0}
property
dim
The dimension of the L2-space, i.e., the number of basis vectors.
Returns:
| Name | Type | Description |
|---|---|---|
dim |
int
|
The dimension of the L2-space, i.e., the number of basis vectors. |
property
writable
name
The name of the L2-space.
Returns:
| Name | Type | Description |
|---|---|---|
name |
Hashable
|
The name of the L2-space. |
property
writable
probability_measure
The probability measure on which the L2-space is defined.
Returns:
| Name | Type | Description |
|---|---|---|
probability_measure |
ProbabilityMeasure
|
The probability measure on which the L2-space is defined. |
property
sample_space
The sample space on which the L2-space is defined.
Returns:
| Name | Type | Description |
|---|---|---|
sample_space |
SampleSpace
|
The sample space on which the L2-space is defined. |
property
writable
sigma_algebra
The sigma-algebra on which the L2-space is defined.
Returns:
| Name | Type | Description |
|---|---|---|
sigma_algebra |
SigmaAlgebra
|
The sigma-algebra on which the L2-space is defined. |
fourier_coefficients(rv)
Compute the Fourier coefficients of a random variable with respect to the basis of the L2-space.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
rv
|
RandomVariable
|
The random variable whose Fourier coefficients are to be computed. |
required |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Returns:
| Name | Type | Description |
|---|---|---|
coefficients |
dict[Hashable, Real]
|
A dictionary mapping the name of each basis vector to the corresponding Fourier coefficient of |
Examples:
>>> from sigalg.core import ProbabilityMeasure, RandomVariable, SampleSpace, SigmaAlgebra
>>> from sigalg.l2 import L2
>>> Omega = SampleSpace().from_sequence(size=3)
>>> F = SigmaAlgebra(sample_space=Omega).from_dict({0: 0, 1: 0, 2: 1})
>>> P = ProbabilityMeasure(sample_space=Omega).from_dict({0: 0.2, 1: 0.5, 2: 0.3})
>>> H = L2(
... sample_space=Omega,
... sigma_algebra=F,
... probability_measure=P,
... )
>>> # Get the Fourier coefficients of X with respect to the basis of H
>>> X = RandomVariable(domain=Omega, name="X").from_dict({0: 2, 1: 2, 2: 3})
>>> coeffs = H.fourier_coefficients(rv=X)
>>> coeffs
{0: 1.6733200530681511, 1: 1.6431676725154982}
>>> # Reconstruct X from its Fourier coefficients and the basis of H
>>> sum(coeffs[basis_name] * basis_vec for basis_name, basis_vec in H.basis.items()).with_name("X_reconstructed")
Random variable 'X_reconstructed':
X_reconstructed
sample
0 2.0
1 2.0
2 3.0
>>> # Define a new probability measure Q that assigns zero probability to an atom in the sigma algebra, and define a new L2-space
>>> Q = ProbabilityMeasure(sample_space=Omega).from_dict({0: 0.2, 1: 0.8, 2: 0.0})
>>> K = L2(
... sample_space=Omega,
... sigma_algebra=F,
... probability_measure=Q,
... )
>>> # Compute the Fourier coefficients of X with respect to the basis of K, and note that there is only one coefficient
>>> K.fourier_coefficients(rv=X)
{0: 2.0}
>>> # Reconstruct X from its Fourier coefficients and the basis of K, and note that the reconstruction differs from the original X on a set of probability zero
>>> (2.0 * K.basis[0]).with_name("X_reconstructed")
Random variable 'X_reconstructed':
X_reconstructed
sample
0 2.0
1 2.0
2 0.0
inner(first, second)
Compute the inner product of two random variables.
Both random variables must be in the L2-space.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
first
|
RandomVariable
|
The first random variable. |
required |
second
|
RandomVariable
|
The second random variable. |
required |
Raises:
| Type | Description |
|---|---|
ValueError
|
If one of the random variables is not in the L2-space. |
Returns:
| Name | Type | Description |
|---|---|---|
inner_product |
Real
|
The inner product of the two random variables. |
Examples:
>>> from sigalg.core import ProbabilityMeasure, RandomVariable, SampleSpace, SigmaAlgebra
>>> from sigalg.l2 import L2
>>> Omega = SampleSpace().from_sequence(size=3)
>>> F = SigmaAlgebra(sample_space=Omega).from_dict({0: 0, 1: 0, 2: 1})
>>> P = ProbabilityMeasure(sample_space=Omega).from_dict({0: 0.2, 1: 0.5, 2: 0.3})
>>> H = L2(
... sample_space=Omega,
... sigma_algebra=F,
... probability_measure=P,
... )
>>> X = RandomVariable(domain=Omega, name="X").from_dict({0: 1, 1: 1, 2: 3})
>>> Y = RandomVariable(domain=Omega, name="Y").from_dict({0: 4, 1: 4, 2: 6})
>>> float(H.inner(X, Y))
8.2
>>> # Example of orthogonal RVs: two indicator functions of disjoint events
>>> A, B = F.to_atoms()
>>> I_A = RandomVariable.indicator_of(A)
>>> I_B = RandomVariable.indicator_of(B)
>>> float(H.inner(I_A, I_B))
0.0
integrate(rv)
Integrate a random variable with respect to the probability measure of the L2-space.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
rv
|
RandomVariable
|
The random variable to be integrated. |
required |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Returns:
| Name | Type | Description |
|---|---|---|
integral |
Real
|
The integral of the random variable with respect to the probability measure of the L2-space. |
Examples:
>>> from sigalg.core import ProbabilityMeasure, RandomVariable, SampleSpace, SigmaAlgebra
>>> from sigalg.l2 import L2
>>> Omega = SampleSpace().from_sequence(size=3)
>>> F = SigmaAlgebra(sample_space=Omega).from_dict({0: 0, 1: 0, 2: 1})
>>> P = ProbabilityMeasure(sample_space=Omega).from_dict({0: 0.2, 1: 0.5, 2: 0.3})
>>> H = L2(
... sample_space=Omega,
... sigma_algebra=F,
... probability_measure=P,
... )
>>> X = RandomVariable(domain=Omega, name="X").from_dict({0: 1, 1: 1, 2: 3})
>>> float(round(H.integrate(X), 2))
1.6
metric(first, second)
Compute the distance between two random variables in the L2-space.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
first
|
RandomVariable
|
The first random variable. |
required |
second
|
RandomVariable
|
The second random variable. |
required |
Raises:
| Type | Description |
|---|---|
ValueError
|
If one of the two random variables is not in the L2-space. |
Returns:
| Name | Type | Description |
|---|---|---|
distance |
Real
|
The distance between the two random variables in the L2-space. |
norm(X)
Compute the norm of a random variable in the L2-space.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
X
|
RandomVariable
|
The random variable whose norm is to be computed. |
required |
Raises:
| Type | Description |
|---|---|
ValueError
|
The random variable must be in the L2-space. |
Returns:
| Name | Type | Description |
|---|---|---|
norm |
Real
|
The norm of the random variable in the L2-space. |
Examples:
>>> from sigalg.core import ProbabilityMeasure, RandomVariable, SampleSpace, SigmaAlgebra
>>> from sigalg.l2 import L2
>>> Omega = SampleSpace().from_sequence(size=3)
>>> F = SigmaAlgebra(sample_space=Omega).from_dict({0: 0, 1: 0, 2: 1})
>>> P = ProbabilityMeasure(sample_space=Omega).from_dict({0: 0.2, 1: 0.5, 2: 0.3})
>>> H = L2(
... sample_space=Omega,
... sigma_algebra=F,
... probability_measure=P,
... )
>>> A, _ = F.to_atoms()
>>> I_A = RandomVariable.indicator_of(A)
>>> # The squared norm of an indicator function is the probability of the corresponding event
>>> float(round(H.norm(I_A) ** 2, 1))
0.7
proj(rv, subspace)
Compute the orthogonal projection of a random variable onto the subspace spanned by a set of random variables.
Let \(H\) be the given \(L^2\)-space, suppose that \(Y\) is a random variable in \(H\), and suppose \(\{X_1,X_2,\ldots,X_n\} \subset H\) spans a subspace \(V\). The random vector \(Y\) is stored in the parameter rv, while the \(X_k\)'s are stored in the parameter subspace. The goal is to compute the orthogonal projection \(\hat{Y}\) of \(Y\) onto \(V\). By definition, this is a minimizer of the squared norm \(\|Y - \hat{Y}\|^2\), over all \(\hat{Y} \in V\). The random variable \(\hat{Y}\) is returned as proj. The method also computes coefficients \(u_1,u_2,\ldots,u_n\) such that
If \(V\) has dimension \(d<n\), then there are infinitely many choices of coefficients \(u_1,u_2,\ldots,u_n\); in this case, the method returns the particular choice of coefficients for which \(\sum_{k=1}^n u_k^2\) is minimized. The coefficients are returned as an np.ndarray in the variable coefficients, in the same order as the random variables in the input list subspace. The method also returns the dimension \(d\) of the subspace \(V\) as the variable dim.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
rv
|
RandomVariable
|
The random variable to be projected. |
required |
subspace
|
list[RandomVariable]
|
A list of random variables spanning the subspace onto which |
required |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Returns:
| Name | Type | Description |
|---|---|---|
proj |
RandomVariable
|
The orthogonal projection of |
coefficients |
ndarray
|
The coefficients of the projection of |
dim |
int
|
The dimension of the subspace spanned by |
Examples:
>>> from sigalg.core import ProbabilityMeasure, RandomVariable, SampleSpace
>>> from sigalg.l2 import L2
>>> # Define a sample space and probability measure
>>> Omega = SampleSpace().from_sequence(size=4)
>>> P = ProbabilityMeasure(sample_space=Omega).from_dict(
... {
... 0: 0.2,
... 1: 0.4,
... 2: 0.2,
... 3: 0.2,
... }
... )
>>> # Define an L2 space with default power set sigma-algebra
>>> H = L2(sample_space=Omega, probability_measure=P)
>>> # For a quadratic regression example, we will project a random variable Y onto the subspace spanned by 1, X, and X^2
>>> one = RandomVariable(domain=Omega, name="one").from_constant(1)
>>> X = RandomVariable(domain=Omega, name="X").from_dict(
... {
... 0: 2.0,
... 1: 3.0,
... 2: 5.0,
... 3: 7.0,
... }
... )
>>> Y = RandomVariable(domain=Omega, name="Y").from_dict(
... {
... 0: 1.0,
... 1: 3.0,
... 2: 2.0,
... 3: 4.0,
... }
... )
>>> # Project Y onto the subspace spanned by 1, X, and X^2
>>> proj, u, dim = H.proj(rv=Y, subspace=[one, X, X**2])
>>> expected_proj = sum([u[k] * X**k for k in range(dim)])
>>> # Check that the projection is correct
>>> print(proj == expected_proj)
True