Parameters¶
Parameters are values which characterize the behavior of a model. When fitting a model, we want to find the values of the parameters which best allow the model to explain the data. However, with Bayesian modeling we want not only to find the single best value for each parameter, but a probability distribution which describes how likely any given value of a parameter is to be the best or true value.
Parameters have both priors (probability distributions which describe how likely we think different values for the parameter are before taking into consideration the current data), and posteriors (probability distributions which describe how likely we think different values for the parameter are after taking into consideration the current data). The prior is set to a specific distribution before fitting the model. While the type of distribution used for the posterior is set before fitting the model, the shape of that distribution (the value of the parameters which define the distribution) is optimized while fitting the model. See the Mathematical Details section for more info.
The Parameter
class can be used to create any probabilistic
parameter.
For convenience, ProbFlow also includes some classes which are special cases of
a Parameter
:
ScaleParameter
- standard deviation parameterCategoricalParameter
- categorical parameterDirichletParameter
- parameter with a Dirichlet posteriorBoundedParameter
- parameter which is bounded between 0 and 1PositiveParameter
- parameter which is always greater than 0DeterministicParameter
- a non-probabilistic parameterMultivariateNormalParameter
- parameter with a multivariate Normal posteriorCenteredParameter
- parameter array with a mean of 0
See the user guide for more information on Parameters.
- class probflow.parameters.BoundedParameter(shape=1, posterior=<class 'probflow.distributions.normal.Normal'>, prior=<probflow.distributions.normal.Normal object>, transform=None, initializer={'loc': <function xavier>, 'scale': <function scale_xavier>}, var_transform={'loc': None, 'scale': <function softplus>}, min: float = 0.0, max: float = 1.0, name='BoundedParameter')[source]¶
Bases:
probflow.parameters.parameter.Parameter
A parameter bounded on either side
This is a convenience class for creating a parameter \(\beta\) bounded on both sides. It uses a logit-normal posterior distribution:
\[\text{Logit}(\beta) = \log \left( \frac{\beta}{1-\beta} \right) \sim \text{Normal}(\mu, \sigma)\]- Parameters
shape (int or List[int]) – Shape of the array containing the parameters. Default =
1
posterior (Distribution class) – Probability distribution class to use to approximate the posterior. Default =
Normal
prior (Distribution object) – Prior probability distribution function which has been instantiated with parameters. Default =
Normal
(0, 1)
transform (callable) – Transform to apply to the random variable. Default is to use a sigmoid transform.
initializer (Dict[str, callable]) – Initializer functions to use for each variable of the variational posterior distribution. Keys correspond to variable names (arguments to the distribution), and values contain functions to initialize those variables given
shape
as the single argument.var_transform (Dict[str, callable]) – Transform to apply to each variable of the variational posterior.
min (float) – Minimum value the parameter can take. Default = 0.
max (float) – Maximum value the parameter can take. Default = 1.
name (str) – Name of the parameter(s). Default =
'BoundedParameter'
Examples
TODO
- bayesian_update()¶
Update priors to match the current posterior
- kl_loss()¶
Compute the sum of the Kullback–Leibler divergences between this parameter’s priors and its variational posteriors.
- property n_parameters¶
Get the number of independent parameters
- property n_variables¶
Get the number of underlying variables
- property posterior¶
This Parameter’s variational posterior distribution
- posterior_ci(ci: float = 0.95, n: int = 10000)¶
Posterior confidence intervals
- Parameters
- Returns
lb (float or |ndarray|) – Lower bound(s) of the confidence interval(s)
ub (float or |ndarray|) – Upper bound(s) of the confidence interval(s)
- posterior_mean()¶
Get the mean of the posterior distribution(s)
- posterior_plot(n: int = 10000, style: str = 'fill', bins: Union[int, list, numpy.ndarray] = 20, ci: float = 0.0, bw: float = 0.075, alpha: float = 0.4, color=None, **kwargs)¶
Plot distribution of samples from the posterior distribution.
- Parameters
n (int) – Number of samples to take from each posterior distribution for estimating the density. Default = 10000
style (str) –
Which style of plot to show. Available types are:
'fill'
- filled density plot (the default)'line'
- line density plot'hist'
- histogram
bins (int or list or
ndarray
) – Number of bins to use for the posterior density histogram (ifstyle='hist'
), or a list or vector of bin edges.ci (float between 0 and 1) – Confidence interval to plot. Default = 0.0 (i.e., not plotted)
bw (float) – Bandwidth of the kernel density estimate (if using
style='line'
orstyle='fill'
). Default is 0.075alpha (float between 0 and 1) – Transparency of fill/histogram
color (matplotlib color code or list of them) – Color(s) to use to plot the distribution. See https://matplotlib.org/tutorials/colors/colors.html Default = use the default matplotlib color cycle
kwargs – Additional keyword arguments are passed to
utils.plotting.plot_dist()
- posterior_sample(n: int = 1)¶
Sample from the posterior distribution.
- Parameters
n (int > 0) – Number of samples to draw from the posterior distribution. Default = 1
- Returns
Samples from the parameter’s posterior distribution. If
n>1
of size(n, self.prior.shape)
. Ifn==1
, of size(self.prior.shape)
.- Return type
- prior_plot(n: int = 10000, style: str = 'fill', bins: Union[int, list, numpy.ndarray] = 20, ci: float = 0.0, bw: float = 0.075, alpha: float = 0.4, color=None)¶
Plot distribution of samples from the prior distribution.
- Parameters
n (int) – Number of samples to take from each prior distribution for estimating the density. Default = 1000
style (str) –
Which style of plot to show. Available types are:
'fill'
- filled density plot (the default)'line'
- line density plot'hist'
- histogram
bins (int or list or
ndarray
) – Number of bins to use for the prior density histogram (ifstyle='hist'
), or a list or vector of bin edges.ci (float between 0 and 1) – Confidence interval to plot. Default = 0.0 (i.e., not plotted)
bw (float) – Bandwidth of the kernel density estimate (if using
style='line'
orstyle='fill'
). Default is 0.075alpha (float between 0 and 1) – Transparency of fill/histogram
color (matplotlib color code or list of them) – Color(s) to use to plot the distribution. See https://matplotlib.org/tutorials/colors/colors.html Default = use the default matplotlib color cycle
- prior_sample(n: int = 1)¶
Sample from the prior distribution.
- Parameters
n (int > 0) – Number of samples to draw from the prior distribution. Default = 1
- Returns
Samples from the parameter prior distribution. If
n>1
of size(n, self.prior.shape)
. Ifn==1
, of size(self.prior.shape)
.- Return type
- property trainable_variables¶
Get a list of trainable variables from the backend
- property variables¶
Variables after applying their respective transformations
- class probflow.parameters.CategoricalParameter(k: int = 2, shape: Union[int, List[int]] = [], posterior=<class 'probflow.distributions.categorical.Categorical'>, prior=None, transform=None, initializer={'probs': <function xavier>}, var_transform={'probs': <function additive_logistic_transform>}, name='CategoricalParameter')[source]¶
Bases:
probflow.parameters.parameter.Parameter
Categorical parameter.
This is a convenience class for creating a categorical parameter \(\beta\) with a Categorical posterior:
\[\beta \sim \text{Categorical}(\mathbf{\theta})\]By default, a uniform prior is used.
TODO: explain that a sample is an int in [0, k-1]
- Parameters
k (int > 2) – Number of categories.
shape (int or List[int]) – Shape of the array containing the parameters. Default =
1
posterior (Distribution class) – Probability distribution class to use to approximate the posterior. Default =
Categorical
prior (Distribution object) – Prior probability distribution function which has been instantiated with parameters. Default =
Categorical
(1/k)
transform (callable) – Transform to apply to the random variable. Default is to use no transform.
initializer (Dict[str, callable]) – Initializer functions to use for each variable of the variational posterior distribution. Keys correspond to variable names (arguments to the distribution), and values contain functions to initialize those variables given
shape
as the single argument.var_transform (Dict[str, callable]) – Transform to apply to each variable of the variational posterior.
name (str) – Name of the parameter(s). Default =
'CategoricalParameter'
Examples
TODO: creating variable
- bayesian_update()¶
Update priors to match the current posterior
- kl_loss()¶
Compute the sum of the Kullback–Leibler divergences between this parameter’s priors and its variational posteriors.
- property n_parameters¶
Get the number of independent parameters
- property n_variables¶
Get the number of underlying variables
- property posterior¶
This Parameter’s variational posterior distribution
- posterior_ci(ci: float = 0.95, n: int = 10000)¶
Posterior confidence intervals
- Parameters
- Returns
lb (float or |ndarray|) – Lower bound(s) of the confidence interval(s)
ub (float or |ndarray|) – Upper bound(s) of the confidence interval(s)
- posterior_mean()¶
Get the mean of the posterior distribution(s)
- posterior_plot(n: int = 10000, style: str = 'fill', bins: Union[int, list, numpy.ndarray] = 20, ci: float = 0.0, bw: float = 0.075, alpha: float = 0.4, color=None, **kwargs)¶
Plot distribution of samples from the posterior distribution.
- Parameters
n (int) – Number of samples to take from each posterior distribution for estimating the density. Default = 10000
style (str) –
Which style of plot to show. Available types are:
'fill'
- filled density plot (the default)'line'
- line density plot'hist'
- histogram
bins (int or list or
ndarray
) – Number of bins to use for the posterior density histogram (ifstyle='hist'
), or a list or vector of bin edges.ci (float between 0 and 1) – Confidence interval to plot. Default = 0.0 (i.e., not plotted)
bw (float) – Bandwidth of the kernel density estimate (if using
style='line'
orstyle='fill'
). Default is 0.075alpha (float between 0 and 1) – Transparency of fill/histogram
color (matplotlib color code or list of them) – Color(s) to use to plot the distribution. See https://matplotlib.org/tutorials/colors/colors.html Default = use the default matplotlib color cycle
kwargs – Additional keyword arguments are passed to
utils.plotting.plot_dist()
- posterior_sample(n: int = 1)¶
Sample from the posterior distribution.
- Parameters
n (int > 0) – Number of samples to draw from the posterior distribution. Default = 1
- Returns
Samples from the parameter’s posterior distribution. If
n>1
of size(n, self.prior.shape)
. Ifn==1
, of size(self.prior.shape)
.- Return type
- prior_plot(n: int = 10000, style: str = 'fill', bins: Union[int, list, numpy.ndarray] = 20, ci: float = 0.0, bw: float = 0.075, alpha: float = 0.4, color=None)¶
Plot distribution of samples from the prior distribution.
- Parameters
n (int) – Number of samples to take from each prior distribution for estimating the density. Default = 1000
style (str) –
Which style of plot to show. Available types are:
'fill'
- filled density plot (the default)'line'
- line density plot'hist'
- histogram
bins (int or list or
ndarray
) – Number of bins to use for the prior density histogram (ifstyle='hist'
), or a list or vector of bin edges.ci (float between 0 and 1) – Confidence interval to plot. Default = 0.0 (i.e., not plotted)
bw (float) – Bandwidth of the kernel density estimate (if using
style='line'
orstyle='fill'
). Default is 0.075alpha (float between 0 and 1) – Transparency of fill/histogram
color (matplotlib color code or list of them) – Color(s) to use to plot the distribution. See https://matplotlib.org/tutorials/colors/colors.html Default = use the default matplotlib color cycle
- prior_sample(n: int = 1)¶
Sample from the prior distribution.
- Parameters
n (int > 0) – Number of samples to draw from the prior distribution. Default = 1
- Returns
Samples from the parameter prior distribution. If
n>1
of size(n, self.prior.shape)
. Ifn==1
, of size(self.prior.shape)
.- Return type
- property trainable_variables¶
Get a list of trainable variables from the backend
- property variables¶
Variables after applying their respective transformations
- class probflow.parameters.CenteredParameter(shape: Union[int, List[int]], center_by: str = 'all', name='CenteredParameter')[source]¶
Bases:
probflow.parameters.parameter.Parameter
A vector of parameters centered at 0.
Uses a QR decomposition to transform a vector of \(K-1\) unconstrained parameters into a vector of \(K\) variables centered at zero (i.e. the mean of the elements in the vector is 0). It starts with a \(K \times K\) matrix \(A\) which has \(1\)-1`s along the bottom - except for the bottom-right element which is \(0\):
\[\begin{split}\mathbf{A} = \begin{bmatrix} 1 & 0 & \dots & 0 & 0 \\ 0 & 1 & & 0 & 0 \\ \vdots & & \ddots & & \vdots \\ 0 & 0 & \dots & 1 & 0 \\ -1 & -1 & \dots & -1 & 0 \\ \end{bmatrix}\end{split}\]The \(QR\) decomposition is performed on this matrix such that
\[\mathbf{A} = \mathbf{Q} \mathbf{R}\]A vector of \(K-1\) unconstrained variables \(\mathbf{u}\) is then transformed into a vector \(\mathbf{v}\) of \(K\) centered variables by
\[\mathbf{v} = \mathbf{Q}_{1:K, 1:K-1} \mathbf{u}\]The prior on the untransformed variables is
\[\mathbf{u} \sim \text{Normal}(0, \frac{1}{\sqrt{1 - \frac{1}{K}}})\]Such that the effective prior on the transformed parameters works out to be
\[\mathbf{v} \sim \text{Normal}(0, 1)\]Prior is fixed!
Note that the prior on the parameters is fixed at \(\text{Normal}(0, 1)\). This is because the true prior is being placed on the untransformed parameters (see above).
- Parameters
d (int or list) – Length of the parameter vector, or shape of the parameter matrix.
center_by (str {'all', 'column', 'row'}) – If
all
(the default), the sum of all parameters in the resulting vector or matrix will be 0. Ifcolumn
, the sum of each column will be 0 (but the sum across rows will not necessarily be 0). Ifrow
, the sum of each row will be 0 (but the sum across columns will not necessarily be 0).name (str) – Name of the parameter(s). Default =
'CenteredParameter'
Examples
TODO
- bayesian_update()¶
Update priors to match the current posterior
- kl_loss()¶
Compute the sum of the Kullback–Leibler divergences between this parameter’s priors and its variational posteriors.
- property n_parameters¶
Get the number of independent parameters
- property n_variables¶
Get the number of underlying variables
- property posterior¶
This Parameter’s variational posterior distribution
- posterior_ci(ci: float = 0.95, n: int = 10000)¶
Posterior confidence intervals
- Parameters
- Returns
lb (float or |ndarray|) – Lower bound(s) of the confidence interval(s)
ub (float or |ndarray|) – Upper bound(s) of the confidence interval(s)
- posterior_mean()¶
Get the mean of the posterior distribution(s)
- posterior_plot(n: int = 10000, style: str = 'fill', bins: Union[int, list, numpy.ndarray] = 20, ci: float = 0.0, bw: float = 0.075, alpha: float = 0.4, color=None, **kwargs)¶
Plot distribution of samples from the posterior distribution.
- Parameters
n (int) – Number of samples to take from each posterior distribution for estimating the density. Default = 10000
style (str) –
Which style of plot to show. Available types are:
'fill'
- filled density plot (the default)'line'
- line density plot'hist'
- histogram
bins (int or list or
ndarray
) – Number of bins to use for the posterior density histogram (ifstyle='hist'
), or a list or vector of bin edges.ci (float between 0 and 1) – Confidence interval to plot. Default = 0.0 (i.e., not plotted)
bw (float) – Bandwidth of the kernel density estimate (if using
style='line'
orstyle='fill'
). Default is 0.075alpha (float between 0 and 1) – Transparency of fill/histogram
color (matplotlib color code or list of them) – Color(s) to use to plot the distribution. See https://matplotlib.org/tutorials/colors/colors.html Default = use the default matplotlib color cycle
kwargs – Additional keyword arguments are passed to
utils.plotting.plot_dist()
- posterior_sample(n: int = 1)¶
Sample from the posterior distribution.
- Parameters
n (int > 0) – Number of samples to draw from the posterior distribution. Default = 1
- Returns
Samples from the parameter’s posterior distribution. If
n>1
of size(n, self.prior.shape)
. Ifn==1
, of size(self.prior.shape)
.- Return type
- prior_plot(n: int = 10000, style: str = 'fill', bins: Union[int, list, numpy.ndarray] = 20, ci: float = 0.0, bw: float = 0.075, alpha: float = 0.4, color=None)¶
Plot distribution of samples from the prior distribution.
- Parameters
n (int) – Number of samples to take from each prior distribution for estimating the density. Default = 1000
style (str) –
Which style of plot to show. Available types are:
'fill'
- filled density plot (the default)'line'
- line density plot'hist'
- histogram
bins (int or list or
ndarray
) – Number of bins to use for the prior density histogram (ifstyle='hist'
), or a list or vector of bin edges.ci (float between 0 and 1) – Confidence interval to plot. Default = 0.0 (i.e., not plotted)
bw (float) – Bandwidth of the kernel density estimate (if using
style='line'
orstyle='fill'
). Default is 0.075alpha (float between 0 and 1) – Transparency of fill/histogram
color (matplotlib color code or list of them) – Color(s) to use to plot the distribution. See https://matplotlib.org/tutorials/colors/colors.html Default = use the default matplotlib color cycle
- prior_sample(n: int = 1)¶
Sample from the prior distribution.
- Parameters
n (int > 0) – Number of samples to draw from the prior distribution. Default = 1
- Returns
Samples from the parameter prior distribution. If
n>1
of size(n, self.prior.shape)
. Ifn==1
, of size(self.prior.shape)
.- Return type
- property trainable_variables¶
Get a list of trainable variables from the backend
- property variables¶
Variables after applying their respective transformations
- class probflow.parameters.DeterministicParameter(shape=1, posterior=<class 'probflow.distributions.deterministic.Deterministic'>, prior=<probflow.distributions.normal.Normal object>, transform=None, initializer={'loc': <function xavier>}, var_transform={'loc': None}, name='DeterministicParameter')[source]¶
Bases:
probflow.parameters.parameter.Parameter
A parameter which takes only a single value (i.e., the posterior is a single point value, not a probability distribution).
- Parameters
shape (int or List[int]) – Shape of the array containing the parameters. Default =
1
posterior (Distribution class) – Probability distribution class to use to approximate the posterior. Default =
Deterministic
prior (Distribution object) – Prior probability distribution function which has been instantiated with parameters. Default =
Normal
(0, 1)
transform (callable) – Transform to apply to the random variable. Default is to use no transformation.
initializer (Dict[str, callable]) – Initializer functions to use for each variable of the variational posterior distribution. Keys correspond to variable names (arguments to the distribution), and values contain functions to initialize those variables given
shape
as the single argument.var_transform (Dict[str, callable]) – Transform to apply to each variable of the variational posterior.
name (str) – Name of the parameter(s). Default =
'PositiveParameter'
Examples
TODO
- bayesian_update()¶
Update priors to match the current posterior
- kl_loss()¶
Compute the sum of the Kullback–Leibler divergences between this parameter’s priors and its variational posteriors.
- property n_parameters¶
Get the number of independent parameters
- property n_variables¶
Get the number of underlying variables
- property posterior¶
This Parameter’s variational posterior distribution
- posterior_ci(ci: float = 0.95, n: int = 10000)¶
Posterior confidence intervals
- Parameters
- Returns
lb (float or |ndarray|) – Lower bound(s) of the confidence interval(s)
ub (float or |ndarray|) – Upper bound(s) of the confidence interval(s)
- posterior_mean()¶
Get the mean of the posterior distribution(s)
- posterior_plot(n: int = 10000, style: str = 'fill', bins: Union[int, list, numpy.ndarray] = 20, ci: float = 0.0, bw: float = 0.075, alpha: float = 0.4, color=None, **kwargs)¶
Plot distribution of samples from the posterior distribution.
- Parameters
n (int) – Number of samples to take from each posterior distribution for estimating the density. Default = 10000
style (str) –
Which style of plot to show. Available types are:
'fill'
- filled density plot (the default)'line'
- line density plot'hist'
- histogram
bins (int or list or
ndarray
) – Number of bins to use for the posterior density histogram (ifstyle='hist'
), or a list or vector of bin edges.ci (float between 0 and 1) – Confidence interval to plot. Default = 0.0 (i.e., not plotted)
bw (float) – Bandwidth of the kernel density estimate (if using
style='line'
orstyle='fill'
). Default is 0.075alpha (float between 0 and 1) – Transparency of fill/histogram
color (matplotlib color code or list of them) – Color(s) to use to plot the distribution. See https://matplotlib.org/tutorials/colors/colors.html Default = use the default matplotlib color cycle
kwargs – Additional keyword arguments are passed to
utils.plotting.plot_dist()
- posterior_sample(n: int = 1)¶
Sample from the posterior distribution.
- Parameters
n (int > 0) – Number of samples to draw from the posterior distribution. Default = 1
- Returns
Samples from the parameter’s posterior distribution. If
n>1
of size(n, self.prior.shape)
. Ifn==1
, of size(self.prior.shape)
.- Return type
- prior_plot(n: int = 10000, style: str = 'fill', bins: Union[int, list, numpy.ndarray] = 20, ci: float = 0.0, bw: float = 0.075, alpha: float = 0.4, color=None)¶
Plot distribution of samples from the prior distribution.
- Parameters
n (int) – Number of samples to take from each prior distribution for estimating the density. Default = 1000
style (str) –
Which style of plot to show. Available types are:
'fill'
- filled density plot (the default)'line'
- line density plot'hist'
- histogram
bins (int or list or
ndarray
) – Number of bins to use for the prior density histogram (ifstyle='hist'
), or a list or vector of bin edges.ci (float between 0 and 1) – Confidence interval to plot. Default = 0.0 (i.e., not plotted)
bw (float) – Bandwidth of the kernel density estimate (if using
style='line'
orstyle='fill'
). Default is 0.075alpha (float between 0 and 1) – Transparency of fill/histogram
color (matplotlib color code or list of them) – Color(s) to use to plot the distribution. See https://matplotlib.org/tutorials/colors/colors.html Default = use the default matplotlib color cycle
- prior_sample(n: int = 1)¶
Sample from the prior distribution.
- Parameters
n (int > 0) – Number of samples to draw from the prior distribution. Default = 1
- Returns
Samples from the parameter prior distribution. If
n>1
of size(n, self.prior.shape)
. Ifn==1
, of size(self.prior.shape)
.- Return type
- property trainable_variables¶
Get a list of trainable variables from the backend
- property variables¶
Variables after applying their respective transformations
- class probflow.parameters.DirichletParameter(k: int = 2, shape: Union[int, List[int]] = [], posterior=<class 'probflow.distributions.dirichlet.Dirichlet'>, prior=None, transform=None, initializer={'concentration': <function pos_xavier>}, var_transform={'concentration': <function softplus>}, name='DirichletParameter')[source]¶
Bases:
probflow.parameters.parameter.Parameter
Dirichlet parameter.
This is a convenience class for creating a parameter \(\theta\) with a Dirichlet posterior:
\[\theta \sim \text{Dirichlet}(\mathbf{\alpha})\]By default, a uniform Dirichlet prior is used:
\[\theta \sim \text{Dirichlet}_K(\mathbf{1}/K)\]TODO: explain that a sample is a categorical prob dist (as compared to CategoricalParameter, where a sample is a single value)
- Parameters
k (int > 2) – Number of categories.
shape (int or List[int]) – Shape of the array containing the parameters. Default =
1
posterior (Distribution class) – Probability distribution class to use to approximate the posterior. Default =
Dirichlet
prior (Distribution object) – Prior probability distribution function which has been instantiated with parameters. Default =
Dirichlet
(1)
transform (callable) – Transform to apply to the random variable. Default is to use no transform.
initializer (Dict[str, callable]) – Initializer functions to use for each variable of the variational posterior distribution. Keys correspond to variable names (arguments to the distribution), and values contain functions to initialize those variables given
shape
as the single argument.var_transform (Dict[str, callable]) – Transform to apply to each variable of the variational posterior.
name (str) – Name of the parameter(s). Default =
'DirichletParameter'
Examples
TODO: creating variable
- bayesian_update()¶
Update priors to match the current posterior
- kl_loss()¶
Compute the sum of the Kullback–Leibler divergences between this parameter’s priors and its variational posteriors.
- property n_parameters¶
Get the number of independent parameters
- property n_variables¶
Get the number of underlying variables
- property posterior¶
This Parameter’s variational posterior distribution
- posterior_ci(ci: float = 0.95, n: int = 10000)¶
Posterior confidence intervals
- Parameters
- Returns
lb (float or |ndarray|) – Lower bound(s) of the confidence interval(s)
ub (float or |ndarray|) – Upper bound(s) of the confidence interval(s)
- posterior_mean()¶
Get the mean of the posterior distribution(s)
- posterior_plot(n: int = 10000, style: str = 'fill', bins: Union[int, list, numpy.ndarray] = 20, ci: float = 0.0, bw: float = 0.075, alpha: float = 0.4, color=None, **kwargs)¶
Plot distribution of samples from the posterior distribution.
- Parameters
n (int) – Number of samples to take from each posterior distribution for estimating the density. Default = 10000
style (str) –
Which style of plot to show. Available types are:
'fill'
- filled density plot (the default)'line'
- line density plot'hist'
- histogram
bins (int or list or
ndarray
) – Number of bins to use for the posterior density histogram (ifstyle='hist'
), or a list or vector of bin edges.ci (float between 0 and 1) – Confidence interval to plot. Default = 0.0 (i.e., not plotted)
bw (float) – Bandwidth of the kernel density estimate (if using
style='line'
orstyle='fill'
). Default is 0.075alpha (float between 0 and 1) – Transparency of fill/histogram
color (matplotlib color code or list of them) – Color(s) to use to plot the distribution. See https://matplotlib.org/tutorials/colors/colors.html Default = use the default matplotlib color cycle
kwargs – Additional keyword arguments are passed to
utils.plotting.plot_dist()
- posterior_sample(n: int = 1)¶
Sample from the posterior distribution.
- Parameters
n (int > 0) – Number of samples to draw from the posterior distribution. Default = 1
- Returns
Samples from the parameter’s posterior distribution. If
n>1
of size(n, self.prior.shape)
. Ifn==1
, of size(self.prior.shape)
.- Return type
- prior_plot(n: int = 10000, style: str = 'fill', bins: Union[int, list, numpy.ndarray] = 20, ci: float = 0.0, bw: float = 0.075, alpha: float = 0.4, color=None)¶
Plot distribution of samples from the prior distribution.
- Parameters
n (int) – Number of samples to take from each prior distribution for estimating the density. Default = 1000
style (str) –
Which style of plot to show. Available types are:
'fill'
- filled density plot (the default)'line'
- line density plot'hist'
- histogram
bins (int or list or
ndarray
) – Number of bins to use for the prior density histogram (ifstyle='hist'
), or a list or vector of bin edges.ci (float between 0 and 1) – Confidence interval to plot. Default = 0.0 (i.e., not plotted)
bw (float) – Bandwidth of the kernel density estimate (if using
style='line'
orstyle='fill'
). Default is 0.075alpha (float between 0 and 1) – Transparency of fill/histogram
color (matplotlib color code or list of them) – Color(s) to use to plot the distribution. See https://matplotlib.org/tutorials/colors/colors.html Default = use the default matplotlib color cycle
- prior_sample(n: int = 1)¶
Sample from the prior distribution.
- Parameters
n (int > 0) – Number of samples to draw from the prior distribution. Default = 1
- Returns
Samples from the parameter prior distribution. If
n>1
of size(n, self.prior.shape)
. Ifn==1
, of size(self.prior.shape)
.- Return type
- property trainable_variables¶
Get a list of trainable variables from the backend
- property variables¶
Variables after applying their respective transformations
- class probflow.parameters.MultivariateNormalParameter(d: int = 1, prior=None, expand_dims: int = - 1, name='MultivariateNormalParameter')[source]¶
Bases:
probflow.parameters.parameter.Parameter
A parameter with a multivariate normal posterior, with full covariance.
TODO: uses the log-Cholesky parameterization (Pinheiro & Bates, 1996).
TODO: support shape?
- Parameters
d (int) – Number of dimensions
prior (Distribution object) – Prior probability distribution function which has been instantiated with parameters. Default =
MultivariateNormal
(0, I)
expand_dims (int or None) – Dimension to expand output samples along.
name (str) – Name of the parameter(s). Default =
'MultivariateNormalParameter'
Examples
TODO
References
Jose C. Pinheiro & Douglas M. Bates. Unconstrained Parameterizations for Variance-Covariance Matrices Statistics and Computing, 1996.
- bayesian_update()¶
Update priors to match the current posterior
- kl_loss()¶
Compute the sum of the Kullback–Leibler divergences between this parameter’s priors and its variational posteriors.
- property n_parameters¶
Get the number of independent parameters
- property n_variables¶
Get the number of underlying variables
- property posterior¶
This Parameter’s variational posterior distribution
- posterior_ci(ci: float = 0.95, n: int = 10000)¶
Posterior confidence intervals
- Parameters
- Returns
lb (float or |ndarray|) – Lower bound(s) of the confidence interval(s)
ub (float or |ndarray|) – Upper bound(s) of the confidence interval(s)
- posterior_mean()¶
Get the mean of the posterior distribution(s)
- posterior_plot(n: int = 10000, style: str = 'fill', bins: Union[int, list, numpy.ndarray] = 20, ci: float = 0.0, bw: float = 0.075, alpha: float = 0.4, color=None, **kwargs)¶
Plot distribution of samples from the posterior distribution.
- Parameters
n (int) – Number of samples to take from each posterior distribution for estimating the density. Default = 10000
style (str) –
Which style of plot to show. Available types are:
'fill'
- filled density plot (the default)'line'
- line density plot'hist'
- histogram
bins (int or list or
ndarray
) – Number of bins to use for the posterior density histogram (ifstyle='hist'
), or a list or vector of bin edges.ci (float between 0 and 1) – Confidence interval to plot. Default = 0.0 (i.e., not plotted)
bw (float) – Bandwidth of the kernel density estimate (if using
style='line'
orstyle='fill'
). Default is 0.075alpha (float between 0 and 1) – Transparency of fill/histogram
color (matplotlib color code or list of them) – Color(s) to use to plot the distribution. See https://matplotlib.org/tutorials/colors/colors.html Default = use the default matplotlib color cycle
kwargs – Additional keyword arguments are passed to
utils.plotting.plot_dist()
- posterior_sample(n: int = 1)¶
Sample from the posterior distribution.
- Parameters
n (int > 0) – Number of samples to draw from the posterior distribution. Default = 1
- Returns
Samples from the parameter’s posterior distribution. If
n>1
of size(n, self.prior.shape)
. Ifn==1
, of size(self.prior.shape)
.- Return type
- prior_plot(n: int = 10000, style: str = 'fill', bins: Union[int, list, numpy.ndarray] = 20, ci: float = 0.0, bw: float = 0.075, alpha: float = 0.4, color=None)¶
Plot distribution of samples from the prior distribution.
- Parameters
n (int) – Number of samples to take from each prior distribution for estimating the density. Default = 1000
style (str) –
Which style of plot to show. Available types are:
'fill'
- filled density plot (the default)'line'
- line density plot'hist'
- histogram
bins (int or list or
ndarray
) – Number of bins to use for the prior density histogram (ifstyle='hist'
), or a list or vector of bin edges.ci (float between 0 and 1) – Confidence interval to plot. Default = 0.0 (i.e., not plotted)
bw (float) – Bandwidth of the kernel density estimate (if using
style='line'
orstyle='fill'
). Default is 0.075alpha (float between 0 and 1) – Transparency of fill/histogram
color (matplotlib color code or list of them) – Color(s) to use to plot the distribution. See https://matplotlib.org/tutorials/colors/colors.html Default = use the default matplotlib color cycle
- prior_sample(n: int = 1)¶
Sample from the prior distribution.
- Parameters
n (int > 0) – Number of samples to draw from the prior distribution. Default = 1
- Returns
Samples from the parameter prior distribution. If
n>1
of size(n, self.prior.shape)
. Ifn==1
, of size(self.prior.shape)
.- Return type
- property trainable_variables¶
Get a list of trainable variables from the backend
- property variables¶
Variables after applying their respective transformations
- class probflow.parameters.Parameter(shape: Union[int, List[int]] = 1, posterior: Type[probflow.utils.base.BaseDistribution] = <class 'probflow.distributions.normal.Normal'>, prior: probflow.utils.base.BaseDistribution = <probflow.distributions.normal.Normal object>, transform: Optional[Callable] = None, initializer: Dict[str, Callable] = {'loc': <function xavier>, 'scale': <function scale_xavier>}, var_transform: Dict[str, Callable] = {'loc': None, 'scale': <function softplus>}, name: str = 'Parameter')[source]¶
Bases:
probflow.utils.base.BaseParameter
Probabilistic parameter(s).
A probabilistic parameter \(\beta\). The default posterior distribution is the
Normal
distribution, and the default prior is aNormal
distribution with a mean of 0 and a standard deviation of 1.The prior for a Parameter can be set to any Distribution object (via the
prior
argument), and the type of distribution to use for the posterior can be set to any Distribution class (using theposterior
argument).The parameter can be given a specific name using the
name
argument. This makes it easier to access specific parameters after fitting the model (e.g. in order to view the posterior distribution).The number of independent parameters represented by this
Parameter
object can be set using theshape
argument. For example, to create a vector of 5 parameters, setshape=5
, or to create a 20x7 matrix of parameters setshape=[20,7]
.- Parameters
shape (int or List[int]) – Shape of the array containing the parameters. Default =
1
posterior (Distribution class) – Probability distribution class to use to approximate the posterior. Default =
Normal
prior (Distribution object) – Prior probability distribution function which has been instantiated with parameters. Default =
Normal
(0,1)
transform (callable) – Transform to apply to the random variable. For example, to create a parameter with an inverse gamma posterior, use
posterior``=:class:`.Gamma
andtransform = lambda x: tf.reciprocal(x)
Default is to use no transform.initializer (Dict[str, callable] or Dict[str, int] or Dict[str, float]) – Initializer functions to use for each variable of the variational posterior distribution. Keys correspond to variable names (arguments to the distribution), and values contain functions to initialize those variables given
shape
as the single argument. Or, keys can be a float or an int, in which case all elements will be initialized to that value.var_transform (Dict[str, callable]) – Transform to apply to each variable of the variational posterior. For example to transform the standard deviation parameter from untransformed space to transformed, positive, space, use
initializer={'scale': tf.random.randn}
andvar_transform={'scale': tf.nn.softplus}
name (str) – Name of the parameter(s). Default =
'Parameter'
- property n_parameters¶
Get the number of independent parameters
- property n_variables¶
Get the number of underlying variables
- property trainable_variables¶
Get a list of trainable variables from the backend
- property variables¶
Variables after applying their respective transformations
- property posterior¶
This Parameter’s variational posterior distribution
- kl_loss()[source]¶
Compute the sum of the Kullback–Leibler divergences between this parameter’s priors and its variational posteriors.
- posterior_sample(n: int = 1)[source]¶
Sample from the posterior distribution.
- Parameters
n (int > 0) – Number of samples to draw from the posterior distribution. Default = 1
- Returns
Samples from the parameter’s posterior distribution. If
n>1
of size(n, self.prior.shape)
. Ifn==1
, of size(self.prior.shape)
.- Return type
- prior_sample(n: int = 1)[source]¶
Sample from the prior distribution.
- Parameters
n (int > 0) – Number of samples to draw from the prior distribution. Default = 1
- Returns
Samples from the parameter prior distribution. If
n>1
of size(n, self.prior.shape)
. Ifn==1
, of size(self.prior.shape)
.- Return type
- posterior_ci(ci: float = 0.95, n: int = 10000)[source]¶
Posterior confidence intervals
- Parameters
- Returns
lb (float or |ndarray|) – Lower bound(s) of the confidence interval(s)
ub (float or |ndarray|) – Upper bound(s) of the confidence interval(s)
- posterior_plot(n: int = 10000, style: str = 'fill', bins: Union[int, list, numpy.ndarray] = 20, ci: float = 0.0, bw: float = 0.075, alpha: float = 0.4, color=None, **kwargs)[source]¶
Plot distribution of samples from the posterior distribution.
- Parameters
n (int) – Number of samples to take from each posterior distribution for estimating the density. Default = 10000
style (str) –
Which style of plot to show. Available types are:
'fill'
- filled density plot (the default)'line'
- line density plot'hist'
- histogram
bins (int or list or
ndarray
) – Number of bins to use for the posterior density histogram (ifstyle='hist'
), or a list or vector of bin edges.ci (float between 0 and 1) – Confidence interval to plot. Default = 0.0 (i.e., not plotted)
bw (float) – Bandwidth of the kernel density estimate (if using
style='line'
orstyle='fill'
). Default is 0.075alpha (float between 0 and 1) – Transparency of fill/histogram
color (matplotlib color code or list of them) – Color(s) to use to plot the distribution. See https://matplotlib.org/tutorials/colors/colors.html Default = use the default matplotlib color cycle
kwargs – Additional keyword arguments are passed to
utils.plotting.plot_dist()
- prior_plot(n: int = 10000, style: str = 'fill', bins: Union[int, list, numpy.ndarray] = 20, ci: float = 0.0, bw: float = 0.075, alpha: float = 0.4, color=None)[source]¶
Plot distribution of samples from the prior distribution.
- Parameters
n (int) – Number of samples to take from each prior distribution for estimating the density. Default = 1000
style (str) –
Which style of plot to show. Available types are:
'fill'
- filled density plot (the default)'line'
- line density plot'hist'
- histogram
bins (int or list or
ndarray
) – Number of bins to use for the prior density histogram (ifstyle='hist'
), or a list or vector of bin edges.ci (float between 0 and 1) – Confidence interval to plot. Default = 0.0 (i.e., not plotted)
bw (float) – Bandwidth of the kernel density estimate (if using
style='line'
orstyle='fill'
). Default is 0.075alpha (float between 0 and 1) – Transparency of fill/histogram
color (matplotlib color code or list of them) – Color(s) to use to plot the distribution. See https://matplotlib.org/tutorials/colors/colors.html Default = use the default matplotlib color cycle
- class probflow.parameters.PositiveParameter(shape=1, posterior=<class 'probflow.distributions.normal.Normal'>, prior=<probflow.distributions.normal.Normal object>, transform=<function softplus>, initializer={'loc': <function xavier>, 'scale': <function scale_xavier>}, var_transform={'loc': None, 'scale': <function softplus>}, name='PositiveParameter')[source]¶
Bases:
probflow.parameters.parameter.Parameter
A parameter which takes only positive values.
This is a convenience class for creating a parameter \(\beta\) which can only take positive values. It uses a normal variational posterior distribution and a softplus transform:
\[\log ( 1 + \exp ( \beta )) \sim \text{Normal}(\mu, \sigma)\]- Parameters
shape (int or List[int]) – Shape of the array containing the parameters. Default =
1
posterior (Distribution class) – Probability distribution class to use to approximate the posterior. Default =
Normal
prior (Distribution object) – Prior probability distribution function which has been instantiated with parameters. Default =
Normal
(0, 1)
transform (callable) – Transform to apply to the random variable. Default is to use a softplus transform.
initializer (Dict[str, callable]) – Initializer functions to use for each variable of the variational posterior distribution. Keys correspond to variable names (arguments to the distribution), and values contain functions to initialize those variables given
shape
as the single argument.var_transform (Dict[str, callable]) – Transform to apply to each variable of the variational posterior.
min (float) – Minimum value the parameter can take. Default = 0.
max (float) – Maximum value the parameter can take. Default = 1.
name (str) – Name of the parameter(s). Default =
'PositiveParameter'
Examples
TODO
- bayesian_update()¶
Update priors to match the current posterior
- kl_loss()¶
Compute the sum of the Kullback–Leibler divergences between this parameter’s priors and its variational posteriors.
- property n_parameters¶
Get the number of independent parameters
- property n_variables¶
Get the number of underlying variables
- property posterior¶
This Parameter’s variational posterior distribution
- posterior_ci(ci: float = 0.95, n: int = 10000)¶
Posterior confidence intervals
- Parameters
- Returns
lb (float or |ndarray|) – Lower bound(s) of the confidence interval(s)
ub (float or |ndarray|) – Upper bound(s) of the confidence interval(s)
- posterior_mean()¶
Get the mean of the posterior distribution(s)
- posterior_plot(n: int = 10000, style: str = 'fill', bins: Union[int, list, numpy.ndarray] = 20, ci: float = 0.0, bw: float = 0.075, alpha: float = 0.4, color=None, **kwargs)¶
Plot distribution of samples from the posterior distribution.
- Parameters
n (int) – Number of samples to take from each posterior distribution for estimating the density. Default = 10000
style (str) –
Which style of plot to show. Available types are:
'fill'
- filled density plot (the default)'line'
- line density plot'hist'
- histogram
bins (int or list or
ndarray
) – Number of bins to use for the posterior density histogram (ifstyle='hist'
), or a list or vector of bin edges.ci (float between 0 and 1) – Confidence interval to plot. Default = 0.0 (i.e., not plotted)
bw (float) – Bandwidth of the kernel density estimate (if using
style='line'
orstyle='fill'
). Default is 0.075alpha (float between 0 and 1) – Transparency of fill/histogram
color (matplotlib color code or list of them) – Color(s) to use to plot the distribution. See https://matplotlib.org/tutorials/colors/colors.html Default = use the default matplotlib color cycle
kwargs – Additional keyword arguments are passed to
utils.plotting.plot_dist()
- posterior_sample(n: int = 1)¶
Sample from the posterior distribution.
- Parameters
n (int > 0) – Number of samples to draw from the posterior distribution. Default = 1
- Returns
Samples from the parameter’s posterior distribution. If
n>1
of size(n, self.prior.shape)
. Ifn==1
, of size(self.prior.shape)
.- Return type
- prior_plot(n: int = 10000, style: str = 'fill', bins: Union[int, list, numpy.ndarray] = 20, ci: float = 0.0, bw: float = 0.075, alpha: float = 0.4, color=None)¶
Plot distribution of samples from the prior distribution.
- Parameters
n (int) – Number of samples to take from each prior distribution for estimating the density. Default = 1000
style (str) –
Which style of plot to show. Available types are:
'fill'
- filled density plot (the default)'line'
- line density plot'hist'
- histogram
bins (int or list or
ndarray
) – Number of bins to use for the prior density histogram (ifstyle='hist'
), or a list or vector of bin edges.ci (float between 0 and 1) – Confidence interval to plot. Default = 0.0 (i.e., not plotted)
bw (float) – Bandwidth of the kernel density estimate (if using
style='line'
orstyle='fill'
). Default is 0.075alpha (float between 0 and 1) – Transparency of fill/histogram
color (matplotlib color code or list of them) – Color(s) to use to plot the distribution. See https://matplotlib.org/tutorials/colors/colors.html Default = use the default matplotlib color cycle
- prior_sample(n: int = 1)¶
Sample from the prior distribution.
- Parameters
n (int > 0) – Number of samples to draw from the prior distribution. Default = 1
- Returns
Samples from the parameter prior distribution. If
n>1
of size(n, self.prior.shape)
. Ifn==1
, of size(self.prior.shape)
.- Return type
- property trainable_variables¶
Get a list of trainable variables from the backend
- property variables¶
Variables after applying their respective transformations
- class probflow.parameters.ScaleParameter(shape=1, posterior=<class 'probflow.distributions.gamma.Gamma'>, prior=<probflow.distributions.gamma.Gamma object>, transform=<function ScaleParameter.<lambda>>, initializer={'concentration': <function full_of.<locals>.init>, 'rate': <function full_of.<locals>.init>}, var_transform={'concentration': <function exp>, 'rate': <function exp>}, name='ScaleParameter')[source]¶
Bases:
probflow.parameters.parameter.Parameter
Standard deviation parameter.
This is a convenience class for creating a standard deviation parameter (\(\sigma\)). It is created by first constructing a variance parameter (\(\sigma^2\)) which uses an inverse gamma distribution as the variational posterior.
\[\frac{1}{\sigma^2} \sim \text{Gamma}(\alpha, \beta)\]Then the variance is transformed into the standard deviation:
\[\sigma = \sqrt{\sigma^2}\]By default, an inverse gamma prior is used:
\[\frac{1}{\sigma^2} \sim \text{Gamma}(5, 5)\]- Parameters
shape (int or List[int]) – Shape of the array containing the parameters. Default =
1
posterior (Distribution class) – Probability distribution class to use to approximate the posterior. Default =
Gamma
prior (Distribution object or
None
) – Prior probability distribution function which has been instantiated with parameters, orNone
for a uniform prior. Default =None
transform (callable) – Transform to apply to the random variable. Default is to use an inverse square root transform (
sqrt(1/x)
)initializer (Dict[str, callable]) – Initializer functions to use for each variable of the variational posterior distribution. Keys correspond to variable names (arguments to the distribution), and values contain functions to initialize those variables given
shape
as the single argument.var_transform (Dict[str, callable]) – Transform to apply to each variable of the variational posterior.
name (str) – Name of the parameter(s). Default =
'ScaleParameter'
Examples
Use
ScaleParameter
to create a standard deviation parameter for aNormal
distribution:TODO
- bayesian_update()¶
Update priors to match the current posterior
- kl_loss()¶
Compute the sum of the Kullback–Leibler divergences between this parameter’s priors and its variational posteriors.
- property n_parameters¶
Get the number of independent parameters
- property n_variables¶
Get the number of underlying variables
- property posterior¶
This Parameter’s variational posterior distribution
- posterior_ci(ci: float = 0.95, n: int = 10000)¶
Posterior confidence intervals
- Parameters
- Returns
lb (float or |ndarray|) – Lower bound(s) of the confidence interval(s)
ub (float or |ndarray|) – Upper bound(s) of the confidence interval(s)
- posterior_mean()¶
Get the mean of the posterior distribution(s)
- posterior_plot(n: int = 10000, style: str = 'fill', bins: Union[int, list, numpy.ndarray] = 20, ci: float = 0.0, bw: float = 0.075, alpha: float = 0.4, color=None, **kwargs)¶
Plot distribution of samples from the posterior distribution.
- Parameters
n (int) – Number of samples to take from each posterior distribution for estimating the density. Default = 10000
style (str) –
Which style of plot to show. Available types are:
'fill'
- filled density plot (the default)'line'
- line density plot'hist'
- histogram
bins (int or list or
ndarray
) – Number of bins to use for the posterior density histogram (ifstyle='hist'
), or a list or vector of bin edges.ci (float between 0 and 1) – Confidence interval to plot. Default = 0.0 (i.e., not plotted)
bw (float) – Bandwidth of the kernel density estimate (if using
style='line'
orstyle='fill'
). Default is 0.075alpha (float between 0 and 1) – Transparency of fill/histogram
color (matplotlib color code or list of them) – Color(s) to use to plot the distribution. See https://matplotlib.org/tutorials/colors/colors.html Default = use the default matplotlib color cycle
kwargs – Additional keyword arguments are passed to
utils.plotting.plot_dist()
- posterior_sample(n: int = 1)¶
Sample from the posterior distribution.
- Parameters
n (int > 0) – Number of samples to draw from the posterior distribution. Default = 1
- Returns
Samples from the parameter’s posterior distribution. If
n>1
of size(n, self.prior.shape)
. Ifn==1
, of size(self.prior.shape)
.- Return type
- prior_plot(n: int = 10000, style: str = 'fill', bins: Union[int, list, numpy.ndarray] = 20, ci: float = 0.0, bw: float = 0.075, alpha: float = 0.4, color=None)¶
Plot distribution of samples from the prior distribution.
- Parameters
n (int) – Number of samples to take from each prior distribution for estimating the density. Default = 1000
style (str) –
Which style of plot to show. Available types are:
'fill'
- filled density plot (the default)'line'
- line density plot'hist'
- histogram
bins (int or list or
ndarray
) – Number of bins to use for the prior density histogram (ifstyle='hist'
), or a list or vector of bin edges.ci (float between 0 and 1) – Confidence interval to plot. Default = 0.0 (i.e., not plotted)
bw (float) – Bandwidth of the kernel density estimate (if using
style='line'
orstyle='fill'
). Default is 0.075alpha (float between 0 and 1) – Transparency of fill/histogram
color (matplotlib color code or list of them) – Color(s) to use to plot the distribution. See https://matplotlib.org/tutorials/colors/colors.html Default = use the default matplotlib color cycle
- prior_sample(n: int = 1)¶
Sample from the prior distribution.
- Parameters
n (int > 0) – Number of samples to draw from the prior distribution. Default = 1
- Returns
Samples from the parameter prior distribution. If
n>1
of size(n, self.prior.shape)
. Ifn==1
, of size(self.prior.shape)
.- Return type
- property trainable_variables¶
Get a list of trainable variables from the backend
- property variables¶
Variables after applying their respective transformations