ProbFlow

ProbFlow is a Python package for building probabilistic Bayesian models with TensorFlow Probability, performing variational inference with those models, and evaluating the models’ inferences. It provides both high-level Layers for building Bayesian neural networks, and low-level modules for constructing custom Bayesian models.

It’s very much still a work in progress.

Getting Started

ProbFlow allows you to quickly and painlessly build, fit, and evaluate custom Bayesian models (or ready-made ones!) which run on top of TensorFlow and TensorFlow Probability.

With ProbFlow, the core building blocks of a Bayesian model are parameters, layers, and probability distributions (and, of course, the input data). Layers define how parameters interact with the independent variables (the features) to predict the probability distribution of the dependent variables (the target).

For example, a simple Bayesian linear regression

\[y \sim \text{Normal}(w x + b, \sigma)\]

can be built with ProbFlow by:

from probflow import Input, Parameter, ScaleParameter, Normal

feature = Input()
weight = Parameter()
bias = Parameter()
noise_std = ScaleParameter()

predictions = weight*feature + bias
model = Normal(predictions, noise_std)

Then, the model can be fit using variational inference, in one line:

# x and y are Numpy arrays or pandas DataFrame/Series
model.fit(x, y)

You can generate predictions for new data:

# x_test is a Numpy array or pandas DataFrame
model.predict(x_test)

Compute probabilistic predictions for new data, with 95% confidence intervals:

model.predictive_distribution_plot(x_test, ci=0.95)
_images/pred_dist.svg

Evaluate your model’s performance using various metrics:

model.metrics('mse')

Inspect the posterior distributions of your fit model’s parameters, with 95% confidence intervals:

model.posterior_plot(ci=0.95)
_images/posteriors.svg

Investigate how well your model is capturing uncertainty by examining how accurate its predictive intervals are:

model.pred_dist_coverage(ci=0.95)

and diagnose where your model is having problems capturing uncertainty:

model.coverage_by(ci=0.95)
_images/coverage.svg

ProbFlow also provides more complex layers, such as those required for building Bayesian neural networks. A multi-layer Bayesian neural network can be built and fit using ProbFlow in only a few lines:

from probflow import Sequential, Dense, ScaleParameter, Normal

predictions = Sequential(layers=[
    Dense(units=128, activation='relu'),
    Dense(units=64, activation='relu'),
    Dense(units=1)
])
noise_std = ScaleParameter()
model = Normal(predictions, noise_std)
model.fit(x, y)

For convenience, ProbFlow also includes several ready-made models for standard tasks (such as linear regressions, logistic regressions, and multi-layer dense neural networks). For example, the above linear regression example could have been done with much less work by using ProbFlow’s ready-made LinearRegression() model:

from probflow import LinearRegression

model = LinearRegression()
model.fit(x, y)

And the multi-layer Bayesian neural net could have been made more easily by using ProbFlow’s ready-made DenseRegression() model:

from probflow import DenseRegression

model = DenseRegression(units=[128, 64, 1])
model.fit(x, y)

Using parameters, layers, and distributions as simple building blocks, ProbFlow allows for the painless creation of more complicated Bayesian models like generalized linear models, neural matrix factorization models, and mixed effects models. Take a look at the Examples section and the User Guide for more!

Installation

Before installing ProbFlow, you’ll first need to install TensorFlow and TensorFlow Probability.

Then, you can use pip to install ProbFlow itself from the GitHub source:

pip install git+http://github.com/brendanhasz/probflow.git

Support

Post bug reports, feature requests, and tutorial requests in GitHub issues.

Contributing

Pull requests are totally welcome! Any contribution would be appreciated, from things as minor as pointing out typos to things as major as writing new layers and distributions.

Why the name, ProbFlow?

Because it’s a package for probabilistic modeling, and it’s built on TensorFlow. ¯\_(ツ)_/¯