| Title: | Boltzmann Machines with MM Algorithms |
|---|---|
| Description: | Provides probability computation, data generation, and model estimation for fully-visible Boltzmann machines. It follows the methods described in Nguyen and Wood (2016a) <doi:10.1162/NECO_a_00813> and Nguyen and Wood (2016b) <doi:10.1109/TNNLS.2015.2425898>. |
| Authors: | Andrew Thomas Jones, Hien Duy Nguyen, and Jessica Juanita Bagnall |
| Maintainer: | Andrew Thomas Jones <[email protected]> |
| License: | GPL-3 |
| Version: | 0.1.4 |
| Built: | 2024-11-04 04:34:36 UTC |
| Source: | https://github.com/andrewthomasjones/boltzmm |
Compute the probability of all 2^n strings of n>1 binary spin variables (i.e. each element is -1 or 1) arising from a fully-visible Boltzmann machine with some specified bias vector and interaction matrix.
allpfvbm(bvec, Mmat)allpfvbm(bvec, Mmat)
bvec |
Vector of length n containing real valued bias parameters. |
Mmat |
Symmetric n by n matrix, with zeros along the diagonal, containing the interaction parameters. |
A vector of the probabilities of all 2^n binary spin vectors under a fully-visible Boltzmann machine with bias vector bvec and interaction matrix Mmat. Probabilities are reported in ascending order of the binary strings; i.e for n=2 the reporting order is (-1,1), (-1,1), (1,-1), and (1,1).
Andrew T. Jones and Hien D. Nguyen
H.D. Nguyen and I.A. Wood (2016), Asymptotic normality of the maximum pseudolikelihood estimator for fully-visible Boltzmann machines, IEEE Transactions on Neural Networks and Learning Systems, vol. 27, pp. 897-902.
# Compute the probability of every length n=3 binary spin vector under bvec and Mmat. bvec <- c(0,0.5,0.25) Mmat <- matrix(0.1,3,3) - diag(0.1,3,3) allpfvbm(bvec,Mmat)# Compute the probability of every length n=3 binary spin vector under bvec and Mmat. bvec <- c(0,0.5,0.25) Mmat <- matrix(0.1,3,3) - diag(0.1,3,3) allpfvbm(bvec,Mmat)
The BoltzMM package allows for computation of probability mass functions of fully-visible Boltzmann machines via pfvbm and allpfvbm.
Random data can be generated using rfvbm. Maximum pseudolikelihood estimation of parameters via the MM algorithm can be conducted using fitfvbm.
Computation of partial derivatives and Hessians can be performed via fvbmpartiald and fvbmHessian.
Covariance estimation and normal standard errors can be computed using fvbmcov and fvbmstderr.
Andrew T. Jones and Hien D. Nguyen
H.D. Nguyen and I.A. Wood (2016), Asymptotic normality of the maximum pseudolikelihood estimator for fully-visible Boltzmann machines, IEEE Transactions on Neural Networks and Learning Systems, vol. 27, pp. 897-902.
H.D. Nguyen and I.A. Wood (2016), A block successive lower-bound maximization algorithm for the maximum pseudolikelihood estimation of fully visible Boltzmann machines, Neural Computation, vol 28, pp. 485-492.
Estimates the bias vector and interaction matrix of a fully-visible Boltzmann machine via maximum pseudolikelihood estimation using an MM algorithm.
fitfvbm(data, bvec, Mmat, delta_crit = 0.001, max_it = 1000L)fitfvbm(data, bvec, Mmat, delta_crit = 0.001, max_it = 1000L)
data |
An N by n matrix, where each of the N rows contains a length n string of spin variables (i.e. each element is -1 or 1). |
bvec |
Initial estimate for a vector of length n containing real valued bias parameters. |
Mmat |
Initial estimate for a symmetric n by n matrix, with zeros along the diagonal, containing the interaction parameters. |
delta_crit |
Real threshold value for the convergence criterion, based on the relative change in the Euclidean distance of parameter estimates from consecutive iterations. |
max_it |
Integer value indicating the maximum number of iterations that the algorithm is to run for. |
A list containing 4 objects: the final log-pseudolikelihood value pll, a vector containing the estimate of the bias parameters bvec, a matrix containing the estimate of the interaction parameters Mmat, and the number of algorithm iterations itt.
Andrew T. Jones and Hien D. Nguyen
H.D. Nguyen and I.A. Wood (2016), A block successive lower-bound maximization algorithm for the maximum pseudolikelihood estimation of fully visible Boltzmann machines, Neural Computation, vol 28, pp. 485-492
# Generate num=1000 random strings of n=3 binary spin variables under bvec and Mmat. num <- 1000 bvec <- c(0,0.5,0.25) Mmat <- matrix(0.1,3,3) - diag(0.1,3,3) data <- rfvbm(num,bvec,Mmat) # Fit a fully visible Boltzmann machine to data, starting from parameters bvec and Mmat. fitfvbm(data,bvec,Mmat)# Generate num=1000 random strings of n=3 binary spin variables under bvec and Mmat. num <- 1000 bvec <- c(0,0.5,0.25) Mmat <- matrix(0.1,3,3) - diag(0.1,3,3) data <- rfvbm(num,bvec,Mmat) # Fit a fully visible Boltzmann machine to data, starting from parameters bvec and Mmat. fitfvbm(data,bvec,Mmat)
Computes the sandwich estimator of the covariance matrix for a maximum pseudolikelihood estimated fully-visible Boltzmann machine.
fvbmcov(data, model, fvbmHess)fvbmcov(data, model, fvbmHess)
data |
An N by n matrix, where each of the N rows contains a length n string of spin variables (i.e. each element is -1 or 1). |
model |
List generated from |
fvbmHess |
A function that computes the Hessian of the parameter elements. Currently, the only implemented method is the default |
The n+choose(n,2) by n+choose(n,2) sandwich covariance matrix, estimated using data and evaluated at the fitted parameter values provided in model. Each row (column) is a unique element of the bias vector and interaction matrix. The rows are arranged in lexicographical order with the bias elements first, followed by the interaction elements. For example, if n=3, the order would be bias[1], bias[2] bias[3], interaction[1,2], interaction[1,3], and interaction[2,3].
Andrew T. Jones and Hien D. Nguyen
H.D. Nguyen and I.A. Wood (2016), Asymptotic normality of the maximum pseudolikelihood estimator for fully-visible Boltzmann machines, IEEE Transactions on Neural Networks and Learning Systems, vol. 27, pp. 897-902.
# Generate num=1000 random strings of n=3 binary spin variables under bvec and Mmat. num <- 1000 bvec <- c(0,0.5,0.25) Mmat <- matrix(0.1,3,3) - diag(0.1,3,3) data <- rfvbm(num,bvec,Mmat) # Fit a fully visible Boltzmann machine to data, starting from parameters bvec and Mmat. model <- fitfvbm(data,bvec,Mmat) # Compute the sandwich covariance matrix using the data and the model. fvbmcov(data,model,fvbmHess)# Generate num=1000 random strings of n=3 binary spin variables under bvec and Mmat. num <- 1000 bvec <- c(0,0.5,0.25) Mmat <- matrix(0.1,3,3) - diag(0.1,3,3) data <- rfvbm(num,bvec,Mmat) # Fit a fully visible Boltzmann machine to data, starting from parameters bvec and Mmat. model <- fitfvbm(data,bvec,Mmat) # Compute the sandwich covariance matrix using the data and the model. fvbmcov(data,model,fvbmHess)
Computes the Hessian with respect to all unique parameter elements of the bias vector and interaction matrix of a fully-visible Boltzmann machine, for some random length n string of spin variables (i.e. each element is -1 or 1) and some fitted parameter values.
fvbmHess(data, model)fvbmHess(data, model)
data |
An N by n matrix, where each of the N rows contains a length n string of spin variables (i.e. each element is -1 or 1). |
model |
List generated from |
The n+choose(n,2) by n+choose(n,2) Hessian matrix, summed over the N rows of data and evaluated at the fitted parameter values provided in model. Each row (column) is a unique element of the bias vector and interaction matrix. The rows are arranged in lexicographical order with the bias elements first, followed by the interaction elements. For example, if n=3, the order would be bias[1], bias[2] bias[3], interaction[1,2], interaction[1,3], and interaction[2,3].
Andrew T. Jones and Hien D. Nguyen
H.D. Nguyen and I.A. Wood (2016), Asymptotic normality of the maximum pseudolikelihood estimator for fully-visible Boltzmann machines, IEEE Transactions on Neural Networks and Learning Systems, vol. 27, pp. 897-902.
# Generate num=1000 random strings of n=3 binary spin variables under bvec and Mmat. num <- 1000 bvec <- c(0,0.5,0.25) Mmat <- matrix(0.1,3,3) - diag(0.1,3,3) data <- rfvbm(num,bvec,Mmat) # Fit a fully visible Boltzmann machine to data, starting from parameters bvec and Mmat. model <- fitfvbm(data,bvec,Mmat) # Compute the Hessian matrix summed over all num rows of data. fvbmHess(data,model)# Generate num=1000 random strings of n=3 binary spin variables under bvec and Mmat. num <- 1000 bvec <- c(0,0.5,0.25) Mmat <- matrix(0.1,3,3) - diag(0.1,3,3) data <- rfvbm(num,bvec,Mmat) # Fit a fully visible Boltzmann machine to data, starting from parameters bvec and Mmat. model <- fitfvbm(data,bvec,Mmat) # Compute the Hessian matrix summed over all num rows of data. fvbmHess(data,model)
Computes the partial derivatives for all unique parameter elements of the bias vector and interaction matrix of a fully-visible Boltzmann machine, for some random length n string of spin variables (i.e. each element is -1 or 1) and some fitted parameter values.
fvbmpartiald(data, model)fvbmpartiald(data, model)
data |
Vector of length n containing binary spin variables. |
model |
List generated from |
A list containing 2 objects: a vector containing the partial derivatives corresponding to the bias parameters bvec, and a matrix containing the partial derivatives corresponding to the interaction parameters Mmat.
Andrew T. Jones and Hien D. Nguyen
H.D. Nguyen and I.A. Wood (2016), Asymptotic normality of the maximum pseudolikelihood estimator for fully-visible Boltzmann machines, IEEE Transactions on Neural Networks and Learning Systems, vol. 27, pp. 897-902.
# Generate num=1000 random strings of n=3 binary spin variables under bvec and Mmat. num <- 1000 bvec <- c(0,0.5,0.25) Mmat <- matrix(0.1,3,3) - diag(0.1,3,3) data <- rfvbm(num,bvec,Mmat) # Fit a fully visible Boltzmann machine to data, starting from parameters bvec and Mmat. model <- fitfvbm(data,bvec,Mmat) # Compute the partial derivatives evaluated at the first observation of data. fvbmpartiald(data,model)# Generate num=1000 random strings of n=3 binary spin variables under bvec and Mmat. num <- 1000 bvec <- c(0,0.5,0.25) Mmat <- matrix(0.1,3,3) - diag(0.1,3,3) data <- rfvbm(num,bvec,Mmat) # Fit a fully visible Boltzmann machine to data, starting from parameters bvec and Mmat. model <- fitfvbm(data,bvec,Mmat) # Compute the partial derivatives evaluated at the first observation of data. fvbmpartiald(data,model)
Computes the normal approximation standard errors from the sandwich estimator of the covariance matrix for a maximum pseudolikelihood estimated fully-visible Boltzmann machine.
fvbmstderr(data, covarmat)fvbmstderr(data, covarmat)
data |
An N by n matrix, where each of the N rows contains a length n string of spin variables (i.e. each element is -1 or 1). |
covarmat |
A covariance matrix generated from |
A list containing 2 objects: a vector containing the standard errors corresponding to the bias parameters bvec_se, and a matrix containing the standard errors corresponding to the interaction parameters Mmat_se.
Andrew T. Jones and Hien D. Nguyen
H.D. Nguyen and I.A. Wood (2016), Asymptotic normality of the maximum pseudolikelihood estimator for fully-visible Boltzmann machines, IEEE Transactions on Neural Networks and Learning Systems, vol. 27, pp. 897-902.
# Generate num=1000 random strings of n=3 binary spin variables under bvec and Mmat. num <- 1000 bvec <- c(0,0.5,0.25) Mmat <- matrix(0.1,3,3) - diag(0.1,3,3) data <- rfvbm(num,bvec,Mmat) # Fit a fully visible Boltzmann machine to data, starting from parameters bvec and Mmat. model <- fitfvbm(data,bvec,Mmat) # Compute the sandwich covariance matrix using the data and the model. covarmat <- fvbmcov(data,model,fvbmHess) # Compute the standard errors of the parameter elements according to a normal approximation. fvbmstderr(data,covarmat)# Generate num=1000 random strings of n=3 binary spin variables under bvec and Mmat. num <- 1000 bvec <- c(0,0.5,0.25) Mmat <- matrix(0.1,3,3) - diag(0.1,3,3) data <- rfvbm(num,bvec,Mmat) # Fit a fully visible Boltzmann machine to data, starting from parameters bvec and Mmat. model <- fitfvbm(data,bvec,Mmat) # Compute the sandwich covariance matrix using the data and the model. covarmat <- fvbmcov(data,model,fvbmHess) # Compute the standard errors of the parameter elements according to a normal approximation. fvbmstderr(data,covarmat)
Tests the hypothesis that the true bias and interaction parameter values are those in nullmodel, given data and model.
fvbmtests(data, model, nullmodel)fvbmtests(data, model, nullmodel)
data |
An N by n matrix, where each of the N rows contains a length n string of spin variables (i.e. each element is -1 or 1). |
model |
List generated from |
nullmodel |
A list containing two elements: a vector of length n |
A list containing 4 objects: a vector containing the z-scores corresponding to the bias parameters bvec_z,a vector containing the p-values corresponding to the bias parameters bvec_p,a matrix containing the z-scores corresponding to the interaction parameters Mmat_z, and a matrix containing the standard errors corresponding to the interaction parameters Mmat_p.
Andrew T. Jones and Hien D. Nguyen
H.D. Nguyen and I.A. Wood (2016), Asymptotic normality of the maximum pseudolikelihood estimator for fully-visible Boltzmann machines, IEEE Transactions on Neural Networks and Learning Systems, vol. 27, pp. 897-902.
# Generate num=1000 random strings of n=3 binary spin variables under bvec and Mmat. num <- 1000; bvec <- c(0,0.5,0.25); Mmat <- matrix(0.1,3,3) - diag(0.1,3,3); data <- rfvbm(num,bvec,Mmat) # Fit a fully visible Boltzmann machine to data, starting from parameters bvec and Mmat. model <- fitfvbm(data,bvec,Mmat) #Propose a null hypothesis model nullmodel <- list(bvec = c(0,0,0), Mmat = matrix(0,3,3)) # Compute z-scores fvbmtests(data,model,nullmodel)# Generate num=1000 random strings of n=3 binary spin variables under bvec and Mmat. num <- 1000; bvec <- c(0,0.5,0.25); Mmat <- matrix(0.1,3,3) - diag(0.1,3,3); data <- rfvbm(num,bvec,Mmat) # Fit a fully visible Boltzmann machine to data, starting from parameters bvec and Mmat. model <- fitfvbm(data,bvec,Mmat) #Propose a null hypothesis model nullmodel <- list(bvec = c(0,0,0), Mmat = matrix(0,3,3)) # Compute z-scores fvbmtests(data,model,nullmodel)
Compute the log pseudolikelihood
log_pl_calc(data, L)log_pl_calc(data, L)
data |
An N by n matrix, where each of the N rows contains a length n string of spin variables (i.e. each element is -1 or 1). |
List |
A list where |
The the log pseudolikelihood of [].
Andrew T. Jones and Hien D. Nguyen
Computes the marginal probabilities (for values = +1 in each coordinate) under under some specified bias vector and interaction matrix, specified by bvec and Mmat, respectively.
marginpfvbm(bvec, Mmat)marginpfvbm(bvec, Mmat)
bvec |
Vector of length n containing real valued bias parameters. |
Mmat |
Symmetric n by n matrix, with zeros along the diagonal, containing the interaction parameters. |
Vector of length n containing the marginal probabilities of +1 in each coordinate.
Andrew T. Jones and Hien D. Nguyen
H.D. Nguyen and I.A. Wood (2016), Asymptotic normality of the maximum pseudolikelihood estimator for fully-visible Boltzmann machines, IEEE Transactions on Neural Networks and Learning Systems, vol. 27, pp. 897-902.
#Compute the marginal probabilities under bvec and Mmat. # Set the parameter values bvec <- c(0,0.5,0.25) Mmat <- matrix(0.1,3,3) - diag(0.1,3,3) marginpfvbm(bvec,Mmat)#Compute the marginal probabilities under bvec and Mmat. # Set the parameter values bvec <- c(0,0.5,0.25) Mmat <- matrix(0.1,3,3) - diag(0.1,3,3) marginpfvbm(bvec,Mmat)
Compute the probability of a string of n>1 binary spin variables (i.e. each element is -1 or 1) arising from a fully-visible Boltzmann machine with some specified bias vector and interaction matrix.
pfvbm(xval, bvec, Mmat)pfvbm(xval, bvec, Mmat)
xval |
Vector of length n containing binary spin variables. |
bvec |
Vector of length n containing real valued bias parameters. |
Mmat |
Symmetric n by n matrix, with zeros along the diagonal, containing the interaction parameters. |
The probability of the random string xval under a fully-visible Boltzmann machine with bias vector bvec and interaction matrix Mmat.
Andrew T. Jones and Hien D. Nguyen
H.D. Nguyen and I.A. Wood (2016), Asymptotic normality of the maximum pseudolikelihood estimator for fully-visible Boltzmann machines, IEEE Transactions on Neural Networks and Learning Systems, vol. 27, pp. 897-902.
# Compute the probability of the vector xval=(-1,1,-1), under bvec and Mmat. xval <- c(-1,1,-1) bvec <- c(0,0.5,0.25) Mmat <- matrix(0.1,3,3) - diag(0.1,3,3) pfvbm(xval,bvec,Mmat)# Compute the probability of the vector xval=(-1,1,-1), under bvec and Mmat. xval <- c(-1,1,-1) bvec <- c(0,0.5,0.25) Mmat <- matrix(0.1,3,3) - diag(0.1,3,3) pfvbm(xval,bvec,Mmat)
Generate N random strings of n>1 binary spin variables (i.e. each element is -1 or 1) arising from a fully-visible Boltzmann machine with some specified bias vector and interaction matrix.
rfvbm(num, bvec, Mmat)rfvbm(num, bvec, Mmat)
num |
Number N of random strings to be generated. |
bvec |
Vector of length n containing real valued bias parameters. |
Mmat |
Symmetric n by n matrix, with zeros along the diagonal, containing the interaction parameters. |
An N by n matrix, where each row contains a random spin variable string from a fully-visible Boltzmann machine with bias vector bvec and interaction matrix Mmat.
The function allpfvbm must be called each time this function is run. Thus, it is much more efficient to generate N strings all at once, than to generate strings one at a time.
Andrew T. Jones and Hien D. Nguyen
H.D. Nguyen and I.A. Wood (2016), Asymptotic normality of the maximum pseudolikelihood estimator for fully-visible Boltzmann machines, IEEE Transactions on Neural Networks and Learning Systems, vol. 27, pp. 897-902.
# Generate num=10 random strings of n=3 binary spin variables under bvec and Mmat. num <- 10 bvec <- c(0,0.5,0.25) Mmat <- matrix(0.1,3,3) - diag(0.1,3,3) rfvbm(num,bvec,Mmat)# Generate num=10 random strings of n=3 binary spin variables under bvec and Mmat. num <- 10 bvec <- c(0,0.5,0.25) Mmat <- matrix(0.1,3,3) - diag(0.1,3,3) rfvbm(num,bvec,Mmat)
A dataset he data from the first sitting of the Senate of the 45th Australian Parliament, until the final sitting of the year 2016. The first division during this period was conducted on the 31st of August 2016, and the last division was performed on the 1st of December 2016. In total, 147 divisions were performed during this period.
Each row represents a division(vote), each column is a party or independent. Data is either "Yes" or "No" depending on the vote. Absences and abstentions are left as NA. See https://hal.archives-ouvertes.fr/hal-01927188v1 for details of data preparation.
data(senate)data(senate)
A data frame with 147 rows (votes) and 9 variables (parties).
Jessica J. Bagnall
www.aph.gov.au/Parliamentary_Business/Statistics/Senate_StatsNet/General/divisions
dim(senate)dim(senate)