Repository navigation
DirichletCalibrator with full l2 regularization throwing error on passing l2_list #5
Description
The DirichletCalibrator is throwing error on passing l2_list for reg_lambda parameter as you originally did in your experiment here and you passed your l2_list as here.
In my case, its throwing error as below
/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.6/site-packages/dirichletcal/calib/fulldirichlet.py in fit(self, X, y, X_val, y_val, *args, **kwargs)
46 reg_norm=self.reg_norm,
47 ref_row=self.ref_row)
---> 48 self.calibrator_.fit(_X, y, *args, **kwargs)
49 final_loss = log_loss(y_val, self.calibrator_.predict_proba(_X_val))
50
/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.6/site-packages/dirichletcal/calib/multinomial.py in fit(self, X, y, *args, **kwargs)
95 reg_mu=self.reg_mu, ref_row=self.ref_row,
96 initializer=self.initializer,
---> 97 reg_format=self.reg_format)
98 else:
99 res = scipy.optimize.fmin_l_bfgs_b(func=_objective, fprime=_gradient,
/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.6/site-packages/dirichletcal/calib/multinomial.py in _newton_update(weights_0, X, XX_T, target, k, method_, maxiter, ftol, gtol, reg_lambda, reg_mu, ref_row, initializer, reg_format)
233 L_list = [raw_np.float(_objective(weights_0, X, XX_T, target, k, method_,
234 reg_lambda, reg_mu, ref_row, initializer,
--> 235 reg_format))]
236
237 weights = weights_0.copy()
/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.6/site-packages/dirichletcal/calib/multinomial.py in _objective(params, *args)
151 else:
152 reg = np.zeros((k, k+1))
--> 153 loss = loss + reg_lambda * np.sum((weights - reg)**2)
154 else:
155 weights_hat = weights - np.hstack([weights[:, :-1] * np.eye(k),
/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.6/site-packages/jax/interpreters/xla.py in _forward_method(attrname, self, fun, *args)
985
986 def _forward_method(attrname, self, fun, *args):
--> 987 return fun(getattr(self, attrname), *args)
988 _forward_to_value = partial(_forward_method, "_value")
989
TypeError: only integer scalar arrays can be converted to a scalar index
Activity
shivamtundele commentedon Apr 14, 2021
Seems like I was assigning it to wrong parameter. I corrected it but then I discovered it still wont work because the reg_lambda_list parameter doesnt exists of FullDirichletCalibrator. You assigned the parameter but the its doesnt exist in the referenced class.
perellonieto commentedon May 20, 2021
Thanks @shivamtundele for pointing out the problem and submitting a pull request with the solution. However, I am afrain that these problems are because of legacy code.
The first version of this library passed lists of hyperparameters to the models, and the models iterated with for loops over them and selected the ones with the best validation results. But at some point, I decidedd that the hyperparameter selection would be better with Scikit-learn tools like GridSearchCV used in the example.
This would simplify overall all the code for the different versions of Dirichlet.
Furthermore, the class DirichletCalibrator was originally used and encapsulated all versions, but at some point it was discontinued and we started ussing the individual versions from /dirichletcal/calib/.
Given that I do not have much time, it may be better to remove the DirichletCalibrator class, and instruct people to use the individual versions directly.