Maximum-likelihood model selection score for binary support vector machines. More...

#include <shark/ObjectiveFunctions/SvmLogisticInterpretation.h>

Inheritance diagram for shark::SvmLogisticInterpretation< InputType >:

Public Types
typedef CVFolds< LabeledData< InputType, unsigned int > >	FoldsType

typedef AbstractKernelFunction< InputType >	KernelType

Public Types inherited from shark::AbstractObjectiveFunction< RealVector, double >
enum	Feature
	List of features that are supported by an implementation. More...

typedef RealVector	SearchPointType

typedef double	ResultType

typedef boost::mpl::if_< std::is_arithmetic< double >, SearchPointType, RealMatrix >::type	FirstOrderDerivative

typedef TypedFlags< Feature >	Features
	This statement declares the member m_features. See Core/Flags.h for details.

typedef TypedFeatureNotAvailableException< Feature >	FeatureNotAvailableException

Public Member Functions
	SvmLogisticInterpretation (FoldsType const &folds, KernelType kernel, bool unconstrained=true, QpStoppingCondition stop_cond=NULL)

std::string	name () const
	From INameable: return the class name.

bool	isFeasible (const SearchPointType &input) const

std::size_t	numberOfVariables () const
	Accesses the number of variables.

double	eval (SearchPointType const &parameters) const

double	evalDerivative (SearchPointType const &parameters, FirstOrderDerivative &derivative) const

Public Member Functions inherited from shark::AbstractObjectiveFunction< RealVector, double >
const Features &	features () const

virtual void	updateFeatures ()

bool	hasValue () const
	returns whether this function can calculate it's function value

bool	hasFirstDerivative () const
	returns whether this function can calculate the first derivative

bool	hasSecondDerivative () const
	returns whether this function can calculate the second derivative

bool	canProposeStartingPoint () const
	returns whether this function can propose a starting point.

bool	isConstrained () const
	returns whether this function can return

bool	hasConstraintHandler () const
	returns whether this function can return

bool	canProvideClosestFeasible () const
	Returns whether this function can calculate thee closest feasible to an infeasible point.

bool	isThreadSafe () const
	Returns true, when the function can be usd in parallel threads.

bool	isNoisy () const
	Returns true, when the function can be usd in parallel threads.

	AbstractObjectiveFunction ()
	Default ctor.

virtual	~AbstractObjectiveFunction ()
	Virtual destructor.

virtual void	init ()

void	setRng (random::rng_type *rng)
	Sets the Rng used by the objective function.

virtual bool	hasScalableDimensionality () const

virtual void	setNumberOfVariables (std::size_t numberOfVariables)
	Adjusts the number of variables if the function is scalable.

virtual std::size_t	numberOfObjectives () const

virtual bool	hasScalableObjectives () const

virtual void	setNumberOfObjectives (std::size_t numberOfObjectives)
	Adjusts the number of objectives if the function is scalable.

std::size_t	evaluationCounter () const
	Accesses the evaluation counter of the function.

AbstractConstraintHandler< SearchPointType > const &	getConstraintHandler () const
	Returns the constraint handler of the function if it has one.

virtual void	closestFeasible (SearchPointType &input) const
	If supported, the supplied point is repaired such that it satisfies all of the function's constraints.

virtual SearchPointType	proposeStartingPoint () const
	Proposes a starting point in the feasible search space of the function.

ResultType	operator() (SearchPointType const &input) const
	Evaluates the function. Useful together with STL-Algorithms like std::transform.

virtual ResultType	evalDerivative (SearchPointType const &input, SecondOrderDerivative &derivative) const
	Evaluates the objective function and calculates its gradient.

Public Member Functions inherited from shark::INameable
virtual	~INameable ()

Protected Attributes
FoldsType	m_folds
	the underlying partitioned dataset.

KernelType *	mep_kernel
	the kernel with which to run the SVM

std::size_t	m_nhp
	for convenience, the Number of Hyper Parameters

std::size_t	m_nkp
	for convenience, the Number of Kernel Parameters

std::size_t	m_numFolds
	the number of folds to be used in cross-validation

std::size_t	m_numSamples
	overall number of samples in the dataset

std::size_t	m_inputDims
	input dimensionality

bool	m_svmCIsUnconstrained
	the SVM regularization parameter C is passed for unconstrained optimization, and the derivative should compensate for that

QpStoppingCondition *	mep_svmStoppingCondition
	the stopping criterion that is to be passed to the SVM trainer.

Protected Attributes inherited from shark::AbstractObjectiveFunction< RealVector, double >
Features	m_features

std::size_t	m_evaluationCounter
	Evaluation counter, default value: 0.

AbstractConstraintHandler< SearchPointType > const *	m_constraintHandler

random::rng_type *	mep_rng

Additional Inherited Members
Protected Member Functions inherited from shark::AbstractObjectiveFunction< RealVector, double >
void	announceConstraintHandler (AbstractConstraintHandler< SearchPointType > const *handler)
	helper function which is called to announce the presence of an constraint handler.

Detailed Description

template<class InputType = RealVector>
class shark::SvmLogisticInterpretation< InputType >

Maximum-likelihood model selection score for binary support vector machines.

: This class implements the maximum-likelihood based SVM model selection procedure presented in the article "Glasmachers and C. Igel. Maximum Likelihood Model Selection for 1-Norm Soft Margin SVMs with Multiple Parameters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010." At this point, only binary C-SVMs are supported.

: This class implements an AbstactObjectiveFunction. In detail, it provides a differentiable measure of how well a C-SVM with given hyperparameters fulfills the maximum-likelihood score presented in the paper. This error measure can then be optimized for externally via gradient-based optimizers. In other words, this class provides a score, not an optimization method or a training algorithm. The C-SVM parameters have to be optimized with regard to this measure

Definition at line 62 of file SvmLogisticInterpretation.h.

Member Typedef Documentation

◆ FoldsType

template<class InputType = RealVector>

typedef CVFolds< LabeledData<InputType, unsigned int> > shark::SvmLogisticInterpretation< InputType >::FoldsType

Definition at line 64 of file SvmLogisticInterpretation.h.

◆ KernelType

template<class InputType = RealVector>

typedef AbstractKernelFunction<InputType> shark::SvmLogisticInterpretation< InputType >::KernelType

Definition at line 65 of file SvmLogisticInterpretation.h.

Constructor & Destructor Documentation

◆ SvmLogisticInterpretation()

template<class InputType = RealVector>

shark::SvmLogisticInterpretation< InputType >::SvmLogisticInterpretation	(	FoldsType const &	folds,
		KernelType *	kernel,
		bool	unconstrained = `true`,
		QpStoppingCondition *	stop_cond = `NULL`
	)

inline

constructor.

Parameters

folds	an already partitioned dataset (i.e., a CVFolds object)
kernel	pointer to the kernel to be used within the SVMs.
unconstrained	whether or not the C-parameter of/for the C-SVM is passed for unconstrained optimization mode.
stop_cond	the stopping conditions which are to be passed to the

Definition at line 83 of file SvmLogisticInterpretation.h.

References shark::AbstractObjectiveFunction< RealVector, double >::HAS_FIRST_DERIVATIVE, shark::AbstractObjectiveFunction< RealVector, double >::HAS_VALUE, shark::AbstractKernelFunction< InputTypeT >::hasFirstParameterDerivative(), shark::AbstractObjectiveFunction< RealVector, double >::IS_CONSTRAINED_FEATURE, shark::AbstractObjectiveFunction< RealVector, double >::m_features, shark::SvmLogisticInterpretation< InputType >::m_folds, shark::SvmLogisticInterpretation< InputType >::m_numFolds, shark::SvmLogisticInterpretation< InputType >::m_svmCIsUnconstrained, shark::SvmLogisticInterpretation< InputType >::mep_kernel, and SHARK_RUNTIME_CHECK.

Member Function Documentation

◆ eval()

template<class InputType = RealVector>

double shark::SvmLogisticInterpretation< InputType >::eval ( SearchPointType const & parameters ) const

inlinevirtual

train a number of SVMs in a cross-validation setting using the hyperparameters passed to this method. the output scores from all validations sets are then concatenated. together with the true labels, these scores can then be used to fit a sigmoid such that it becomes as good as possible a model for the class membership probabilities given the SVM output scores. This method returns the negative likelihood of the best fitting sigmoid, given a set of SVM hyperparameters.

Parameters

parameters the SVM hyperparameters to use for all C-SVMs

Reimplemented from shark::AbstractObjectiveFunction< RealVector, double >.

Definition at line 130 of file SvmLogisticInterpretation.h.

References shark::LabeledData< InputT, LabelT >::append(), shark::Classifier< Model >::decisionFunction(), shark::LabeledData< InputT, LabelT >::inputs(), shark::LabeledData< InputT, LabelT >::labels(), shark::SvmLogisticInterpretation< InputType >::m_folds, shark::SvmLogisticInterpretation< InputType >::m_nhp, shark::SvmLogisticInterpretation< InputType >::m_nkp, shark::SvmLogisticInterpretation< InputType >::m_numFolds, shark::SvmLogisticInterpretation< InputType >::m_svmCIsUnconstrained, shark::SvmLogisticInterpretation< InputType >::mep_kernel, shark::SvmLogisticInterpretation< InputType >::mep_svmStoppingCondition, shark::IParameterizable< VectorType >::setParameterVector(), SHARK_RUNTIME_CHECK, shark::QpConfig::sparsify(), shark::QpConfig::stoppingCondition(), shark::CSvmTrainer< InputType, CacheType >::train(), shark::CVFolds< DatasetTypeT >::training(), shark::transformInputs(), and shark::CVFolds< DatasetTypeT >::validation().