Optimization algorithms: the Newton Method

Predictive Statistics and Machine Learning aim at building models with parameters such that the final output/prediction is as close as possible to the actual value. This implies the optimization of an objective function, which might be either minimized (like loss functions) or maximized (like Maximum Likelihood function). The idea behind optimization routine is starting from […]

Multivariate Differential Calculus and Optimization-Part 2

In my previous article, I introduced some concepts which are necessary if we want to set an optimization problem in a multivariate environment. Here, we will first dwell on how to check the smoothness of a surface (which is the main assumption to deploy an optimization task), then we will see how to look for […]

Multivariate Differential Calculus and Optimization-Part 1

Differential calculus is a powerful tool to find the optimal solution to a given task. When I say ‘optimal solution’, I’m referring to the result of the optimization of a given function, called objective function. This result might be either a maximum (namely, if your objective function describes your revenues) or a minimum (namely, if […]