Logistic Regression and Optimization

Article BigQuant

Summary

This tutorial introduces binary classification using logistic regression, with stock direction as its example. It explains encoding outcomes as two labels, interpreting the model output as a probability, and applying a threshold to produce a class. Logistic regression maps a linear predictor through a logistic function so predictions stay within the probability range, addressing a limitation of applying ordinary linear regression directly to classification. The model parameters are described as minimizing cross-entropy, which generally requires iterative optimization rather than a simple closed-form solution.

The article compares gradient descent, which uses gradients and a learning rate, with Newton–Raphson, which uses curvature information. It then discusses local minima and saddle points in higher-dimensional optimization, introducing momentum, AdaGrad, and Adam as refinements. These are conceptual explanations rather than a tested trading model: no feature set, validation design, or investment performance results are provided. Some explanations simplify optimizer behavior, so practical use requires attention to convergence, scaling, and out-of-sample evaluation.

Key ideas

Logistic regression estimates the probability of a binary outcome by applying a logistic function to a linear predictor.
Its parameters are fitted by minimizing cross-entropy, commonly through iterative optimization.
Gradient descent uses a learning rate, while Newton–Raphson adjusts updates using second-order curvature information.
Momentum, AdaGrad, and Adam adapt updates to address optimization challenges such as slow progress or saddle points.
The tutorial provides no evidence that a particular model or optimizer produces profitable trading results.

Logistic Regression and Gradient-Based Optimization for Trading Models

Summary

Key ideas

Tags