SGD Learning Model

This module contains the implementation of the Stochastic Gradient Descent algorithm for online linear regression.

Stochastic Gradient Descent Class

class source.model.incremental.StochasticGradientDescent(num_variables, learning_rate)

Bases: object

Stochastic Gradient Descent (SGD) algorithm for online linear regression.

Parameters:

  • num_variables (int) – Number of variables including the constant.

  • learning_rate (float) – Step size (eta) for gradient updates.

Initialize the StochasticGradientDescent object.

Parameters:

  • num_variables (int) – Number of variables including the constant.

  • learning_rate (float) – Step size (eta) for gradient updates.

__init__(num_variables, learning_rate)

Initialize the StochasticGradientDescent object.

Parameters:

  • num_variables (int) – Number of variables including the constant.

  • learning_rate (float) – Step size (eta) for gradient updates.

fit(observations, labels)

Fit the model to a set of observations and labels.

Parameters:

  • observations (list of numpy arrays) – List of observation vectors.

  • labels (list of floats) – List of true labels corresponding to the observations.

predict(observation)

Predict the value of a new observation.

Parameters:

observation (numpy array) – Observation column vector.

Returns:

prediction – Predicted value for the observation.

Return type:

float

update(observation, label)

Update the model with a new observation and label.

Parameters:

  • observation (numpy array) – Observation column vector.

  • label (float) – True label corresponding to the observation.

Example Usage

from source.model.incremental import StochasticGradientDescent
import numpy as np

# Initialize model
model = StochasticGradientDescent(num_features=3, learning_rate=1e-5)

# Simulate streaming data
for _ in range(100):

   # Generate random data
   X_new = np.random.rand(1, 3)  # New feature vector
   y_new = np.random.rand().reshape(-1, 1)  # New target value

   # Make prediction
   y_pred = model.predict(X_new)
   print(f"Predicted: {y_pred}, Actual: {y_new}")

   # Update model with new data
   model.update(X_new, y_new)
   print(f"Updated Weights: {model.weights}")