Recurrent neural network model card

Problem type

Sequence modeling: classification, forecasting, tagging, generation.

Sequences of vectors $[B, T, d_x]$ .

A hidden state per step; predictions from the last or every step.

Read one step at a time and keep a running summary (the hidden state), updated with the same weights at every step.

Task loss over outputs, with $h_t = \tanh(W_x x_t + W_h h_{t-1} + b)$ ; trained by backpropagation through time.

Unroll over the sequence, backpropagate, clip gradients; truncated BPTT for long sequences.

Scale inputs; pad and mask variable-length sequences; time-ordered splits.

Relevant information can be carried in a fixed-size state; dependencies are not too long.

Hidden size, layers, learning rate, gradient clipping threshold, sequence length.

Short sequences; streaming with constant memory; teaching sequence modeling.

Long-range dependencies (vanishing gradients); large-scale language modeling.

Constant memory per step; natural for streaming; few parameters.

Vanishing/exploding gradients; sequential (slow) training; limited memory span.

$O(T \cdot d_h(d_x + d_h))$ per sequence, sequential in $T$ .

Task metrics on time-ordered validation data.

NaN losses without clipping; forgetting early inputs; leakage from shuffled time-series splits.

python

import torch.nn as nn
rnn = nn.RNN(input_size=16, hidden_size=64, batch_first=True)