How does non-sequential data look and how do we process it?

What propeties does sequential data have?

We want to build a model that considers the sequentiality of the data.

Data representation

• One-hot vectors
• Embeddings “someone else” gave us

Encode music using the Fourier Transform (fft) to get spectrogram (which can be considered as image arepresentation).

We can later decode the predictions using the inverse fft.

We saw examples of sequential data:

• Text
• Video frames
• Audio
• Actions
• (and you can imagine other real-world data which we could abstract into representation of sequences)

So far we used very simplified schematics of what the [model] is … let’s explore this in more detail.

We usually have some model parameters that we can set so that the model does what we want from it. We usually define what we want using a loss function on the predicted outputs and the desired output values.

Let’s say that someone told us that these values for the parameters are going to work the best:

W = [1, -1, 0, 0]

b = [0.1, 0.1, 0.1, 0.1]

V = [1, 1, -1, 1]

This is to illustrate that by using parameters to influence what happens to the input (W, b) and what is kept in the memory (V)…

We can influence the behaviour of the model. Iteratively change parameters (W, b, V) so that the loss gets smaller.

Find parameters which will minimize the distance between the prediction our model is giving and the labels we have (loss).

Different scenarios for sequential modelling - depending on the task and the dataset we might be using different model variants.

Classification:

• We want to classify the input data into its class.
• For example: input = mail text, label = spam / not-spam

Regression:

• We want to assign a continuous value to the input data.
• For example: input = movie frames, label = expected IMDb rating.

Generation:

• We want to use the model to predict a believable continuation to what we show it.

Type of the model you would use depends on your data and the task you want to solve…

Examples from real-world research