A simple introduction to Quantum enhanced SVM | by Matheus Cammarosano Hidalgo | Oct, 2023

Learn how to mix fascinating quantum computing properties with a traditional Machine Studying approach

Though quantum computer systems are usually not but accessible for everybody, Quantum Machine Studying (QML) is a promising subject of examine because it makes use of the intrinsic probabilistic nature of quantum techniques to develop fashions. Proper now, knowledge scientists around the globe try to grasp methods to leverage the quantum paradigm to provide higher, scalable fashions. It’s not doable to quantify when this may occur as a result of it additionally is determined by the evolution of quantum {hardware}, however there may be accelerated progress on this matter.

In my final research I used to be attempting to design Variational Quantum Classifiers (VQCs), as you would see in a previous post I wrote. That is an fascinating case of examine in case you are beginning to examine QML like me.

Nonetheless, recently I additionally began to check a quantum strategy to the Help Vector Machine (SVM) and I used to be intrigued by how SVM might be translated into the quantum world.

As I used to be learning VQCs, I used to be very biased and I used to be attempting to guess how the SVM might be translated right into a parameterizable quantum circuit, however I discovered that the quantum enhancement right here works in another way, which was a pleasant shock and helped me to open my thoughts about this topic.

On this put up I begin with a short introduction of SVM adopted by methods to do a Quantum Machine Studying (QML) strategy to this method and I end with an instance of quantum enhanced SVM (QSVM) utilizing the Titanic dataset.

I current right here the SVM centered on classification issues, that’s, the Help Vector Classifier (SVC). The target of SVC is to discover a hyperplane that separates knowledge from completely different lessons with the very best margin. This doesn’t appear very useful at first, proper?

However what is that this hyperplane that separates the lessons? Suppose we’re working with knowledge in a two-dimensional vector area and now we have two lessons, as in Determine 1.

On this instance now we have knowledge factors from 2 completely different lessons and we are able to simply draw a line separating each. Our stable line is the hyperplane that separates our knowledge with the very best margin, as seen by the dashed traces. Thus, SVM tries to seek out the very best separator.

It’s possible you’ll suppose that my instance was too naive and a line is a really specific case of a hyperplane, which is a legitimate level. What if our two-dimensional knowledge appears like Determine 2?

On this case we are able to’t draw a line that separates our knowledge appropriately. If we take a look at this determine, we may draw a circle as an excellent separator. Nonetheless, this form is neither a line nor a airplane, so SVM isn’t in a position to straight clear up this downside. Nonetheless, that is the best SVM trick and the half the place a excessive dimensional hyperplane happens!

What if now we have the transformation of this knowledge into the next dimension vector area? As:

So we’d may draw a airplane:

Which separates the 2 lessons optimally, as seen in Determine 3:

In our case, the perform f is what we name the kernel, which tasks knowledge into the next dimensional area, which makes it simpler to discover a hyperplane that may appropriately establish knowledge from completely different lessons.

This was a really transient introduction about kernels and SVM, in case you are keen on extra clarification about SVM I like to recommend you to learn these two posts (1 and 2), that are actually good introductions to SVM and I used each of them as references on this put up.

You could be pondering now that my instance was very handy to elucidate the kernel idea, however how in actual life will we discover a appropriate kernel that solves our issues? There are some kernels which are very versatile and are very useful to resolve an excellent variety of issues, just like the Radial Foundation Operate (RBF), which is the default possibility of scikit-learn’s SVC. In case you are keen on studying extra about this kernel, I like to recommend this post. An necessary element about kernels such because the RBF is that they aren’t described by an analytic perform, however as a similarity matrix between knowledge factors primarily based on the kernel.

Nonetheless, what if we need to be extra artistic? If in case you have learn my previous posts you would possibly do not forget that some of the fascinating properties of quantum computing is the exponential relation between qubits and quantum states. Thus, a quantum system is a really fascinating candidate for an excellent kernel, because it tends to drive our system in direction of a high-dimensional vector area, relying on the amount of qubits we’re utilizing.

Quantum kernels are normally outlined by an similarity matrix primarily based on a quantum circuit, that could be parameterizable or not. Each Pennylane and Qiskit have built-in features that create kernels that can be utilized in scikit-learn’s SVC.

The challenge of a quantum kernel has some steps:

Embedding knowledge into quantum states

Designing a quantum circuit that could be parameterizable or not

At this stage, it’s extremely really useful to work with some extent of superposition and entanglement between states to acquire the very best that quantum computing can present.

Constructing the similarity matrix

Right here we work with the unitary U(|x>) that we constructed within the final step and its adjoint to design a similarity matrix.

Right here we’re designing a easy quantum kernel with Pennylane to make use of it with an SVC from scikit-learn for the Titanic Classification dataset, the place we need to predict whether or not an individual survived the Titanic tragedy primarily based on variables comparable to age, gender and boarding class.

In our instance we’re utilizing the next variables:

• is_child: if the age of the particular person is lower than 12 (boolean)
• Pclass_1: if the particular person boarded within the first-class (boolean)
• Pclass_2: if the particular person boarded within the second class (boolean)
• Sex_female: if the gender of the particular person is feminine (boolean)

As you may see, it is a quite simple mannequin with 4 boolean variables. We’re embedding our knowledge into quantum states utilizing quantum embedding (Foundation Embedding), making use of Hadamard gates to use superposition into our qubits and CNOT gates to generate entanglement.

It is a easy and non-parameterizable ansatz, however it generates superposition and entanglement between our variables.

The code to create the kernel and SVM is right here:

import pennylane as qml
from pennylane import numpy as np
from sklearn.model_selection import train_test_split
import pandas as pd
from sklearn.metrics import accuracy_score
from sklearn.metrics import f1_score, precision_score, recall_score
from sklearn.svm import SVC
num_qubits = 4
def layer(x):
qml.BasisEmbedding(x, wires=vary(num_qubits))
for j, wire in enumerate(wires):
qml.Hadamard(wires=[wire])
if j != num_qubits-1:
qml.CNOT(wires=[j, j+1])
else:
qml.CNOT(wires=[j, 0])
def ansatz(x, wires):
layer(x)
adjoint_ansatz = qml.adjoint(ansatz)
dev = qml.gadget("default.qubit", wires=num_qubits, photographs=None)
wires = dev.wires.tolist()
@qml.qnode(dev, interface="autograd")
def kernel_circuit(x1, x2):
ansatz(x1, wires=wires)
adjoint_ansatz(x2, wires=wires)
return qml.probs(wires=wires)
def kernel(x1, x2):
return kernel_circuit(x1, x2)[0]
df_train = pd.read_csv('practice.csv')
df_train['Pclass'] = df_train['Pclass'].astype(str)
df_train = pd.concat([df_train, pd.get_dummies(df_train[['Pclass', 'Sex', 'Embarked']])], axis=1)
X_train, X_test, y_train, y_test = train_test_split(df_train.drop(columns=['Survived']), df_train['Survived'], test_size=0.10, random_state=42, stratify=df_train['Survived'])
X_train['Age'] = X_train['Age'].fillna(X_train['Age'].median())
X_test['Age'] = X_test['Age'].fillna(X_test['Age'].median())
X_train['is_child'] = X_train['Age'].map(lambda x: 1 if x < 12 else 0)
X_test['is_child'] = X_test['Age'].map(lambda x: 1 if x < 12 else 0)
cols_model = ['is_child', 'Pclass_1', 'Pclass_2', 'Sex_female']
X_train = X_train[cols_model]
X_test = X_test[cols_model]
X_train = np.array(X_train.values, requires_grad=False)
init_kernel = lambda x1, x2: kernel(x1, x2)
Ok = qml.kernels.square_kernel_matrix(X_train, init_kernel, assume_normalized_kernel=True)
svm = SVC(kernel=lambda X1, X2: qml.kernels.kernel_matrix(X1, X2, init_kernel)).match(X_train, y_train)
X_test = np.array(X_test.values, requires_grad=False)
predictions = svm.predict(X_test)
accuracy_score(y_test, predictions)
precision_score(y_test, predictions)
recall_score(y_test, predictions)
f1_score(y_test, predictions, common='macro')
svm1 = SVC(gamma='auto', kernel='rbf')
svm1.match(X_train, y_train)
y_pred = svm1.predict(X_test)
accuracy_score(y_test, y_pred)
precision_score(y_test, y_pred)
recall_score(y_test, y_pred)
f1_score(y_test, y_pred, common='macro')

The outcomes are:

As you may see, the SVC with the RBF kernel outperformed our SVC with quantum kernel. Our quantum strategy had good precision, which signifies that we managed to keep away from false positives at an excellent price, however our recall wasn’t so good, implying that we bought a major variety of false negatives.

If you wish to learn extra about SVMs with quantum kernel, these posts are good references: 1, 2 and these texts from Pennylane concerning the topic: 3 and 4.

Quantum kernels generally is a highly effective instrument to extend SVM efficiency. Nonetheless, as we may see in our instance, a SVM with a easy quantum kernel isn’t in a position to outperform SVM with an RBF kernel. Quantum kernels require cautious design so as to be aggressive with classical strategies.

I’ve been deepening my research to design parameterizable quantum kernels and I hope to have excellent news on this topic quickly.