Can still plot model and prune model in efficient kan?

[yhaddbb]

I notice that most image classification tasks are based on efficient-kan instead of original kan. I want to know if it is possible to plot and prune the efficient-kan just like the examples in original kan.

[Blealtan]

Plotting is definitely possible but not planned yet. The parameters in spline_weight is the coefficients of the splines, for now dumping that out and you will be able to plot the grid on your own.

For the pruning, it still needs to validate if my "efficient" way of doing sparsifying regularization works as expected. If not, the trained network might be badly redundant, unlike the cases in original paper.

[yhaddbb]

Plotting is definitely possible but not planned yet. The parameters in spline_weight is the coefficients of the splines, for now dumping that out and you will be able to plot the grid on your own.

For the pruning, it still needs to validate if my "efficient" way of doing sparsifying regularization works as expected. If not, the trained network might be badly redundant, unlike the cases in original paper.

Thank you for your reply

[477810383]

Plotting is definitely possible but not planned yet. The parameters in spline_weight is the coefficients of the splines, for now dumping that out and you will be able to plot the grid on your own.

For the pruning, it still needs to validate if my "efficient" way of doing sparsifying regularization works as expected. If not, the trained network might be badly redundant, unlike the cases in original paper.

Specifically, how should we use the parameter ‘spline_weight’ to draw the shape of the activation function on each edge?

[FakeEnd]

I write some code to visualize the activation function on each edge, but it seems not right. If somebody knows how to modify it, welcome to chat. 😄

import numpy as np
import matplotlib.pyplot as plt
from scipy.interpolate import BSpline

def visualize_kan(weight):
    # define B-spline parameters
    grid_size = 5
    spline_order = 3
    weights = weight

    # define knot vector
    knot_vector = np.concatenate(([-1] * spline_order, np.linspace(-1, 1, grid_size), [1] * spline_order))

    # define parameter range
    t = np.linspace(-1, 1, 100)

    # create B-spline object
    spline = BSpline(knot_vector, weights, spline_order)

    # calculate B-spline curve values
    spline_values = spline(t)
    
    # add bias
    silu = nn.SiLU()
    bias = silu(torch.tensor(t))
    
    spline_values = spline_values + bias.numpy()

    # plot B-spline curve
    plt.figure(figsize=(8, 6))
    plt.plot(t, spline_values, label='B-spline curve')
    plt.scatter(np.linspace(-1, 1, len(weights)), weights, color='red', label='Control points')
    plt.title('B-spline Curve')
    plt.xlabel('t')
    plt.ylabel('Value')
    plt.legend()
    plt.grid(True)
    plt.show()

for layer in kan_model.layers:
    for i in range(5):
        for j in range(2):
            visualize_kan(layer.spline_weight[i][j].detach().numpy())

[link24tech]

mark