Support sparse matrix

[unnonouno]

Especially in NLP, we use sparse information such as in EmbedID. Its current implementation uses dense matrix to store gradient. When minibatch size is small, almost all elements of gradient are zero. Sparse matrices are more efficient in such situation.

• Use sparse matrices such as scikit-sparse in Chainer
• Make sparse matrices with CUDA using cuSPARSE

[Crissman]

Sparse matrices incorporated into CuPy, but not available yet from Chainer.

[Crissman]

Considering as part of future roadmap.

[unnonouno]

We now have cupy.sparse even for GPU. We have some ideas to use it in Chainer.

• use sparse matrices for efficiency on some function such as EmbedID
• make a function that uses sparse matrices on both train and test time such as sparse linear function
• provide a feature that sparsify trained linear model or normalization to sparsify linear model, and use sparse matrices on test time

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity occurs. Thank you for your contributions.

[stale]

This issue is closed as announced. Feel free to re-open it if needed.