Wrong reconstruction of tucker decomposition

[mhelal]

Hi

I am following the same example in: http://tensorly.org/stable/user_guide/tensor_decomposition.html

ValueError                                Traceback (most recent call last)
<ipython-input-93-1acc231c5b9d> in <module>
----> 1 print(tl.tucker_to_tensor(core, factors))

~\Anaconda3\lib\site-packages\tensorly\tucker_tensor.py in tucker_to_tensor(tucker_tensor, skip_factor, transpose_factors)
     61     """
     62     core, factors = tucker_tensor
---> 63     return multi_mode_dot(core, factors, skip=skip_factor, transpose=transpose_factors)
     64 
     65 

~\Anaconda3\lib\site-packages\tensorly\tenalg\__init__.py in dynamically_dispatched_fun(*args, **kwargs)
    114 
    115         if hasattr(current_backend, name):
--> 116             fun = getattr(current_backend, name)(*args, **kwargs)
    117         else:
    118             warnings.warn(f'tenalg: defaulting to core tenalg backend, {name}'

~\Anaconda3\lib\site-packages\tensorly\tenalg\core_tenalg\n_mode_product.py in multi_mode_dot(tensor, matrix_or_vec_list, modes, skip, transpose)
    127             res = mode_dot(res, T.conj(T.transpose(matrix_or_vec)), mode - decrement)
    128         else:
--> 129             res = mode_dot(res, matrix_or_vec, mode - decrement)
    130 
    131         if T.ndim(matrix_or_vec) == 1:

~\Anaconda3\lib\site-packages\tensorly\tenalg\core_tenalg\n_mode_product.py in mode_dot(tensor, matrix_or_vector, mode, transpose)
     65 
     66         else:
---> 67             raise ValueError('Can only take n_mode_product with a vector or a matrix.'
     68                              'Provided array of dimension {} not in [1, 2].'.format(T.ndim(matrix_or_vector)))
     69 

ValueError: Can only take n_mode_product with a vector or a matrix.Provided array of dimension 0 not in [1, 2].

I tried another example, and it is reconstructing one value only:

X = tl.tensor(np.arange(64).reshape((4, 4, 4)), dtype=tl.float32)
from tensorly.decomposition import tucker

core, factors = tucker(X, rank=[2, 2, 2])
[f.shape for f in factors]
out: [(4, 2), (4, 2), (4, 2)]
factors
out: [array([[19.717047  , -1.4151516 ],
        [36.01058   , -0.5987119 ],
        [52.304077  ,  0.21773142]], dtype=float32),
 array([[-0.4115961 ,  1.5758767 ],
        [-0.46890688,  1.2336237 ],
        [-0.5262176 ,  0.8913703 ],
        [-0.5835284 ,  0.5491172 ]], dtype=float32),
 array([[-0.70674   ,  2.601153  ],
        [-0.74887186,  2.2431436 ]], dtype=float32)]

core
out: array([[[ 2.9157446e+02,  5.6701660e-02],
        [-1.4241800e-02,  8.3631551e-01]],

       [[ 4.4879317e-03, -4.3396211e+00],
        [ 1.7547552e+01, -9.3107128e-01]]], dtype=float32)

# reconstruct a full tensor from our Tucker decomposition:
reconstructed = tl.tucker_to_tensor(core, factors)
reconstructed

out: array(27.425539, dtype=float32)

Also I tried not to compress and passed None to ranks, and this is what I get:

core, factors = tucker(X, rank=None)
# reconstruct a full tensor from our Tucker decomposition:
reconstructed = tl.tucker_to_tensor(core, factors)
reconstructed
out: 

ValueError                                Traceback (most recent call last)
<ipython-input-121-1a5004abd119> in <module>
      2 
      3 #from tensorly import tucker_to_tensor
----> 4 reconstructed = tl.tucker_to_tensor(core, factors)
      5 #reconstructed = tucker_to_tensor(core, factors)
      6 reconstructed

~\Anaconda3\lib\site-packages\tensorly\tucker_tensor.py in tucker_to_tensor(tucker_tensor, skip_factor, transpose_factors)
     60        full tensor of shape ``(factors[0].shape[0], ..., factors[-1].shape[0])``
     61     """
---> 62     core, factors = tucker_tensor
     63     return multi_mode_dot(core, factors, skip=skip_factor, transpose=transpose_factors)
     64 

ValueError: too many values to unpack (expected 2)

So, your example gives an error, and another simple example does not reconstruct the tensor, or without rank give another error. Any help please to be able to continue will be appreciated,

Kind Regards,

Manal

[JeanKossaifi]

Hi @mhelal ,
There is indeed an error on the doc, it should be rank, not ranks. Aside from this, everything works as expected. If you do not want to compress, you need to pass rank='same', None is not accepted.

[JeanKossaifi]

Does it answer your question @mhelal ?

[mhelal]

Thank you and sorry for replying late,

The last two problems are both solved by converting:
reconstructed = tl.tucker_to_tensor(core, factors)
to

reconstructed = tl.tucker_to_tensor((core, factors))

such that the error "ValueError: too many values to unpack (expected 2)" is solved by sending 2 values to unpack in the "tucker_tensor" argument

The problem of compressing and using a smaller rank than the original tensor is as follows:

tensor = tl.tensor([[ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
                        [ 0.,  0.,  0.,  0.,  1.,  1.,  1.,  1.,  0.,  0.,  0.,  0.],
                        [ 0.,  0.,  0.,  0.,  1.,  1.,  1.,  1.,  0.,  0.,  0.,  0.],
                        [ 0.,  0.,  0.,  0.,  1.,  1.,  1.,  1.,  0.,  0.,  0.,  0.],
                        [ 0.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  0.],
                        [ 0.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  0.],
                        [ 0.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  0.],
                        [ 0.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  0.],
                        [ 0.,  0.,  0.,  0.,  1.,  1.,  1.,  1.,  0.,  0.,  0.,  0.],
                        [ 0.,  0.,  0.,  0.,  1.,  1.,  1.,  1.,  0.,  0.,  0.,  0.],
                        [ 0.,  0.,  0.,  0.,  1.,  1.,  1.,  1.,  0.,  0.,  0.,  0.],
                        [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.]])
core, factors = tucker(tensor, rank=[2, 3])
# The core is a smaller tensor of size (2, 3):
core.shape

--------------------------------------------------------------------------
ArpackError                               Traceback (most recent call last)
<ipython-input-42-f14051cef28d> in <module>
----> 1 core, factors = tucker(tensor, rank=[2, 3])
      2 # The core is a smaller tensor of size (2, 3):
      3 core.shape

~\Anaconda3\lib\site-packages\tensorly\decomposition\_tucker.py in tucker(tensor, rank, fixed_factors, n_iter_max, init, svd, tol, random_state, mask, verbose)
    203         rank = validate_tucker_rank(tl.shape(tensor), rank=rank)
    204 
--> 205         core, factors = partial_tucker(tensor, modes, rank=rank, n_iter_max=n_iter_max, init=init,
    206                             svd=svd, tol=tol, random_state=random_state, mask=mask, verbose=verbose)
    207         return TuckerTensor((core, factors))

~\Anaconda3\lib\site-packages\tensorly\decomposition\_tucker.py in partial_tucker(tensor, modes, rank, n_iter_max, init, tol, svd, random_state, verbose, mask)
    107         for index, mode in enumerate(modes):
    108             core_approximation = multi_mode_dot(tensor, factors, modes=modes, skip=index, transpose=True)
--> 109             eigenvecs, _, _ = svd_fun(unfold(core_approximation, mode), n_eigenvecs=rank[index], random_state=random_state)
    110             factors[index] = eigenvecs
    111 

~\Anaconda3\lib\site-packages\tensorly\backend\core.py in partial_svd(self, matrix, n_eigenvecs, random_state, **kwargs)
    911                 V = np.dot(matrix.T.conj(), U * np.where(np.abs(S) <= np.finfo(S.dtype).eps, 0, 1/S)[None, :])
    912             else:
--> 913                 S, V = scipy.sparse.linalg.eigsh(
    914                     np.dot(matrix.T.conj(), matrix), k=n_eigenvecs, which='LM', v0=v0
    915                 )

~\Anaconda3\lib\site-packages\scipy\sparse\linalg\eigen\arpack\arpack.py in eigsh(A, k, M, sigma, which, v0, ncv, maxiter, tol, return_eigenvectors, Minv, OPinv, mode)
   1685     with _ARPACK_LOCK:
   1686         while not params.converged:
-> 1687             params.iterate()
   1688 
   1689         return params.extract(return_eigenvectors)

~\Anaconda3\lib\site-packages\scipy\sparse\linalg\eigen\arpack\arpack.py in iterate(self)
    571                 self._raise_no_convergence()
    572             else:
--> 573                 raise ArpackError(self.info, infodict=self.iterate_infodict)
    574 
    575     def extract(self, return_eigenvectors):

ArpackError: ARPACK error 3: No shifts could be applied during a cycle of the Implicitly restarted Arnoldi iteration. One possibility is to increase the size of NCV relative to NEV. 

There are discussions online from other libraries about do not use ARPACK so you can use that instead of QSQP as a workaround, So I am not sure yet how to compress while decomposing, and then reconstruct from the core and factors and check the errors.

Thank you again,

Manal

[JeanKossaifi]

I actually have no issues with the above snippet - are you using the latest version of TensorLy/Scipy?

[merajhashemi]

The code does occasionally fail; setting the random seed would make it more clear (e.g., np.random.seed(0) fails while np.random.seed(6) works fine). I noticed the issue is a bug in tl.partial_svd.

import tensorly as tl
import numpy as np
np.random.seed(1)

a = tl.dot(tl.randn((4, 2)), tl.randn((2, 4)))
u, s, v = tl.partial_svd(a, n_eigenvecs=3)
print(u)

returns:

array([[-0.56637993, -0.29798997,         nan],
       [ 0.53373738, -0.33025035,         nan],
       [ 0.2412373 , -0.82224119,         nan],
       [-0.57977819, -0.35504382,         nan]])

[merajhashemi]

This is now fixed in master. @mhelal can you try installing the latest version from github (pip install git+https://github.com/tensorly/tensorly) and also I think you need to update scipy as the error message you posted shows you're using an old version (pip install -U scipy).

[mhelal]

Hi

I did both install commands: Requirement already satisfied: numpy<1.23.0,>=1.16.5 in c:\users\mh20afd\anaconda3\lib\site-packages (from scipy) (1.21.1)

and

Name: tensorly
Version: 0.6.1

and it is finally working, thank you very much for your support,

Kind Regards,

Manal