by Pavel Holoborodko on November 3, 2013
We continue to compare our toolbox with Maple and Symbolic Math Toolbox (VPA). This time we focus on main operations in dense linear algebra – solvers and factorizations.
Every case/routine is tested on a random 100 x 100
matrix. Average timing over ten tests is used.
Comparison of linear system solver (“\”) in quadruple precisionMatrix Type | Timing (sec) | Speed-up (times) |
---|
VPA | Maple | Advanpix | Over VPA | Over Maple |
---|
General Real | 0.99 | 0.58 | 0.02 | 42.22 | 24.67 |
Symmetric Positive Definite | 0.99 | 0.63 | 0.01 | 77.10 | 49.31 |
General Complex | 3.13 | 2.43 | 0.08 | 38.34 | 29.74 |
Hermitian Positive Definite | 3.72 | 2.56 | 0.05 | 70.85 | 48.82 |
In average Advanpix solver is faster by 60 times over VPA and 40 times over Maple.
Comparison of matrix factorization in quadruple precisionFactorization | Timing (sec) | Speed-up (times) |
---|
VPA | Maple | Advanpix | Over VPA | Over Maple |
---|
A is a general real matrix: |
[L,U] = lu(A) | 1.67 | 0.61 | 0.02 | 78.26 | 28.29 |
[Q,R] = qr(A) | 3.46 | 2.60 | 0.08 | 44.44 | 33.46 |
[U,S,V] = svd(A) | 30.85 | 20.99 | 0.30 | 101.66 | 69.17 |
S = svd(A) | 15.88 | 2.51 | 0.07 | 223.54 | 35.35 |
[V,D] = eig(A) | 57.25 | 29.54 | 0.72 | 79.20 | 40.86 |
lambda = eig(A) | 21.21 | 26.90 | 0.34 | 62.31 | 79.01 |
A is a general complex matrix: |
[L,U] = lu(A) | 2.74 | 2.56 | 0.07 | 41.63 | 38.89 |
[Q,R] = qr(A) | 10.99 | 11.39 | 0.25 | 43.38 | 44.99 |
[U,S,V] = svd(A) | 65.56 | 57.53 | 0.81 | 80.96 | 71.04 |
S = svd(A) | 25.28 | 10.22 | 0.26 | 98.24 | 39.71 |
[V,D] = eig(A) | 89.30 | 129.54 | 2.85 | 31.34 | 45.46 |
lambda = eig(A) | 33.14 | 113.99 | 1.29 | 25.67 | 88.32 |
As for factorizations, Advanpix toolbox outperforms VPA by 75 times and Maple by 50 times in average. Worth to note the massive speed gain of singular value decomposition (SVD) in particular.
Read More
by Pavel Holoborodko on October 3, 2013
MATLAB doesn’t support sparse matrices computing with high precision.
As a result, we have no other way but compare our performance with famous mathematical packages Maple and Mathematica, which do have support for sparse matrices with arbitrary precision elements.
Table below show timing comparisons of solving sparse linear systems of moderate size (min 1000x1000
) by Advanpix toolbox and Maple 17
using quadruple precision.
Advanpix vs. Maple: Sparse LU, 34 digits (quadruple precision).Matrix Name | Size | NNZ | Advanpix (sec) | Maple (sec) | Speed Gain (times) |
---|
nnc1374 | 1374 x 1374 | 8606 | 0.2009 | 20.858 | 103.8 |
orsreg_1 | 2205 x 2205 | 14133 | 1.4409 | 22.948 | 15.9 |
watt__1 | 1856 x 1856 | 11360 | 0.6724 | 649.308 | 965.7 |
watt__1 | 1856 x 1856 | 11360 | 0.6723 | 342.282 | 509.1 |
lns_3937 | 3937 x 3937 | 25407 | 1.3501 | 432.482 | 320.3 |
lnsp3937 | 3937 x 3937 | 25407 | 1.2554 | 433.324 | 345.2 |
sherman1 | 1000 x 1000 | 3750 | 0.0686 | 2.184 | 31.8 |
sherman2 | 1080 x 1080 | 23094 | 1.6683 | 231.724 | 138.9 |
sherman3 | 5005 x 5005 | 20033 | 2.0002 | 335.824 | 167.9 |
sherman4 | 1104 x 1104 | 3786 | 0.0583 | 1.778 | 30.5 |
sherman5 | 3312 x 3312 | 20793 | 0.8152 | 81.152 | 99.5 |
Read More
by Pavel Holoborodko on September 5, 2013
Newest version of toolbox includes sparse linear system solver – function mldivide
or operator "\"
.
Solver analyzes input matrix and automatically uses the most suitable decomposition:
- Cholesky
LDLT
factorization is applied if input matrix is symmetrical/hermitian and has real positive diagonal. LU
factorization is applied if LDLT
failed or directly if matrix is obviously non-SPD.
(Algorithm is based on ideas from SuperLU and uses column approximate minimum degree permutation)QR
factorization for rectangular matrices to handle least squares problems (beta).
(Left-looking QR
decomposition, also uses colamd pre-ordering)
All methods are capable to compute with arbitrary precision, in real and complex cases. Tables below present timing for solving sparse linear systems with quadruple precision (34
decimal digits).
See the timing tests
by Pavel Holoborodko on August 2, 2013
Besides sparse matrices support latest version of toolbox also includes significant changes in performance optimization of the matrix / array manipulation operations. This post is attempt to highlight some of the key points & reasons of the development.
As we noted before [1, 2], MATLAB has somewhat limited object oriented capabilities. The good feature that MATLAB has in this area – it allows elements of dense matrix / n-dim array to be of custom-type (e.g. multiprecision number). In this case MATLAB automatically takes care of basic manipulation operations (indexing, assigning, concatenation, deletion, etc.).
This nice feature removes huge amount of work from toolbox’s developers – so that we can concentrate on mathematical functionality rather than spending time re-inventing the wheel.
Although we have been using this functionality from the start, we decided to abandon it in the recent version. The reason is simple: MATLAB’s implementation of array manipulation operations (with custom objects as elements) suffers from enormous loss in performance!
Read More
by Pavel Holoborodko on July 31, 2013
Starting from version 3.5.0
toolbox has support for sparse matrices with multiple precision elements.
Please note, this is the first beta version with limited functionality – at the moment toolbox can only create and display multiprecision sparse matrices:
>> S = zeros(7,3);
>> S(2,1) = eps;
>> S(5,1) = sqrt(2);
>> S(3,2) = pi;
>> S(2,3) = exp(1);
>> S(5,3) = log(2);
>> S(6,3) = sqrt(-1);
>> S = sparse(S)
S =
(2,1) 2.22044604925031e-16
(5,1) 1.4142135623731
(3,2) 3.14159265358979
(2,3) 2.71828182845905
(5,3) 0.693147180559945
(6,3) 0 + 1i
>> A = mp(S) % Convert S to multiprecision sparse matrix of 34 digits (default)
% Note accuracy of all constants are extended automatically.
A =
(2,1) 1.925929944387235853055977942584927e-34
(5,1) 1.414213562373095048801688724209698
(3,2) 3.141592653589793238462643383279503
(2,3) 2.718281828459045235360287471352662
(5,3) 0.693147180559945309417232121458177
(6,3) 0 + 1i
From now on we will extend sparse matrices support to the point of full transparency & flexibility as we have done for dense matrices.
Next steps are: element-wise arithmetic and mathematical functions, basic matrix-matrix and matrix-vector operations. Major goal after that will be to add iterative Krylov subspace methods.
Although such (currently) basic support may seem to be trivial, its implementation is very far from elementary. In fact, sparse matrices support requires by order of magnitude more efforts than dense matrices. Below we outline the major reasons why it is so.
Read More
by Pavel Holoborodko on July 22, 2013
Recently I have got new notifications from researchers about toolbox usage in their works. Pre-prints of the papers are available by links below:
Thank you for letting me know.
by Pavel Holoborodko on July 19, 2013
It is not a secret that MATLAB has many undocumented (or deliberately hidden) features and commands. There are even excellent website & book devoted specifically to this topic: Undocumented Matlab
However most of the findings are related to MATLAB language itself and investigations on undocumented MEX API seems to be missing (or scarce at least).
During development of our toolbox we have found lots of hidden functions which can be helpful for creating speed-efficient extensions for MATLAB using native languages.
Here we want to explain some of them in details and provide complete list of undocumented MEX API functions.
Read More
by Pavel Holoborodko on July 11, 2013
After almost 2 years of email based communication with users we have decided to open publicly visible forum. Where anyone can leave feedback, ask questions, request new features or report bugs (there are no bugs in toolbox of course :)).
Hopefully this will make question resolution more efficient and less time consuming for everybody. Please follow this link (or use “Support” menu in navigation bar above): Advanpix Support Forum
Don’t forget to write something there, vote for requests or give us your unique ideas on further toolbox improvement.
by Denys Dutykh on March 13, 2013
Dear readers of the Advanpix blog,
My name is Denys Dutykh, a guest blogger today, and I would like to share my experience with the Multiprecision toolbox.
My research interests lie in the fields of applied mathematics and hydrodynamics. More precisely, with my collaborators we study mathematically free surface flows or, in other words, water waves.
Read More
by Pavel Holoborodko on January 20, 2013
In order to improve performance we have made important changes to the toolbox recently.
Now toolbox is heavily optimized for computations with quadruple precision.
In more details, new core engine of toolbox (written in Assembler/C++) has two different variants for every routine:
- The first variant is truly precision independent, capable of computing results with any required precision (same as before).
- The other is highly optimized for quadruple precision (128-bit or 34 decimal digits) computations.
Read More