I want to create a number of random matrices, but they are really big to fit in memory, so I'd like to find a way to reproduce them across computers, so that when I need to send them to another machine, I'd just need to send the code. Here is how I want to do it:
[code]num_of_iters = 10; K = 200; for iter = 1:num_of_iters parfor j = 1:K R = make_random_R(iter,j,.....); % Do something end end[/code] What I'm worried about is the parfor loop, I need to be able to reproduce the random matrices no matter what the order of indices in the parfor is. So I decided to use a MATLAB stream for this:

Save the global stream
Create a new stream, set the seed and appropriate substream (which depends on iter and j)
Do the math
Put back the global stream
Here is my code (the variables n,p,R_type control how the random matrices are made, but they are not important, and K is the same variable as the one from above, I need it in the line substream_id = (iter - 1) * K + j;) :
[code]function [R] = make_random_R(iter,j,n,K,p,R_type) % Data as code % R_type: 'posneg' or 'normdist' % 1 <= iter <= 100 % 1 <= j <= K % K: Number of classifiers % n: Number of observations assert(strcmp(R_type,'posneg') || strcmp(R_type,'normdist'),'R_type must be posneg or normdist'); assert(iter >= 1,'Error: iter >= 1 not satisfied'); assert((1 <= j) && (j <= K),'Error: 1 <= j <= K not satisfied'); assert(K > 0,'Error: K > 0 not satisfied'); globalStream = RandStream.getGlobalStream; globalState = globalStream.State; stream=RandStream('mlfg6331_64','Seed',1); substream_id = (iter - 1) * K + j; stream.Substream = substream_id; RandStream.setGlobalStream(stream); switch R_type case 'posneg' q0=ceil(2*log(n)/0.25^2)+1; if (q0 < p) q = q0; else q = ceil(p/2); end R = randi([0 1],p,q); R(R == 0) = -1; case 'normdist' q = 2*ceil(log2(p)); R = normrnd(0,1,[p,q]); end RandStream.setGlobalStream(globalStream); globalStream.State = globalState; end[/code] Tried some code and here it is:
[code]>> iter = 2; >> j = 3; >> n=100; >> K=10; >> p=6; >> R_type = 'normdist'; >> for j=1:K j make_ran >> parfor j=1:K j make_random_R(iter,j,n,K,p,R_type) end Starting parallel pool (parpool) using the 'local' profile ... connected to 4 workers. ans = 7 ans = -0.3660 0.8816 1.1754 -0.4987 -1.8612 -0.3683 0.9504 -0.3067 -0.5156 -0.2383 -1.1661 0.3622 2.0743 -0.4195 0.5021 0.3954 0.2415 -0.4552 -0.0474 -0.1645 -0.1725 -0.4938 -0.2559 0.2188 1.0735 0.3660 0.1043 0.4403 -0.3166 1.1241 -1.0421 -1.4528 -0.4976 -0.7166 -1.1328 -2.0260 ans = 2 ans = -1.6629 0.0213 -1.8138 -0.4375 0.3575 -0.0353 0.6653 -1.2662 -0.3977 -0.6540 -1.2131 0.4858 0.3421 1.1266 -0.6066 -1.2095 1.5496 -0.9341 0.2145 0.7192 -2.2087 0.7597 -0.0110 -1.1282 -0.3511 -0.7305 -0.1143 0.0242 0.2431 -0.8612 0.5875 1.2665 -2.1943 -0.4879 0.0120 -1.1539 ans = 1 ans = -0.5300 2.4077 -0.3478 1.8695 -1.1327 -1.0734 -0.2540 -1.1265 0.3152 0.4265 1.2777 0.0959 0.5005 -0.7557 0.6194 1.5873 0.0961 -1.9216 0.7275 0.5420 -0.6237 -0.2228 0.8915 0.4644 0.8131 -0.1492 0.9232 0.8410 -0.0637 2.1163 -1.1995 0.2338 -1.3726 0.1604 -0.1855 1.3826 ans = 8 ans = -0.5146 2.2106 2.7200 -1.2136 1.0004 1.3089 0.7225 0.2746 -0.8798 0.2978 -0.8490 1.6744 1.1998 -0.0363 1.9105 -0.7747 -0.8707 -0.6823 0.6801 1.3194 -0.0685 0.5944 1.5078 -1.6821 0.0876 1.2150 -0.0747 0.0324 -1.1552 0.0966 -0.0624 -0.3874 -0.5356 0.6353 1.4090 -1.1014 ans = 6 ans = 0.5866 -1.0222 -0.2168 0.8582 1.4360 0.0699 2.0677 -0.4740 -0.8763 1.7827 0.1930 -1.2167 -0.3941 -0.5441 0.3719 -0.0609 0.7138 -1.0920 0.3622 -0.0459 -0.0221 0.2030 -0.7695 -0.8963 -0.1986 -0.2560 0.6666 0.4831 -1.2028 -0.9423 0.1656 1.2006 -1.1131 0.7704 -0.6906 -1.3143 ans = 5 ans = -0.5782 -0.3634 1.5381 -1.3173 -0.9493 0.8480 1.5921 -0.4069 0.7795 -0.3390 -0.1071 0.4201 -0.0184 0.2865 -0.1139 -0.1171 0.2288 0.5511 0.1787 0.7583 0.3994 1.0457 0.3291 -0.9150 0.3641 -0.6420 -0.2096 0.7761 0.4022 -0.7478 0.1165 0.7142 0.7029 -1.1195 0.0905 0.6810 ans = 4 ans = 0.1246 -0.3173 0.8068 0.6485 -0.8572 0.2275 0.3674 -0.0507 -0.9196 0.6161 -0.5821 -0.4291 -1.0142 -1.1614 -2.5438 1.5915 2.0356 0.4535 -0.2111 -0.3974 0.0376 0.3825 -1.9702 1.5318 -0.3890 0.9210 -0.0635 0.3248 1.8666 -0.0160 1.3908 -0.7204 -0.6772 -0.0713 0.0569 0.5929 ans = 3 ans = -0.1602 0.6891 0.4725 0.0277 -2.0510 -2.2440 -0.7497 1.8225 -0.4433 0.4090 0.9021 -1.6683 0.0659 0.3909 0.2043 0.9065 1.4630 0.3091 -0.3886 0.6715 -0.9742 -0.5468 0.2890 0.5625 -0.4558 0.4770 -0.1888 -0.6504 0.3281 1.3767 0.3983 0.5834 0.9360 0.8604 -0.9776 0.6755 ans = 10 ans = -0.4843 -0.4512 0.7544 0.7585 -0.4417 -0.0208 1.8537 -1.6935 -2.7067 -0.5077 0.9616 -1.7904 -1.6943 -1.0988 0.1208 -0.8100 1.8778 1.1654 1.1759 -0.7087 -1.2673 -0.1381 -0.0710 0.5343 0.2589 -0.5128 -0.3970 0.6737 0.8097 2.7024 -0.8933 0.2810 0.8117 -0.5428 -0.8782 1.1746 ans = 9 ans = 0.0254 -0.7993 1.5164 1.2921 -1.1013 1.8556 -0.6280 0.9374 -0.1962 0.1685 -0.5079 0.4333 -0.3962 -0.9977 0.6971 -1.0310 -1.1997 -2.1391 0.7179 1.0177 -0.8874 -0.6732 0.7295 1.4448 -1.1793 -1.3210 1.5292 0.2280 1.9337 1.0901 -0.0926 0.1798 -1.1740 0.3447 2.4578 0.4170[/code] I wonder if the code is correct, and does it retain the state of the global stream after the function call? Please help me, thank you very much Small Bio