/[MITgcm]/MITgcm_contrib/llc_hires/llc_4320/readme.txt
ViewVC logotype

Contents of /MITgcm_contrib/llc_hires/llc_4320/readme.txt

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph


Revision 1.7 - (show annotations) (download)
Mon Nov 4 16:06:42 2013 UTC (11 years, 8 months ago) by dimitri
Branch: MAIN
Changes since 1.6: +1 -0 lines
File MIME type: text/plain
adding "export MPI_NUM_MEMORY_REGIONS=256"
to avoid out-of-memory crashes

1 For interactive session, Ivy Bridge nodes:
2 qsub -I -q devel -l select=300:ncpus=20:model=ivy,walltime=02:00:00 -m abe -M email
3 qsub -I -q normal -l select=300:ncpus=20:model=ivy,walltime=8:00:00 -m abe -M email
4 qsub -I -q long -l select=300:ncpus=20:model=ivy,walltime=120:00:00 -m abe -M email
5 qsub -I -q long -l select=300:ncpus=20:model=ivy,min_walltime=30:00,max_walltime=120:00:00 -m abe -M email
6
7 ==============
8
9 cd ~/llc_4320
10 cvs co MITgcm_code
11 cvs co MITgcm_contrib/llc_hires/llc_4320
12 cd MITgcm
13 module purge
14 module load comp-intel/2012.0.032 mpi-sgi/mpt.2.08r7 netcdf/4.0
15 mkdir build run
16 lfs setstripe -c -1 run
17 cd build
18 cp ../../MITgcm_contrib/llc_hires/llc_4320/code/SIZE.h_72x72x29297 SIZE.h
19 ../tools/genmake2 -of \
20 ../../MITgcm_contrib/llc_hires/llc_4320/code-async/linux_amd64_ifort+mpi_ice_nas -mpi -mods \
21 '../../MITgcm_contrib/llc_hires/llc_4320/code ../../MITgcm_contrib/llc_hires/llc_4320/code-async'
22 make depend
23 make -j 8
24 cd ../run
25 ln -sf ../build/mitgcmuv .
26 ln -sf /nobackup/dmenemen/tarballs/llc_4320/run_template/* .
27 ln -sf /nobackup/dmenemen/forcing/ECMWF_operational/* .
28 cp ../../MITgcm_contrib/llc_hires/llc_4320/input/* .
29 mv data.exch2_72x72x29297 data.exch2
30 export MPI_NUM_MEMORY_REGIONS=256
31 mpiexec -n 35000 ./mitgcmuv
32
33 ==============
34
35 look at output
36
37 for ts=[0 120 600:10:980 1080:120:2280]
38 fld=quikread_llc(['Eta.' myint2str(ts,10) '.data'],4320);
39 clf,quikplot_llc(fld),caxis([-2.5 2]),thincolorbar
40 title(ts)
41 pause(.1)
42 end
43
44 ==============
45
46 to determine empty tiles:
47 grep Empty STDOUT.*
48
49 =============
50
51 memory requirements:
52 nPx sNx sNy nSx cpu node0 total
53 3744 180 180 2 san 22,106,128kb 5,195,641,224kb - node ran out of memory and crashed
54 5616 120 120 3 san - node ran out of memory and crashed
55 7488 180 180 1 san
56
57 =============
58
59 2 = 2
60 3 = 3
61 2*2 = 4
62 5 = 5
63 2*3 = 6
64 2*2*2 = 8
65 3*3 = 9
66 2*5 = 10
67 2*2*3 = 12
68 3*5 = 15
69 2*2*2*2 = 16
70 2*3*3 = 18
71 2*2*5 = 20
72 2*2*2*3 = 24
73 3*3*3 = 27
74 2*3*5 = 30
75 2*2*2*2*2 = 32
76 2*2*3*3 = 36
77 2*2*2*5 = 40
78 3*3*5 = 45
79 2*2*2*2*3 = 48
80 2*3*3*3 = 54
81 2*2*3*5 = 60
82 2*2*2*3*3 = 72 * 60
83 2*2*2*2*5 = 80 * 54
84 2*3*3*5 = 90 * 48
85 2*2*2*2*2*3 = 96 * 45
86 2*2*3*3*3 = 108 * 40
87 2*2*2*3*5 = 120 * 36
88 3*3*3*5 = 135 * 32
89 2*2*2*2*3*3 = 144 * 30
90 2*2*2*2*2*5 = 160 * 27
91 2*2*3*3*5 = 180 * 24
92 2*2*2*3*3*3 = 216 * 20
93 2*2*2*2*3*5 = 240 * 18
94 2*3*3*3*5 = 270 * 16
95 2*2*2*2*2*3*3 = 288 * 15
96 2*2*2*3*3*5 = 360 * 12
97 2*2*2*2*3*3*3 = 432 * 10
98 2*2*2*2*2*3*5 = 480 * 9
99 2*2*3*3*3*5 = 540 * 8
100 2*2*2*2*3*3*5 = 720 * 6
101 2*2*2*2*2*3*3*3 = 864 * 5
102 2*2*2*3*3*3*5 = 1080 * 4
103 2*2*2*2*2*3*3*5 = 1440 * 3
104 2*2*2*2*3*3*3*5 = 2160 * 2

  ViewVC Help
Powered by ViewVC 1.1.22