MSC.Nastran 2004

Sun Platform FAQs

Table of Contents

1. What's new in MSC.Nastran 2004 on the Sun platform?

2. Where can I download  "Product Updates" for  MSC.Nastran 2004?

3. Where can I download the Sun Performance Library for UltraSPARC I and II processors?

4. Where can I download HPC Tools 5 Software (HPC 5.0) required for MSC.Nastran 2004's Distributed Memory Parallel (DMP) capability?

5. How can I calculate the best  MSC.Nastran Memory Settings (command line keyword "mem=")?

6. What are some guidelines for choosing the best file allocation settings?

7. What are some of the MSC.Nastran 2001 vs. 2004 performance related changes?

8. What tools are available for evaluating/comparing MSC.Nastran performance?

9. What are the advantages of the 64-bit MSC.Nastran binary?

1.  What's new in MSC.Nastran 2004 on the Sun platform ?

• MSC.Nastran 2004 now targets the UltraSPARC III processor family for maximum performance.  This allows MSC.Nastran to take advantage of the many UltraSPARC III hardware capabilities, such as: hardware prefetch, intrinsic level parallelism, enhanced cache architecture, a highly scalable SMP capability, and an advanced error detection and correction system.

• Other MSC.Nastran 2004 features:

1.     Upgraded to use the Sun ONE Studio 7 Fortran 90 compiler. 

2.     Built at the Solaris 8 HW 2/02 Operating System.

MSC.Nastran 2004 is certified to run at Solaris 8 HW 2/02 and later versions of the Solaris 8 OS.  It is also certified to run at all versions of the Solaris 9 OS.

3.     Enhanced to use the Sun Performance Library.

MSC.Nastran 2004 is the first release to use the Sun Performance Library (libsunperf.so). This library is a set of optimized, high-speed mathematical subroutines for solving linear algebra and other numerically intensive problems.

4.     Upgraded to Sun HPC Cluster Tools 5 Software (HPC 5.0). 

(see HPC 5.0 download instructions)

5.     Several new I/O methods have been added for additional performance. 

 (see I/O methods)

6.     An optional 64-bit MSC.Nastran binary is now available.

 (see 64bit MSC.Nastran binary advantages)

Back to Top

2.  Where can I download product updates for the MSC.Nastran 2004 ?

• These libraries can be downloaded from MSC.Software's "Product Update" site:

http://www.mscsoftware.com/support/software_updates

Note:  This site currently has an update for missing files needed for MSC.Nastran 2004 (64-bit binary only).  This update does not affect the MSC.Nastran 32-bit binary (which is the default on MSC.Nastran 2004).  You will need to install this update if you get a fatal message indicating missing files when you invoke the 64-bit binary on the MSC.Nastran command line (i.e., exe=.../solaris/solaris64/ultra.analysis.64).

• Installation Instructions:

The file, V2004_Nastran_solaris64_so.tar.gz , should be downloaded and it's contents placed in your MSC.Nastran 2004 64-bit installation directory:

/<install_directory>/msc2004/solaris/solaris64

Back to Top

3.  Where can I download the Sun Performance Library for UltraSPARC I and II processors?

•  These libraries can be downloaded from MSC.Software's "Product Update" site:

http://www.mscsoftware.com/support/software_updates

Note:  MSC.Nastran 2004 now targets the UltraSPARC III processor family for maximum performance, but continues to be compatible with UltraSPARC I and II processors.  Those with UltraSPARC I and II processors can obtain up to 7% performance (elapsed time) improvements by using processor targeted "performance libraries"--these can be downloaded and installed within your MSC.Nastran environment.  

• Installation Instructions:

For the 32-bit library, the file V2004_libsunperf.solaris32.tar.gz, should be downloaded, uncompressed (gunzip), untar'd,  and placed in your MSC.Nastran 2004 32-bit installation directory:

/<install_directory>/msc2004/solaris

For the 64-bit library, the file V2004_libsunperf.solaris64.tar.gz, should be downloaded, uncompressed (gunzip), untar'd, and placed in your MSC.Nastran 2004 64-bit installation directory:

/<install_directory>/msc2004/solaris/solaris64

Note: These files will replace the UltraSPARC III targeted files in these directories.

Back to Top

4.  Where can I download the Sun HPC Cluster Tools 5 Software (HPC 5.0) required for MSC.Nastran 2004's  Distributed Memory Parallel (DMP) capability?

• The HPC 5.0 installation files can be found at:

http://www.sun.com/servers/hpc/software/tryandbuy.html

• Installation instructions:

Download the Sun HPC Cluster Tools 5 Software Installation Guide available on the above site.

Hints:   Although detailed instructions are available in the above guide here is some information that may help during your installation:

• You can  install the HPC software using either the graphical installation tool, ctgui, or the corresponding command-line interface (CLI) tools.  Both ctgui and the CLI tools require superuser privileges to execute.

• The ctgui program can be found at:

   /<your_install_directory>/HPC5.0/hpc/Product/Install_Utilities/bin

Notes:

(1)  During installation you will be given a choice of various connection methods.  If you experience problems with one of these methods then choose the "rsh" method (remember to add the appropriate entry in your .rhosts file if you use this method).

(2) After you have installed the software you must perform a final step to add the nodes to a partition "all" using the following utility:

/opt/SUNWhpc/sbin/part_initialize

(3) Verify installation activation using the utility:

mpinfo -N

Here's some sample output from mpinfo -N

opt/SUNWspro_s1s9/prod/bin >mpinfo -N 
NAME     UP    PARTITION  OS    OSREL NCPU FMEM FSWP  LOAD1 LOAD5 LOAD15 
beaufort  y      all     SunOS   5.8   2   5754 24915 0.00  0.00  0.01 
chalupa   y      all     SunOS   5.8   4   4306 22639 2.00  1.72  0.97 
delagoa   y      all     SunOS   5.8   2   6984 16686 0.00  0.00  0.01 

 Note: Under the column "PARTITION" you must see the partition name "all".  If you see  "-" in this column you will get an error when you attempt to run MSC.Nastran ("...mprun "object" not found..").

Back to Top

5.  How can I calculate the best MSC.Nastran Memory Settings (command line keyword "mem=") ?

The ESTIMATE program provided with MSC.Nastran (described in the MSC.visualNastran Installation/Operations Guide) can be used to estimate memory and disk requirements prior to execution of the MSC.Nastran analysis run. However, in a variety of cases its estimates are not always accurate. The problem with inaccurate estimates is that performance can be adversely affected. This occurs most often when the memory estimate is too low, forcing MSC.Nastran's numerical methods to choose "spill" algorithms which are designed for limited memory space (these are significantly slower than other algorithms where larger portions of the matrices can be held in memory). Thus, providing sufficient memory to eliminate "spill" can result in significant performance improvements – sometimes resulting in up to 50% reductions in elapsed time.

With this in mind, making an initial run to get memory requirements is the best way to see what settings will work best for your job.

The following steps describe how to examine the output from a MSC.Nastran job to determine the optimum memory setting.

Step #1: (reading the MSC.Nastran .f04 file):

After your MSC.Nastran job is complete you will find several output files generated by MSC.Nastran. We're going to be looking at the .f04 file since it contains a variety of runtime information—including which MSC.Nastran modules were executed along with memory and disk usage. The most significant memory requirement statistics is printed out by the decomposition operation . Below is one example of this output in an .f04 output file – the important information is labeled "MEMORY REQR'D TO AVOID SPILL=" and "MEMORY AVAILABLE=" .

Note: Since several "decompositions" may occur in a run search for all of them and then use the largest value specified to "...AVOID SPILL=".

 
11:21:42 4:52 15300.0 0.0 200.7 0.0 FREQRS 311 FRRD1 BEGN
...
*** USER INFORMATION MESSAGE 4157 (DFMSYN)
PARAMETERS FOR SPARSE DECOMPOSITION OF DATA 	 BLOCK SCRATCH ( TYPE=CDP ) FOLLOW
MATRIX SIZE = 525027 ROWS                  	 NUMBER OF NONZEROES = 13137502 TERMS
NUMBER OF ZERO COLUMNS = 0                 	 NUMBER OF ZERO DIAGONAL TERMS = 0
CPU TIME ESTIMATE = 4951 SEC               	 I/O TIME ESTIMATE = 25 SEC
MINIMUM MEMORY REQUIREMENT = 6275 K WORDS  	 MEMORY AVAILABLE = 108468 K WORDS
MEMORY REQR'D TO AVOID SPILL = 70121 K WORDS     MEMORY USED BY BEND = 12227 K WORDS
EST. INTEGER WORDS IN FACTOR = 51910 K WORDS     EST. NONZERO TERMS = 108351 K TERMS
ESTIMATED MAXIMUM FRONT SIZE = 3455 TERMS        RANK OF UPDATE = 32

Note: The following output  from running ESTIMATE (i.e., nast2004 xl0tdf1 mem=estimate out=xltd0 rcf=nast2004rc sdir=/tmp ) on this job shows how ESTIMATE isn't always able to give the optimum memory requirement. The decomposition requirement to avoid 'spill' is shown above as 70121 K WORDS or 280mb, but ESTIMATE only calculated 122MB:

Starting ESTIMATE, please wait...

*** USER INFORMATION MESSAGE (pgm: nastran, fn: estimate_job_requirements)

Estimated DOF      = 529257

Estimated memory = 122.0MB

Estimated disk    =  4205.4MB

Note: ESTIMATE had some difficulty analyzing this job.

Note: In addition to the above "decomp" memory requirement there are other "executive system" memory requirements that use a portion of the amount you specify for "mem=".  These other memory areas are listed near the top of the .f04 file (see below).

--The following output was taken from a job run with the following command line: nast2004 xl0tdf1 mem=450mb out=xltd0 rcf=nast2004rc sdir=$SDIR1

** MASTER DIRECTORIES ARE LOADED IN MEMORY.

USER OPENCORE (HICORE)      = 108478947 WORDS

EXECUTIVE SYSTEM WORK AREA  = 447997 WORDS

MASTER(RAM)                 = 58913 WORDS

SCRATCH(MEM) AREA           = 6553700 WORDS ( 100 BUFFERS)

BUFFER POOL AREA (GINO/EXEC)= 2425239 WORDS ( 37 BUFFERS)

TOTAL MSC.NASTRAN MEMORY LIMIT = 117964796 WORDS

Step #2: (setting the memory for your run).

Here is the important memory related "decomposition" output:

MEMORY REQR'D TO AVOID SPILL = 70121 K WORDS'

Memory Available             = 108468 K WORDS

You can see that this job will not spill since there is sufficient "Memory Available".   However, if "Memory Available" had been less than 70121 K words than you would have needed to increase the memory setting for this job by the difference between the two numbers and then re-run the job with this larger  "mem=" setting.

Back to Top

6.  What are some guidelines for choosing the best file allocation settings?

Since I/O time can be a significant part of a MSC.Nastran run, it's important to configure your job with the best I/O settings for your particular machine and MSC.Nastran job configuration.

The three topics in this section will discuss methods for optimizing your MSC.Nastran I/O performance.

Note: This discussion assumes you're running the job as a scratch run (i.e., scr=yes specified on the MSC.Nastran command line). However, the same recommendations would apply if this were a permanent database run (i.e., scr=no)--the only difference being that you would have to apply the following recommendations to not only the SCRATCH DBSET (SCRATCH/SCR300), but also to any additional permanent database files allocated to your run (i.e., DBALL DBSET).

Keep in mind that you should always give the highest priority to assigning the SCRATCH DBSET to your fastest filesystem since the Scratch DBSET (especially the SCR300 files) generates the most I/O activity compared to any other DBSET.

Topic #1: (Swap filesystem (/tmp))

As a rule of thumb, the swap filesystem (/tmp) should be used whenever possible since in most cases it provides I/O speeds that are faster than your standard disk-based filesystems.  This is based on studies using a variety of solution sequences and disk configurations. To give you an idea of the variations you might see using different I/O methods (see figure1.1).

With this in mind lets look at a few examples and some recommended file allocation settings:

Case #1:

• Machine Configuration:

SunBlade 2000, 8GB Memory, 16GB swap space (/tmp)

MSC.Nastran Job's High-Water file requirements:

SCRATCH DBSET (Total 6GB)--

SCRATCH: 3GB

SCR300   : 3GB

• Recommended MSC.Nastran settings:

To avoid excessive paging you should keep the usage of the /tmp below the physical memory (8GB).  Since the total SCRATCH DBSET space of 6GB is much less than the available 8GB of memory you can assign the entire SCRATCH DBSET to /tmp:

nastran <input_file> sdir=/tmp

Note: The above sdir setting directs both the SCRATCH (3GB) and SCR300(3GB) database files to /tmp.

Case #2

• Machine Configuration:

SunBlade 2000, 8GB Memory, 16GB swap space (/tmp)

MSC.Nastran Job's High-Water file requirements:

SCRATCH DBSET- (Total 20GB)

SCRATCH: 10GB

SCR300   : 10GB

• Recommended MSC.Nastran settings:

Even though the total Scratch DBSET size is larger than the available 8GB Memory and 16GB /tmp you should still point as much as possible of your MSC.Nastran I/O to this /tmp filesystem.  To optimize the use of the limited /tmp space you'll need to partition the Scratch DBSET between the /tmp and your other non-/tmp filesystems.

Here are the MSC.Nastran's FMS commands (INIT and ASSIGN) that you could use to do this partitioning:

INIT SCRATCH LOGICAL=(slow1(10gb),fast2(5gb)),

scr300=(fast1(10gb),slow2(5gb))

assign fast1 = '/tmp/input.fast1_1'

assign fast2 = '/tmp/input.fast2_1'

assign slow1 = 'input.slow1_1'

assign slow2 = 'input.slow2_1'

Note: The decision to use /tmp (i.e., fast1/fast2) or your standard filesystem (slow1/slow2) will depend on which file, scratch or scr300 does the most I/O (see Topic #3 (advanced techniques), Step #3).

Topic #2 (new/updated I/O methods):

MSC.Nastran 2004 has several new/updated I/O methods. These methods can provide significant performance improvement (elapsed time reductions) in certain cases.  You will have to experiment with these to find the one that gives you the best performance since they are highly sensitive to your hardware configuration and MSC.Nastran job characteristics.  However, one of these methods stands out, and can improve performance in almost all cases if you have sufficient memory to hold the entire DBSET file(s) in memory. This is the "buffio" method (described in detail within the MSC.visualNastran Installation /Operations Guide).

Hint: To ensure that your buffio "wsize,wnum" allocation setting was applied you can substitute "buffio=yes" with "buffio=must" in the "sysfield=" setting shown below.  If you don't specify "must" the space you requested may be silently reduced if it exceeds the available memory on your machine.  By using "buffio=must" the job will immediately terminate with an error message if the requested space is not available.

Here are examples of jobs that benefited from using buffio.

Job #1 (using buffio reduced elapsed time by 20%):

Machine Configuration:

Sun Fire V440, 16GB Memory

Job configuration:

MSC.Nastran 2004, Sol 101 (statics), 2.5 million DOF, 64bit MSC.Nastran executable

buffio setting: sysfield=scr*(buffio=yes),scratch(wsize=-5GB,wnum=2),scr300(wsize=-1GB,wnum=2)

Note: This sysfield setting would appear as follows on your MSC.Nastran command line:

nastran <input_file> scr=yes exe=/msc/msc2004/solaris/solaris64/analysis.utlra.64 sysfield=scr*(buffio=yes),scratch(wsize=-5GB, wnum=2),scr300(wsize=-1GB,wunum=2)

The following excerpt from the .f04 file shows the SCRATCH (9.773 GB) and SCR300 (1.235 GB) file sizes used to determine the above "wsize,wnum" settings and also verification that the buffio settings of "scratch(wsize=-5GB, wnum=2),scr300(wsize=-1GB,wunum=2)".. were set correctly:

 
 *** DATABASE USAGE STATISTICS ***
 
 +---------------- LOGICAL DBSETS ----------------+   +------------------------------ DBSET FILES -----------------------------+
  DBSET     ALLOCATED  BLOCKSIZE       USED   USED     FILE      ALLOCATED  ALLOCATED    HIWATER       HIWATER  I/O TRANSFERRED
             (BLOCKS)    (WORDS)   (BLOCKS)      %                (BLOCKS)       (GB)   (BLOCKS)          (GB)             (GB)
 
  MASTER         5000      32768         40   0.80     MASTER         5000       0.61         40         0.005            0.120
  DBALL       2000000      32768          4   0.00     DBALL       2000000     244.14          4         0.000            0.001
  OBJSCR         5000       8192        187   3.74     OBJSCR         5000       0.15        187         0.006            0.012
  SCRATCH     4000100      32768         27   0.00    (MEMFILE         100       0.01        100         0.012            0.000)
                                                       SCRATCH     2000000     244.14      80064         9.773           38.289
                                                       SCR300      2000000     244.14      10121         1.235           31.626
                                                                                                                 ==============
                                                                                                          TOTAL:         70.048
 
 *** BUFFER POOL AND SCRATCH 300 USAGE STATISTICS ***
 
 +----------------- BUFFER POOL -----------------+      +-------------------------- SCRATCH 300 --------------------------+ 
   OPTION        BLOCKS      BLOCKS      BLOCKS           OPTION    HIWATER               SUB_DMAP        DMAP     OPN/CLS
  SELECTED     ALLOCATED     REUSED     RELEASED         SELECTED  (BLOCKS)   DAY_TIME      NAME         MODULE    COUNTER
  GINO,EXEC          37      108108      472428              2       10121    18:09:17    SEKR0      53  MCE2         15
 
 
 *** SUMMARY OF PHYSICAL FILE I/O ACTIVITY ***
 
 ASSIGNED PHYSICAL FILE NAME                                   RECL (BYTES)  READ/WRITE COUNTS   WSIZE  (WNUM)  MAP-I/O CNT
 ------------------------------------------------------------  -----------  -------------------  -------------  -----------
 /tmp/xxafst0.T20869_39.SCRATCH                                     131072     221003/92658           5GB ( 2)            0 (B)
 /tmp/xxafst0.T20869_39.OBJSCR                                       32768        151/226          N/A            N/A          
 /tmp/xxafst0.T20869_39.MASTER                                      131072         43/942          N/A            N/A          
 /tmp/xxafst0.T20869_39.DBALL                                       131072          3/9            N/A            N/A          
 /tmp/xxafst0.T20869_39.SCR300                                      131072     111796/147288          1GB ( 2)            0 (B)
 /dpl/msc/msc2004/solaris/SSS.MASTERA                                32768         72/0            N/A            N/A          
 /dpl/msc/msc2004/solaris/SSS.MSCOBJ                                 32768        344/0            N/A            N/A          
 
 
 *** SUMMARY OF PHYSICAL FILE I/O TIMING INFORMATION ***
                                                                                OPEN/READ/WRITE TIMES (IN SECONDS)
 ASSIGNED PHYSICAL FILE NAME                                       ELAPSED TIMES          USER CPU TIMES        SYSTEM CPU TIMES  
 ------------------------------------------------------------  ---------------------  ---------------------  ---------------------
 /tmp/xxafst0.T20869_39.SCRATCH                                    0./33.47/33.39         0./32.89/14.29         0./0.43/19.02    
 /tmp/xxafst0.T20869_39.SCR300                                     0./16.66/21.81         0./16.63/19.37          0./0.16/2.67    
                                                               ---------------------  ---------------------  ---------------------
                                                   TOTALS:             105.33                 83.18                  22.28        
 

Job #2 (using buffio reduced elapsed time by 14%):

Machine Configuration:

Sun Fire V6800, 96GB Memory

Job configuration:

MSC.Nastran 2004, Sol 103 (Normal Modes), 1.5 million DOF, 64bit MSC.Nastran executable

buffio setting:  sysfield = scr*(buffio=yes), scratch(wsize=-4GB,wnum=1), scr300(wsize=-1GB,wnum=1)

Note: This sysfield setting would appear as follows on your MSC.Nastran command line:

nastran <input_file> scr=yes exe=/msc/msc2004/solaris/solaris64/analysis.utlra.64  sysfield=scr*(buffio=yes),scratch(wsize=-4GB,wnum=1),scr300(wsize=-1GB,wnum=1)

The following excerpt from this job's .f04 file shows the SCRATCH (3.617 GB) and SCR300 (0.460 GB) file sizes used to determine the above "wsize,wnum" settings:

*** DATABASE USAGE STATISTICS ***
 
 +---------------- LOGICAL DBSETS ----------------+   +------------------------------ DBSET FILES -----------------------------+
  DBSET     ALLOCATED  BLOCKSIZE       USED   USED     FILE      ALLOCATED  ALLOCATED    HIWATER       HIWATER  I/O TRANSFERRED
             (BLOCKS)    (WORDS)   (BLOCKS)      %                (BLOCKS)       (GB)   (BLOCKS)          (GB)             (GB)
 
  MASTER         5000      65536         50   1.00     MASTER         5000       1.22         50         0.012           10.823
  DBALL       2000000      65536          3   0.00     DBALL       2000000     488.28          3         0.001            0.002
  OBJSCR         5000       8192        202   4.04     OBJSCR         5000       0.15        202         0.006            0.741
  SCRATCH     4000100      65536        179   0.00    (MEMFILE         100       0.02        100         0.024            0.000)
                                                       SCRATCH     2000000     488.28      14816         3.617          612.492
                                                       SCR300      2000000     488.28       1884         0.460          231.592
                                                                                                                 ==============
                                                                                                          TOTAL:        855.649



 

Topic #3 (advanced technique):

This topic describes a non-standard technique for improving performance on MSC.Nastran's distributed memory parallel (DMP) jobs.  It can provide over 10% reductions in the elapsed time of your MSC.Nastran DMP jobs.  It involves using the INIT and ASSIGN FMS commands to partition the SCRATCH DBSET for each of the separate MSC.Nastran processes started as part of the DMP job.  The INIT/ASSIGN statement settings are determined by evaluating which MSC.Nastran files would benefit most from the faster /tmp filesystem.  Currently there is no standard way to use INIT/ASSIGN statements on a DMP job and the Installation / Operations guide says these FMS commands are "not supported".   However, with a little effort you can get around this by performing the following steps:

The steps in this example assume a 2-cpu machine but they can also be applied to any number of CPU's.

Step #1 (save the intermediate submittal script):

Edit your MSC.Nastran command line by adding the flag "-n" as follows:

nastran input mem=450mb -n out=input_dmp2 rcf=nast2004rc dmp=2

Note: The -n flag will save the submittal script that the MSC.Nastran command program generates prior to actually running the MSC.Nastran job.  You can then edit this file and submit it (the MSC.Nastran job) manually.  Once you've run the above MSC.Nastran command with the -n flag , you'll need to edit the file (script) it generates so that it will point to two separate MSC.Nastran input files (where each input file will contain its own file assignments in order to optimize the use of the available /tmp filesystem. This will be done by generating/allocating "unique" database files for each parallel job--the FMS INIT/ASSIGN statements you'll need to add to these input files is described in Step #3).

For this example we'll name these two MSC.Nastran input files:

input_1.dat

input_2.dat

Here's the output you'll see when you submit the above MSC.Nastran command with the -n flag:

MSC.Nastran V2004.0 (SUN SPARC SunOS 5.8) Mon Oct 6 08:41:04 2003

Determining available hosts, please wait...

DMP task 1: host="beaufort.la.na.mscsoftware.com" sdir="/tmp"

DMP task 2: host="beaufort.la.na.mscsoftware.com" sdir="/tmp"

No-execute flag set.

Job script: ./input_dmp2.T2635_4

Job command: ./input_dmp2.T2635_4

Step #2 (edit the Job Script):

Edit the above file specified next to "Job Script":

./input_dmp2.T2635_4.

You need to edit this file so that it will refer to two separate MSC.Nastran input files instead of just the standard single file.

Here are the 2 changes you need to make:

• Change lines 123-131 from:

123 i=0

124 while (( i < 2 )) ; do

125 if [[ $i -eq 0 && ${jidvis[0]} = yes ]] ; then

126 jid=./input.dat

127 elif [[ $i -gt 0 && ${jidvis[$i]} = yes ]] ; then

128 jid=./input.dat

129 else

130 jid=${outs[$i]}.t$i.dat

131 fi

• To:  (note: you'll be changing the jid=input.dat references to the two separate input files)

123 i=0

124 while (( i < 2 )) ; do

125 if [[ $i -eq 0 && ${jidvis[0]} = yes ]] ; then

126 jid=./input_1.dat

127 elif [[ $i -gt 0 && ${jidvis[$i]} = yes ]] ; then

128 jid=./input_2.dat

129 else

130 jid=${outs[$i]}.t$i.dat

131 fi

Step #3:(create two input files (input_1.dat, input_2.dat):

In this step you'll need to copy the original input.dat file and make two new files with the following INIT/ASSIGN statements added.

• /input_1.dat

$ The following is based on the fact

$ that limited memory and /tmp (swap) filesystem space

$ will benefit the scr300 the most.

INIT SCRATCH LOGICAL=(slow1(3gb),fast2(3gb)),

scr300=(fast1(3gb),slow2(3gb))

assign fast1 = '/tmp/input.fast1_1'

assign fast2 = '/tmp/input.fast2_1'

assign slow1 = 'input.slow1_1'

assign slow2 = 'input.slow2_1'

• /input_2.dat

$

$ The following is based on the fact

$ that limited memory and /tmp (swap) filesystem space

$ will benefit the scr300 the most.

INIT SCRATCH LOGICAL=(slow1(3gb),fast2(3gb)),

scr300=(fast1(3gb),slow2(3gb))

assign fast1 = '/tmp/input.fast1_2'

assign fast2 = '/tmp/input.fast2_2'

assign slow1 = 'input.slow1_2'

assign slow2 = 'input.slow2_2'

Note: The fast and slow logical files used in the above INIT/ASSIGNs and their sizes is based on evaluating the following .f04 output.  In this particular example the I/O on the SCR300 file is 15x more than the SCRATCH.   And since there is limited /tmp and physical memory on this 8GB memory machine than the intent is to assign as much as possible of the scr300 file to the /tmp filesystem.  This is especially important in this example because with DMP=2, each of the 2 processes will allocate 6GB each for the SCRATCH DBSET.  In this case the more heavily used scr300 file should be given as much of the /tmp space as possible so that it doesn't have to compete for space with the less used 3GB SCRATCH file.

Here's the .f04 output showing the SCRATCH and SCR300 file usage:

 
 *** DATABASE USAGE STATISTICS ***
 
 +---------------- LOGICAL DBSETS ----------------+   +------------------------------ DBSET FILES -----------------------------+
  DBSET     ALLOCATED  BLOCKSIZE       USED   USED     FILE      ALLOCATED  ALLOCATED    HIWATER       HIWATER  I/O TRANSFERRED
             (BLOCKS)    (WORDS)   (BLOCKS)      %                (BLOCKS)       (GB)   (BLOCKS)          (GB)             (GB)
 
  MASTER         5000      65536         36   0.72     MASTER         5000       1.22         36         0.009            0.246
  DBALL       2000000      65536          3   0.00     DBALL       2000000     488.28          3         0.001            0.002
  OBJSCR         5000       8192        197   3.94     OBJSCR         5000       0.15        197         0.006            0.013
  SCRATCH     4000100      65536         46   0.00    (MEMFILE         100       0.02        100         0.024            0.000)
                                                       SCRATCH     2000000     488.28      10911         2.664           12.798
                                                       SCR300      2000000     488.28      11340         2.769          190.032
                                                                                                                 ==============
                                                                                                          TOTAL:        203.090
 
 *** BUFFER POOL AND SCRATCH 300 USAGE STATISTICS ***
 
 +----------------- BUFFER POOL -----------------+      +-------------------------- SCRATCH 300 --------------------------+ 
   OPTION        BLOCKS      BLOCKS      BLOCKS           OPTION    HIWATER               SUB_DMAP        DMAP     OPN/CLS
  SELECTED     ALLOCATED     REUSED     RELEASED         SELECTED  (BLOCKS)   DAY_TIME      NAME         MODULE    COUNTER
  GINO,EXEC          37       71811      828055              2       11340    20:20:14    FREQRS    311  FRRD1        14
 
 
 *** SUMMARY OF PHYSICAL FILE I/O ACTIVITY ***
 
 ASSIGNED PHYSICAL FILE NAME                                   RECL (BYTES)  READ/WRITE COUNTS   WSIZE  (WNUM)  MAP-I/O CNT
 ------------------------------------------------------------  -----------  -------------------  -------------  -----------
 /tmp/xl0tdf1.T24044_24.SCRATCH                                     262144      33561/18861        N/A            N/A          
 /tmp/xl0tdf1.T24044_24.OBJSCR                                       32768        164/246          N/A            N/A          
 /tmp/xl0tdf1.T24044_24.MASTER                                      262144         45/963          N/A            N/A          
 /tmp/xl0tdf1.T24044_24.DBALL                                       262144          1/6            N/A            N/A          
 /tmp/xl0tdf1.T24044_24.SCR300                                      262144     515920/262450       N/A            N/A          
 /dpl/msc/msc2004/solaris/SSS.MASTERA                                32768         72/0            N/A            N/A          
 /dpl/msc/msc2004/solaris/SSS.MSCOBJ                                 32768        363/0            N/A            N/A          
 
 
 ASSIGNED PHYSICAL FILE NAME                    RECL (BYTES) SIZE (BLOCKS) FLAGS    READ/WRITE COUNT   WSIZE  (WNUM)  MAP-I/O CNT
 ---------------------------------------------  -----------  ------------  -----  -------------------  -------------  -----------
 ./xltd0.xdb                                           4096          3166    L G       1015/3213         N/A           N/A          

    FLAG VALUES ARE --1G1G1G
      G   FILE ON FILESYSTEM SUPPORTING FILES LARGER THAN 2GB
      L   FILE HAS BEEN LOCKED
      M   FILE MAPPING USED TO PROCESS FILE
      S   SUN AIO INTERFACE USED TO PROCESS FILE

Note: The above READ/WRITE COUNTS for the ...SCR300 file shows the much higher  I/O activity (up to 15x) than the ...SCRATCH file.

Machine Configuration for the above job:

SunBlade 2000, 8GB Memory, 16GB swap space (/tmp)

MSC.Nastran Job's High-Water file requirements:

Scratch DBSET (Total 6GB)--

Scratch: 3GB

SCR300: 3GB

Step #4 (submit the MSC.Nastran job):

Now submit the script with the changes you made in Step #2: 

./input_dmp2.T2635_4

Note: If you will be submitting this script multiple times then you'll need to rename input_dmp2.T2635_4, otherwise it will delete itself at the end of the MSC.Nastran job.

Back to Top

Figure 1.1 (I/0 method comparison--elapsed times in seconds)

Where:

vxfs -> using Veritas File Sytem for scratch files

tmp -> using /tmp for scratch files

mapio -> using the parameter sysfield=scr*(mapio=yes,wsize=-16MB)

buffio -> using the parameter

sysfield="scr*(buffio=yes),scratch(wsize=**,wnum=**),scr300(wsize=**,wnum=**)"

Note: The buffio tests used the 64-bit MSC.Nastran binary because of the 4GB memory limit that exists with the 32-bit MSC.Nastran binary (i.e., buffio requires that MSC.Nastran be able to allocate at least enough memory to hold the total of all the (wsize * wnum) areas for each DBSET specified on the sysfield command).

Machine Configurations:

• Sun Fire 6800

24 UltraSPARC-III+ @1200MHz/8MB/150MHz

96 GB memory

4 T3 ( 9 x 36GB )

Solaris 9 8/03 s9s_u4wos_08 SPARC Generic_112233-07

Sun HPC CT 5.0

VXFS 3.5 for Solaris

• Sun Fire V880

8 UltraSPARC-III+ @1200MHz/8MB/150MHz

32 GB memory

7 36GB internal disks

Solaris 9 8/03 s9s_u4wos_08 SPARC Generic_112233-08

Sun HPC CT 5.0

VXVM 3.5 for Solaris

VXFS 3.5 for Solaris

Back to Top

7.  What are some of the MSC.Nastran 2001 vs. 2004 performance related changes?

The following four changes in MSC.Nastran 2004  may result in runs showing increased elapsed times compared to MSC.Nastran 2001.

1. The default value for parameter K6ROT changed from zero (0.0) in MSC.Nastran 2001 to a non-zero value in MSC.Nastran 2004:                                     

This change to a non-zero value will result in increased matrix sizes if you have QUAD4/TRIA3 elements in your model. You can verify this by searching in the .f04 file for "DECOMPOSITION" and then locating the matrix size "NUMBER OF NONZEROES".  This K6ROT change may result in up to 50% more non-zero matrix terms being used in the decomposition module than the same run using MSC.Nastran 2001 (assuming, of course, that you didn't explicitly set K6ROT to a non-zero value in your  MSC.Nastran 2001 run).

These extra terms can result in up to 50% more floating point calculations performed in the decomposition phase of the run, and since decomposition is a large part of most jobs this fact combined with the overall larger matrix size can add up to 60% to a job's total CPU time compared with MSC.Nastran 2001.

For example, here are two .f04 file output listings that show these extra nonzero terms in the decomposition matrices:

• This .f04 excerpt is from a Sol 108 run on MSC.Nastran 2004 : {k6rot set by default to a non-zero value}

21:03:05     0:16     1099.0     0.0     12.6     0.0     FREQRS     308     FRRD1 BEGN

*** USER INFORMATION MESSAGE 4157 (DFMSYN)

PARAMETERS FOR SPARSE DECOMPOSITION OF DATA BLOCK SCRATCH ( TYPE=CDP ) FOLLOW

MATRIX SIZE = 75871 ROWS NUMBER OF NONZEROES = 2072599 TERMS

• And here is an .f04 excerpt from the same Sol 108 run on MSC.Nastran 2004 : {with k6rot =0.0 - the default on MSC.Nastran 2001}

15:53:21     0:16     1053.0     0.0     12.4     0.0     FREQRS     306     FRRD1 BEGN

*** USER INFORMATION MESSAGE 4157 (DFMSYN)

PARAMETERS FOR SPARSE DECOMPOSITION OF DATA BLOCK SCRATCH ( TYPE=CDP ) FOLLOW

MATRIX SIZE = 62994 ROWS NUMBER OF NONZEROES = 1443719 TERMS

2. An additional decomposition may occur in order to improve initial computations for transient analysis runs.

This may occur if there are forces (spcd 's) at time=0 (in MSC.Nastran 2001 these spcd's were ignored and then applied at the next time step).

Summary: If you have SPCD cards in your input file an extra matrix decomposition will now occur on MSC.Nastran 2004 runs (affects solutions: 108,109,112, 200).

3. The new MMFIND SubDmap may now be called in MSC.Nastran 2004.

This new SubDmap call occurs when the READ module encounters three shifts that fail during the eigenvalue process--the DMAP will now call MMFIND to remove the "mechanisms" that may be causing this problem.

4. The DMAP functions that allow sparse data recovery may be unnecessarily executed in SOL 101 and ANALYSIS=STATICS in SOL 200 when data recovery for the entire model is requested.

Avoidance: If data recovery for the entire model is being requested (i.e.DISP=ALL, STRESS=ALL) the best avoidance is to include param,sparsedr,no to completely avoid the additional operations. The following dmap avoidance will allow automatic selection to occur but may not beat V2001
performance for all conditions:

compile fdrmgen
alter 'call spdr1'
return $
compile sedisp
alter 'SDR1 USET'(,-1)
call genrga uset,goa,gm,vgdr,goq/
rga,rgq/
true/atqset/seid $
dbstatus rga//s,n,norga $
if ( norga>0 ) then
mpyad rga,uas,/ugi $
else $
alter 'SDR1 USET'
endif $
endalter

Back to Top
 

8.  What tools are available for evaluating/comparing MSC.Nastran 2004 performance?

A performance evaluation/comparison utility (F04REPRT) is provided with MSC.Nastran.   It compares the performance between one or more runs and is useful for isolating which MSC.Nastran modules are responsible for the change.  A detailed description of this utility is in the MSC.Nastran Installation / Operations Guide.

Here is a sample of the output from the F04REPRT utility—note that it shows that the "DECOMPOSITION" module (DCMP) is 164 cpu seconds slower on the xxafst_old run:

% f04reprt    xxafst_old.f04   xxafst_new.f04

Old file: xxafst_old.f04

Old Machine ID: Sun Microsystems Com UltraSPARC/900 (beau SunOS 5.8

Old file BUFFSIZE: 32769, MEM: 100.00 MW

New File: xxafst_new.f04

New Machine ID: Sun Microsystems Com UltraSPARC/900 (beau SunOS 5.8

New file BUFFSIZE: 32769, MEM: 100.00 MW

 
Module      xxafst_new     xxafst_old     Diff       Ratio  Comment
 -------- -------------- -------------- ---------- --------
 DCMP             3369.2         3533.5     -164.3     0.95
 EMA                69.7           70.0       -0.3     1.00
 SSG3               57.2           57.2        0.0     1.00
 MCE2               47.7           46.4        1.3     1.03
 EMG                44.2           43.8        0.4     1.01
 EMA                44.1           44.2       -0.1     1.00
 UPARTN             18.6           18.4        0.2     1.01
 XSORT              16.0           16.0        0.0     1.00
 IFP                12.0           12.0        0.0     1.00
 TA1                11.8           12.0       -0.2     0.98
 TA1                10.7           10.8       -0.1     0.99
 VECPLOT            10.0           10.1       -0.1     0.99
 UPARTN              7.8            7.5        0.3     1.04
 UPARTN              6.0            6.1       -0.1     0.98
 MCE1                5.2            4.9        0.3     1.06  FASTER
 GPSP                4.7            4.8       -0.1     0.98
 SSG1                3.9            3.9        0.0     1.00
 VECPLOT             3.8            3.7        0.1     1.03
 GP4                 3.0            2.9        0.1     1.03
 SSG2                2.8            2.8        0.0     1.00
 UMERGE1             2.7            2.8       -0.1     0.96
 SDR1                2.5            2.5        0.0     1.00
 GP2                 2.4            2.5       -0.1     0.96
 GP2                 2.4            2.5       -0.1     0.96
 UMERGE1             2.3            2.2        0.1     1.05
 UPARTN              2.0            1.9        0.1     1.05  FASTER
 GP1                 1.7            1.7        0.0     1.00
 GP1                 1.6            1.6        0.0     1.00
 TASNP2              1.3            1.3        0.0     1.00
 RMDUPBLK            1.2            1.2        0.0     1.00
 MATGEN              0.9            0.9        0.0     1.00
 PLTSET              0.8            0.8        0.0     1.00
 OUTPRT              0.8            0.8        0.0     1.00
  ----------------------------------------------------------
 EXIT             3779.8         3942.6     -162.8     0.96
 ELAPSED          6117           6318       -201       0.97
 EXIT/ELAPSED      0.618          0.624

Module xxafst_new xxafst_old Diff Ratio Comment

Back to Top

9.  What are the advantages of the 64-bit MSC.Nastran binary?

The two main advantages of using the 64-bit binary are:

1. You can increase your MSC.Nastran memory setting to "mem=8GB"  (the 32-bit binary limit is "mem=4GB").
2. You can now take better advantage of the "buffio"  I/O option since you can specify much larger (wsize,wnum) settings than is possible using the 32-bit MSC.Nastran binary.   This effectively allows you to use all of your machine's physical memory when specifying the size of your "buffio" space, thus maximizing the performance improvement possible with this I/O method.  (see " Topic #2" for a discussion on using the buffio method")

Note: The 64-bit binary may run slower than the 32-bit binary on some runs--however, if you can make use of the above 64-bit features (i.e., increased memory capability (up to 8GB) and/or  the larger "buffio" settings) then use of these features will typically provide performance that exceeds the 32-bit binary.

Back to Top

Author-Dale Layfield.
Copyright © 2003  [SUN Microsystems]. All rights reserved.
Revised: -> 11/03/03 -> .