How Do I Determine FEA Software Quality?
What Managers Need to Know About Finite Element Analysis What questions should I
ask?
Better, stronger, lighter, safer, less costly--in less time. But how
do you get there from here? Computer simulation tools have been automated
to a level that produces significant competitive advantage for companies
designing mechanical products--and a significant competitive disadvantage
for those without them. Finite element analysis (FEA), based on solid
geometric modeling, provides the foundation for predicting product performance
throughout the entire design and manufacturing process and into the hands
of customers. Better, stronger, lighter, safer, less costly, in less time--through
Predictive Engineering.
Stronger, lighter, safer, better--these are attributes of product quality. Giving your products these characteristics provides you competitive advantage and makes your products more desirable. You use FEA to help ensure that these and other quality characteristics are designed and built into your products. How do you ensure that your FEA program has the same high quality as your products?
What is quality?
There are several definitions of quality. One definition is "free from error." But, that definition is very narrow. Another definition (Webster's New World Dictionary, Simon & Schuster, New York, 1982) is "the degree of excellence which a thing possesses." But what degree? And what is excellence?
We offer another definition that we will use throughout this paper:
Quality is meeting the needs and expectations of the customer.
In other words, quality is in the mind of the beholder--the customer. You might believe that your product is good, but if it does not meet the needs and expectations of your customers, then it does not represent quality for them. The same holds true for FEA software--it must meet your needs and expectations.
OK, what are my needs and expectations of FEA software?
Your needs and expectations of FEA software are basically the same as for any other product: You want a reliable, easy-to-use product that is also a good value. In addition, the software must contain the necessary functionality to enable you to accurately analyze and optimize your structure, thereby decreasing time and effort in the design cycle. The FEA software must also interface to design, modeling, and other analysis software that you use. These attributes can be further defined according to software quality factors. What are software quality factors?
Software quality factors (G. Gordon Schulmeyer and James I. McManus, Handbook of Software Quality Assurance, Van Nostrand Reinhold Company, New York, 1987) define attributes that any software--including FEA programs--should have. These quality factors represent many of your needs and expectations, including:
- Correctness
- Reliability
- Testability
- Usability
- Efficiency
- Interoperability
- Portability
- Flexibility
- Maintainability
- Reusability
- Integrity
Other quality factors are also important , as we'll see later.
Correctness--isn't that the most important quality factor?
In many ways it is. Correctness means that the program satisfies your specifications and meets your expectations--in other words, that it is the right tool for your job. If you need to predict your product's performance under a dynamic (time varying) environment and your FEA program contains only static analysis (time invariant), then you do not have the right tool for the job. Correctness, then, is a quality factor that only you can determine. How do I determine reliability?
Reliability is a measure of the FEA program's precision...the ability to get the correct answer when given the correct input. Measuring reliability is the responsibility of both the FEA vendor and the user.
The FEA vendor should run numerous tests in order to ensure reliability. These tests should include textbook problems for which there are known solutions. However, the tests must also include complex, real world problems because those are the kinds of models you create. The vendor should provide, upon request, results of these tests as well as the test input files. These tests should not be run only once; they should be run each time a new version is released, and on each type of computer platform.
You could also measure reliability using the kinds of models typical to your
designs. This kind of testing is often called acceptance testing;
you will not accept the software into production use until it has passed
your tests. Depending on the depth of your FEA vendor's test and quality
assurance procedures, you may choose to bypass your acceptance testing
and put the software into production use immediately.
If the software cannot be easily tested then it is difficult to assess its reliability. The effort required to test the software is called testability . The number and types of test problems run indicate the vendor's diligence and the software's testability. But the software must not only be right...it must be easy to use.
Absolutely--it must be easy to use. The degree to which it can be easily used is called usability. This is a quality factor that is rather subjective and one that you have to establish for yourself...you cannot take the vendor's word for it. Besides, all vendors say their software is easy to use! What may be easy for one person may be difficult for another, which is why you have to decide for yourself. Is the speed of solution important?
Not only must right answers be computed in an easy-to-use manner...they must be computed quickly and with a minimum of computer resources (memory and disk space). The measurement of speed and required resources is called efficiency . You want the software to get the answer quickly--after all, the purpose of computers is to reduce analysis time. Ideally, your FEA software is designed to take advantage of your advanced computer. Similarly, the software provides the solution with a minimum amount of memory and disk space. Solution efficiency is a function of how well the software has been designed. I use other software in addition to FEA. How is that accounted for in the quality factors?
Interoperability is the ability to interface, couple, and integrate with
other software. Common programs that work with FEA software include solid
modeling, CAD, kinematics, plotting, and testing programs. Your FEA program
must be able to "talk to" these programs.
What about the other quality
factors? They are also important. Portability means being able to use the software on multiple operating systems or hardware platforms. Flexibility is a measure of how easy it is to add new functionality. Maintainability is a similar measure for fixing errors. Reusability concerns software use in multiple applications--for example, it can be thought of as a measure of how much the program is general purpose versus special purpose. And integrity is a measure of your confidence that no one has inadvertently or purposely modified or changed the program or the data such that the analysis is adversely affected. What other factors are important?
Learnability is a measure of how easy the program is to learn. This is somewhat distinct from usability in that usability is about application of the mechanics (i.e., the ease of operation) whereas learnability is about the underlying concepts and theories. Learnability involves training and documentation. Training should be not only for the mechanics but also for the underlying principles, with an emphasis on applications. Training is best done using workshop examples, because engineers learn with their hands and fingers and not only with their eyes and ears.
User documentation must be complete, well written, and organized in a manner that makes it easy to find information. There are two broad classes of documentation: tutorial and reference. Tutorial documentation is for a first time or occasional user of either the software or a particular analytical capability, and is meant to be read from front to back. It should include numerous examples. Reference documentation, on the other hand, is used to learn about a specific feature or detail.
Availability is another important factor--the program must be available on your specific computer type. Likewise, updated versions, including enhancements and error fixes, must also be available on a timely basis on your computer. You need the FEA vendor's commitment of continued support for your computer type.
Accountability means owning up to the fact that the software is not perfect...and
telling you about it. FEA vendors should publish error reports that describe
the error and show ways to work around them. Nobody is perfect, and that
goes for FEA vendors. Reputable vendors acknowledge that fact, document
the errors, and help you avoid them...and in the meantime work to correct
the errors.
Compatibility measures the degree to which different versions of the software work together. For example, if a program is upward compatible , later versions will support the current version. Conversely, downward compatible means that the current version supports previous versions. Upward and downward compatibility is important because as new versions of the software are available you do not want to recreate and reanalyze your designs from scratch. The FEA software should have a proven track record of compatibility.
What vendor attributes are important from a quality standpoint?
Your FEA vendor must provide software that meets your needs, including the quality factors described in this paper. In addition, your vendor must be a stable company--the vendor's going out of business does you no good. Your vendor must be knowledgeable and must provide good technical support. You are probably best off with a vendor who has a large and satisfied installed base and proven track record, because they are proof and assurance of good software quality. In short, quality software comes from a quality vendor. How can I assess a vendor's commitment to quality?
All vendors say they are committed to quality. To judge for yourself, look at the size of the installed base (especially of users in your industry), clarity and depth of the user documentation, number of the development and support personnel and their knowledge and experience, and overall company track record. And look for a formal quality assurance (QA) program. What is a formal software QA program?
A formal software QA program defines in writing the process for ensuring software quality.
Software QA involves several functions, including:
- Software management
- Configuration management, change control, project management, and project documentation.
- Software engineering
- Software design methodologies, CASE (Computer Aided Software Engineering) tools, and independent verification and validation.
- Quality Control
- Quality programs and procedures, quality metrics, standards and methods, design reviews, and code walk-throughs.
The function of management is to apply the people and tools to the quality process. The organized approach that software QA gives to the development process produces software of consistent quality at lowest cost.
Several organizations specify software QA procedures, standards, and regulations. Some of these organizations are:
- Nuclear Regulatory Commission (NRC)
- American National Standards Institute (ANSI)
- Institute of Electrical and Electronic Engineers (IEEE)
- International Standards Organization (ISO)
- Federal Aviation Administration (FAA)
Many of the procedures, standards, and regulations defined by these organizations are similar. For example, NRC 10 CFR 50 regulations contain many of the same specifications found in ISO 9000. Each define standards for software design, internal documentation, testing, and corrective action. Software vendors that comply with these regulations submit themselves to audits--internal and external--to assure compliance. These regulations cover the entire software life cycle.
What is the software life cycle?The software life cycle covers the life of the program, including requirements, design, implementation, porting, testing, release and distribution, and maintenance.
The requirements phase defines the functionality, performance, and user
interface. The design phase describes the software architecture,
components, and data flow. Coding and the writing of user documentation
take place in the implementation phase. Software is converted to
run on multiple computer platforms in the porting phase. Software
accuracy and performance is evaluated in the testing phase. Software
and user documentation are packaged and shipped in the release and
distribution phase. The maintenance phase covers training,
support, and error corrections.
Because the software life cycle begins with defining the requirements, it is essential that the requirements be properly addressed. FEA vendors use current customers, prospects, competitive analysis, and market research as sources of these needs and expectations. All of the vendor's organizations--marketing, sales, development, support, and shipping--contribute to obtaining and addressing the requirements. So quality really begins with me?
That's right--your needs define the overall requirements. Quality FEA software meets these needs.
When adopting FEA as an integral component of your design philosophy, you must not only consider your immediate needs, you must also anticipate your future requirements. While today's needs may involve only the static analysis of simple parts, success in this area will generate requirements for assembly modeling and more advanced analysis. Your FEA software must be able to accommodate and facilitate your analysis expansion.
As the benefits of increased automation grow, thus do the demands for functionality, performance, reliability, and usability...in other words, the demand for quality!
Glossary of Software Quality Terms- Acceptance testing
- Customer testing prior to putting the FEA software into production use.
- Accountability
- Assuming responsibility for documenting and correcting errors.
- Availability
- Being able to get the FEA software on your platform.
- Compatibility
- Current, past, and previous
versions of the software support each other.
- Correctness
- The software does what you need it to do--the right tool for the job.
- Efficiency
- Fast running software that uses little memory and disk space.
- Flexibility
- Easy to add new functionality.
- Integrity
- Controlling access to software and data by authorized people.
- Interoperability
- Interfacing and integrating FEA software with solids modeling, CAD, and other software.
- Learnability
- Grasping the underlying concepts and theories and applying them wisely.
- Maintainability
- Being able to correct and enhance the software, easily and efficiently.
- Portability
- Having the software run on multiple computer types and operating systems.
- Quality
- Meeting the needs and expectations of the customer.
- Reliability
- Getting an accurate answer.
- Reusability
- General purpose software for multiple analysis applications.
- Software development cycle
- All activities related to the FEA software, from design to release to maintenance.
- Testability
- The software can be easily tested and verified.
- Usability
- Ease of operation.
