Questions And Answers

To All Your Questions

1. What does CEH Solutions specialize in?

Meeting DO-254 Simple approach for various developed and COTS Complex Devices

Meeting DO-254 approval for various COTS Intellectual Programs (IP)

Cost Effective Verification Independence

Fast Track Low Cost to Completion & Project Oversight

Functional Testing Including Internal Logic/Gate Level on Targeted Devices

Meeting Level A Elemental Analysis Requirements

2. How does the process meet DO-254?

The process provides an opportunity to meet DO-254 simple definition while preserving the ability to meet level A Appendix B, Elemental Analysis,  requirements management, traceability, independent verification, and dissimilar modeling. This includes a reduction in the number of documents typically from 15 to 4.

Our process provides confidence and evidence that design errors are rule out in advance by separating a complex implementation into simpler elements at the level that the designer generated it.

3. What are the customer benefits?

Minimize cost of future change

 Faster debugging results

Capable of conducting pin, functional, and gate level testing

Schedule reduction.

Development and cost of change cost reduction.

4. How does CEH process support cost of change and obsolescence?

Cost of change and part obsolescence are inherently addressed with minimum cost and impact. The process allows the customers to migrate/ transport the architect easily to a replacement device by simply rerunning the formal testing on the new device. The cost of this effort is reduced to the amount of time it takes to modify the development board, load the behavior into the new device and press the test button. This is consistent with the need for regression testing in a much simper and cost effective solution. Cost of change is reduced to a fraction of the cost one would normally occur.

5. What are the types of devices supported by the process?

         PLD’s such as FPGA, CLPD, PAL, support development of ASICs using PLD test beds.

6. What kinds of problems have your process found?

Configuration Management Corruption

Pin - state sensitivity at startup (floating, tri-state, low, high)

Requirements inconsistency

Functional Timing/Race Conflicts

Functional failures

Simulation/Synthesis errors

Latent safety hidden failure condition

Place & Route errors

PLD speed sensitivity at slower rate than guaranteed by mfg

Unfavorable tool optimization changes due to HDL synthesis

7. Has your process been reviewed by the FAA as an acceptable method?

The innovative process has supported 14 successful FAA approval for Level A-& B FPGA, PAL, and CPLD type devices over the past 3 years.

8. How does this map to DO-254?

Section Description 1.6, Paragraph 3, 6 Complexity Considerations Figure 2-3 Decision Making Process for Selecting the Hardware Design Assurance Strategy 5.1.2 Requirements Capture Activity 6.2, 6.3 Verification Process and Methods 7.0 – 7.3 Configuration Management 8.2 - Process Assurance activities 10.1.1 Plans For Hardware Aspects of Certification 10.1.4 Verification Plan 10.2.3 Verification Standards 10.4 -10.4.5 Verification Data 10.5 Acceptance Criteria 10.6 Problem Report 10.7 Configuration Management Records 10.8 Process Assurance Records 10.9 Accomplishment Summary 11.1.5 Additional Configuration Considerations 11.2 Commercial-Off-The-Shelf Component Usage 11.4 Tool Assessment 3.3 (1) Advanced Verification – elemental Analysis

9. What design assurance levels is your process support?

         Level A, B, C

10.  How long is a typical program?

         2 – 3  months including development board design, debugging, and final documentation.

11.  Do you use qualified tools?

Our model based and signal acquisition tools are independently verified at three different stages of the process and the tools have been used consistently over 14 programs with 100% success.

12.  Does the process address board integration influences due to the intended operating environment?

 The process depends on board level testing as integrated into the final design to address DO-160 (environmental)            qualification approvals in the intended operating environment beyond device level.

13.  Does the process address gate/logic misbehavior?

Gate and logic misbehavior can occur and has occurred in several projects. These types of conditions are discovered during the testing process.

14.  Does the process identify altered behavior due to pin assignment placement, place and route, and/or  changes?

 In many cases, the testing has identified where pin location and proximity to other signals were causing incorrect                behavior. This is typically discovered by erratic outputs, incorrect outputs or during the place and route activity.

15.  Does the process address compiler optimization?

Compiler optimization can and will create the most effective solution per the compiler constraints. However, our process   can detect the functional influence through compiling each function multiple time. Therefore, one can make an assertion      that compiler optimization will not cause a concern when no behavioral conditions are discovered through combination     and permutation testing. CEH has not found an adverse compiler optimization error in testing over 6 million combinations over 1 devices using multiple device manufacturers.

16.  Does the process address foreseeable conditions as required by CFR Title 14, X.1309 System Safety?

Foreseeable conditions are addressed through combination and permutation testing of each of the functional models.         The test processes addresses corner cases, robustness, and normal operation .When assembled into a production            package, foreseeable conditions are bounded to performance and board interface conditions.

17.  How does the process support requirements traceability?

 Requirements traceability is achieved by identifying each of the functional modules and their test cases in the test                procedure and the higher level requirement(s) the module supports.

18.  How does your process satisfy DO-254 independent verification requirements?

DO-254 requires an intellectual separation of effort between the designer and the verifier.

DO-254 states ” Independence - Separation of responsibilities which ensures the accomplishment of objective                evaluation. Refers to intellectual independence, such as another individual, and not departmental or company                     independence.

For verification, independence is achieved by evaluation of the technical correctness of the data by means, either someone or something, other than those used to produce the data.

For process assurance, independence is achieved by evaluation of process compliance by means, either someone or something, other than those used to perform the process.” (RTCA/DO-254, April 2000)

CEH provides independent verification of someone or something inherently through our involvement and provide             independent reviews as well as verify the design implementation meets the requirement thus satisfying DO-254                 independent verification requirement.

19.  How does CEH process support the FAA’s requirement for reproducibility 10-20 years later?

CEH process supports this through archiving all software, test equipment and development board.

20.  How does the process meet configuration management requirements?

CEH works wit the customer to ensure the final configuration is placed under the customer configuration control                process. This achieves two purposes. It ensures that the configuration files were not get corrupted when placed in an        independent archive system and it verifies the process for reproducibility is achievable and repeatable.

21.  Does the process take into consideration robustness testing?

 Robustness testing is inherently built into the combination and permutation testing. No additional effort is necessary.

22.  How do you know if the device was programmed correctly?

The ability to ensure the device is programmed correctly is a combination of tool usage, dissimilar testing approach. If       the programming file was retrieved out of cm control, one can consider the use of checksums (lest preferred), CRC          values, hash values to ensure no bits have sifted within the NETLIST and RTL file during cm control and retrieval.

23.  Do you stress the device during your test?

Our test method does not stress the device over environmental conditions.  We can however, input various voltage conditions to determine if there is an impact to the test results.

24.  Where do you obtain the device’s internal architecture information when you have to test a COTS piece of                   hardware?

Typically we will contact the supplier or vendor and gather as much information as possible. If the hardware is something like an FPGA we would expect to receive the designer’s specifications and documentation along with the hardware being submitted for test.

25.  How do you run set up and hold violations over temperature?

Our test method does not stress the device over environmental conditions.  We can however, input various voltage conditions to determine if there is an impact to the test results.

26.  How does this approach provide feedback or address the solution if a state machine portrays an error?

Since the simple approach we use to breaks the device’s functionality down into logical portions the root cause of the error can easily be traced back to the offending portion of the architecture in most cases. The approach exercises all combinations thus capable of providing a single unique bit pattern feedback to the designers for pinpoint troubleshooting.

27.  How do you address the state of unused pins and/or logic cell/gates on a device during functional testing?

Typically the designer’s documentation will identify the state the unused pin shall remain in during all modes of device operation. Our process does not meet exhaustive testing at the line level.  You can only show the functional behavior is operating correctly for complex processors. To test the device our simple test approach will assign the specification value conditions to be checked against the results.  If this condition changes during the test it will be identified as differences between what was expected and the actual hardware performance. A report will be generated for the customers review and evaluation.

The need to address unused pins or logic cell/gates has not been necessary. DO-254 requires abnormal, robustness, and normal testing to be performed under “intended as designed”. Internal miswirings (compilers, place & route errors, etc.) or pin cross couplings are considered a random event which cannot be deterministically defined by definition.

28.  Does your process support exhaustive testing on processors?

The shear complexity of a processor and the unlimited cross functionality internal to these typically proprietary functions limits the applicability of our approach.  Our methods typically cannot be used on these type devices.

29.  Does CEH Solutions, Inc. maintain the only formal copy of the data and test report?

We deliver the data and report to the customer for release into their documentation system.  Therefore, the only official copy with respect to the FAA is being maintained and controlled by the customer’s CM system. CEH Solutions, Inc. also will maintain a copy for the future reference but it is only an official copy to the customer and not necessarily the FAA.

30.  How does CEH Solutions’ simple test method detect unintended internal connections the designer may not be            aware of?

Yes, in limited scope. The limitations are based on the number of physical pins available to be used to map the internal functional outputs to the physical pin output for monitoring and analysis. Unintended temporary connections (coupling) as a result of environmental conditions such as EMI/EMC or HIRF are considered random in nature. Our process and lab equipment is not capable of producing EMI/EMC/HIRF/SEU conditions to allow monitoring and are considered board level/system integration requirements. However, our tests are capable of being run under these conditions under the correct test environment.

31. Does CEH Solutions, Inc. perform design work?

Our primary focus is on independent verification testing. Depending on the specifics of design work being requested, CEH would be willing to consider such work through a staff augmentation approach with work direction provided by the customer. CEH is not in the position to provide on-site contractor labor that is traditional to a temporary job placement service.