Compiler Qualification Service

Compiler test suite

The compiler is a very sophisticated software. Therefore, compiler testing requires test cases that cover a huge number of combinations, which is the number of astronomical combinations.

We strive to expand our testing program every day, and the resulting compiler quality verification test suite is the largest in the world.

  Number of test programs Corresponding standards Test contents
C 490,000
  • ISO/IEC 9899:1990 (C90)
  • ISO/IEC 9899:1999 (C99)
  • ISO/IEC 9899:2011 (C11)
  • Constants (integers, characters, floating point numbers, etc.)
  • Variables (extern, auto, etc.)
  • Arrays (one-dimensional, two-dimensional, multidimensional, variable-length array, etc.)
  • Structures/unions (Bit field, designated initializer, etc.)
  • Operators (four arithmetic operations, logical operators, bit operators, etc.)
  • Functions (arguments, declarations, etc.)
  • Pointers (variables, functions, arrays, pointers, etc.)
  • Control structure (switch-case, for, etc.)
  • _Generic selection

..... and more
C++ 380,000
  • ISO/IEC 14882:2003 (C++03)
  • ISO/IEC 14882:2011 (C++11)
  • ISO/IEC 14882:2014 (C++14)
  • ISO/IEC 14882:2017 (C++17)

The contents of the test program are diverse, from simple operation checks to those that include complex algorithms.

  • Variables (auto, decltype, etc.)
  • Type declarations (in-block declarations, etc.)
  • Type qualifiers (such as using macros for const constants)
  • Classes (public/private, friend, inheritance, etc.)
  • Templates (function, class, variable, variable-length argument, etc.)
  • Operators (:: operators, reference operators, new/delete, etc.)
  • Control structure (The range-based for statement, Initializing statements for if and switch statements, etc.)
  • Casts (function format, etc.)
  • Functions (overload, inline, lambda expressions, etc.)

..... and more
Fixed point 80,000
  • ISO TR
  • ACE
It supports both specifications such as modwrap peculiar to ISO TR and circ peculiar to ACE specification.
  • Four arithmetic operations and compound operations using fixed-point type
  • Verification of overflow processing using overflow type specifier (saturation/modwrap)
  • Verification of fixed-point type conversion processing

..... and more

Refers to the fixed-point part of Embedded-C

（The above content is current as of March 2023）

Roadmap or Release History

  Language Standard Number of test cases Total number of lines
C ISO/IEC 9899:1990 (C90)
ISO/IEC 9899:1999 (C99) 		494,000 Source 250 million lines
ISO/IEC 9899:2011 (C11) 1,600 Source 150,000 lines
ISO/IEC 9899:2018 (C17) *Be available in 2023
C++ ISO/IEC 14882:2003 (C++03) 152,000 Source 65 million lines
ISO/IEC 14882:2011 (C++11) 172,000 Source 43 million lines
ISO/IEC 14882:2014 (C++14) 59,000 Source 10 million lines
ISO/IEC 14882:2017 (C++17) 10,000 Source 1.5 million lines
ISO/IEC 14882:2020 (C++20) *Be available in 2024

In the evaluation of the C++ compiler tool, the test cases for the C compiler are also used for inspection.