//header
//center

Description of the test format

Only a few information is required to describe the tests used for the test campaign. For this edition, two distinct formats will be proposed, both of them containing very similar information:

  • The first one, based on .ini files, is very basic, in order to be easy to parse, and has been used for the first edition of the contest. It is inspired by Windows .ini files.
  • The other one is a XML-based format, containing the same information that the previous format, and also some meta description of the tests (domain application, segmentation, scalability type...)

As the information essential to launch the tests are the same whatever the description file type, you can choose any of these description files if you make executable scripts.

.ini based description file

Syntax of .ini description file

Below is presented a skeleton of the description of a test. Each description is stored in a file named xxxxx.test, where xxxxx is the name of the test.

    
[testname] ... [modelspath] ... [imagespath] ... [models] ... [images] ...

Description of .ini description file

  • The testname section contains a single line, which is the name of the test. Please note that this name is important, as all filenames of result files (generated by the participants) must include this name (see naming conventions for the syntax).
  • The modelspath section contains an single line, which is the access path to the models directory. This path is expressed relatively to the directory containing the current .test file. Please note that the path is appended neither by / nor \, as the tests can be performed under several OS (Windows, Linux, ...)
  • The imagepath section contains an single line, which is the access path to the test image directory. This path is expressed relatively to the directory containing the current .test file. Please note that the path is appended neither by / nor \textbackslash, as the tests can be performed under several OS (Windows, Linux, ...)
  • The models section describes all the model files used for the current test. Each file corresponds to a line within this section. The filenames are expressed without any prefix path and without any extension. Each model file will be supplied in 3 formats: BMP, TIF and VEC. Please just add the extension of the format you consider to the filename to obtain the full filename.
  • The images section describes the test image files used for the current test. Each file corresponds to a line within this section. The filenames are expressed without any prefix path and without any extension. Each image will be supplied in 2 formats: BMP and TIF. Please just add the extension of the format you consider to the filename to obtain the full filename.

Warning for the .ini description file

Please note that several test images are contained in a single .test file. In the case where a recognition application crashes during a test, your application must be able to recover and continue the next tests.

Example of .ini description file

    
[testname] testgrec [modelspath] models [imagespath] testgrec [models] ArchitecturalA ArchitecturalB ArchitecturalC ArchitecturalD ArchitecturalE ArchitecturalF ArchitecturalG ArchitecturalH ArchitecturalI ArchitecturalJ ElectricalA ElectricalB ElectricalC ElectricalD ElectricalE ElectricalF ElectricalG ElectricalH ElectricalI ElectricalJ [images] testgrec-image1 testgrec-image2 testgrec-image3 testgrec-image4 testgrec-image5

XML based description file

DTD of XML-based description file (download it!)

<!ELEMENT test (testname, metadesc+, modelspath, imagespath, model+, testimage+)>
<!ELEMENT testname (#PCDATA)>
<!ELEMENT metadesc EMPTY>
<!ELEMENT modelspath (#PCDATA)>
<!ELEMENT imagespath (#PCDATA)>
<!ELEMENT model (#PCDATA)>
<!ELEMENT testimage (#PCDATA)>
<!ATTLIST metadesc  property  CDATA         #REQUIRED
                    value     CDATA         #REQUIRED>

  

Description of the DTD

  • The testname element contains the name of the test. Please note that this name is important, as all filenames of result files (generated by the participants) must include this name (see naming conventions for the syntax).
  • The metadesc elements contain useful meta information about the test, described as some pairs property/value,
  • The modelspath element contains the access path to the models directory. This path is expressed relatively to the directory containing the current .xml file. Please note that the path is appended neither by / nor \, as the tests can be performed under several OS (Windows, Linux, ...)
  • The imagepath element contains the access path to the test image directory. This path is expressed relatively to the directory containing the current .xml file. Please note that the path is appended neither by / nor \textbackslash, as the tests can be performed under several OS (Windows, Linux, ...)
  • The models elements describes all the model files used for the current test. Each file corresponds to a line within this section. The filenames are expressed without any prefix path and without any extension. Each model file will be supplied in 3 formats: BMP, TIF and VEC. Please just add the extension of the format you consider to the filename to obtain the full filename.
  • The images elements describes the test image files used for the current test. Each file corresponds to a line within this section. The filenames are expressed without any prefix path and without any extension. Each image will be supplied in 2 formats: BMP and TIF. Please just add the extension of the format you consider to the filename to obtain the full filename.

Warning for the .xml description file

Please note that several test images are contained in a single .xml file. In the case where a recognition application crashes during a test, your application must be able to recover and continue the next tests.

Name of the .xml description file

The name of the .xml description file is xxxxx.xml, where xxxxx is the name of the file. Please do not confuse it with the file xxxxx.gt.xml (when available) which describes the test too, but also describes the associated ground-truth.

Example of .xml description file

<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE test SYSTEM "GRECTestDescription.dtd">
<test>
  <testname>testgrec</testname>
  <metadesc property="domain" value="architecture"/>
  <metadesc property="domain" value="electronic"/>
  <metadesc property="representation" value="bitmap"/>
  <metadesc property="acquisition" value="offline"/>
  <modelspath>models</modelspath>
  <imagespath>testgrec</imagespath>
  <model>ArchitecturalA<model>
  <model>ArchitecturalB<model>
  <model>ArchitecturalC<model>
  <model>ArchitecturalD<model>
  <model>ArchitecturalE<model>
  <model>ArchitecturalF<model>
  <model>ArchitecturalG<model>
  <model>ArchitecturalH<model>
  <model>ArchitecturalI<model>
  <model>ArchitecturalJ<model>
  <model>ElectricalA<model>
  <model>ElectricalB<model>
  <model>ElectricalC<model>
  <model>ElectricalD<model>
  <model>ElectricalE<model>
  <model>ElectricalF<model>
  <model>ElectricalG<model>
  <model>ElectricalH<model>
  <model>ElectricalI<model>
  <model>ElectricalJ<model>
  <testimage>testgrec-image1</testimage>
  <testimage>testgrec-image2</testimage>
  <testimage>testgrec-image3</testimage>
  <testimage>testgrec-image4</testimage>
  <testimage>testgrec-image5</testimage>
</test>