Field Extension Development

Fields are the components used to represent fields within structured data. Fields are made up of two components, a frontend and a backend. The role of the frontend is to provide information on how to render the Field and the Field properties within the Message Template Designer. The backend is the actual implementation of the Field used by the Engine during test run time.

Frontend Implementation

Implementing a field’s frontend requires working with JSON objects. The relevant files are stored under the frontend folder of the Field template generated by the SDK. These files control how different areas of the Message Template Designer render the field. The table below summarizes the role of each file.

File	Description
render.json	This file controls how the field is displayed and, if the field has a default value, how it is represented.
skeleton.json	All fields are represented by a basic data structure. This structure includes the name of the field, the field properties and any metadata. This file is the skeleton of the previously mentioned data structure.
properties.json	This file supplements the `skeleton.json` file by providing information to the Message Template Designer on how to work with each property within the skeleton and how to visually render those properties on the user interface.

More details on each of the above files is provided under the sections below.

render.json

The file contains two primary keys, render and value as shown in the example below.

{
    "render": {
        "background": "#333333",
        "text": "#ffffff",
        "display": "hex"
    },
    "value": {
        "load": "DecimalArrayToHexString",
        "save": "HexStringToDecimalArray",
        "format": "^(0x[0-9a-fA-F]{2}[,]?[\\s]*)*$",
        "valueOverride": "value"
    }
}

The render key must contain two properties, the background, and text. Just as shown in the example above, the expected value of these properties is an HTML color code. The Message Template Designer uses colors to help visually distinguish different fields within structured data (fields). These properties control what color the field should be represented with.

The render key supports an additional, optional property called display. This property controls how the value of the field visually represented on the user interface. The possible values are below.

Value	Description
decimal	Display each byte of a field as a decimal number.
hex	Display each byte of a field in hexadecimal format, just as standard hex editors do. This is the default.
string	Display each byte of a field as ASCII. This is useful for fields that operate with ASCII strings.

The properties under the value key describe how the Message Template Designer should handle the value of a field. The table below summarizes the properties supported.

Property	Description
load	Defines which method to use to represent the field’s value loaded from the skeleton or Message Template to the user.
save	Defines which method to use to convert the value provided by the user to the format expected by GUARDARA.
format (optional)	In case a field operates with a default value (has the value key set in the skeleton), the field may want to perform validation on the value entered by the user to ensure it is correctly formatted. This can be done using the `format` property which accepts a regular expression.
valueOverride (optional)	Fields that do not operate with a default value, such as support fields that perform operations based on the value of other fields should tell the Message Template Designer not to attempt to obtain the value of the `value` key. Instead, use a different property from the skeleton that is available (e.g. `name` or `block`).

The load and save methods are necessary as how GUARDARA expect field values usually is a lot different than how a user would expect to work with a field’s value. GUARDARA, at least the built-in fields, expect any values, be it a number, a string or binary data, to be represented as a list of decimal numbers.

The following methods are available for the save property:

StringToDecimalArray: tells GUARDARA that it should convert a string to a list of decimal numbers when saving the field’s value
HexStringToDecimalArray

The following methods are available for the load property:

DecimalArrayToString: will instruct GUARDARA to convert the field’s value to ASCII text when displaying it in the editor pane
DecimalArrayToHexString

As can be seen, there are two variants for each, the HexString and the simple String variant. The HexString variant is mainly used with fields where it feels much natural to work with hexadecimal values, such as the Byte and Word fields. The String variants are used by fields such as String and Delimiter.

skeleton.json

The skeleton.json file describes the field in a format that is expected by the field’s backend and frontend components. The core structure can be seen below.

{
    "field": "name",
    "title": ""
}

The below table describes the above properties.

Property	Description
field	The actual name of the field. The SDK generates it based on the name provided when generating the extension. This is the name that is being used by GUARDARA to refer to the field when running tests.
title	The name of the field.

Additional properties specific for the field implementation may be included. Please note, these have to be compatible with the backend’s properties.json file.

The skeleton of the built-in String field for example looks like the below.

{       
    "field": "string",
    "title": "",
    "value": [],
    "max_length": 0,
    "size": 0,
    "padding": 0,
    "ascii": true,
    "test": true,
    "expose": false,
    "custom_library": ""
}

As can be seen, the object contains several parameters and their default values. Extension skeletons should always include all possible parameters and their default values. There certain parameters that can be considered special, such as expose. More information on these will be provided when discussing the backend component implementation.

Capabilities

The meta key of the field skeleton can be used to define certain field (in)capabilities. The only supported (in)capability at the moment is no-transform to indicate the field does not support Transforms. Configuring such a field with this capability would look similar to what is shown below.

{       
    "field": "custom",
    "title": "",
    "name": "",
    "value": []     
    "meta": {
        "capabilities": ["no-transform"]
    }
}

properties.json

The properties.json file is a user interface specific extension of the field skeleton. It is mainly responsible of controlling the rendering and processing of field parameters.

The properties object of the built-in String field for example looks like the below.

{       
    "name": {
        "title: "Name",
        "description": "Field names make it easier to navigate messages.",
        "type": "string", 
        "mandatory": true,
        "min_length": 1,
        "max_length": 36
    },
    "value": {
        "title": "Value", 
        "description": "Value as simple text. Example: 'String Value'.",
        "type": "string",
        "mandatory": true,
        "min_length": 1
    },
    "max_length": {
        "title": "Maximum Length",
        "description": "Maximize the length of the generated mutations in number of bytes.",
        "type": "integer",
        "mandatory": false,
        "default": 0
    },
    "size": {
        "title": "Fixed Length",
        "description": "Set fixed length for the generated mutations. If the mutation is shorter then it will be padded with padding bytes.", 
        "type": "integer",
        "mandatory": false,
        "default": 0
    },  
    "padding": {
        "title": "Padding",
        "description": "Padding byte in decimal format. Used to expand mutations to match the length of the size property. Ignored if size is set to 0.",
        "type": "integer",
        "mandatory": false,
        "default": 0,
        "min_value": 0,
        "max_value": 255
    },
    "ascii": {
        "title": "ASCII",
        "description": "Whether to render the field as ASCII or binary.",
        "type": "boolean",
        "mandatory": false,
        "default": true
    },
    "test": {
        "title": "Mutate",
        "description": "When enabled the field's value will be mutated.",
        "type": "boolean",
        "mandatory": false,
        "default": true
    },
    "expose": {
        "title": "Expose in Session",
        "description": "Fields exposed in the session can be referred to from anywhere.",
        "type": "boolean",
        "mandatory": false,
        "default": false
    },
    "custom_library": {
        "title": "Custom Library",
        "description": "Each entry in the file should be on a new line.",
        "type": "file",
        "mandatory": false
    }
}

There are some common, mandatory attributes. These are summarized by the table below.

Attribute	Description
title	Each property rendered by the editor pane has a title. This is a free-form text but it is recommended to keep it short.
description	Each attribute has a description. This is also free-form text but it can be longer than the title. It can be used in different ways, for example, to provide more context, to explain on how the value of the form control affect the field or to describe the expected format of the form control value.
type	Specifies the form control type. The editor panel renders the appropriate form control based on the type specified within the properties file. For the supported values please refer to the Supported Type Attribute Values section of this document.

The additional attributes supported are listed in the table below.

Attribute	Description
mandatory	Defines whether the attribute is mandatory or not. If the attribute is mandatory GUARDARA expects it to be present in the skeleton.
min_length	Defines the minimum length of the attribute’s value. Please note, non-string values (such as integers and Booleans) are converted to string during validation.
max_length	Defines the maximum length of the attribute’s value. Please note, non-string values (such as integers and Booleans) are converted to string during validation.
min_value	The minimum value of a numeric field.
max_value	The maximum value of a numeric field.
values	Defines a list of allowed values.
format	Validates the attribute’s value against the provided regular expression. Please note, non-string values (such as integers and Booleans) are converted to string during validation.
default	The default value of the attribute.
condition	Allows to render a property only if the condition evaluates to true. See the example provided under Conditional Rendering of Properties.
advanced	If `true`, the field editor will only render the propery if the user checks the Advanced checkbox listed under the field properties.

Supported Type Attribute Values

The table below summarizes the supported values of the type attribute.

Value	Description
string	Renders a standard input fields that allows the user to enter arbitrary text.
integer	Renders an input field that allows to enter numeric values.
boolean	Renders a select field to choose from the values `Yes` (true) and `No` (false).
select	Allows the user to select from a list of options defined via the `values` property.
block	Renders a select field that allows the user to select from a list of Group fields within the same Message Template.
file	Renders a button in the Editor Panel that allows to select a file. The content of the file is then stored as the value of the property as a string.

select Type Attribute Example

The following example creates a field attribute with the type select. As can be seen, the values property is a list of objects, each defining a selectable item.

"format": {
    "title": "Format",
    "description": "Defines the format of the rendered field. It can be rendered as a 'binary' byte value or as an 'ascii' number.",
    "type": "select",
    "values": [
        {text: "Binary", value: "binary"},
        {text: "ASCII", value: "ascii"}
    ],
    "mandatory": true,
    "default": "binary"
}

Conditional Rendering of Properties

It is possible to conditionally render field properties, based on another property’s value. The example below illustrates this.

mode: {
    title: "Mode",
    description: "Specify whether to manually configure ('Manual') or calculate dynamically ('Dynamic').",
    type: "select",
    values: [
        {text: 'Manual', value: 'manual'},
        {text: 'Dynamic', value: 'dynamic'}
    ],
    mandatory: true
},
min: {
    title: "Minimum Value",
    description: "The minimum value.",
    type: "integer",
    mandatory: false,
    min_value: 0,
    condition: "mode == 'manual'"
},
formula: {
    title: "Formula",
    description: "The formula to calculate the minimum value by.",
    type: "string",
    mandatory: false,
    condition: "mode == 'dynamic'"
}

As can be seen, the value of the condition attribute were set differently for the min and formula field properties. At the time of rendering these properties, the current value of the mode property will be compared to the value defined in the condition. If the values are equal, only then the property will be rendered. As of this:

If the user selected Manual mode, the Minimum Value property will be visible
If the user selected Dynamic mode, the Formula property will be visible

The condition attribute supports the ==, !=, >, <, >= and <= comparison operators.

Backend Implementation

The backend component of a field is a Python module. The SDK generates the module skeleton under the backend/g-field-${NAME} directory, where ${NAME} is the sanitized Extension name. Please note, the directory layout generated by the SDK should not be altered.

Backend Methods

The main file of the field is called field.py and can be found under backend/g-field-${NAME}/g_field_${NAME}. This file contains information about each of the methods to be implemented.

properties.json

The field backend component has its own dedicated properties.json file. The frontend’s and backend’s properties file are strongly related as they describe the same thing to different audiences.

Putting things into context:

The frontend’s properties.json file tells JavaScript code how to render a form dynamically.
The frontend’s skeleton.json file defines the data structure to be created/populated by the UI based on the changes made by the user on the rendered form. This is the data structure that is expected by the backend component.
The backend’s properties.json file defines what properties are expected/supported, and how these should be validated and process.

The following is an example of how the Engine expects a field definition within a Message Template.

{
    "id": "7eed35fe-a995-489e-9b9f-8dc2c0305cd5",
    "field": "my_field",
    "title": "Example Field",
    "value": [0x41, 0x41],
    "test": true,
    "expose": false
    "meta": {}
}

The key properties are:

The id property is mandatory and must be unique. Even though the id is not included in the skeleton.json file, it is mandatory to be specified.
The value of the field key (in the above example: “my_field”) tells the Engine that it should load and instantiate the field “my_field”.
The title is the user-given names of the field and provided when created/edited the field in the Message Template Designer.
The meta object can hold arbitrary meta-information.

The additional properties can be considered as the parameters of the field. Each field must have a properties.json file that define what parameters are expected/supported and their characteristics. The below is an example of a backend properties.json file that defines the 4 properties of our example field.

{
    "name": {
        "type": "str",
        "mandatory": true
    },
    "value": {
        "type": "list",
        "mandatory": true
    },
    "test": {
        "type": "bool",
        "default": false
    },
    "expose": {
        "type": "bool",
        "default": false
    }
}

As can be seen, the name of each property is defined as a key within the JSON object. To each of these properties, there are a set of attributes specified.

The name, value and test properties are mandatory, even though the value and test properties are not required to be exposed via the user interface.

The following table summarizes each of the attributes supported for defining a property:

Attribute	Mandatory	Type	Description
type	Yes	String	Defines a the type of the attribute’s value using a standard Python type, such as `str` and `list`.
mandatory	No	Boolean	If set to `true` the Engine will expect the property to be present.
default	No	As required by `type`	Allows to define a default value in case the property is not mandatory and it was not defined.

The value of the properties can be accessed from anywhere within the field as shown below.

self.get('name')

Initialization

The __init__ method of the field should be used to perform any initialization as required by the field.

Either way, the field should be implemented so that the total number of mutations can be calculated during field initialization, and that it is set in the field meta property as shown below.

self['meta']['mutation']['total’] = total_mutations

Generic Rules

There are some generic rules all fields should comply with. Non-compliant fields can have a significant impact on test generation and reporting. The rules are:

Non-mutable fields having their test property set to False must set their completed property to True and set the meta mutation counters to 0:
```
self['meta']['mutation']['total'] = 0
self['meta']['mutation']['index'] = 0
```
Non-mutable fields should not change their values when their mutate method is called.
Non-mutable fields should not change their meta mutation counters when their mutate method is called.
The first mutation produced by a field should always be the default value provided by the user.
After calling a field’s reset method, it should render its default value, even without a call to the mutate method.
The number of mutations generated by the field should be equal to the total mutation counter in the meta.
At the time a field completes, it must have its meta counters equal.

Support Fields

Support fields are fields that operate or depend on another field’s (usually a Group) value. The sizer and hash built-in fields are good examples. Non-mutating support fields should have self.ignore and self['ignore'] set to True.

It is extremely important that support fields that access the value of another field do so by calling the field’s render method as:

field.render(peek=True)

This is because there are certain fields, such as the built-in increment field, that increments its internal counter within the render method. Passing peek=True instructs the field not to increment its counter as it is just simply being observed.

Working with the Session

Exposing Fields in the Session

A Field can expose it's current value in the Session. One way it can do it is by calling the inherited expose(...) method as shown below.

self.expose(raw_value, rendered_value)

If the Field wishes to update a Session Variables that was not created with the ID of the Field, it can do it by adding the desired Session Variable ID as the 3rd argument:

self.expose(raw_value, rendered_value, expose_id)

The self.session is a Python dict that fields can read or manipulate. As of this, a field can expose its current value, or in fact any data in the session simply by updating it:

if self.get('expose') is True:
    self.session[self.get('id')] = {
        "raw": raw_value,
        "rendered": rendered_value
    }

The above example exposes both the raw and rendered value of the field under the field’s ID. As fields can apply transforms to field values, the value that has been processed by the transforms should be stored under the rendered key. The non-transformed mutation can be exposed via the raw key. If exposing only the value that has been processed by the transforms it is highly recommended to set it as the value of both keys. Similarly, if there are no Transforms attached to the field, the rendered key's value should be set to the value of the raw key's value.

Accessing Session Variables

Session variables are exposed to all fields via self.session. A field can access session variables any time as shown below.

value = self.session.get(variable_id)

Then, the raw and rendered value can be extracted as:

rendered = value.get('raw’)
rendered = value.get('rendered’)

Accessing Other Fields Exposed in the Session

Fields exposed via the session can be accessed exactly the same way as the session variables discussed above.

value = self.session.get(variable_id)

As variable_id the ID of the field is expected.

Please note, accessing the value of other fields via the session should only be done runtime, thus from the mutate or render methods of the field.

Field Lookup

The parent field (FieldInterface) implements and exposes the search and search_by_id methods to all fields. These methods can be used by fields to obtain a handle on any field included in the Message Template by the name of the field. This can be useful in scenarios where, for example, the field depends on the value of another field.

To get a handle on a Field by it's name:

handle = self.search(field_name)

To get a handle on a Field by it's ID:

handle = self.search_by_id(field_id)

The handle allows the field to call both the render and mutate method of the target field. For example:

handle = self.search(field_name)
value_of_field = handle.render()

The Group field is a special field as its render method is a Python generator. This means, if the handle is for a Group field, the following has to be done.

handle = self.search(group_field_name)
value_of_field = next(handle.render())

Something's not right? Let us know!

Field Extension Development

Frontend Implementation​

render.json​

skeleton.json​

Capabilities​

properties.json​

Supported Type Attribute Values​

select Type Attribute Example​

Conditional Rendering of Properties​

Backend Implementation​

Backend Methods​

properties.json​

Initialization​

Generic Rules​

Support Fields​

Working with the Session​

Exposing Fields in the Session​

Accessing Session Variables​

Accessing Other Fields Exposed in the Session​

Field Lookup​

Frontend Implementation

render.json

skeleton.json

Capabilities

properties.json

Supported Type Attribute Values

select Type Attribute Example

Conditional Rendering of Properties

Backend Implementation

Backend Methods

properties.json

Initialization

Generic Rules

Support Fields

Working with the Session

Exposing Fields in the Session

Accessing Session Variables

Accessing Other Fields Exposed in the Session

Field Lookup