Deployment

Architecture

GUARDARA is made up of the following key components.

Component	Description
Manager	The Manager provides interfaces to run tests, view test results and manage the entire deployment from a central location. The Manager is a set of microservices implementing both the user interface and the API. The API can be used to automate tests and integrate GUARDARA with automation frameworks and into CI/CD pipelines.
Engines	The Engine is a service responsible for performing the tests configured via the Manager. Each Engine is capable of testing one or more targets.

A GUARDARA deployment consists of a Manager and one or more Engines. This page walks you through the deployment process of both components.

Deployment Options

All components of GUARDARA can be deployed on a single host, but it is also possible to build a flexible, enterprise-scale deployment. This section discusses these deployment options.

Single-host Deployment

It is possible to deploy the Manager and an Engine on a single host. This deployment scenario is suitable for security consultants, researchers and small businesses.

Illustration 1: Example single host deployment

Even with a single Engine deployed, it is possible to test multiple targets simultaneously. The test targets can be applications and services deployed locally or remotely.

Distributed Deployment

GUARDARA's architecture allows the implementation of a highly-scalable, distributed setups. Multiple Engines can be deployed to test any number of targets, and the testing operations can be managed from a centralized location.

Illustration 2: Example distributed deployment

There is no one-size-fits-all deployment architecture as the requirements may vary from business to business. Therefore, you are responsible for designing and implementing a deployment that best fits your specific requirements.

For a distributed deployment to be functional, firewalls must configured appropriately. The table below discusses the different network connections required by GUARDARA.

Source	Destination	Port Number	Protocol	Description
Manager	Docker Hub	5000	HTTPS	Only required when installing GUARDARA Manager.
Users / Automation	Manager	8443	HTTPS	Access to the user interface and the API.
Engines	Manager	8444	Secure WebSocket	Communications channel required so that GUARDARA Manager can communicate with the Engines.

In addition to the above, the connection between the Engines and the test targets should be set up as appropriate for the environment.

Manual Installation

This section walks you through how to deploy GUARDARA manually.

Manager Installation

The GUARDARA Manager is a web-based, multi-user interface that allows administration of the GUARDARA infrastructure, creating and managing tests, and viewing the test results. The Manager exposes an API that can be used to automate tests and integrate GUARDARA into CI/CD pipelines.

Operating System

While the GUARDARA should run on most Linux distributions without any issues, we recommend deploying on a Debian-based Linux distribution, such as Ubuntu or Kali Linux.

Resource Allocation

Make sure to have at least 4 GB RAM and 4 cores dedicated to GUARDARA.

Deployment

Installing Manager, the management interface of GUARDARA is a fairly straightforward process, as shown below.

unzip ./guardara-latest.zip
sudo ./install.sh -i

As the installer script automatically installs all the dependencies, you must run the installer script with root privileges.

Once the script is finished, you should be able to access GUARDARA's setup pages on port 8443. Please note that it may take a minute or two for the system to become operational.

Open a web browser to access the setup pages. Follow the instructions to initialize your deployment.

tip

You may want to check out the different functionalities provided by the installer script by running the script with the -h argument.

Engine Deployment

The component of GUARDARA responsible for test execution is called the Engine. The Engine is the service that performs test case generation, test delivery and target monitoring during a test run. A GUARDARA deployment can have any number of Engines deployed. Each Engine is capable of running multiple tests simultaneously.

tip

If you are new to GUARDARA, you can simplify your deployment by deploying the Engine on the same host as the Manager. Don't worry about resources for now, as you can scale up your deployment later.

Operating System

The Linux version of the Engine should run on most Linux distributions; however, we recommend running it on a Debian-based Linux distribution such as Ubuntu or Kali Linux.

Memory Requirements

The memory requirements vary based on the complexity and size of the Message Templates used by the Project(s) and the number of tests running simultaneously. Therefore, the recommended way to estimate the memory requirements is to multiply the number of simultaneous tests by 256 MB. For example, 2.5 GB of memory should ideally be available to test ten targets simultaneously.

CPU Requirements

Running tests is a CPU-intensive operation; therefore, the CPU requirements vary based on the number of simultaneous tests. As a reference, our tests show that the Engine can run around 20 tests simultaneously on an Intel Core i7-5820K CPU (3.3GHz, 12 CPU cores). As of this, we recommend deploying the Engine on a machine with a modern CPU with at least 4 cores.

Deployment

Register an Engine

Deploying an Engine requires registering it within the Manager. The process is the following:

1. Go to the Settings / Engines page within the Manager.
2. Register your new Engine by clicking the + button and completing the form.

Setting up the Engine

Once the Engine is registered and the deployment bundle is ready, a blue Download button appears next to the Engine.

1. Click the Download button, then select your platform to download the Engine deployment bundle.
2. Decompress the deployment bundle into a directory.

After following the instructions above, the Engine can be started as:

engine \
    -a ${MANAGER_ADDRESS} \
    -p ${MANAGER_PORT} \
    --cert-ca ./credentials/cabundle.crt \
    --cert ./credentials/service.crt \
    --key ./credentials/private.key

The MANAGER_PORT is by default 8444.

In most cases, the engine does not require special permissions and thus can be executed as an unprivileged user. The use of the RAW network driver, for example, is an exception as it requires root privileges.

Extension Support

If you wish to extend the Engine's capabilities using extensions, you have to have the following installed on the host running the Engine:

apt-get install -y python3.9 python3.9-dev python3.9-distutils python3-pip

info

Depending on the extension, additional packages may be required. For example, non-pure Python extensions may require specific components to be compiled on the system. In that case, you must have gcc and possibly additional libraries installed.

Make sure the system is configured so that Python 3.9 is the default Python interpreter.

In addition to the above, install the latest version of pip and setuptools by running the below commands:

python3 -m pip install -U pip
python3 -m pip install setuptools setuptools_rust

Depending on your operating system and deployment, you may have to set up the PATH environment variable for the user running the Engine so that the Engine can run pip and the Python executable without knowing its exact location.

Unattended Installation

GUARDARA can be deployed without user interaction to support automated deployments, for example, when installing on AWS using Terraform or CloudFormation. This section discusses how to implement the required configuration.

You can find an example Terraform configuration to deploy GUARDARA on AWS on GitLab.

Manager Deployment

Create Cloud-Init Configuration

Create a file named cloud-init.yml. This file will contain the initial configuration for GUARDARA. The file should be located right next to the installer script. Make sure to replace the variables (${.*}) shown below with actual values, such as the FQDN or IP address of the system, the administrator's initial credentials.

setup:
  fqdn: ${FQDN}
  admin_email: ${EMAIL}
  admin_password: ${RANDOM_PASSWORD}

The following optional settings are also supported under the setup key:

• smtp_server: The hostname or IP address of the SMTP server.
• smtp_port: The port number the SMTP server is listening on.
• smtp_tls: Whether to enable or disable TLS. Default true.
• smtp_username: The username to use to authenticate to the SMTP server.
• smtp_password: The password to use to authenticate to the SMTP server.

Run Installer

Run the installer script to deploy GUARDARA.

sudo ./install.sh -i

After the installer script completes, the system becomes accessible within a few minutes at https://${SYSTEM_IP_ADDRESS}:8443/.

danger

Make sure to change the administrator's password at the first login.

Engine Deployment

Log into GUARDARA and create a new API key on the Settings / API Keys page with the scopes engine:write and engine:read assigned. The generated API key can be used to register a new engine, fetch the list of registered engines and download their deployment bundle.

Register an Engine

The following request illustrated using curl can be used to register a new Engine. In this example, we register an Engine called "Example".

curl -k \
    -X POST \
    -H "Authorization: Bearer ${API_KEY}" \
    -H "Content-Type: application/json" \
    https://${MANAGER_ADDRESS}:8443/api/engine/engine \
    --data '{"name":"Example"}'

The request blocks until the Engine's registration is completed. After, you will see a response similar to what is shown below.

{
    "name": "Example",
    "createdAt": "2022-06-29T20:14:08.708Z",
    "updatedAt": "2022-06-29T20:14:08.708Z",
    "id": "1e72d42a-b8c8-437b-a2be-fe39d2515511"
}

We need the id of the Engine to be able to download the deployment bundle. We will refer to this in the following examples as ${ENGINE_ID}. Alternatively, we can obtain a list of Engines with their ID as discussed under the List Registered Engines section below.

List Registered Engines

The following curl example illustrates how to fetch the list of registered engines from the API. As discussed earlier, you may not need to perform this step if you already obtained the Engine ID when registering the Engine.

curl -k \
    -H "Authorization: Bearer ${API_KEY}" \
    https://${MANAGER_ADDRESS}:8443/api/engine/engine

The response is a list of object, each representing a registered engine. An example is shown below.

[{
    "name": "Destroyer",
    "createdAt": "2022-06-28T11:28:09.505Z",
    "updatedAt": "2022-06-28T11:28:09.505Z",
    "id": "00000000-0000-0000-0000-000000000000"
}, {
    "name": "Example",
    "createdAt": "2022-06-29T19:46:16.616Z",
    "updatedAt": "2022-06-29T19:46:24.758Z",
    "id": "1e72d42a-b8c8-437b-a2be-fe39d2515511"
}]

We need the id of the Engine we want to download the deployment bundle of. We will refer to this in the following examples as ${ENGINE_ID}.

Please note that downloading the built-in Engine (Destroyer) installer bundle is not a good idea as that Engine cannot run tests. Make sure to download an Engine you have registered yourself.

Download Deployment Bundle

First, we initialize the download by performing the request below.

Linux

curl -k \
    -H "Authorization: Bearer ${API_KEY}" \
    https://${MANAGER_ADDRESS}:8443/api/engine/engine/${ENGINE_ID}/download/linux

macOS

curl -k \
    -H "Authorization: Bearer ${API_KEY}" \
    https://${MANAGER_ADDRESS}:8443/api/engine/engine/${ENGINE_ID}/download/macos

The request blocks until the Engine's deployment bundle is ready. Once ready, you will see a response similar to the below.

Linux

/bundles/engine-${ENGINE_ID}-linux.zip

macOS

/bundles/engine-${ENGINE_ID}-macos.zip

Of course, ${ENGINE_ID} will be the actual ID of the requested Engine.

Finally, we can download the Engine deployment bundle with the request below.

Linux

curl -k \
    -H "Authorization: Bearer ${API_KEY}" \
    https://${MANAGER_ADDRESS}:8443/api/engine/engine/bundles/engine-${ENGINE_ID}-linux.zip \
    --output ./engine-linux.zip

macOS

curl -k \
    -H "Authorization: Bearer ${API_KEY}" \
    https://${MANAGER_ADDRESS}:8443/api/engine/engine/bundles/engine-${ENGINE_ID}-macos.zip \
    --output ./engine-macos.zip

Deploying the Engine

Copy the Engine deployment bundle to the system where you want it to run. Then, decompress the archive and run the engine as shown below.

engine \
    -a ${MANAGER_ADDRESS} \
    -p ${MANAGER_PORT} \
    --cert-ca ./credentials/cabundle.crt \
    --cert ./credentials/service.crt \
    --key ./credentials/private.key

Set up an environment variable named MANAGER_ADDRESS that points to the hostname or IP address the Manager was deployed on. Alternatively, replace ${MANAGER_ADDRESS} in the above example with the hostname or IP address of the Manager.

By default, the Manager exposes the port 8444 to communicate with Engines. The Engines will connect to this port. Therefore, the MANAGER_PORT should be set to 8444.

Maintenance

Upgrading

To update your GUARDARA deployment, execute the installer script and answer Y for the Uninstall question. When prompted, answer N to remove data managed by GUARDARA. Next, the installer script removes all GUARDARA Docker containers and images but preserves all data. The script prompts whether to deploy GUARDARA or not. Answer Y to the question and wait until the latest version is deployed.

sudo ./install.sh
...
This script  will install GUARDARA Manager to your computer. As part
of the process the following actions will be performed:
...
Would you like to proceed? (Y/N) y
Initializing environment...
Installing dependencies...
GUARDARA appears to be already installed. Uninstall? (Y/N) y
Would you like to remove all data managed by GUARDARA? (Y/N) n
Deploy GUARDARA Manager? (Y/N) y

Once the upgrade has been completed, it is best to download and deploy any external Engines again, as your old Engines may no longer be compatible with the latest version of GUARDARA.

Certificate Rotation

The Manager is a set of microservices. Each of these services communicates over TLSv1.2. Mutual certificate authentication is in place. The certificates can be rotated using the install.sh script as shown below.

sudo ./install.sh -r

Please note that the Manager is not available during the certificate rotation process. The process removes custom frontend certificates as well. If you had a custom frontend certificate deployed, you have to deploy those again by following the instructions provided under the Custom Frontend Certificate section below.

Custom Frontend Certificate

If you would like GUARDARA to present your own certificate to users (recommended), a reverse proxy must be set up to proxy all requests between the users and GUARDARA Manager.

Alternatively, you can overwrite the certificate GUARDARA presents to the users. To do this, make sure the guardara_service_nginx container is running and issue the following commands:

docker cp ${CERT_FILE_LOCATION} guardara_service_nginx:/credentials/server/service.crt
docker cp ${PRIVATE_KEY_LOCATION} guardara_service_nginx:/credentials/server/private.key

The next step is to restart the service:

docker restart guardara_service_nginx

Troubleshooting

Manager: Error saving credentials

On macOS you may encounter the below error when running the installer script.

[ERROR] Failed to authenticate to GUARDARA Registry.

Looking at the install.debug file, you may see the below error.

Error saving credentials: error storing credentials - err: exit status 1, out: `Post "http://ipc/registry/credstore-updated": dial unix backend.sock: connect: no such file or directory`

Try removing the "credStore": "desktop" line from ~/.docker/config.json and run the installer script again.

Manager: Unexpected EOF During Install

If the internet connection is slow or unreliable, you may encounter this error during the Manager deployment process. If the quality of the internet connection cannot be improved, it may be possible to work around the problem by reducing the concurrent downloads when pulling Docker containers. This can be achieved by updating the Docker daemon’s configuration.

The following example shows how this can be done on Ubuntu Linux.

1. Open the file /lib/systemd/system/docker.service and locate the line starting with the text ExecStart=.
2. Add --max-concurrent-downloads=2 to the command line arguments
3. Save the file
4. Execute the command systemctl daemon-reload to reload the configuration
5. Execute the command systemctl restart docker to restart Docker
6. Try to install GUARDARA again. Run sudo ./install.sh -r to make sure all GUARDARA related data is removed. Then, install GUARDARA by running sudo ./install.sh -i.
7. If the above did not solve the issue, try changing max-concurrent-downloads to 1.

Please note, the more we restrict the concurrent downloads, the longer it takes to deploy GUARDARA.

Manager: Failed to Install Dependencies Error

The installer script attempts to install all the dependencies required to get GUARDARA Manager up and running. However, due to reasons outside our control, this process may fail. The best place to look to determine the root cause of the failure is the install.debug file. If it turns out we made a mistake or missed something, please let us know, and we will address the problem.

Engine: Test Run Fail on MacOS

Currently, GUARDARA includes a MacOS Engine (Apple M1), which you can download from the Settings / Engines page. Please note, that this Engine will be removed in the future. Until then, if you wish to use it, you may (or may not) encounter a cryptic error message shown below.

bjc[33204]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.

The solution is to make sure you have the following environment variable set before you start the engine.

export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

Something's not right? Let us know!