1. Local Deployment

Here you’ll learn how to quickly set up a local deployment of the FAIR Data Point on your development system. This will allow you to play around with the FDP and try out different configurations, without needing to worry about too many details.

Warning

This local deployment is intended for offline testing only. If you are going to expose your FDP to the public internet, please have a look at the Production Deployment section.

1.1. Prerequisites

To set up an FDP stack, using containers described in the Components section, we use Docker Compose or an equivalent tool. If you don’t have Docker Compose yet, follow the Docker installation instructions first. Alternatively, you could install another tool that supports the compose specification.

1.2. Quickstart

Set up

Here’s how to get started quickly with a minimal FDP stack that has no data persistence (ephemeral):

Clone the FAIRDataTeam/compose repository from GitHub:
```
git clone https://github.com/FAIRDataTeam/compose.git
```
The FAIRDataTeam/compose repository contains the latest compose files for a variety of FDP configurations and versions, such as persistent and ephemeral (i.e. non-persistent) configurations. These compose files are used by our development team for testing FDP deployments locally. As such, they represent a good starting point for reproducing any issues that you may encounter. See the FAIRDataTeam/compose readme for more information.
Change into the directory for the ephemeral/v1 stack:
```
cd compose/fdp/ephemeral/v1
```
This directory contains a compose configuration that defines a minimal stack consisting of the mongo, fdp, and fdp-client containers. Here ephemeral implies “non-persistent data” and v1 refers to the latest major version of the fdp and fdp-client components.

The fdp is configured to use an in-memory triple store, and mongo data is stored only in the container. There are no persistent volumes or bind mounts, so all data is lost when the stack is torn down.

If you do need persistent data storage, you can try the persistent/v1 configuration instead. That configuration includes a graphdb triple store and uses volumes for persistence of all data.
Set up the stack:
```
docker compose up -d
```
This downloads the required Docker images, if necessary, and starts the containers in the proper order.

Play around

Once all containers are up, and healthy, you can start playing around with the FDP.

For example:

Use curl http://localhost to see the machine readable FDP metadata
Visit http://localhost in your favorite web browser to try the FDP client interface (also see the Authentication and Usage sections)
Visit http://localhost/swagger-ui/index.html in the browser to inspect the interactive API documentation

Tear down

Once you’re done playing with your FDP, here’s how to remove every trace:

Make sure you are (still) in the directory corresponding to the running stack, in our case ephemeral/v1.
Tear down the stack:
```
docker compose down
```
If you’re running a persistent configuration, this command does not remove the persistent volumes. If you do want to remove the persistent volumes, it is most convenient to use docker compose down --volumes. Alternatively you could use docker volume rm <volume-name>.
If you really want to remove every trace of the FDP, you’ll need to remove the containers and corresponding images as well. If not, you can leave them in place for the next time.

1.3. Deep dive

A more detailed description of the compose configurations from the FAIRDataTeam/compose repo can be found below. First we describe the basic stack components, then we show you how to use them in composing a stack.

Stack components

The compose repo fdp/components dir contains basic configurations for the components described in the Components section. These components are listed below:

minimal fdp v1 config

  fdp:
    image: "fairdata/fairdatapoint:${FDP_VERSION:-1.18}"
    restart: no
    environment:
      SERVER_PORT: 8080
      INSTANCE_CLIENTURL: http://localhost
      INSTANCE_PERSISTENTURL: http://localhost
    depends_on:
      mongo:
        condition: service_healthy
    healthcheck:
      test: wget --quiet --spider http://127.0.0.1:8080 || exit 1
      start_interval: 3s
      start_period: 30s

minimal fdp-client v1 config

  fdp-client:
    image: "fairdata/fairdatapoint-client:${FDP_CLIENT_VERSION:-1.18}"
    restart: no
    ports:
      - "127.0.0.1:80:80"
    environment:
      FDP_HOST: fdp:8080
    depends_on:
      fdp:
        condition: service_healthy
    healthcheck:
      test: wget --quiet --spider http://127.0.0.1 || exit 1
      start_interval: 3s
      start_period: 30s

minimal mongo config

  mongo:
    image: "mongo:${MONGO_VERSION:-8.0}"
    restart: no
    healthcheck:
      test: |
        [ $(mongosh --quiet --host localhost:27017 --eval "db.runCommand('ping').ok") = 1 ] || exit 1
      start_interval: 3s
      start_period: 30s

minimal graphdb config

  graphdb:
    image: "ontotext/graphdb:${GRAPHDB_VERSION:-10.8.8}"
    restart: no
    ports:
      # expose to localhost so we can use the graphdb web ui
      - "127.0.0.1:7200:7200"
    healthcheck:
      # test if reachable (also works if there are no repositories yet)
      test: "curl http://localhost:7200/rest/monitor/infrastructure --silent --fail || exit 1"
      start_interval: 3s
      start_period: 30s

  graphdb-init:
    # this service uses the graphdb rest api to create repositories defined in the repos/*.json files
    # (safe to call if repositories exist)
    # https://graphdb.ontotext.com/documentation/10.8/manage-repos-with-restapi.html#create-a-repository
    # https://graphdb.ontotext.com/documentation/10.8/manage-repos-with-curl.html
    image: "curlimages/curl:${CURL_VERSION:-latest}"
    restart: no
    volumes:
      - ".:/templates"
    environment:
      # space-separated list of repo names
      REPO_NAMES: fdp
    # the following curl request returns status 201 if the repo does not exist, or status 400 otherwise
    command:
      - sh
      - -c
      - |
        for name in $$REPO_NAMES
        do
        # substitute "repo-name" by variable name in graphdb repo template
        data=$$(sed s/repo-name/$$name/g /templates/graphdb-repo-template.json)
        # post repo definition to graphdb rest api
        curl http://graphdb:7200/rest/repositories --verbose --silent --header "Content-Type: application/json" --data "$$data"
        done
    depends_on:
      graphdb:
        condition: service_healthy

The graphdb-init service, defined above, requires a template file in order to create an initial GraphDB repository. Here’s a minimal example based on the GraphDB API docs:

graphdb repo template

{
  "id": "repo-name",
  "title": "",
  "type": "graphdb",
  "params": {
    "defaultNS": {
      "name": "defaultNS",
      "label": "Default namespaces for imports(';' delimited)",
      "value": ""
    },
    "imports": {
      "name": "imports",
      "label": "Imported RDF files(';' delimited)",
      "value": ""
    }
  }
}

Alternatively, it is possible to remove the template file and the graphdb-init service, and create the required repository manually, using the GraphDB web interface. See GraphDB instructions for more info. Note that you would need to do this before starting the fdp container.

Composing a stack

The components described above are the basic building blocks for our compose stack. We use the compose-file include element to build our actual compose file.

Ephemeral

For example, the following compose file defines the minimal ephemeral FDP stack used in the quickstart:

ephemeral/v1/compose.yml

include:
  - ../../components/db/mongo.yml
  - ../../components/v1/fdp.yml
  - ../../components/v1/fdp-client.yml

By itself, this file is not very informative. In order to inspect the complete configuration resulting from the above, you can use the config command:

docker compose config

It merges the Compose files set by -f flags, resolves variables in the Compose file, and expands short-notation into the canonical format.

Persistent

A stack with persistent data requires a bit more configuration. In this case we add graphdb as an external triple store:

persistent/v1/compose.yml

include:
  - ../../components/db/graphdb.yml
  - ../../components/db/mongo.yml
  - ../../components/v1/fdp.yml
  - ../../components/v1/fdp-client.yml

In addition, we need to extend and/or override the configuration of the basic components. This is achieved using a compose.override.yml file, as explained in the compose-file merge docs:

persistent/v1/compose.override.yml

# note docker compose automatically loads compose.yml *and* compose.override.yml (if present)
services:
  # configure fdp to use graphdb instead of the in-memory triple store
  fdp:
    environment:
      REPOSITORY_TYPE: 4
      REPOSITORY_GRAPHDB_URL: http://graphdb:7200
      REPOSITORY_GRAPHDB_REPOSITORY: fdp
    depends_on:
      # depend on initialized graphdb (in addition to mongo)
      graphdb-init:
        condition: service_completed_successfully
  # use docker managed volumes for data persistence
  graphdb:
    volumes:
      - graphdbdata:/opt/graphdb/home
  graphdb-init:
    environment:
      # space-separated list of repo names
      REPO_NAMES: fdp
  mongo:
    volumes:
      - mongodbdata:/data/db

volumes:
  graphdbdata:
  mongodbdata:

FDP configuration options

The FAIRDataPoint backend is a Java application based on the Spring framework. Spring applications can be configured in several ways, as described in the Spring Boot externalized configuration docs. The FAIRDataPoint app itself uses application.yml files for the default configuration. In the FAIRDataTeam/compose repo, we use environment variables to override and/or extend this default configuration for the fdp service. For example, the default server.port Spring configuration value is overridden using the SERVER_PORT environment variable. This is convenient because we don’t need to include multiple files.

Nevertheless, it is also possible to override fdp configuration using an application.yml file. To do this we need to mount our custom application.yml file as follows:

   fdp:
       ...
       volumes:
         - ./application.yml:/fdp/application.yml:ro
         ...