Skip to main content

Local Development

System Requirements​

Authentication​

The default username and password is unstract.
See backend README.md for more details.

Git pre-commit hook​

We use pre-commit which has hooks to perform linting and static code analysis among other checks. These hooks from pre-commit tool are installed under Git pre-commit hook in local, and can also be run during CI. See currently active hooks.

# Install Git pre-commit hook.
./dev-env-cli.sh -p

# Run Git pre-commit hook.
./dev-env-cli.sh -r

The pre-commit hooks can also be run directly as follows, once the virtual env of the respective service is activated in current shell:

  • Using pre-commit tool directly

    # Run all pre-commit hooks
    pre-commit run

    # Run specific pre-commit hook
    pre-commit run flake8

    # Run mypy pre-commit hook for selected folder
    pre-commit run mypy --files prompt-service/**/*.py

    # Run mypy for selected folder
    mypy prompt-service/**/*.py

  • Using pdm to run the scripts

    # Run all pre-commit hooks
    pdm run pre-commit run

    # Run specific pre-commit hook
    pdm run pre-commit run flake8

    # Run mypy pre-commit hook for selected folder
    pdm run pre-commit run mypy --files prompt-service/**/*.py

    # Run mypy for selected folder
    pdm run mypy prompt-service/**/*.py

Run selected services alone directly on Docker Host​

For seamless contribution, you might want to run some services alone directly on Docker Host while letting others to continue running in Docker containers.
For example, one typical use case is running backend and/or frontend service(s) alone directly on Host, to have a fast feedback loop while working on them.

Allow Traefik running in Docker container to talk to the Host​

  • cp docker/sample.proxy_overrides.yaml docker/proxy_overrides.yaml
    Now override the default Traefik routes by adding desired routes in docker/proxy_overrides.yaml.
  • NOTE:
    Services running directly on Host will be reachable at host.docker.internal:<port>.

Allow services running directly on Host to talk to dependency services running in Docker containers​

Update config of services running directly on Host by modifying host names of dependency components as follows:

  • Replace as <original_host_name>.localhost.
    Do this ONLY if container port is exposed on Host.
  • OR use container IPs directly. 
    docker network inspect unstract-network
  • OR run docker/scripts/resolve_container_svc_from_host.sh.
    Do this ONLY if container port is NOT exposed on Host OR if you prefer to keep the dependency components host names unchanged.
# IMPORTANT! Recreate Traefik container for the route overrides to take effect.
docker compose up -d reverse-proxy

Dual usage behaviors of some host names​

In the case of some services, the same host name config can cause reachability issues based on who is invoking that service.

For example, backend/.env has a PROMPT_HOST config which contains the host name of prompt-service. Now note that the same config is also supplied to a Tool configuration for spawning corresponding Tool Docker container. Now if both backend and prompt-service are running directly on Host and hence if the value of PROMPT_HOST is localhost, this will inadvertently prevent Tool Docker containers from reaching prompt-service at all. This is because the host name localhost inside a Docker container points to the container itself, and thus will never reach the Host where prompt-service is running.

In such cases, one solution would be to define a host name in /etc/hosts and use the same.

# `/etc/hosts` entry to make this host name resolve to the Host IP address.
<host_local_ip>    unstract_prompt_service

Generate encryption key​

Adapter credentials are symmetrically encrypted by the platform using an encryption key.
This is configured under ENCRYPTION_KEY in both backend/.env and platform-service/.env files.

# Generate an encryption key in required format.
$(python -c "import secrets, base64; print(base64.urlsafe_b64encode(secrets.token_bytes(32)).decode())")

NOTE: Do backup the encryption key in use. Its loss or change will make all existing adapters inaccessible!