Develop bandheart.com

3. Docker for laravel octane, queue(horizon) and scheduling, PostgresSQL. Nginx proxy. Local development

Video version
(Leave your feedback on YouTube)

This guide is for local development. Do not use it in production!

TL; DR

Docker - must-have skill. Mandatory for PHP developers

Use laravel sail or laradock for inspired not for work

Use fixed image version with patch!

Linux alpine - small, secure good solutions for you containers.

Use nginx proxy even on local developing process

Do not forget .dockerignore

Do not use default network

Use top level volumes

docker compose can get variables from .env file

Consider using Postgres

Docker does not store anything. Use volume

Know docker

For a backend developer, knowledge of Docker is mandatory. Accept it, master it, and move on.

If you have more than one container, use Docker Compose.

Docker Compose was developed as a standalone solution and has grown into an extension for Docker (Docker Compose without the hyphen). The extension is official, fixing all the risks of the original, and most discussions about the vulnerability of using Compose in production should be renewed.

laravel sail and laradoc - ready-made solutions that offer to dokerize Laravel project in Docker out of the box.

I don't recommend using them out of the box. There are many reasons:

  1. These solutions don't allow you and your team to understand the service-oriented approaches. When you're looking for a solution, When you're looking for a solution, you don't even think that you can take some service written on any technology and use it. You are limited by php
  2. My practice shows, these solutions often goes in production. Blindly copied and pushed to production. But if we don't use these solutions in production, consistency is broken, and the project has two environments: one on developers' machines and one on the server.
  3. These have problems with updates and extensions. The versioning of Docker images works on the same principle as package manager versioning. If you haven't specified the patch version, the latest version will be installed. However, in these packages, the patch version is never specified. This means that the version for a new developer and a developer who started a week ago may be different. And of course, in production, you also don't know which version you have. You can't confidently say whether the PHP vulnerability everyone is talking about has been fixed or not.

Use fixed image version with patch!

  1. Optimization and security. Deploying containers in Ubuntu is bad idea. It's heavy, it has many dependencies that you don't need, and it potentially could have security vulnerabilities. We argue about whether Yarn or npm is faster... and then we put them on Ubuntu...

Instead, I recommend Linux Alpine. It's small, lightweight, and fast

Linux alpine - small, secure good solutions for you containers.

But these are cases when I recommended use it:

  1. You're a beginner and learning - use them. But it's better to learn how to set up your local machine.
  2. You need something to test or demonstrate quickly without thinking about the future of it.
  3. You need to peek at something. Look into the source code is goode ideas. Most likely, they're more correct than what you'll find in mine blog.

Docker compose advices

I won't go into documenting here. Instead, I'll provide a set of general recommendations that, based on my subjective experience, developers often forget.

  1. Docker doesn't save any data; all data exists only as long as it's running. Use top-level volumes. 2This allows you to store service data (and shares between services) without worrying about where to store them.
  2. Docker Compose automatically creates a local default network and includes all services in it. Don't use it. Limit visibility of each service. A service should only have access to the services it interacts with. This minimizes the impact of a hack some service. Make sure to do this even for local development. Who responsible for deployment (hypothetical DevOps) is unlikely to understand how your code interacts and just copy your local setup.
  3. healthcheck allows you to add a command to check if a service is running correctly. It important for the next point and allows complete abstraction from the local development host machine, in my case - for installing dependencies. If the container is started without dependencies, it allows developer to install them, and if everything is installed, mark itself as healthy. And don't rely on healthcheck as a tracking indicator; if something fails during operation, Docker won't show it. But DevOps will use your functions for checking.
  4. depends_on: condition: service_healthy - launches a service only after the service it depends on has fully started. Not critical for local work but helps avoid your bug tracker being flooded with errors because services started but didn't work at the same time in production.
  5. yaml reference &some_name + yaml merge >> - make it easy to deploy services based on the same images without duplicating code.
  6. Docker Compose works with the .env file - use it.
  7. Open only necessary ports to the host. If you need to open a port to networks, use expose.

DB service

If you don't know why you need something else, use relational databases. NoSQL, graph, columnar, and all the others only if you know why project needed it.

If you're still on MySQL, drop it. Postgres is incredibly cool; even if you're a beginner, I recommend starting with it right away.

In my setup, I have PostGIS - it's Postgres with the plugin installed for working with geolocation. There are many plans for geolocation. If you don't need it, just use Postgres. You can always install plugins later.

docker-compose.yml
db:
image: postgis/postgis:16-3.4-alpine
restart: unless-stopped
container_name: bh-db
environment:
  POSTGRES_DB: ${DB_DATABASE}
  POSTGRES_USER: ${DB_USERNAME}
  POSTGRES_PASSWORD: ${DB_PASSWORD}
ports:
  - 5432:5432
volumes:
  - db_data:/var/lib/postgresql/data
networks:
  - bh-db-network
healthcheck:
  test: pg_isready -U ${DB_USERNAME} -d ${DB_DATABASE}
  interval: 5s
  timeout: 10s
  retries: 20
  • DB_DATABASE, DB_USERNAME та DB_PASSWORD - docker compose will read from the .env configuration file. Database will be created automatically with these parameters on first startup.
  • container_name: bh-db - bh-db - will be used as app-host name. We use it to connect laravel
  • test: pg_isready -U ${DB_USERNAME} -d ${DB_DATABASE} - check if database is successfully started. I didn't face any problems but plugins, version migration can affect.

PHP service

Container with PHP (app). Here we have a choice:

  • PhpFPM - popular, tested, safe option.
  • Nginx-unit - sometimes hyped, advertised as a superfast solution. I personally have never seen it in practice, so it's at your own risk.
  • Laravel Octane. This is my choice for the project. Not just because of the speed; I believe it will give roughly the same speed as PHP FPM with opcache. I want to grove as a specialist. learning to write on octane is the next step. Also, I got project ideas for which Octane will be suitable. So, this is purely subjective choice!

If we choose Octane, then we need to make another choice, the engine on which Octane will work. FrankenPHP, RoadRunner or Swoole.

FrankenPHP, at the time of decision-making, is a new engine, and I don't want to be an alpha tester.

Between RoadRunner and Swoole - my subjective choice is Swoole, because of its unique capabilities; perhaps I'll change my mind in the future.

With Swoole, again, you need to make a choice - Open Swoole or regular. For now, there isn't much difference; there are internal conflicts between commands, but they are not difference for my project's needs.

You can find the official Docker image here. And since this is a PHP image with a high probability of needing various PHP extensions, in a standard situation, I would recommend creating a Dockerfile that installs extensions on top of the image FROM phpswoole/swoole...

But after reading the documentation and looking at the source code, you can see that a lot is already installed there. Many things and many unnecessary things. For me, that's MySQL. If we work step by step, we also need to remove Redis, socket, etc.

So, I take the official source code as a basis. Remove what we don't need. Install dependencies that are missing. Update what I can update.

docker/php/Dockerfile
FROM php:8.3.6-cli-alpine3.18

COPY --from=composer:2.7.4 /usr/bin/composer /usr/bin/

RUN \
    set -ex && \
    apk update && \
    apk add --no-cache libstdc++ libpq && \
    apk add --no-cache --virtual .build-deps $PHPIZE_DEPS curl-dev linux-headers postgresql-dev openssl-dev pcre-dev pcre2-dev zlib-dev && \
    apk add --no-cache supervisor && \
    apk add --no-cache supercronic && \
    pecl channel-update pecl.php.net && \
    docker-php-ext-install sockets && \
    pecl install redis &&  \
    docker-php-ext-enable redis && \
    docker-php-source extract && \
    mkdir /usr/src/php/ext/swoole && \
    curl -sfL https://github.com/swoole/swoole-src/archive/v5.1.2.tar.gz -o swoole.tar.gz && \
    tar xfz swoole.tar.gz --strip-components=1 -C /usr/src/php/ext/swoole && \
    docker-php-ext-configure swoole \
        --enable-swoole-pgsql \
        --enable-openssl      \
        --enable-sockets --enable-swoole-curl && \
    docker-php-ext-install -j$(nproc) swoole && \
    docker-php-ext-configure pcntl --enable-pcntl && \
    docker-php-ext-install pcntl && \
    rm -f swoole.tar.gz && \
    docker-php-source delete && \
    apk del .build-deps

# TODO not use node in producation
RUN apk add --no-cache nodejs npm git

COPY docker/php/supervisord/supervisord.conf /etc/

EXPOSE 8000

WORKDIR "/var/www/"

ENTRYPOINT ["supervisord", "--nodaemon", "--configuration", "/etc/supervisord.conf"]

Here pay attention to:

  • EXPOSE 8000 - makes port 8000 available to other services within the network. This port is closed for the host.
  • pcntl - extension for parallel operation of processes. Required for laravel octane
  • RUN apk add --no-cache nodejs 'npm' -noda is required for watch mod laravel octane. Do not install it to prod version Git will allow you to run commitlint and branch name validator from the container. So we are 100% independent of the developer environment. Also do not install in prod!
  • supervisord та ENTRYPOINT - here you need to understand that Docker works as long as the main Docker process is active. In our case, this process is php artisan ocrane:start, but let me remind you that we are deploying the project locally and don't want to depend on the developer's local machine, which means the developer must be able to manipulate packages, install/update on the raised service. That's why we have a supervisord as the main team, and inside run our commands.

Supervisor - a process management utility. Runs, restarts, monitors execution.

docker/php/supervisord/supervisord.conf
[supervisord]
nodaemon=true
logfile=/var/www/storage/logs/supervisord_%(ENV_PROCESS)s.log
pidfile=/var/www/storage/logs/supervisord_%(ENV_PROCESS)s.pid

[program:php]
directory=/var/www/
command: %(ENV_COMMAND)s 
startretries: 20

Step by step:

  • nodaemon=true - cannot run supervisor in docker in daemon mode. It will return a startup code and the container will stop.
  • logfile, pidfile - it makes sense to redirect to the director with the logs. By default it will be generated in the root directory and disturb
  • command: %(ENV_COMMAND)s - command what we have to run. For app service - php artisan octane:start. In my case there is a need to pass the command through environment variables. so %(ENV_COMMAND)s. This step will become clear on queue services.
  • startretries - the number of attempts to run the command. Each attempt is delayed by the number of seconds of the previous attempts. That is, the supervisor will try to run the command every 1,2,3,6... 20 seconds before giving an error. It's not ideal, but unfortunately I haven't found a proper timeout implementation in the supervisor, so use this for now
docker-compose.yml
app: &app
build:
  context: ./
  dockerfile: docker/php/Dockerfile
user: ${UID}:${GID}
restart: unless-stopped
container_name: bh-app
environment:
  COMMAND: php artisan octane:start --watch --host=0.0.0.0 --port=8000
  PROCESS: app
depends_on:
  db:
    condition: service_healthy
  redis:
    condition: service_healthy
volumes:
  - ./:/var/www
  - bh_app_composer_data:/.composer
  - bh_app_npm_data:/.npm
networks:
  - bh-db-network
  - bh-webserver-network
  - bh-redis-network
healthcheck:
  test: curl -s http://localhost:8000/up >/dev/null || exit 1
  interval: 5s
  timeout: 10s
  retries: 20

Pay attention:

  • build: ... - app image we build (for now). All others - we use ready-made ones
  • &app - anchor|alias|reference to service (or code block). Using in Queue|Schedule services
  • user: ${UID}:${GID} - run the container with local user rights. All files created in the service can be easily edited on the host. Please, please, think when you see solutions on the web. rights 777 THIS IS NOT OPTION!.
Push your user id and group id to .env
echo UID=$(id -u) >> .env
echo GID=$(id -g) >> .env
  • volumes: - ./:/var/www - all files (except described in .dockerignore) from directory in host в ROOT_DIR inside container. "watch mode" in dev build
  • COMMAND: php artisan octane:start --watch --host=0.0.0.0 --port=8000 - command to supervisor. Host property is required. Docker do not see default artisan host
  • PROCESS: app - process name. Used for identify supervisord log files
  • test: curl -s http://localhost:8000/up >/dev/null || exit 1 - laravel 11 up function. Write your own if your version below
  • bh_app_composer_data:/.composer and bh_app_npm_data:/.npm - store composer and npm cache. Not necessarily, but useful allows you to stop the container without losing the cache, and the need to do something with the rights of the cache directories.

Webserver service

Here it is important to understand - we can work without webserver, especially locally. BUT on the prod it is mandatory, which means all problems of interaction, configuration, etc. better to know at the development stage.

There is no nginx competitor, apache hasn't even been considered for a bunch of years.

Take the configuration directly from the laravel octane documentation. The logic is simple - nginx provides all the static resource for all requests - octane. But I still need to make changes for project needs:

  • Remove favicon and robot.txt. API should not use it
  • resolver 127.0.0.11 - Docker DNS standard, nginx will not catch app host without this
  • proxy_pass http://bh-app:8000$suffix - app host (container name)
docker/nginx/conf.d/app.conf
map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

server {
  listen 80;
  listen [::]:80;
  server_tokens off;
  root /var/www/public;

  index index.php;

  charset utf-8;

  location /index.php {
    try_files /not_exists @octane;
  }

  location / {
    try_files $uri $uri/ @octane;
  }

  access_log /var/log/nginx/nginx-access.log;
  error_log  /var/log/nginx/nginx-error.log error;

  error_page 404 /index.php;

  resolver 127.0.0.11;

  location @octane {
    set $suffix "";

    if ($uri = /index.php) {
        set $suffix ?$query_string;
    }

    proxy_http_version 1.1;
    proxy_set_header Host $http_host;
    proxy_set_header Scheme $scheme;
    proxy_set_header SERVER_PORT $server_port;
    proxy_set_header REMOTE_ADDR $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;

    proxy_pass http://bh-app:8000$suffix;
  }
}

Redis service

Cache service. For Laravel. Redis is best choose for Laravel (in my case). Use it for same broadcast driver, and driver for queues.

Fast key-value database.

The only thing I recommend is to start immediately with a password, again so that the person responsible for the release does not forget. And if necessary, the team could open a port to the world with lower risks.

docker-compose.yml
redis:
  image: redis:7.2.4-alpine
  restart: unless-stopped
  container_name: bh-redis
  command:
    - 'redis-server'
    - '--requirepass ${REDIS_PASSWORD}'
  volumes:
    - redis_data:/data
  networks:
    - bh-redis-network
  healthcheck:
    test: redis-cli ping
    interval: 5s
    timeout: 10s
    retries: 20

Queue service

Here, I must say, I not recommend installing everything at once. If the project doesn't have queues, scheduled tasks, or caching needs - don't implement them.

With a high probability, you'll activate these services. But I often see configured services that are not used in the project. Someone created them with confidence that they would definitely be used... And they are not deleted because everyone is afraid, as no one knows where it's used.

Also, I understand that both queue and schedule services could be run inside app service, with supervisor. But in Docker, there's the principle of one service - one process. This allows you to limit resources for a specific service (e.g., queue), monitor its state, etc.

Go back to queue. Laravel has ready-made Laravel Horizon. I recommended it.

Just one advice (at this stage) - the documentation suggests hooking the composer update command to publish Horizon resources. Instead, I suggest adding public/vendor to .gitignore and adding the command to publish also in install. This is what I recommend for all similar services. Because this code doesn't have any payload, it's always installed with composer packages, it always requires publication, and sending a bunch of generated code for review is not okay, especially when it should be reviewed. Someone might sneak in their script into the Horizon view, knowing now one no one is going to check code...

composer.json
{
  "post-install-cmd": [
    "@php artisan vendor:publish --tag=laravel-assets --ansi --force"
  ],
  "post-update-cmd": [
    "@php artisan vendor:publish --tag=laravel-assets --ansi --force"
  ]
}
docker-compose.yml
horizon:
<<: *app
container_name: bh-horizon
environment:
  COMMAND: php artisan horizon
  PROCESS: horizon
networks:
  - bh-db-network
  - bh-redis-network
healthcheck:
  test: php artisan horizon:status | grep -q 'is running'  # TODO try spatie LARAVEL-HEALTH
  interval: 5s
  timeout: 10s
  retries: 20
  • <<: *app - merge with &app all property duplicated from &app, other - overwritten. Same as app image, restart rules, depends conditions etc. But container name, environment, networks, та healthcheck - new one.
  • test: php artisan horizon:status | grep -q 'is running - bad solution, it is better to try something like. Going to change it in feature

Schedule service

Every minute run command.

docker/php/schedule/crontab
* * * * * cd /var/www && php artisan schedule:run >> /dev/null 2>&1

Here's one advice: instead of the cron, recommend use supercronic. It's a cron like utility designed to work inside containers, allowing the use of environment variables, managing output, and ability run as active process.

Personally, I'm more interested in the possibility of integration with Sentry. But let's discuss that another time.

docker-compose.yml
schedule:
<<: *app
container_name: bh-schedule
environment:
  COMMAND: supercronic -quiet /var/www/docker/php/schedule/crontab 
  PROCESS: schedule
networks:
  - bh-db-network
  - bh-redis-network
healthcheck:
  test: supercronic -test /var/www/docker/php/schedule/crontab | grep -q 'is valid'  # TODO try spatie LARAVEL-HEALTH
  interval: 5s
  timeout: 10s
  retries: 2

Dockerignore

Developers often forget about .dockerignore. Remember - anything that Docker doesn't need to work should not be pushed there.

In production containers, also avoid adding tests, stubs, mocks, etc. Don't pull unnecessary things into production.

.dockerignore
**.git
.idea
**.gitignore
**.gitattributes
node_modules/
public/hot/
public/storage/
storage/*.key
storage/app/*
storage/framework/cache/*
storage/framework/sessions/*
storage/framework/views/*
storage/logs/*
.env.example
.phpunit.result.cache
npm-debug.log
yarn-error.log
Dockerfile
docker-compose.yml
branchlint.config.json
commitlint.config.js
README.md
.husky
.styleci.yml
.editorconfig
.phpstorm.meta.php
_ide_helper.php
_ide_helper_models.php
.gitlab-ci.yml
psysh
setup-local.sh
_ide_helper_actions.php

Readme та ENV

This instruction is the only one your team will read. This is the reality 🥲

A little advice - type command in MD format, do sh. This will allow command to simply click commands run button directly from the IDE

README.md
## Setup local
1) Copy environments
```sh
cp .env.example .env
```
2) Add current user identity data to environments
```sh
echo UID=$(id -u) >> .env
echo GID=$(id -g) >> .env
```
3) Run docker
```sh
docker compose up -d --build
```
4) Make cache dirs for package managers
```sh
docker compose exec -u root app install -o $(id -u) -g $(id -g) -d "/.npm" &&
docker compose exec -u root app install -o $(id -u) -g $(id -g) -d "/.composer"
```
5) Install composer dependency
```sh 
docker compose exec app composer install
```
6) Install npm dependency
```sh 
docker compose exec app npm i
```
7) Generate app key
```sh 
docker compose exec app php artisan key:generate
```
8) #if services still unhealthy - restart docker
```sh 
docker compose up -d --build
```

### Entry points
* [http://localhost/](http://localhost/) - application
* [http://localhost/horizon](http://localhost/horizon) - queue manager

One moment

docker compose exec app npm i

Install npm dependencies from app container. Usually, they are almost never updated for the backend. But changes to the lock file may arrive depending on node version on the local machine. Of course, they will be sent to the review, which is not ok, needs extra attention.

And don't forget to sync the .env.example

.env.example
DB_CONNECTION=pgsql
DB_HOST=bh-db
DB_PORT=5432
DB_DATABASE=band_h
DB_USERNAME=bh_db_user
DB_PASSWORD=bh_db_password

BROADCAST_DRIVER=redis
CACHE_DRIVER=redis
QUEUE_CONNECTION=redis


REDIS_HOST=bh-redis
REDIS_PASSWORD=redis_password

OCTANE_SERVER=swoole

And I can run linters from docker container:

.husky/pre-push
docker exec -t bh-app npx validate-branch-name

.husky/commit-msg
docker exec -t bh-app npx --no -- commitlint --edit $1

Got my respect to those who have read this and thought: "The author spent a lot of time talking about versioning, everything is the same for everyone.... And in the end, he builds app services from scratch for everyone." See the next lesson.

Help to survive