Speed Up Drupal 8 Docker Development with Docker Sync

2019-08-20

When running Drupal 8 and Docker to develop locally, we often use volumes to map our Drupal files and modules from our local machine to our container so we can edit our files in our local IDE and run the files with our container. But it turns out with large frameworks like Drupal 8 mapping these files with volumes causes our sites to run extremely slow. We can solve this with an open source tool called Docker Sync.

Prerequisits

I am going to use a pre-configured Drupal 8 Docker image along with a set of files for a default Drupal 8 site. While I am using Drupal 8 development as an example, this technique will help speed up any sites running with large volumes mapped. I am going to assume you are familiar with Docker and Docker Compose.

Initial Setup

First of all lets setup the site and witness how slow it is. My local file system is setup to allow Docker to map volumes to my local machine, the folder structure looks this:

project
- Drupal8Site
  - core
  - modules
  - profiles
  - etc…
- docker-compose.yml

My docker-compose.yml file is used to launch my Drupal 8 site with a mapped volume, specifically see the volumes section under drupal_dev. Additionally I launch a MySql container for a database:

version: '2.1'

services:

  drupal_dev_db:    
    image: mysql:5.7
    container_name: "drupal_dev_db"
    environment:
       MYSQL_ROOT_PASSWORD: drupal
       
    
  drupal_dev_:     
    image: drupal
    container_name: "drupal_dev"
    ports:
      - 8099:80
    volumes:
      - /project/Drupal8Site:/var/www/html
    links:
      - drupal_dev_db

Running docker-compose up will at this point get us our site running at http://localhost:8099 with our mapped files that we can edit, but if you load the site in a browser, you will experience the painfully slow load time. Here is a screenshot of my Chrome network tab in the inspector, loading the basic install.php page took 18 seconds. Painfully slow!

Click the image for a larger version

Introducing Docker Sync

Now that we’ve seen how slow Drupal 8 can be when running from a volume in a Docker container, we are going to speed things up with Docker Sync. Drupal Sync is an open source project that helps speed up mapped volumes by creating an intermediate container that listes for changes on your local volume, and transfers files into your container when changed. This all happens extremely fast, and allows for the containers to run files at native OS speeds without any of the overhead of mapped volumes.

Installing Docker Sync

Docker Sync Is a Ruby gem project and can be easily installed with the following command:

gem install docker-sync

I am running Mac OSX for this article, in which Ruby comes installed by default. If you do not have RubyGem installed, see this article on Installing RubyGem.

Once installed, we will configure Docker Sync to sync our Drupal 8 files from our mapped volume.

Configure Docker Sync

First up we will configure a docker-sync.yml file to tell Docker Sync what files to sync and how to sync them. Note that each site you work on will need one of these docker-sync.yml files, and there are many different ways to configure seperate sites. Here is an example:

version: "2"

options:
  verbose: true
syncs:
  drupal_dev-sync: # tip: add -sync and you keep consistent names as a convention
    src: './site'
    # sync_strategy: 'native_osx' # not needed, this is the default now
    sync_excludes: ['ignored_folder', '.ignored_dot_folder']

Save this file in the same directory level as your docker-compose.yml so that your directory structure looks like this:

project
- Drupal8Site
  - core
  - modules
  - profiles
  - etc…
- docker-compose.yml
- docker-sync.yml

Docker-Sync.yml File

The main thing to look at here is the syncs directive, where we have drupal_dev-sync. This is the name of our sync that we will use later. Notice the comment (left in from the example) that suggests always including -sync, which will help you differenciate your containers. (More on this later).

Note: As mentioned there are many ways to configure syncing your volume data, with only a default docker-sync.yml file shown that works out of the box for OSX. For more examples check the Docker Sync Boilerplate repository.

Now that we have our Docker Sync file configured, we’ll let our containers know about Docker Sync with external volumes.

Updating Docker Compose

We’ll need to modify our docker-compose.yml file to use Docker Sync. This is done by changing our volume mappings to use the top level volumes directive in and to modify the sub level volumes directive for the drupal_dev container. Here is what our new docker-compose.yml file looks like:

version: '2.1'

services:

  drupal_dev_db:    
    image: mysql:5.7
    container_name: "drupal_dev_db"
    environment:
       MYSQL_ROOT_PASSWORD: drupal
       
    
  drupal_dev:     
    image: drupal
    container_name: "drupal_dev"
    ports:
      - 8099:80
    volumes:
      - drupal_dev-sync:/var/www/html:nocopy
    links:
      - drupal_dev_db
volumes:
  drupal_dev-sync:
    external: true

First we change the volume mapping to use drupal_dev-sync:/var/www/html:nocopy. Notice the :nocopy option at the end, this is important.

Then the new volumes options defined at the end of the file, drupal_dev-sync. Remember our naming conventioned mentioned earlier, where we add -sync to the end of our container name? This is the connection between the two.

Essentially here we are telling Docker to use an external container to control how our volumes are mapped, with drupal_dev-sync doing the mapping work.

Note: Mapping volumes in Docker is a full subject on its own. For more information, Docker has great documenation on this subject here: https://docs.docker.com/storage/volumes/.

With both the Docker Sync config ready and Docker Compose up to date, we are ready to launch the site.

Launch the Site

First, launch your containers using docker-compose exactly like you normally would:

$docker-compose up -d
Creating network "drupal_dev_default" with the default driver
Creating drupal_dev_db ... done
Creating drupal_dev    ... done

With your containers launched, next launch the sync container:

$docker-sync start
ok  Starting native_osx for sync drupal_dev-sync
          ok  drupal_dev-sync container not running
          ok  creating drupal_dev-sync container
          ok  Starting precopy
          ...

This will kick off the sync process, you’ll notice at the end of the output:

...
doing initial sync with unison
Unison 2.51.2 (ocaml 4.06.1): Contacting server...
Looking for changes

After all files are copied into your container you will see the command finish with the following:

success  Sync container started
success  Starting Docker-Sync in the background

And if you run docker ps to see a list of containers, you’ll see our Drupal and MySql container, as well as our new sync container. And again, with the -sync added to the name for clarity:

$docker ps
... drupal_debug-sync
... drupal_dev
... drupal_dev_db

Now your site should be running much quicker!

Note: I know 11 seconds load time is not that fast… but way faster than 18 seconds in the first test! Your site will be much faster after install.

Shortcuts

Docker Sync comes with a few handy commands to speed up typing all of those commands:

To launch the full stack and sync:

docker-sync-stack start

To remove all containers, similar to docker-compose down:

docker-sync-stack clean

Final Thoughts and Next Steps

Having Docker Sync running to sync files from mapped volumes has saved the day when doing local Drupal 8 development. With my initial setup using the Drupal 8 site was unbearable with the 20+ second load time for each page and I would have ditched mapped volumes without this solution.

Next I am going to look at debugging with Visual Studio code, and explore how the debugger can connect with mapped volumes.

Sources

Check out the Docker Sync Github here: https://github.com/EugenMayer/docker-sync.