Docker for Windows: Watch Bindings
Docker has changed the way how we build and deploy our apps, by letting developers to spawn development or production environment on any (or almost any) machine just by typing docker-compose up
. However, initially Docker was not available for Windows users until February 2016 when Docker for Windows was released. Unfortunately, despite support of most features that Docker has and seamless integration with Windows-host’s environment, Docker for Windows (DfW) has its limitations. In particular you can’t use development server’s watch mode of most of modern frameworks (e.g. Jekyll, Angular CLI, etc.), since DfW does not notify container about any file changes you make from Windows. In this post we will describe workaround for this problem.
Problem Investigation
The above mentioned problem seems to be known for a long time, but yet not solved. To understand the root cause of it let’s dive into implementation details of host directory bindings in DfW. It’s not a secret that Docker for Windows runs Docker daemon inside Hyper-V virtual machine (VM) running Linux guest OS. To make your local files available to Docker daemon DfW shares your hard drive with VM using Samba/CIFS protocol. Unfortunately, CIFS implementation in Linux kernel doesn’t support inotify events. Thus if you modify file inside mounted folder with your favorite text editor running on Windows host, Linux guest OS will not be able to get appropriate inotify event. That is why watch mode (auto rebuild) of most development servers will not work with DfW.
Available Solutions
Probably the cleanest possible solution to this problem is improvement of CIFS support in Linux kernel, however this sounds like a big chunk of work, thus it probably wouldn’t be done soon. So far user of Docker Community Forums suggests to use polling (i.e. continuous scanning mounted directory for changes). However, as justinmchase notes, this workaround has poor performance.
Proposed Solution
As a workaround for the described problem we have developed a script that runs on Windows host and monitors all mounted directories. Once file is changed the script rewrites file permissions of changed file inside container to trigger inotify event. The triggered event causes development server to perform rebuild.
This script can be installed with pip (both Python 2 & 3 are supported).
pip install docker-windows-volume-watcher
Then you can just run the script without any arguments:
docker-volume-watcher
The script will inspect all running containers and start notifying containers about changes in mounted directories. The script will also listen container start/stop events and update the list of watched directories.
To monitor only bindings of container container_name
execute:
dos
docker-volume-watcher container_name
To monitor only binding of container_name
to host directory C:\some\directory
execute:
docker-volume-watcher container_name C:\some\directory
You can also specify wildcards with *
and ?
characters. For example: monitor only bindings of containers with names containing myproject
to directories starting with C:\project\folder\
.
docker-volume-watcher *myproject* C:\project\folder\*
Use flag -v
to enable verbose output: the script will report start/stop events of eligible containers and print all detected file changes.
Implementation highlights
The script is fairly simple, you can view its source on GitHub. So let’s just discuss a few interesting implementation details.
File Watching
To monitor host directories mounted by containers we use watchdog package.
Container Notification
At the first place we tried to notify containers about file changes by executing inside container:
touch /changed/file
Unfortunately, this approach leads to infinite event loop: touch to changed file triggers new watchdog event, which makes script to touch file again and trigger new watchdog event. This problem can be solved by introducing some time-out between consequent change events of a single file. However, this approach feels too hacky.
Instead of touching file we read its permissions by running stat
util and then we rewrite its permissions with chmod
. This approach has two advantages:
- Since Windows knows nothing about Unix file permissions, the host OS doesn’t notice any change, thus no watchdog event is triggered.
- No actual change is made, since we rewrite file permissions with the same value, however rewrite triggers inotify event inside container and makes development server to perform rebuild.
Unfortunately, you can’t use this approach to report file deletion event to container. However, supported file creation and modification events cover most of use cases.
Interaction with Docker Daemon
We use Docker Python package to interact with Docker daemon. This package has pretty neat interface enabling you to perform all operations you can do with Docker CLI. To monitor container start/stop events we use events()
method of DockerClient
object. This method returns blocking generator yielding events. Unfortunately, wait for the next event can’t be interrupted by user hitting Ctrl+C
(see discussion at Stack Overflow). To solve this problem we employ since
and until
arguments of events()
method to listen for events in 2 second timeframes.
delta = timedelta(seconds=2)
since = datetime.utcnow()
until = datetime.utcnow() + delta
filters = {'event': ['start', 'stop'], 'type': 'container'}
while True:
for event in self.client.events(since=since, until=until, decode=True, filters=filters):
self.__handle_event(event)
since = until
until = datetime.utcnow() + delta
Please note that for each while
iteration we set since
equal to previous value of until
, thus we don’t miss any event. Also since we limit events()
method with timeframe, the script terminates in 2 seconds after user hits Ctrl+C
.