Blog Archives

Playing with Docker, Silex, Python, Node and WebSockets

I’m learning Docker. In this post I want to share a little experiment that I have done. I know the code looks like over-engineering but it’s just an excuse to build something with docker and containers. Let me explain it a little bit.

The idea is build a Time clock in the browser. Something like this:

Clock

Yes I know. We can do it only with js, css and html but we want to hack a little bit more. The idea is to create:

  • A Silex/PHP frontend
  • A WebSocket server with socket.io/node
  • A Python script to obtain the current time

WebSocket server will open 2 ports: One port to serve webSockets (socket.io) and another one as a http server (express). Python script will get the current time and it’ll send it to the webSocket server. Finally one frontend(silex) will be listening to WebSocket’s event and it will render the current time.

That’s the WebSocket server (with socket.io and express)

var
    express = require('express'),
    expressApp = express(),
    server = require('http').Server(expressApp),
    io = require('socket.io')(server, {origins: 'localhost:*'})
    ;

expressApp.get('/tic', function (req, res) {
    io.sockets.emit('time', req.query.time);
    res.json('OK');
});

expressApp.listen(6400, '0.0.0.0');

server.listen(8080);

That’s our Python script

from time import gmtime, strftime, sleep
import httplib2

h = httplib2.Http()
while True:
    (resp, content) = h.request("http://node:6400/tic?time=" + strftime("%H:%M:%S", gmtime()))
    sleep(1)

And our Silex frontend

use Silex\Application;
use Silex\Provider\TwigServiceProvider;

$app = new Application(['debug' => true]);
$app->register(new TwigServiceProvider(), [
    'twig.path' => __DIR__ . '/../views',
]);

$app->get("/", function (Application $app) {
    return $app['twig']->render('index.twig', []);
});

$app->run();

using this twig template

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <title>Docker example</title>
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">
    <link href="css/app.css" rel="stylesheet">
    <script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
    <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
</head>
<body>
<div class="site-wrapper">
    <div class="site-wrapper-inner">
        <div class="cover-container">
            <div class="inner cover">
                <h1 class="cover-heading">
                    <div id="display">
                        display
                    </div>
                </h1>
            </div>
        </div>
    </div>
</div>
<script src="//localhost:8080/socket.io/socket.io.js"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
<script>
var socket = io.connect('//localhost:8080');

$(function () {
    socket.on('time', function (data) {
        $('#display').html(data);
    });
});
</script>
</body>
</html>

The idea is to use one Docker container for each process. I like to have all the code in one place so all containers will share the same volume with source code.

First the node container (WebSocket server)

FROM node:argon

RUN mkdir -p /mnt/src
WORKDIR /mnt/src/node

EXPOSE 8080 6400

Now the python container

FROM python:2

RUN pip install httplib2

RUN mkdir -p /mnt/src
WORKDIR /mnt/src/python

And finally Frontend contailer (apache2 with Ubuntu 16.04)

FROM ubuntu:16.04

RUN locale-gen es_ES.UTF-8
RUN update-locale LANG=es_ES.UTF-8
ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update -y
RUN apt-get install --no-install-recommends -y apache2 php libapache2-mod-php
RUN apt-get clean -y

COPY ./apache2/sites-available/000-default.conf /etc/apache2/sites-available/000-default.conf

RUN mkdir -p /mnt/src

RUN a2enmod rewrite
RUN a2enmod proxy
RUN a2enmod mpm_prefork

RUN chown -R www-data:www-data /mnt/src
ENV APACHE_RUN_USER www-data
ENV APACHE_RUN_GROUP www-data
ENV APACHE_LOG_DIR /var/log/apache2
ENV APACHE_LOCK_DIR /var/lock/apache2
ENV APACHE_PID_FILE /var/run/apache2/apache2.pid
ENV APACHE_SERVERADMIN admin@localhost
ENV APACHE_SERVERNAME localhost

EXPOSE 80

Now we’ve got the three containers but we want to use all together. We’ll use a docker-compose.yml file. The web container will expose port 80 and node container 8080. Node container also opens 6400 but this port is an internal port. We don’t need to access to this port outside. Only Python container needs to access to this port. Because of that 6400 is not mapped to any port in docker-compose

version: '2'

services:
  web:
    image: gonzalo123/example_web
    container_name: example_web
    ports:
     - "80:80"
    restart: always
    depends_on:
      - node
    build:
      context: ./images/php
      dockerfile: Dockerfile
    entrypoint:
      - /usr/sbin/apache2
      - -D
      - FOREGROUND
    volumes:
     - ./src:/mnt/src

  node:
    image: gonzalo123/example_node
    container_name: example_node
    ports:
     - "8080:8080"
    restart: always
    build:
      context: ./images/node
      dockerfile: Dockerfile
    entrypoint:
      - npm
      - start
    volumes:
     - ./src:/mnt/src

  python:
      image: gonzalo123/example_python
      container_name: example_python
      restart: always
      depends_on:
        - node
      build:
        context: ./images/python
        dockerfile: Dockerfile
      entrypoint:
        - python
        - tic.py
      volumes:
       - ./src:/mnt/src

And that’s all. We only need to start our containers

docker-compose up --build -d

and open our browser at: http://localhost to see our Time clock

Full source code available within my github account

Sending automated emails with PHP, Swiftmailer and Twig

I’m the one of hosts of a Coding Dojo in my city called Katayunos. Katayunos is the mix of the word Kata (coding kata) and “Desayuno” (breakfast in Spanish). A group of brave programmers meet together one Saturday morning and after having breakfast we pick one coding kata and we practise TDD and pair programming. It’s something difficult to explain to non-geek people (why the hell we wake up early one Saturday morning to do this) but if you are reading this post probably it sounds good:).

My work as host is basically pick the place and encourage people to join to the Coding Dojo. One way of doing this (besides twitter buzz) is take my address book and send one bulk email to all of them inviting to join us. I don’t like this kind of mails. They look like spam, so I prefer to send a personalized email. This email has a common part (the place location, the hour, the event description, …) and the personalized part. I can do it manually, the list isn’t so huge, but definitely that’s not cool. Because of that I have done a little script to perform this operation. I can do a simple PHP script but we are speaking about announcing a event about TDD, SOLID and things like that, so I must use the “right way”. Let’s start.

I manage my list of contacts within a spreadsheet. In this spreadsheet I have the name, the email and a one paragraph with the personalized part to each one of my contact. I can easily export this spreadsheet to a csv document like this:

Peter Parker, spiderman@gmail.com, "Lorem ipsum dolor sit amet, ..."
Clark Kent, superman@gmail.com, "consectetur adipisicing elit, ..."
Juan López Fernández, superlopez@gmail.com, "sed do eiusmod tempor incididunt .."

So first of all I need to parse this file.

class Parser
{
    private $data;

    public function createFromCsvFile($path)
    {
        $handle = fopen($path, "r");
        while (($data = fgetcsv($handle)) !== false) {
            $this->data[] = [
                'name'  => trim($data[0]),
                'email' => trim($data[1]),
                'body'  => isset($data[2]) ? trim($data[2]) : null,
            ];
        }
    }

    public function getData()
    {
        return $this->data;
    }
}

Easy. Now I want to send this parsed array by email. Because of that I will include Swiftmailer in my composer.json file.

My email will also be one template and one personalized part. We will use Twig to manage the template.

"require": {
        "swiftmailer/swiftmailer": "v5.0.2",
        "twig/twig": "v1.13.2",
}

Now we will create a class to wrap the needed code to send emails

class Mailer
{
    private $swiftMailer;
    private $swiftMessage;

    function __construct(Swift_Mailer $swiftMailer, Swift_Message $swiftMessage)
    {
        $this->swiftMailer  = $swiftMailer;
        $this->swiftMessage = $swiftMessage;
    }

    public function sendMessage($to, $body)
    {
        $this->swiftMessage->setTo($to);
        $this->swiftMessage->setBody(strip_tags($body));
        $this->swiftMessage->addPart($body, 'text/html');

        return $this->swiftMailer->send($this->swiftMessage);
    }
}

Our Mailer class sends mails. Our Parser class parses one csv file. Now we need something to join those two classes: the Spammer class. Spammer class will take one parsed array and it will send one by one the mails using Mailer class.

class Spammer
{
    private $twig;
    private $mailer;

    function __construct(Twig_Environment $twig, Mailer $mailer)
    {
        $this->twig       = $twig;
        $this->mailer     = $mailer;
    }

    public function sendEmails($data)
    {
        foreach ($data as $item) {
            $to = $item['email'];
            $this->mailer->sendMessage($to, $this->twig->render('mail.twig', $item));
        }
    }
}

Ok with this three classes I can easily send my emails. This script is a console script and we also want pretty console colours and this kind of stuff. symfony/console to the rescue. But I’ve a problem now. I want to write one message when one mail is sent and another one when something wrong happens. If I want to do that I need to change my Spammer class. But my Spammer class does’t know anything about my console Command. If I inject the console command into my Spammer class I will violate the Demeter law, and that’s a sin. What can we do? Easy: The mediator pattern. We can write one implementation of mediator pattern but we also can use symfony/event-dispatcher, a well done implementation of this pattern. We change our Spammer class to:

use Symfony\Component\EventDispatcher\EventDispatcher;

class Spammer
{
    private $twig;
    private $mailer;
    private $dispatcher;

    function __construct(Twig_Environment $twig, Mailer $mailer, EventDispatcher $dispatcher)
    {
        $this->twig       = $twig;
        $this->mailer     = $mailer;
        $this->dispatcher = $dispatcher;
    }

    public function sendEmails($data)
    {
        foreach ($data as $item) {
            $to = $item['email'];
            try {
                $this->mailer->sendMessage($to, $this->twig->render('mail.twig', $item));
                $this->dispatcher->dispatch(MailEvent::EVENT_MAIL_SENT, new MailEvent\Sent($to));
            } catch (\Exception $e) {
                $this->dispatcher->dispatch(MailEvent::EVENT_SENT_ERROR, new MailEvent\Error($to, $e));
            }
        }
    }
}

Now can easily build of console command class:

use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Output\OutputInterface;
use Symfony\Component\EventDispatcher\EventDispatcher;

class SpamCommand extends Command
{
    private $parser;
    private $dispatcher;

    protected function configure()
    {
        $this->setName('spam:run')
            ->setDescription('Send Emails');
    }

    protected function execute(InputInterface $input, OutputInterface $output)
    {
        $output->writeln("Sending mails ...");
        $this->dispatcher->addListener(MailEvent::EVENT_MAIL_SENT, function (MailEvent\Sent $event) use ($output) {
                $output->writeln("<info>Mail sent to</info>: <fg=black;bg=cyan>{$event->getTo()}</fg=black;bg=cyan>");
            }
        );

        $this->dispatcher->addListener(MailEvent::EVENT_SENT_ERROR, function (MailEvent\Error $event) use ($output) {
                $output->writeln("<error>Error sending mail to</error>: <fg=black;bg=cyan>{$event->getTo()}</fg=black;bg=cyan> Error: " . $event->getException()->getMessage());
            }
        );

        $this->spammer->sendEmails($this->parser->getData());
        $output->writeln("End");
    }

    public function setSpammer(Spammer $spammer)
    {
        $this->spammer = $spammer;
    }

    public function setParser(Parser $parser)
    {
        $this->parser = $parser;
    }

    public function setDispatcher(EventDispatcher $dispatcher)
    {
        $this->dispatcher = $dispatcher;
    }
}

With all this parts we can build our script. Our classes are decoupled. That’s good but setting up the dependencies properly can be hard. Because of that we will use symfony/dependency-injection. With symfony DIC we can set up our dependency tree within a yaml file:

Our main services.yml

imports:
  - resource: conf.yml
  - resource: mail.yml
  - resource: twig.yml

parameters:
  base.path: .

services:
  parser:
    class: Parser
    calls:
      - [createFromCsvFile, [%mail.list%]]

  mailer:
    class: Mailer
    arguments: [@swift.mailer, @swift.message]

  spam.command:
    class: SpamCommand
    calls:
      - [setParser, [@parser]]
      - [setDispatcher, [@dispatcher]]
      - [setSpammer, [@spammer]]

  spammer:
    class: Spammer
    arguments: [@twig, @mailer, @dispatcher]

  dispatcher:
    class: Symfony\Component\EventDispatcher\EventDispatcher

I like to separate the configuration files to reuse those files between projects and to make them more readable.

One for twig:

parameters:
  twig.path: %base.path%/templates
  twig.conf:
    auto_reload: true

services:
  twigLoader:
    class: Twig_Loader_Filesystem
    arguments: [%twig.path%]

  twig:
    class: Twig_Environment
    arguments: [@twigLoader, %twig.conf%]

another one for swiftmailer:

services:
  swift.message:
    class: Swift_Message
    calls:
      - [setSubject, [%mail.subject%]]
      - [setFrom, [%mail.from.mail%: %mail.from.name%]]

  swift.transport:
    class: Swift_SmtpTransport
    arguments: [%mail.smtp.host%, %mail.smtp.port%, %mail.smtp.encryption%]
    calls:
      - [setUsername, [%mail.smtp.username%]]
      - [setPassword, [%mail.smtp.password%]]

  swift.mailer:
    class: Swift_Mailer
    arguments: [@swift.transport]

and the last one for the configuration parameters:

parameters:
  mail.do.not.send.mails: false

  mail.list: %base.path%/mailList.csv
  mail.subject: mail subject
  mail.from.name: My Name
  mail.from.mail: my_email@mail.com

  mail.smtp.username: my_smtp_username
  mail.smtp.password: my_smtp_password
  mail.smtp.host: smtp.gmail.com
  mail.smtp.port: 465
  mail.smtp.encryption: ssl

Now we can build our script.

use Symfony\Component\Console\Application;
use Symfony\Component\DependencyInjection\ContainerBuilder;
use Symfony\Component\Config\FileLocator;
use Symfony\Component\DependencyInjection\Loader\YamlFileLoader;

$container = new ContainerBuilder();
$loader = new YamlFileLoader($container, new FileLocator(__DIR__ . '/conf'));
$loader->load('services.yml');

$container->setParameter('base.path', __DIR__);

$application = new Application();
$application->add($container->get('spam.command'));
$application->run();

n

And that’s all. My colleagues of the next Katayuno will be invited in a “SOLID” way :).
Source code is available in my github account.

BTW: Do you want to organize one Katayuno in your city? It’s very easy. Feel free to contact me for further information.