Category Archives: js

Working with SAPUI5 locally (part 3). Adding more services in Docker

In the previous project we moved one project to docker. The idea was to move exactly the same functionality (even without touching anything within the source code). Now we’re going to add more services. Yes, I know, it looks like overenginering (it’s exactly overenginering, indeed), but I want to build something with different services working together. Let start.

We’re going to change a little bit our original project. Now our frontend will only have one button. This button will increment the number of clicks but we’re going to persists this information in a PostgreSQL database. Also, instead of incrementing the counter in the backend, our backend will emit one event to a RabbitMQ message broker. We’ll have one worker service listening to this event and this worker will persist the information. The communication between the worker and the frontend (to show the incremented value), will be via websockets.

With those premises we are going to need:

  • Frontend: UI5 application
  • Backend: PHP/lumen application
  • Worker: nodejs application which is listening to a RabbitMQ event and serving the websocket server (using socket.io)
  • Nginx server
  • PosgreSQL database.
  • RabbitMQ message broker.

As the previous examples, our PHP backend will be server via Nginx and PHP-FPM.

Here we can see to docker-compose file to set up all the services

version: '3.4'

services:
  nginx:
    image: gonzalo123.nginx
    restart: always
    ports:
    - "8080:80"
    build:
      context: ./src
      dockerfile: .docker/Dockerfile-nginx
    volumes:
    - ./src/backend:/code/src
    - ./src/.docker/web/site.conf:/etc/nginx/conf.d/default.conf
    networks:
    - app-network
  api:
    image: gonzalo123.api
    restart: always
    build:
      context: ./src
      dockerfile: .docker/Dockerfile-lumen-dev
    environment:
      XDEBUG_CONFIG: remote_host=${MY_IP}
    volumes:
    - ./src/backend:/code/src
    networks:
    - app-network
  ui5:
    image: gonzalo123.ui5
    ports:
    - "8000:8000"
    restart: always
    volumes:
    - ./src/frontend:/code/src
    build:
      context: ./src
      dockerfile: .docker/Dockerfile-ui5
    networks:
    - app-network
  io:
    image: gonzalo123.io
    ports:
    - "9999:9999"
    restart: always
    volumes:
    - ./src/io:/code/src
    build:
      context: ./src
      dockerfile: .docker/Dockerfile-io
    networks:
    - app-network
  pg:
    image: gonzalo123.pg
    restart: always
    ports:
    - "5432:5432"
    build:
      context: ./src
      dockerfile: .docker/Dockerfile-pg
    environment:
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_DB: ${POSTGRES_DB}
      PGDATA: /var/lib/postgresql/data/pgdata
    networks:
    - app-network
  rabbit:
    image: rabbitmq:3-management
    container_name: gonzalo123.rabbit
    restart: always
    ports:
    - "15672:15672"
    - "5672:5672"
    environment:
      RABBITMQ_ERLANG_COOKIE:
      RABBITMQ_DEFAULT_VHOST: /
      RABBITMQ_DEFAULT_USER: ${RABBITMQ_DEFAULT_USER}
      RABBITMQ_DEFAULT_PASS: ${RABBITMQ_DEFAULT_PASS}
    networks:
    - app-network
networks:
  app-network:
    driver: bridge

We’re going to use the same docker files than in the previous post but we also need new ones for worker, database server and message queue:

Worker:

FROM node:alpine

EXPOSE 8000

WORKDIR /code/src
COPY ./io .
RUN npm install
ENTRYPOINT ["npm", "run", "serve"]

The worker script is simple script that serves the socket.io server and emits a websocket within every message to the RabbitMQ queue.

var amqp = require('amqp'),
  httpServer = require('http').createServer(),
  io = require('socket.io')(httpServer, {
    origins: '*:*',
  }),
  pg = require('pg')
;

require('dotenv').config();
var pgClient = new pg.Client(process.env.DB_DSN);

rabbitMq = amqp.createConnection({
  host: process.env.RABBIT_HOST,
  port: process.env.RABBIT_PORT,
  login: process.env.RABBIT_USER,
  password: process.env.RABBIT_PASS,
});

var sql = 'SELECT clickCount FROM docker.clicks';

// Please don't do this. Use lazy connections
// I'm 'lazy' to do it in this POC 🙂
pgClient.connect(function(err) {
  io.on('connection', function() {
    pgClient.query(sql, function(err, result) {
      var count = result.rows[0]['clickcount'];
      io.emit('click', {count: count});
    });

  });

  rabbitMq.on('ready', function() {
    var queue = rabbitMq.queue('ui5');
    queue.bind('#');

    queue.subscribe(function(message) {
      pgClient.query(sql, function(err, result) {
        var count = parseInt(result.rows[0]['clickcount']);
        count = count + parseInt(message.data.toString('utf8'));
        pgClient.query('UPDATE docker.clicks SET clickCount = $1', [count],
          function(err) {
            io.emit('click', {count: count});
          });
      });
    });
  });
});

httpServer.listen(process.env.IO_PORT);

Database server:

FROM postgres:9.6-alpine
COPY pg/init.sql /docker-entrypoint-initdb.d/

As we can see we’re going to generate the database estructure in the first build

CREATE SCHEMA docker;

CREATE TABLE docker.clicks (
clickCount numeric(8) NOT NULL
);

ALTER TABLE docker.clicks
OWNER TO username;

INSERT INTO docker.clicks(clickCount) values (0);

With the RabbitMQ server we’re going to use the official docker image so we don’t need to create one Dockerfile

We also have changed a little bit our Nginx configuration. We want to use Nginx to serve backend and also socket.io server. That’s because we don’t want to expose different ports to internet.

server {
    listen 80;
    index index.php index.html;
    server_name localhost;
    error_log  /var/log/nginx/error.log;
    access_log /var/log/nginx/access.log;
    root /code/src/www;

    location /socket.io/ {
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_pass "http://io:9999";
    }

    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

    location ~ \.php$ {
        try_files $uri =404;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass api:9000;
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param PATH_INFO $fastcgi_path_info;
    }
}

To avoid CORS issues we can also use SCP destination (the localneo proxy in this example), to serve socket.io also. So we need to:

  • change our neo-app.json file
  • "routes": [
        ...
        {
          "path": "/socket.io",
          "target": {
            "type": "destination",
            "name": "SOCKETIO"
          },
          "description": "SOCKETIO"
        }
      ],
    

    And basically that’s all. Here also we can use a “production” docker-copose file without exposing all ports and mapping the filesystem to our local machine (useful when we’re developing)

    version: '3.4'
    
    services:
      nginx:
        image: gonzalo123.nginx
        restart: always
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-nginx
        networks:
        - app-network
      api:
        image: gonzalo123.api
        restart: always
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-lumen
        networks:
        - app-network
      ui5:
        image: gonzalo123.ui5
        ports:
        - "80:8000"
        restart: always
        volumes:
        - ./src/frontend:/code/src
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-ui5
        networks:
        - app-network
      io:
        image: gonzalo123.io
        restart: always
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-io
        networks:
        - app-network
      pg:
        image: gonzalo123.pg
        restart: always
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-pg
        environment:
          POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
          POSTGRES_USER: ${POSTGRES_USER}
          POSTGRES_DB: ${POSTGRES_DB}
          PGDATA: /var/lib/postgresql/data/pgdata
        networks:
        - app-network
      rabbit:
        image: rabbitmq:3-management
        restart: always
        environment:
          RABBITMQ_ERLANG_COOKIE:
          RABBITMQ_DEFAULT_VHOST: /
          RABBITMQ_DEFAULT_USER: ${RABBITMQ_DEFAULT_USER}
          RABBITMQ_DEFAULT_PASS: ${RABBITMQ_DEFAULT_PASS}
        networks:
        - app-network
    networks:
      app-network:
        driver: bridge
    

    And that’s all. The full project is available in my github account

    Advertisements

    Working with SAPUI5 locally (part 2). Now with docker

    In the first part I spoke about how to build our working environment to work with UI5 locally instead of using WebIDE. Now, in this second part of the post, we’ll see how to do it using docker to set up our environment.

    I’ll use docker-compose to set up the project. Basically, as I explain in the first part, the project has two parts. One backend and one frontned. We’re going to use exactly the same code for the frontend and for the backend.

    The frontend is build over a localneo. As it’s a node application we’ll use a node:alpine base host

    FROM node:alpine
    
    EXPOSE 8000
    
    WORKDIR /code/src
    COPY ./frontend .
    RUN npm install
    ENTRYPOINT ["npm", "run", "serve"]
    

    In docker-compose we only need to map the port that we´ll expose in our host and since we want this project in our depelopemet process, we also will map the volume to avoid to re-generate our container each time we change the code.

    ...
      ui5:
        image: gonzalo123.ui5
        ports:
        - "8000:8000"
        restart: always
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-ui5
        volumes:
        - ./src/frontend:/code/src
        networks:
        - api-network
    

    The backend is a PHP application. We can set up a PHP application using different architectures. In this project we’ll use nginx and PHP-FPM.

    for nginx we’ll use the following Dockerfile

    FROM  nginx:1.13-alpine
    
    EXPOSE 80
    
    COPY ./.docker/web/site.conf /etc/nginx/conf.d/default.conf
    COPY ./backend /code/src
    

    And for the PHP host the following one (with xdebug to enable debugging and breakpoints):

    FROM php:7.1-fpm
    
    ENV PHP_XDEBUG_REMOTE_ENABLE 1
    
    RUN apt-get update && apt-get install -my \
        git \
        libghc-zlib-dev && \
        apt-get clean
    
    RUN apt-get install -y libpq-dev \
        && docker-php-ext-configure pgsql -with-pgsql=/usr/local/pgsql \
        && docker-php-ext-install pdo pdo_pgsql pgsql opcache zip
    
    RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
    
    RUN composer global require "laravel/lumen-installer"
    ENV PATH ~/.composer/vendor/bin:$PATH
    
    COPY ./backend /code/src
    

    And basically that’s all. Here the full docker-compose file

    version: '3.4'
    
    services:
      nginx:
        image: gonzalo123.nginx
        restart: always
        ports:
        - "8080:80"
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-nginx
        volumes:
        - ./src/backend:/code/src
        - ./src/.docker/web/site.conf:/etc/nginx/conf.d/default.conf
        networks:
        - api-network
      api:
        image: gonzalo123.api
        restart: always
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-lumen-dev
        environment:
          XDEBUG_CONFIG: remote_host=${MY_IP}
        volumes:
        - ./src/backend:/code/src
        networks:
        - api-network
      ui5:
        image: gonzalo123.ui5
        ports:
        - "8000:8000"
        restart: always
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-ui5
        networks:
        - api-network
    
    networks:
      api-network:
        driver: bridge
    

    If we want to use this project you only need to:

    • clone the repo fron github
    • run ./ui5 up

    With this configuration we’re exposing two ports 8080 for the frontend and 8000 for the backend. We also are mapping our local filesystem to containers to avoid to regenerate our containers each time we change the code.

    We also can have a variation. A “production” version of our docker-compose file. I put production between quotation marks because normally we aren’t going to use localneo as a production server (please don’t do it). We’ll use SCP to host the frontend.

    This configuration is just an example without filesystem mapping, without xdebug in the backend and without exposing the backend externally (Only the frontend can use it)

    version: '3.4'
    
    services:
      nginx:
        image: gonzalo123.nginx
        restart: always
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-nginx
        networks:
        - api-network
      api:
        image: gonzalo123.api
        restart: always
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-lumen
        networks:
        - api-network
      ui5:
        image: gonzalo123.ui5
        ports:
        - "8000:8000"
        restart: always
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-ui5
        networks:
        - api-network
    
    networks:
      api-network:
        driver: bridge
    

    And that’s all. You can see the all the source code in my github account

    Happy logins. Only the happy user will pass

    Login forms are bored. In this example we’re going to create an especial login form. Only for happy users. Happiness is something complicated, but at least, one smile is more easy to obtain, and all is better with one smile :). Our login form will only appear if the user smiles. Let’s start.

    I must admit that this project is just an excuse to play with different technologies that I wanted to play. Weeks ago I discovered one library called face_classification. With this library I can perform emotion classification from a picture. The idea is simple. We create RabbitMQ RPC server script that answers with the emotion of the face within a picture. Then we obtain on frame from the video stream of the webcam (with HTML5) and we send this frame using websocket to a socket.io server. This websocket server (node) ask to the RabbitMQ RPC the emotion and it sends back to the browser the emotion and a the original picture with a rectangle over the face.

    Frontend

    As well as we’re going to use socket.io for websockets we will use the same script to serve the frontend (the login and the HTML5 video capture)

    <!doctype html>
    <html>
    <head>
        <title>Happy login</title>
        <link rel="stylesheet" href="css/app.css">
    </head>
    <body>
    
    <div id="login-page" class="login-page">
        <div class="form">
            <h1 id="nonHappy" style="display: block;">Only the happy user will pass</h1>
            <form id="happyForm" class="login-form" style="display: none" onsubmit="return false;">
                <input id="user" type="text" placeholder="username"/>
                <input id="pass" type="password" placeholder="password"/>
                <button id="login">login</button>
                <p></p>
                <img id="smile" width="426" height="320" src=""/>
            </form>
            <div id="video">
                <video style="display:none;"></video>
                <canvas id="canvas" style="display:none"></canvas>
                <canvas id="canvas-face" width="426" height="320"></canvas>
            </div>
        </div>
    </div>
    
    <div id="private" style="display: none;">
        <h1>Private page</h1>
    </div>
    
    <script src="https://code.jquery.com/jquery-3.2.1.min.js" integrity="sha256-hwg4gsxgFZhOsEEamdOYGBf13FyQuiTwlAQgxVSNgt4=" crossorigin="anonymous"></script>
    <script src="https://unpkg.com/sweetalert/dist/sweetalert.min.js"></script>
    <script type="text/javascript" src="/socket.io/socket.io.js"></script>
    <script type="text/javascript" src="/js/app.js"></script>
    </body>
    </html>
    

    Here we’ll connect to the websocket and we’ll emit the webcam frame to the server. We´ll also be listening to one event called ‘response’ where server will notify us when one emotion has been detected.

    let socket = io.connect(location.origin),
        img = new Image(),
        canvasFace = document.getElementById('canvas-face'),
        context = canvasFace.getContext('2d'),
        canvas = document.getElementById('canvas'),
        width = 640,
        height = 480,
        delay = 1000,
        jpgQuality = 0.6,
        isHappy = false;
    
    socket.on('response', function (r) {
        let data = JSON.parse(r);
        if (data.length > 0 && data[0].hasOwnProperty('emotion')) {
            if (isHappy === false && data[0]['emotion'] === 'happy') {
                isHappy = true;
                swal({
                    title: "Good!",
                    text: "All is better with one smile!",
                    icon: "success",
                    buttons: false,
                    timer: 2000,
                });
    
                $('#nonHappy').hide();
                $('#video').hide();
                $('#happyForm').show();
                $('#smile')[0].src = 'data:image/png;base64,' + data[0].image;
            }
    
            img.onload = function () {
                context.drawImage(this, 0, 0, canvasFace.width, canvasFace.height);
            };
    
            img.src = 'data:image/png;base64,' + data[0].image;
        }
    });
    
    navigator.getMedia = (navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia);
    
    navigator.getMedia({video: true, audio: false}, (mediaStream) => {
        let video = document.getElementsByTagName('video')[0];
        video.src = window.URL.createObjectURL(mediaStream);
        video.play();
        setInterval(((video) => {
            return function () {
                let context = canvas.getContext('2d');
                canvas.width = width;
                canvas.height = height;
                context.drawImage(video, 0, 0, width, height);
                socket.emit('img', canvas.toDataURL('image/jpeg', jpgQuality));
            }
        })(video), delay)
    }, error => console.log(error));
    
    $(() => {
        $('#login').click(() => {
            $('#login-page').hide();
            $('#private').show();
        })
    });
    

    Backend
    Finally we’ll work in the backend. Basically I’ve check the examples that we can see in face_classification project and tune it a bit according to my needs.

    from rabbit import builder
    import logging
    import numpy as np
    from keras.models import load_model
    from utils.datasets import get_labels
    from utils.inference import detect_faces
    from utils.inference import draw_text
    from utils.inference import draw_bounding_box
    from utils.inference import apply_offsets
    from utils.inference import load_detection_model
    from utils.inference import load_image
    from utils.preprocessor import preprocess_input
    import cv2
    import json
    import base64
    
    detection_model_path = 'trained_models/detection_models/haarcascade_frontalface_default.xml'
    emotion_model_path = 'trained_models/emotion_models/fer2013_mini_XCEPTION.102-0.66.hdf5'
    emotion_labels = get_labels('fer2013')
    font = cv2.FONT_HERSHEY_SIMPLEX
    
    # hyper-parameters for bounding boxes shape
    emotion_offsets = (20, 40)
    
    # loading models
    face_detection = load_detection_model(detection_model_path)
    emotion_classifier = load_model(emotion_model_path, compile=False)
    
    # getting input model shapes for inference
    emotion_target_size = emotion_classifier.input_shape[1:3]
    
    
    def format_response(response):
        decoded_json = json.loads(response)
        return "Hello {}".format(decoded_json['name'])
    
    
    def on_data(data):
        f = open('current.jpg', 'wb')
        f.write(base64.decodebytes(data))
        f.close()
        image_path = "current.jpg"
    
        out = []
        # loading images
        rgb_image = load_image(image_path, grayscale=False)
        gray_image = load_image(image_path, grayscale=True)
        gray_image = np.squeeze(gray_image)
        gray_image = gray_image.astype('uint8')
    
        faces = detect_faces(face_detection, gray_image)
        for face_coordinates in faces:
            x1, x2, y1, y2 = apply_offsets(face_coordinates, emotion_offsets)
            gray_face = gray_image[y1:y2, x1:x2]
    
            try:
                gray_face = cv2.resize(gray_face, (emotion_target_size))
            except:
                continue
    
            gray_face = preprocess_input(gray_face, True)
            gray_face = np.expand_dims(gray_face, 0)
            gray_face = np.expand_dims(gray_face, -1)
            emotion_label_arg = np.argmax(emotion_classifier.predict(gray_face))
            emotion_text = emotion_labels[emotion_label_arg]
            color = (0, 0, 255)
    
            draw_bounding_box(face_coordinates, rgb_image, color)
            draw_text(face_coordinates, rgb_image, emotion_text, color, 0, -50, 1, 2)
            bgr_image = cv2.cvtColor(rgb_image, cv2.COLOR_RGB2BGR)
    
            cv2.imwrite('predicted.png', bgr_image)
            data = open('predicted.png', 'rb').read()
            encoded = base64.encodebytes(data).decode('utf-8')
            out.append({
                'image': encoded,
                'emotion': emotion_text,
            })
    
        return out
    
    logging.basicConfig(level=logging.WARN)
    rpc = builder.rpc("image.check", {'host': 'localhost', 'port': 5672})
    rpc.server(on_data)
    

    Here you can see in action the working prototype

    Maybe we can do the same with another tools and even more simple but as I said before this example is just an excuse to play with those technologies:

    • Send webcam frames via websockets
    • Connect one web application to a Pyhon application via RabbitMQ RPC
    • Play with face classification script

    Please don’t use this script in production. It’s just a proof of concepts. With smiles but a proof of concepts 🙂

    You can see the project in my github account

    Pomodoro with ESP32. One “The Melee – Side by side” project

    Last weekend there was a great event called The Melee – Side by side (Many thanks to @ojoven and @diversius).

    The event was one kind of Hackathon where a group of people meet together one day, to share our side projects and to work together (yes. We also have a lunch and beers also :). The format of the event is just a copy of the event that our colleagues from Bilbao called “El Comité“.

    @ibaiimaz spoke about one project to create one collaborative pomodoro where the people of one team can share their status and see the status of the rest of the team. When I heard pomodoro and status I immediately thought in one servo moving a flag and some LEDs turning on and off. We had a project. @penniath and @tatai also joined us. We also had a team.

    We had a project and we also had a deadline. We must show a working prototype at the end of the day. That means that we didn’t have too many time. First we decided the mockup of the project, reducing the initial scope (more ambitious) to fit it within our time slot. We discuss intensely for 10 minutes and finally we describe an ultra detailed blueprint. That’s the full blueprint of the project:

    It was time to start working.

    @penniath and @tatai worked in the Backend. It must be the responsible of the pomodoro timers, listen to MQTT events and create an API for the frontend. The backend also must provide a WebSockets interface to allow real time events within the frontend. They decided to use node and socket.io for the WebSockets. You can see the source code here.

    @ibaiimaz started with the frontend. He decided to create an Angular web application listening to socket.io events to show the status of the pomodoro. You can see the source code here.

    Finaly I worked with the hardware. I created a prototype with one ESP32, two RGB LEDs, one button, one servo and a couple of resistors.

    That’s the source code.

    #include <WiFi.h>
    #include <PubSubClient.h>
    
    int redPin_g = 19;
    int greenPin_g = 17;
    int bluePin_g = 18;
    
    int redPin_i = 21;
    int greenPin_i = 2;
    int bluePin_i = 4;
    
    #define SERVO_PIN 16
    
    const int buttonPin = 15;
    int buttonState = 0;
    
    int channel = 1;
    int hz = 50;
    int depth = 16;
    
    const char* ssid = "SSID";
    const char* password = "password";
    const char* server = "192.168.1.105";
    const char* topic = "/pomodoro/+";
    const char* clientName = "com.gonzalo123.esp32";
    
    WiFiClient wifiClient;
    PubSubClient client(wifiClient);
    
    void wifiConnect() {
      Serial.print("Connecting to ");
      Serial.println(ssid);
    
      WiFi.begin(ssid, password);
    
      while (WiFi.status() != WL_CONNECTED) {
        delay(500);
        Serial.print("*");
      }
    
      Serial.print("WiFi connected: ");
      Serial.println(WiFi.localIP());
    }
    
    void mqttReConnect() {
      while (!client.connected()) {
        Serial.print("Attempting MQTT connection...");
        if (client.connect(clientName)) {
          Serial.println("connected");
          client.subscribe(topic);
        } else {
          Serial.print("failed, rc=");
          Serial.print(client.state());
          Serial.println(" try again in 5 seconds");
          delay(5000);
        }
      }
    }
    
    void callback(char* topic, byte* payload, unsigned int length) {
      Serial.print("Message arrived [");
      Serial.print(topic);
    
      String data;
      for (int i = 0; i < length; i++) {
        data += (char)payload[i];
      }
    
      int value = data.toInt();
    
      if (strcmp(topic, "/pomodoro/gonzalo") == 0) {
        Serial.print("[gonzalo]");
        switch (value) {
          case 1:
            ledcWrite(1, 3400);
            setColor_g(0, 255, 0);
            break;
          case 2:
            setColor_g(255, 0, 0);
            break;
          case 3:
            ledcWrite(1, 6400);
            setColor_g(0, 0, 255);
            break;
        }
      } else {
        Serial.print("[ibai]");
        switch (value) {
          case 1:
            setColor_i(0, 255, 0);
            break;
          case 2:
            setColor_i(255, 0, 0);
            break;
          case 3:
            setColor_i(0, 0, 255);  // green
            break;
        }
      }
    
      Serial.print("] value:");
      Serial.println(data);
    }
    
    void setup()
    {
      Serial.begin(115200);
    
      pinMode(buttonPin, INPUT_PULLUP);
      pinMode(redPin_g, OUTPUT);
      pinMode(greenPin_g, OUTPUT);
      pinMode(bluePin_g, OUTPUT);
    
      pinMode(redPin_i, OUTPUT);
      pinMode(greenPin_i, OUTPUT);
      pinMode(bluePin_i, OUTPUT);
    
      ledcSetup(channel, hz, depth);
      ledcAttachPin(SERVO_PIN, channel);
      wifiConnect();
      client.setServer(server, 1883);
      client.setCallback(callback);
    
      delay(1500);
    }
    
    void mqttEmit(String topic, String value)
    {
      client.publish((char*) topic.c_str(), (char*) value.c_str());
    }
    
    void loop()
    {
      if (!client.connected()) {
        mqttReConnect();
      }
    
      client.loop();
    
      buttonState = digitalRead(buttonPin);
      if (buttonState == HIGH) {
        mqttEmit("/start/gonzalo", (String) "3");
      }
    
      delay(200);
    }
    
    void setColor_i(int red, int green, int blue)
    {
      digitalWrite(redPin_i, red);
      digitalWrite(greenPin_i, green);
      digitalWrite(bluePin_i, blue);
    }
    
    void setColor_g(int red, int green, int blue)
    {
      digitalWrite(redPin_g, red);
      digitalWrite(greenPin_g, green);
      digitalWrite(bluePin_g, blue);
    }
    

    The MQTT server (a mosquitto server) was initially running in my laptop but as well as I had one Raspberry Pi Zero also in my bag we decided to user the Pi Zero as a server and run mosquitto MQTT server with Raspbian. Everything is better with a Raspberry Pi. @tatai helped me to set up the server.

    Here you can see the prototype in action

    That’s the kind of side projects that I normally create alone but definitely it’s more fun to do it with other colleagues even it I need to wake up early one Saturday morning.

    Source code of ESP32 here.

    Playing with Ionic, Lumen, Firebase, Google maps, Raspberry Pi and background geolocation

    I wanna do a simple pet project. The idea is to build a mobile application. This application will track my GPS location and send this information to a Firebase database. I’ve never play with Firebase and I want to learn a little bit. With this information I will build a simple web application hosted in my Raspberry Pi. This web application will show a Google map with my last location. I will put this web application in my TV and anyone in my house will see where I am every time.

    That’s the idea. I want a MVP. First the mobile application. I will use ionic framework. I’m big fan of ionic.

    The mobile application is very simple. It only has a toggle to activate-deactivate the background geolocation (sometimes I don’t want to be tracked :).

    <ion-header>
        <ion-navbar>
            <ion-title>
                Ionic Blank
            </ion-title>
        </ion-navbar>
    </ion-header>
    
    <ion-header>
        <ion-toolbar [color]="toolbarColor">
            <ion-title>{{title}}</ion-title>
            <ion-buttons end>
                <ion-toggle color="light"
                            checked="{{isBgEnabled}}"
                            (ionChange)="changeWorkingStatus($event)">
                </ion-toggle>
            </ion-buttons>
        </ion-toolbar>
    </ion-header>
    
    <ion-content padding>
    </ion-content>
    

    And the controller:

    import {Component} from '@angular/core';
    import {Platform} from 'ionic-angular';
    import {LocationTracker} from "../../providers/location-tracker/location-tracker";
    
    @Component({
        selector: 'page-home',
        templateUrl: 'home.html'
    })
    export class HomePage {
        public status: string = localStorage.getItem('status') || "-";
        public title: string = "";
        public isBgEnabled: boolean = false;
        public toolbarColor: string;
    
        constructor(platform: Platform,
                    public locationTracker: LocationTracker) {
    
            platform.ready().then(() => {
    
                    if (localStorage.getItem('isBgEnabled') === 'on') {
                        this.isBgEnabled = true;
                        this.title = "Working ...";
                        this.toolbarColor = 'secondary';
                    } else {
                        this.isBgEnabled = false;
                        this.title = "Idle";
                        this.toolbarColor = 'light';
                    }
            });
        }
    
        public changeWorkingStatus(event) {
            if (event.checked) {
                localStorage.setItem('isBgEnabled', "on");
                this.title = "Working ...";
                this.toolbarColor = 'secondary';
                this.locationTracker.startTracking();
            } else {
                localStorage.setItem('isBgEnabled', "off");
                this.title = "Idle";
                this.toolbarColor = 'light';
                this.locationTracker.stopTracking();
            }
        }
    }
    

    As you can see, the toggle button will activate-deactivate the background geolocation and it also changes de background color of the toolbar.

    For background geolocation I will use one cordova plugin available as ionic native plugin

    Here you can see read a very nice article explaining how to use the plugin with ionic. As the article explains I’ve created a provider

    import {Injectable, NgZone} from '@angular/core';
    import {BackgroundGeolocation} from '@ionic-native/background-geolocation';
    import {CONF} from "../conf/conf";
    
    @Injectable()
    export class LocationTracker {
        constructor(public zone: NgZone,
                    private backgroundGeolocation: BackgroundGeolocation) {
        }
    
        showAppSettings() {
            return this.backgroundGeolocation.showAppSettings();
        }
    
        startTracking() {
            this.startBackgroundGeolocation();
        }
    
        stopTracking() {
            this.backgroundGeolocation.stop();
        }
    
        private startBackgroundGeolocation() {
            this.backgroundGeolocation.configure(CONF.BG_GPS);
            this.backgroundGeolocation.start();
        }
    }
    

    The idea of the plugin is send a POST request to a url with the gps data in the body of the request. So, I will create a web api server to handle this request. I will use my Raspberry Pi3. to serve the application. I will create a simple PHP/Lumen application. This application will handle the POST request of the mobile application and also will serve a html page with the map (using google maps).

    Mobile requests will be authenticated with a token in the header and web application will use a basic http authentication. Because of that I will create two middlewares to handle the the different ways to authenticate.

    <?php
    require __DIR__ . '/../vendor/autoload.php';
    
    use App\Http\Middleware;
    use App\Model\Gps;
    use Illuminate\Contracts\Debug\ExceptionHandler;
    use Illuminate\Http\Request;
    use Laravel\Lumen\Application;
    use Laravel\Lumen\Routing\Router;
    
    (new Dotenv\Dotenv(__DIR__ . '/../env/'))->load();
    
    $app = new Application(__DIR__ . '/..');
    $app->singleton(ExceptionHandler::class, App\Exceptions\Handler::class);
    $app->routeMiddleware([
        'auth'  => Middleware\AuthMiddleware::class,
        'basic' => Middleware\BasicAuthMiddleware::class,
    ]);
    
    $app->router->group(['middleware' => 'auth', 'prefix' => '/locator'], function (Router $route) {
        $route->post('/gps', function (Gps $gps, Request $request) {
            $requestData = $request->all();
            foreach ($requestData as $poi) {
                $gps->persistsData([
                    'date'             => date('YmdHis'),
                    'serverTime'       => time(),
                    'time'             => $poi['time'],
                    'latitude'         => $poi['latitude'],
                    'longitude'        => $poi['longitude'],
                    'accuracy'         => $poi['accuracy'],
                    'speed'            => $poi['speed'],
                    'altitude'         => $poi['altitude'],
                    'locationProvider' => $poi['locationProvider'],
                ]);
            }
    
            return 'OK';
        });
    });
    
    return $app;
    

    As we can see the route /locator/gps will handle the post request. I’ve created a model to persists gps data in the firebase database:

    <?php
    
    namespace App\Model;
    
    use Kreait\Firebase\Factory;
    use Kreait\Firebase\ServiceAccount;
    
    class Gps
    {
        private $database;
    
        private const FIREBASE_CONF = __DIR__ . '/../../conf/firebase.json';
    
        public function __construct()
        {
            $serviceAccount = ServiceAccount::fromJsonFile(self::FIREBASE_CONF);
            $firebase       = (new Factory)
                ->withServiceAccount($serviceAccount)
                ->create();
    
            $this->database = $firebase->getDatabase();
        }
    
        public function getLast()
        {
            $value = $this->database->getReference('gps/poi')
                ->orderByKey()
                ->limitToLast(1)
                ->getValue();
    
            $out                 = array_values($value)[0];
            $out['formatedDate'] = \DateTimeImmutable::createFromFormat('YmdHis', $out['date'])->format('d/m/Y H:i:s');
    
            return $out;
        }
    
        public function persistsData(array $data)
        {
            return $this->database
                ->getReference('gps/poi')
                ->push($data);
        }
    }
    

    The project is almost finished. Now we only need to create the google map.

    That’s the api

    <?php
    $app->router->group(['middleware' => 'basic', 'prefix' => '/map'], function (Router $route) {
        $route->get('/', function (Gps $gps) {
            return view("index", $gps->getLast());
        });
    
        $route->get('/last', function (Gps $gps) {
            return $gps->getLast();
        });
    });
    

    And the HTML

    <!DOCTYPE html>
    <html>
    <head>
        <meta name="viewport" content="initial-scale=1.0, user-scalable=no">
        <meta charset="utf-8">
        <title>Locator</title>
        <style>
            #map {
                height: 100%;
            }
    
            html, body {
                height: 100%;
                margin: 0;
                padding: 0;
            }
        </style>
    </head>
    <body>
    <div id="map"></div>
    <script>
    
        var lastDate;
        var DELAY = 60;
    
        function drawMap(lat, long, text) {
            var CENTER = {lat: lat, lng: long};
            var contentString = '<div id="content">' + text + '</div>';
            var infowindow = new google.maps.InfoWindow({
                content: contentString
            });
            var map = new google.maps.Map(document.getElementById('map'), {
                zoom: 11,
                center: CENTER,
                disableDefaultUI: true
            });
    
            var marker = new google.maps.Marker({
                position: CENTER,
                map: map
            });
            var trafficLayer = new google.maps.TrafficLayer();
    
            trafficLayer.setMap(map);
            infowindow.open(map, marker);
        }
    
        function initMap() {
            lastDate = '{{ $formatedDate }}';
            drawMap({{ $latitude }}, {{ $longitude }}, lastDate);
        }
    
        setInterval(function () {
            fetch('/map/last', {credentials: "same-origin"}).then(function (response) {
                response.json().then(function (data) {
                    if (lastDate !== data.formatedDate) {
                        drawMap(data.latitude, data.longitude, data.formatedDate);
                    }
                });
            });
        }, DELAY * 1000);
    </script>
    <script async defer src="https://maps.googleapis.com/maps/api/js?key=my_google_maps_key&callback=initMap">
    </script>
    </body>
    </html>
    

    And that’s all just enough for a weekend. Source code is available in my github account

    Authenticate OpenUI5 applications and Lumen backends with Amazon Cognito and JWT

    Today I want to create an UI5/OpenUI5 boilerplate that plays with Lumen backends. Simple, isn’t it? We only need to create a Lumen API server and connect our OpenUI5 application with this API server. But today I also want to create a Login also. The typical user/password input form. I don’t want to build it from scratch (a user database, oauth provider or something like that). Since this days I’m involved with Amazon AWS projects I want to try Amazon Cognito.

    Cognito has a great javaScript SDK. In fact we can do all the authentication flow (create users, validate passwords, change password, multifactor authentication, …) with Cognito. To create this project first I’ve create the following steps within Amazon AWS Cognito Console: Create a user pool with required attributes (email only in this example), without MFA and only allow administrators to create users. I’ve also created a App client inside this pool, so I’ve got a UserPoolId and a ClientId.

    Let’s start with the OpenUI5 application. I’ve created an small application with one route called “home”. To handle the login process I will work in Component.js init function. The idea is check the cognito session. If there’s an active one (that’s means a Json Web Token stored in the local storage) we’ll display to “home” route and if there isn’t we’ll show login one.

    sap.ui.define([
            "sap/ui/core/UIComponent",
            "sap/ui/Device",
            "app/model/models",
            "app/model/cognito"
        ], function (UIComponent, Device, models, cognito) {
            "use strict";
    
            return UIComponent.extend("app.Component", {
    
                metadata: {
                    manifest: "json"
                },
    
                init: function () {
                    UIComponent.prototype.init.apply(this, arguments);
                    this.setModel(models.createDeviceModel(), "device");
                    this.getRouter().initialize();
    
                    var targets = this.getTargets();
                    cognito.hasSession(function (err) {
                        if (err) {
                            targets.display("login");
                            return;
                        }
                        targets.display("home");
                    });
                },
    
                /* *** */
            });
        }
    );
    

    To encapsulate the cognito operations I’ve create a model called cognito.js. It’s not perfect, but it allows me to abstract cognito stuff in the OpenUI5 application.

    sap.ui.define([
            "app/conf/env"
        ], function (env) {
            "use strict";
    
            AWSCognito.config.region = env.region;
    
            var poolData = {
                UserPoolId: env.UserPoolId,
                ClientId: env.ClientId
            };
    
            var userPool = new AWSCognito.CognitoIdentityServiceProvider.CognitoUserPool(poolData);
            var jwt;
    
            var cognito = {
                getJwt: function () {
                    return jwt;
                },
    
                hasSession: function (cbk) {
                    var cognitoUser = cognito.getCurrentUser();
                    if (cognitoUser != null) {
                        cognitoUser.getSession(function (err, session) {
                            if (err) {
                                cbk(err);
                                return;
                            }
                            if (session.isValid()) {
                                jwt = session.idToken.getJwtToken();
                                cbk(false, session)
                            } else {
                                cbk(true);
                            }
                        });
                    } else {
                        cbk(true);
                    }
                },
    
                getCurrentUser: function () {
                    return userPool.getCurrentUser();
                },
    
                signOut: function () {
                    var currentUser = cognito.getCurrentUser();
                    if (currentUser) {
                        currentUser.signOut()
                    }
                },
    
                getUsername: function () {
                    var currentUser = cognito.getCurrentUser();
                    return (currentUser) ? currentUser.username : undefined;
                },
    
                getUserData: function (user) {
                    return {
                        Username: user,
                        Pool: userPool
                    };
                },
    
                getCognitoUser: function (user) {
                    return new AWSCognito.CognitoIdentityServiceProvider.CognitoUser(cognito.getUserData(user));
                },
    
                authenticateUser: function (user, pass, cbk) {
                    var authenticationData = {
                        Username: user,
                        Password: pass
                    };
    
                    var authenticationDetails = new AWSCognito.CognitoIdentityServiceProvider.AuthenticationDetails(authenticationData);
                    var cognitoUser = new AWSCognito.CognitoIdentityServiceProvider.CognitoUser(cognito.getUserData(user));
    
                    cognitoUser.authenticateUser(authenticationDetails, cbk);
    
                    return cognitoUser;
                }
            };
    
            return cognito;
        }
    );
    

    The login route has the following xml view:

    <core:View
            xmlns:core="sap.ui.core"
            xmlns:f="sap.ui.layout.form"
            xmlns="sap.m"
            controllerName="app.controller.Login"
    >
        <Image class="bg"></Image>
        <VBox class="sapUiSmallMargin loginForm">
            <f:SimpleForm visible="{= ${/flow} === 'login' }">
                <f:toolbar>
                    <Toolbar>
                        <Title text="{i18n>Login_Title}" level="H4" titleStyle="H4"/>
                    </Toolbar>
                </f:toolbar>
                <f:content>
                    <Label text="{i18n>Login_user}"/>
                    <Input placeholder="{i18n>Login_userPlaceholder}" value="{/user}"/>
                    <Label text="{i18n>Login_pass}"/>
                    <Input type="Password" placeholder="{i18n>Login_passPlaceholder}" value="{/pass}"/>
                    <Button type="Accept" text="{i18n>OK}" press="loginPressHandle"/>
                </f:content>
            </f:SimpleForm>
            
            <f:SimpleForm visible="{= ${/flow} === 'PasswordReset' }">
                <f:toolbar>
                    <Toolbar>
                        <Title text="{i18n>Login_PasswordReset}" level="H4" titleStyle="H4"/>
                    </Toolbar>
                </f:toolbar>
                <f:content>
                    <Label text="{i18n>Login_verificationCode}"/>
                    <Input type="Number" placeholder="{i18n>Login_verificationCodePlaceholder}" value="{/verificationCode}"/>
                    <Label text="{i18n>Login_newpass}"/>
                    <Input type="Password" placeholder="{i18n>Login_newpassPlaceholder}" value="{/newPass}"/>
                    <Button type="Accept" text="{i18n>OK}" press="newPassVerificationPressHandle"/>
                </f:content>
            </f:SimpleForm>
            
            <f:SimpleForm visible="{= ${/flow} === 'newPasswordRequired' }">
                <f:toolbar>
                    <Toolbar>
                        <Title text="{i18n>Login_PasswordReset}" level="H4" titleStyle="H4"/>
                    </Toolbar>
                </f:toolbar>
                <f:content>
                    <Label text="{i18n>Login_newpass}"/>
                    <Input type="Password" placeholder="{i18n>Login_newpassPlaceholder}" value="{/newPass}"/>
                    <Button type="Accept" text="{i18n>OK}" press="newPassPressHandle"/>
                </f:content>
            </f:SimpleForm>
        </VBox>
    </core:View>
    

    It has three different stages: “login”, “PasswordReset” and “newPasswordRequired”
    “login” is the main one. In this stage the user can input his login credentials. If credentials are OK then we’ll display home route.
    The first time a user log in in the application with the password provided by the administrator, Cognito will force to change the password. Then We’ll show newPasswordRequired flow. I’m not going to explain each step. We developers prefer code than texts. That’s the code:

    sap.ui.define([
            "app/controller/BaseController",
            "sap/ui/model/json/JSONModel",
            "sap/m/MessageToast",
            "app/model/cognito"
        ], function (BaseController, JSONModel, MessageToast, cognito) {
            "use strict";
    
            var cognitoUser;
            return BaseController.extend("app.controller.Login", {
                model: {
                    user: "",
                    pass: "",
                    flow: "login",
                    verificationCode: undefined,
                    newPass: undefined
                },
    
                onInit: function () {
                    this.getView().setModel(new JSONModel(this.model));
                },
    
                newPassPressHandle: function () {
                    var that = this;
                    var targets = this.getOwnerComponent().getTargets();
                    var attributesData = {};
                    sap.ui.core.BusyIndicator.show();
                    cognitoUser.completeNewPasswordChallenge(this.model.newPass, attributesData, {
                        onFailure: function (err) {
                            sap.ui.core.BusyIndicator.hide();
                            MessageToast.show(err.message);
                        },
                        onSuccess: function (data) {
                            sap.ui.core.BusyIndicator.hide();
                            that.getModel().setProperty("/flow", "login");
                            targets.display("home");
                        }
                    })
                },
    
                newPassVerificationPressHandle: function () {
                    var that = this;
                    var targets = this.getOwnerComponent().getTargets();
                    sap.ui.core.BusyIndicator.show();
                    cognito.getCognitoUser(this.model.user).confirmPassword(this.model.verificationCode, this.model.newPass, {
                        onFailure: function (err) {
                            sap.ui.core.BusyIndicator.hide();
                            MessageToast.show(err);
                        },
                        onSuccess: function (result) {
                            sap.ui.core.BusyIndicator.hide();
                            that.getModel().setProperty("/flow", "PasswordReset");
                            targets.display("home");
                        }
                    });
                },
    
                loginPressHandle: function () {
                    var that = this;
                    var targets = this.getOwnerComponent().getTargets();
                    sap.ui.core.BusyIndicator.show();
                    cognitoUser = cognito.authenticateUser(this.model.user, this.model.pass, {
                        onSuccess: function (result) {
                            sap.ui.core.BusyIndicator.hide();
                            targets.display("home");
                        },
    
                        onFailure: function (err) {
                            sap.ui.core.BusyIndicator.hide();
                            switch (err.code) {
                                case "PasswordResetRequiredException":
                                    that.getModel().setProperty("/flow", "PasswordReset");
                                    break;
                                default:
                                    MessageToast.show(err.message);
                            }
                        },
    
                        newPasswordRequired: function (userAttributes, requiredAttributes) {
                            sap.ui.core.BusyIndicator.hide();
                            that.getModel().setProperty("/flow", "newPasswordRequired");
                        }
                    });
                }
            });
        }
    );
    

    The home route is the main one. It asumes that there’s an active Cognito session enabled.

    <mvc:View
            controllerName="app.controller.Home"
            xmlns="sap.m"
            xmlns:mvc="sap.ui.core.mvc"
            xmlns:semantic="sap.m.semantic">
        <semantic:FullscreenPage
                id="page"
                semanticRuleSet="Optimized"
                showNavButton="false"
                title="{i18n>loggedUser}: {/userName}">
            <semantic:content>
                <Panel width="auto" class="sapUiResponsiveMargin" accessibleRole="Region">
                    <headerToolbar>
                        <Toolbar height="3rem">
                            <Title text="Title"/>
                        </Toolbar>
                    </headerToolbar>
                    <content>
                        <Text text="Lorem ipsum dolor st amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat"/>
                        <Button text="{i18n>Hello}" icon="sap-icon://hello-world" press="helloPress"/>
                    </content>
                </Panel>
            </semantic:content>
            <semantic:customFooterContent>
                <Button text="{i18n>LogOff}" icon="sap-icon://visits" press="onLogOffPress"/>
            </semantic:customFooterContent>
        </semantic:FullscreenPage>
    </mvc:View>
    

    It shows the Cognito login name. It alos has a simple logff button and also one button that calls to the backend.

    sap.ui.define([
            "app/controller/BaseController",
            "sap/ui/model/json/JSONModel",
            "sap/m/MessageToast",
            "app/model/cognito",
            "app/model/api"
        ], function (BaseController, JSONModel, MessageToast, cognito, api) {
            "use strict";
    
            return BaseController.extend("app.controller.Home", {
                model: {
                    userName: ""
                },
    
                onInit: function () {
                    this.model.userName = cognito.getUsername();
                    this.getView().setModel(new JSONModel(this.model));
                },
    
                helloPress: function () {
                    api.get("/api/hi", {}, function (data) {
                        MessageToast.show("Hello user " + data.userInfo.username + " (" + data.userInfo.email + ")");
                    });
                },
    
                onLogOffPress: function () {
                    cognito.signOut();
                    this.getOwnerComponent().getTargets().display("login");
                }
            });
        }
    );
    

    To handle ajax requests I’ve create an api model. This model injects jwt inside every request.

    sap.ui.define([
        "sap/m/MessageToast",
        "app/model/cognito"
    ], function (MessageToast, cognito) {
        "use strict";
    
        var backend = "";
    
        return {
            get: function (uri, params, cb) {
                params = params || {};
                params._jwt = cognito.getJwt();
                sap.ui.core.BusyIndicator.show(1000);
    
                jQuery.ajax({
                    type: "GET",
                    contentType: "application/json",
                    data: params,
                    url: backend + uri,
                    cache: false,
                    dataType: "json",
                    async: true,
                    success: function (data, textStatus, jqXHR) {
                        sap.ui.core.BusyIndicator.hide();
                        cb(data);
                    },
                    error: function (data, textStatus, jqXHR) {
                        sap.ui.core.BusyIndicator.hide();
                        switch (data.status) {
                            case 403: // Forbidden
                                MessageToast.show('Auth error');
                                break;
                            default:
                                console.log('Error', data);
                        }
                    }
                });
            }
        };
    });
    

    That’s the frontend. Now it’s time to backend. Our Backend will be a simple Lumen server.

    use App\Http\Middleware;
    use Illuminate\Contracts\Debug\ExceptionHandler;
    use Laravel\Lumen\Application;
    
    (new Dotenv\Dotenv(__DIR__ . "/../env/"))->load();
    
    $app = new Application();
    
    $app->singleton(ExceptionHandler::class, App\Exceptions\Handler::class);
    
    $app->routeMiddleware([
        'cognito' => Middleware\AuthCognitoMiddleware::class,
    ]);
    
    $app->register(App\Providers\RedisServiceProvider::class);
    
    $app->group([
        'middleware' => 'cognito',
        'namespace'  => 'App\Http\Controllers',
    ], function (Application $app) {
        $app->get("/api/hi", "DemoController@hi");
    });
    
    $app->run();
    

    As you can see I’ve created a middelware to handle the authentication. This middleware will check the jwt provided by the frontend. We will use “spomky-labs/jose” library to validate the token.

    namespace App\Http\Middleware;
    
    use Closure;
    use Illuminate\Http\Request;
    use Jose\Factory\JWKFactory;
    use Jose\Loader;
    use Monolog\Logger;
    use Symfony\Component\Cache\Adapter\RedisAdapter;
    
    class AuthCognitoMiddleware
    {
        public function handle(Request $request, Closure $next)
        {
            try {
                $payload = $this->getPayload($request->get('_jwt'), $this->getJwtWebKeys());
                config([
                    "userInfo" => [
                        'username' => $payload['cognito:username'],
                        'email'    => $payload['email'],
                    ],
                ]);
            } catch (\Exception $e) {
                $log = app(Logger::class);
                $log->alert($e->getMessage());
    
                return response('Token Error', 403);
            }
    
            return $next($request);
        }
    
        private function getJwtWebKeys()
        {
            $url      = sprintf(
                'https://cognito-idp.%s.amazonaws.com/%s/.well-known/jwks.json',
                getenv('AWS_REGION'),
                getenv('AWS_COGNITO_POOL')
            );
            $cacheKey = sprintf('JWKFactory-Content-%s', hash('sha512', $url));
    
            $cache = app(RedisAdapter::class);
    
            $item = $cache->getItem($cacheKey);
            if (!$item->isHit()) {
                $item->set($this->getContent($url));
                $item->expiresAfter((int)getenv("TTL_JWK_CACHE"));
                $cache->save($item);
            }
    
            return JWKFactory::createFromJKU($url, false, $cache);
        }
    
        private function getPayload($accessToken, $jwtWebKeys)
        {
            $loader  = new Loader();
            $jwt     = $loader->loadAndVerifySignatureUsingKeySet($accessToken, $jwtWebKeys, ['RS256']);
            $payload = $jwt->getPayload();
    
            return $payload;
        }
    
        private function getContent($url)
        {
            $ch = curl_init();
            curl_setopt_array($ch, [
                CURLOPT_RETURNTRANSFER => true,
                CURLOPT_URL            => $url,
                CURLOPT_SSL_VERIFYPEER => true,
                CURLOPT_SSL_VERIFYHOST => 2,
            ]);
            $content = curl_exec($ch);
            curl_close($ch);
    
            return $content;
        }
    }
    

    To validate jwt Cognito tokens we need to obtain JwtWebKeys from this url

    https://cognito-idp.my_aws_region.amazonaws.com/my_aws_cognito_pool_id/.well-known/jwks.json

    That means that we need to fetch this url within every backend request, and that’s not cool. spomky-labs/jose allows us to use a cache to avoid fetch the request again and again. This cache is an instance of something that implementes the interface Psr\Cache\CacheItemPoolInterface. I’m not going to create a Cache from scratch. I’m not crazy. I’ll use symfony/cache here with a Redis adapter

    And basically that’s all. Full application in my github

    Playing with IoT, MQTT, Arduino and Raspberry Pi. Building a dashboard with OpenUI5

    I’ve been playing with MQTT in previous posts. Today I want to build a simple dashboard. Basically because I’ve got a 3.5inch display for my Raspberry Py and I want to use it. The idea is set up my Rasperry Pi as a web kiosk and display the MQTT variables in real time using websockets. Let’s start.

    Set up Raspberry Pi as a web kiosk is pretty straightforward. You only need to follow instructions detailed here. Now we will prepare the MQTT inputs. Today we’re going to reuse one example of previous post. A potentiometer controlled by a nodemcu microcontroller connected to our MQTT server via Wifi.

    We also will build another circuit using a Arduino board and a ethernet Shield.

    With this circuit we’ll register the temperature (using a LM35 temperature sensor), a photo resistor (CDS) to show the light level and a relay to switch on/off a light bulb. The Idea of the circuit is emit the temperature and light level to mosquitto mqtt server and listen to switch status form mqtt server to fire the relay. That’s the arduino code

    #include <SPI.h>
    #include <Ethernet.h>
    #include <PubSubClient.h>
    
    const int photocellPin = 1;
    const int tempPin = 0;
    const int relayPin = 9;
    bool lightStatus = false;
    
    const byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
    
    // mqtt configuration
    const char* mqttServer = "192.168.1.104";
    const int mqttPort = 1883;
    const String topicLightChange = "sensors/arduino/light/change";
    const String topicLightStatus = "sensors/arduino/light/status";
    const String topicTemp = "sensors/arduino/temperature/room1";
    const String topicLight = "sensors/arduino/light/room1";
    const char* clientName = "com.gonzalo123.arduino";
    
    EthernetClient ethClient;
    PubSubClient client(ethClient);
    
    void mqttReConnect() {
      while (!client.connected()) {
        Serial.print("Attempting MQTT connection...");
        if (client.connect(clientName)) {
          Serial.println("connected");
          client.subscribe(topicLightChange.c_str());
        } else {
          Serial.print("failed, rc=");
          Serial.print(client.state());
          Serial.println(" try again in 5 seconds");
          delay(5000);
        }
      }
    }
    
    void mqttEmit(String topic, String value) {
      if (client.publish((char*) topic.c_str(), (char*) value.c_str())) {
        //Serial.print("Publish ok (topic: ");
        //Serial.print(topic);
        //Serial.print(", value: ");
        //Serial.print(value);
        //Serial.println(")");
      } else {
        Serial.println("Publish failed");
      }
    }
    
    void callback(char* topic, byte* payload, unsigned int length) {
      Serial.print("Message arrived [");
      Serial.print(topic);
      Serial.print("] payload: ");
      String data;
      for (int i = 0; i < length; i++) {
        data += (char)payload[i];
      }
    
      if (strcmp(topic, topicLightChange.c_str()) == 0) {
        lightStatus = (data == "1") ? true : false;
        Serial.print(data);
      }
      
      Serial.println("");
    }
    
    void setup()
    {
      Serial.begin(9600);
      pinMode(relayPin, OUTPUT);
      digitalWrite(relayPin, LOW);
      
      client.setServer(mqttServer, mqttPort);
      client.setCallback(callback);
      if (Ethernet.begin(mac) == 0) {
        Serial.println("Failed to configure Ethernet using DHCP");
      }
    
      delay(1500);
    }
    
    void loop()
    {
      if (!client.connected()) {
        mqttReConnect();
      }
    
      client.loop();
    
      if (lightStatus == 1) {
        digitalWrite(relayPin, HIGH);
      } else {
        digitalWrite(relayPin, LOW);
      }
      mqttEmit(topicLightStatus, lightStatus ? "1" : "0");
      mqttEmit(topicLight, (String) analogRead(photocellPin));
      mqttEmit(topicTemp, (String) ((5.0 * analogRead(tempPin) * 100.0) / 1024.0));
    
      delay(500);
    }
    

    Now we’re going to work with dashboard. This days I’m working with OpenUI5 within various projects and because of that we’ll use this library to build the dashboard. we’ll build something like this:

    Basically it’s a view

    <mvc:View
            controllerName="gonzalo123.controller.Controller"
            height="100%"
            width="100%"
            xmlns="sap.m"
            xmlns:mvc="sap.ui.core.mvc"
            xmlns:app="http://schemas.sap.com/sapui5/extension/sap.ui.core.CustomData/1"
    >
        <IconTabBar expandable="false"
                    stretchContentHeight="true"
                    class="sapUiResponsiveContentPadding">
            <items>
                <IconTabFilter icon="sap-icon://bbyd-dashboard">
                    <TileContainer>
                        <StandardTile
                                icon="sap-icon://explorer"
                                number="{/potentiometer}"
                                numberUnit="%"
                                title="{i18n>potentiometer}"/>
                        <StandardTile
                                icon="sap-icon://temperature"
                                number="{/temperature}"
                                numberUnit="ºC"
                                title="{i18n>temperature}"/>
                        <StandardTile
                                icon="sap-icon://lightbulb"
                                number="{/light/level}"
                                title="{i18n>light}"/>
                    </TileContainer>
                </IconTabFilter>
                <IconTabFilter icon="sap-icon://lightbulb">
                    <Page showHeader="false"
                          enableScrolling="true">
                        <List>
                            <InputListItem label="{i18n>light}">
                                <Switch state="{/light/status}"
                                        change="onStatusChange"/>
                            </InputListItem>
                        </List>
                    </Page>
                </IconTabFilter>
            </items>
        </IconTabBar>
    </mvc:View>
    

    And a controller:

    sap.ui.define([
            'jquery.sap.global',
            'sap/ui/core/mvc/Controller',
            'sap/ui/model/json/JSONModel',
            "sap/ui/model/resource/ResourceModel",
            'gonzalo123/model/io'
        ],
    
        function (jQuery, Controller, JSONModel, ResourceModel, io) {
            "use strict";
    
            io.connect("//192.168.1.104:3000/");
    
            return Controller.extend("gonzalo123.controller.Controller", {
                model: new JSONModel({
                    light: {
                        status: false,
                        level: undefined
                    },
                    potentiometer: undefined,
                    temperature: undefined
                }),
    
                onInit: function () {
                    var model = this.model;
                    io.on('mqtt', function (data) {
                        switch (data.topic) {
                            case 'sensors/arduino/temperature/room1':
                                model.setProperty("/temperature", data.payload);
                                break;
                            case 'sensors/arduino/light/room1':
                                model.setProperty("/light/level", data.payload);
                                break;
                            case 'sensors/nodemcu/potentiometer/room1':
                                model.setProperty("/potentiometer", data.payload);
                                break;
                            case 'sensors/arduino/light/status':
                                model.setProperty("/light/status", data.payload == "1");
                                break;
                        }
                    });
    
                    this.getView().setModel(this.model);
    
                    var i18nModel = new ResourceModel({
                        bundleName: "gonzalo123.i18n.i18n"
                    });
    
                    this.getView().setModel(i18nModel, "i18n");
                },
    
                onStatusChange: function () {
                    io.emit('mqtt', {
                        topic: 'sensors/arduino/light/change',
                        payload: (this.getView().getModel().oData.light.status ? "1" : "0")
                    });
                }
            });
        }
    );
    

    The real time part we need a gateway between websockets and mqtt data. We’ll use socket.io. Here is the server:

    var mqtt = require('mqtt');
    var mqttClient = mqtt.connect('mqtt://192.168.1.104');
    var httpServer = require('http').createServer();
    io = require('socket.io')(httpServer, {origins: '*:*'});
    
    io.on('connection', function(client){
        client.on('mqtt', function(msg){
            console.log("ws", msg);
            mqttClient.publish(msg.topic, msg.payload.toString());
        })
    });
    
    mqttClient.on('connect', function () {
        mqttClient.subscribe('sensors/#');
    });
    
    mqttClient.on('message', function (topic, message) {
        console.log("mqtt", topic, message.toString());
        io.sockets.emit('mqtt', {
            topic: topic,
            payload: message.toString()
        });
    });
    
    httpServer.listen(3000, '0.0.0.0');
    

    Hardware

    • 1 Arduino Uno
    • 1 NodeMCU (V3)
    • 1 potentiometer
    • 1 Servo (SG90)
    • 1 Raspberry Pi 3
    • 3.5inch Display Hat for Raspberry Pi
    • LM35
    • CDS
    • pull down resistor

    Source code available in my github account

    Playing with RabbitMQ, PHP and node

    I need to use RabbitMQ in one project. I’m a big fan of Gearman, but I must admit Rabbit is much more powerful. In this project I need to handle with PHP code and node, so I want to build a wrapper for those two languages. I don’t want to re-invent the wheel so I will use existing libraries (php-amqplib and amqplib for node).

    Basically I need to use three things: First I need to create exchange channels to log different actions. I need to decouple those actions from the main code. I also need to create work queues to ensure those works are executed. It doesn’t matter if work is executed later but it must be executed. And finally RPC commands.

    Let’s start with the queues. I want to push events to a queue in PHP

    use G\Rabbit\Builder;
    $server = [
        'host' => 'localhost',
        'port' => 5672,
        'user' => 'guest',
        'pass' => 'guest',
    ];
    $queue = Builder::queue('queue.backend', $server);
    $queue->emit(["aaa" => 1]);
    

    and also with node

    var server = {
        host: 'localhost',
        port: 5672,
        user: 'guest',
        pass: 'guest'
    };
    
    var queue = builder.queue('queue.backend', server);
    queue.emit({aaa: 1});
    

    And I also want to register workers to those queues with PHP and node

    use G\Rabbit\Builder;
    $server = [
        'host' => 'localhost',
        'port' => 5672,
        'user' => 'guest',
        'pass' => 'guest',
    ];
    Builder::queue('queue.backend', $server)->receive(function ($data) {
        error_log(json_encode($data));
    });
    
    var server = {
        host: 'localhost',
        port: 5672,
        user: 'guest',
        pass: 'guest'
    };
    
    var queue = builder.queue('queue.backend', server);
    queue.receive(function (data) {
        console.log(data);
    });
    

    Both implementations use one builder. In this case we are using Queue:

    namespace G\Rabbit;
    use PhpAmqpLib\Connection\AMQPStreamConnection;
    use PhpAmqpLib\Message\AMQPMessage;
    class Queue
    {
        private $name;
        private $conf;
        public function __construct($name, $conf)
        {
            $this->name = $name;
            $this->conf = $conf;
        }
        private function createConnection()
        {
            $server = $this->conf['server'];
            return new AMQPStreamConnection($server['host'], $server['port'], $server['user'], $server['pass']);
        }
        private function declareQueue($channel)
        {
            $conf = $this->conf['queue'];
            $channel->queue_declare($this->name, $conf['passive'], $conf['durable'], $conf['exclusive'],
                $conf['auto_delete'], $conf['nowait']);
        }
        public function emit($data = null)
        {
            $connection = $this->createConnection();
            $channel = $connection->channel();
            $this->declareQueue($channel);
            $msg = new AMQPMessage(json_encode($data),
                ['delivery_mode' => 2] # make message persistent
            );
            $channel->basic_publish($msg, '', $this->name);
            $channel->close();
            $connection->close();
        }
        public function receive(callable $callback)
        {
            $connection = $this->createConnection();
            $channel = $connection->channel();
            $this->declareQueue($channel);
            $consumer = $this->conf['consumer'];
            if ($consumer['no_ack'] === false) {
                $channel->basic_qos(null, 1, null);
            }
            $channel->basic_consume($this->name, '', $consumer['no_local'], $consumer['no_ack'], $consumer['exclusive'],
                $consumer['nowait'],
                function ($msg) use ($callback) {
                    call_user_func($callback, json_decode($msg->body, true), $this->name);
                    $msg->delivery_info['channel']->basic_ack($msg->delivery_info['delivery_tag']);
                    $now = new \DateTime();
                    echo '['.$now->format('d/m/Y H:i:s')."] {$this->name}::".$msg->body, "\n";
                });
            $now = new \DateTime();
            echo '['.$now->format('d/m/Y H:i:s')."] Queue '{$this->name}' initialized \n";
            while (count($channel->callbacks)) {
                $channel->wait();
            }
            $channel->close();
            $connection->close();
        }
    }
    
    var amqp = require('amqplib/callback_api');
    
    var Queue = function (name, conf) {
        return {
            emit: function (data, close=true) {
                amqp.connect(`amqp://${conf.server.user}:${conf.server.pass}@${conf.server.host}:${conf.server.port}`, function (err, conn) {
                    conn.createChannel(function (err, ch) {
                        var msg = JSON.stringify(data);
    
                        ch.assertQueue(name, conf.queue);
                        ch.sendToQueue(name, new Buffer(msg));
                    });
                    if (close) {
                        setTimeout(function () {
                            conn.close();
                            process.exit(0)
                        }, 500);
                    }
                });
            },
            receive: function (callback) {
                amqp.connect(`amqp://${conf.server.user}:${conf.server.pass}@${conf.server.host}:${conf.server.port}`, function (err, conn) {
                    conn.createChannel(function (err, ch) {
                        ch.assertQueue(name, conf.queue);
                        console.log(new Date().toString() + ' Queue ' + name + ' initialized');
                        ch.consume(name, function (msg) {
                            console.log(new Date().toString() + " Received %s", msg.content.toString());
                            if (callback) {
                                callback(JSON.parse(msg.content.toString()), msg.fields.routingKey)
                            }
                            if (conf.consumer.noAck === false) {
                                ch.ack(msg);
                            }
                        }, conf.consumer);
                    });
                });
            }
        };
    };
    
    module.exports = Queue;
    

    We also want to emit messages using an exchange

    Emiter:

    use G\Rabbit\Builder;
    $server = [
        'host' => 'localhost',
        'port' => 5672,
        'user' => 'guest',
        'pass' => 'guest',
    ];
    $exchange = Builder::exchange('process.log', $server);
    $exchange->emit("xxx.log", "aaaa");
    $exchange->emit("xxx.log", ["11", "aaaa"]);
    $exchange->emit("yyy.log", "aaaa");
    
    var builder = require('../../src/Builder');
    
    var server = {
        host: 'localhost',
        port: 5672,
        user: 'guest',
        pass: 'guest'
    };
    
    var exchange = builder.exchange('process.log', server);
    
    exchange.emit("xxx.log", "aaaa");
    exchange.emit("xxx.log", ["11", "aaaa"]);
    exchange.emit("yyy.log", "aaaa");
    

    and receiver:

    use G\Rabbit\Builder;
    $server = [
        'host' => 'localhost',
        'port' => 5672,
        'user' => 'guest',
        'pass' => 'guest',
    ];
    Builder::exchange('process.log', $server)->receive("yyy.log", function ($routingKey, $data) {
        error_log($routingKey." - ".json_encode($data));
    });
    
    var server = {
        host: 'localhost',
        port: 5672,
        user: 'guest',
        pass: 'guest'
    };
    
    var exchange = builder.exchange('process.log', server);
    
    exchange.receive("yyy.log", function (routingKey, data) {
        console.log(routingKey, data);
    });
    

    And that’s the PHP implementation:

    namespace G\Rabbit;
    use PhpAmqpLib\Connection\AMQPStreamConnection;
    use PhpAmqpLib\Message\AMQPMessage;
    class Exchange
    {
        private $name;
        private $conf;
        public function __construct($name, $conf)
        {
            $this->name = $name;
            $this->conf = $conf;
        }
        private function createConnection()
        {
            $server = $this->conf['server'];
            return new AMQPStreamConnection($server['host'], $server['port'], $server['user'], $server['pass']);
        }
        public function emit($routingKey, $data = null)
        {
            $connection = $this->createConnection();
            $channel = $connection->channel();
            $conf = $this->conf['exchange'];
            $channel->exchange_declare($this->name, 'topic', $conf['passive'], $conf['durable'], $conf['auto_delete'],
                $conf['internal'], $conf['nowait']);
            $msg = new AMQPMessage(json_encode($data), [
                'delivery_mode' => 2, # make message persistent
            ]);
            $channel->basic_publish($msg, $this->name, $routingKey);
            $channel->close();
            $connection->close();
        }
        public function receive($bindingKey, callable $callback)
        {
            $connection = $this->createConnection();
            $channel = $connection->channel();
            $conf = $this->conf['exchange'];
            $channel->exchange_declare($this->name, 'topic', $conf['passive'], $conf['durable'], $conf['auto_delete'],
                $conf['internal'], $conf['nowait']);
            $queueConf = $this->conf['queue'];
            list($queue_name, ,) = $channel->queue_declare("", $queueConf['passive'], $queueConf['durable'],
                $queueConf['exclusive'], $queueConf['auto_delete'], $queueConf['nowait']);
            $channel->queue_bind($queue_name, $this->name, $bindingKey);
            $consumerConf = $this->conf['consumer'];
            $channel->basic_consume($queue_name, '', $consumerConf['no_local'], $consumerConf['no_ack'],
                $consumerConf['exclusive'], $consumerConf['nowait'],
                function ($msg) use ($callback) {
                    call_user_func($callback, $msg->delivery_info['routing_key'], json_decode($msg->body, true));
                    $now = new \DateTime();
                    echo '['.$now->format('d/m/Y H:i:s').'] '.$this->name.':'.$msg->delivery_info['routing_key'].'::', $msg->body, "\n";
                    $msg->delivery_info['channel']->basic_ack($msg->delivery_info['delivery_tag']);
                });
            $now = new \DateTime();
            echo '['.$now->format('d/m/Y H:i:s')."] Exchange '{$this->name}' initialized \n";
            while (count($channel->callbacks)) {
                $channel->wait();
            }
            $channel->close();
            $connection->close();
        }
    }
    

    And node:

    var amqp = require('amqplib/callback_api');
    
    var Exchange = function (name, conf) {
        return {
            emit: function (routingKey, data, close = true) {
                amqp.connect(`amqp://${conf.server.user}:${conf.server.pass}@${conf.server.host}:${conf.server.port}`, function (err, conn) {
                    conn.createChannel(function (err, ch) {
                        var msg = JSON.stringify(data);
                        ch.assertExchange(name, 'topic', conf.exchange);
                        ch.publish(name, routingKey, new Buffer(msg));
                    });
                    if (close) {
                        setTimeout(function () {
                            conn.close();
                            process.exit(0)
                        }, 500);
                    }
                });
            },
            receive: function (bindingKey, callback) {
                amqp.connect(`amqp://${conf.server.user}:${conf.server.pass}@${conf.server.host}:${conf.server.port}`, function (err, conn) {
                    conn.createChannel(function (err, ch) {
                        ch.assertExchange(name, 'topic', conf.exchange);
                        console.log(new Date().toString() + ' Exchange ' + name + ' initialized');
                        ch.assertQueue('', conf.queue, function (err, q) {
    
                            ch.bindQueue(q.queue, name, bindingKey);
    
                            ch.consume(q.queue, function (msg) {
                                console.log(new Date().toString(), name, ":", msg.fields.routingKey, "::", msg.content.toString());
                                if (callback) {
                                    callback(msg.fields.routingKey, JSON.parse(msg.content.toString()))
                                }
                                if (conf.consumer.noAck === false) {
                                    ch.ack(msg);
                                }
                            }, conf.consumer);
                        });
                    });
                });
            }
        };
    };
    
    module.exports = Exchange;
    

    Finally we want to use RPC commands. In fact RPC implementations is similar than Queue but in this case client will receive an answer.

    Client side

    use G\Rabbit\Builder;
    $server = [
        'host' => 'localhost',
        'port' => 5672,
        'user' => 'guest',
        'pass' => 'guest',
    ];
    echo Builder::rpc('rpc.hello', $server)->call("Gonzalo", "Ayuso");
    
    var builder = require('../../src/Builder');
    
    var server = {
        host: 'localhost',
        port: 5672,
        user: 'guest',
        pass: 'guest'
    };
    
    var rpc = builder.rpc('rpc.hello', server);
    rpc.call("Gonzalo", "Ayuso", function (data) {
        console.log(data);
    });
    

    Server side:

    use G\Rabbit\Builder;
    $server = [
        'host' => 'localhost',
        'port' => 5672,
        'user' => 'guest',
        'pass' => 'guest',
    ];
    Builder::rpc('rpc.hello', $server)->server(function ($name, $surname) use ($server) {
        return "Hello {$name} {$surname}";
    });
    
    var builder = require('../../src/Builder');
    
    var server = {
        host: 'localhost',
        port: 5672,
        user: 'guest',
        pass: 'guest'
    };
    
    var rpc = builder.rpc('rpc.hello', server);
    
    rpc.server(function (name, surname) {
        return "Hello " + name + " " + surname;
    });
    

    And Implementations:

    namespace G\Rabbit;
    use PhpAmqpLib\Connection\AMQPStreamConnection;
    use PhpAmqpLib\Message\AMQPMessage;
    class RPC
    {
        private $name;
        private $conf;
        public function __construct($name, $conf)
        {
            $this->name = $name;
            $this->conf = $conf;
        }
        private function createConnection()
        {
            $server = $this->conf['server'];
            return new AMQPStreamConnection($server['host'], $server['port'], $server['user'], $server['pass']);
        }
        public function call()
        {
            $params = (array)func_get_args();
            $response = null;
            $corr_id = uniqid();
            $connection = $this->createConnection();
            $channel = $connection->channel();
            $queueConf = $this->conf['queue'];
            list($callback_queue, ,) = $channel->queue_declare("", $queueConf['passive'], $queueConf['durable'],
                $queueConf['exclusive'], $queueConf['auto_delete'], $queueConf['nowait']);
            $consumerConf = $this->conf['consumer'];
            $channel->basic_consume($callback_queue, '', $consumerConf['no_local'], $consumerConf['no_ack'],
                $consumerConf['exclusive'], $consumerConf['nowait'], function ($rep) use (&$corr_id, &$response) {
                    if ($rep->get('correlation_id') == $corr_id) {
                        $response = $rep->body;
                    }
                });
            $msg = new AMQPMessage(json_encode($params), [
                'correlation_id' => $corr_id,
                'reply_to'       => $callback_queue,
            ]);
            $channel->basic_publish($msg, '', $this->name);
            while (!$response) {
                $channel->wait();
            }
            return json_decode($response, true);
        }
        public function server(callable $callback)
        {
            $connection = $this->createConnection();
            $channel = $connection->channel();
            $queueConf = $this->conf['queue'];
            $channel->queue_declare($this->name, $queueConf['passive'], $queueConf['durable'], $queueConf['exclusive'],
                $queueConf['auto_delete'], $queueConf['nowait']);
            $now = new \DateTime();
            echo '['.$now->format('d/m/Y H:i:s')."] RPC server '{$this->name}' initialized \n";
            $channel->basic_qos(null, 1, null);
            $consumerConf = $this->conf['consumer'];
            $channel->basic_consume($this->name, '', $consumerConf['no_local'], $consumerConf['no_ack'],
                $consumerConf['exclusive'],
                $consumerConf['nowait'], function ($req) use ($callback) {
                    $response = json_encode(call_user_func_array($callback, array_values(json_decode($req->body, true))));
                    $msg = new AMQPMessage($response, [
                        'correlation_id' => $req->get('correlation_id'),
                        'delivery_mode'  => 2, # make message persistent
                    ]);
                    $req->delivery_info['channel']->basic_publish($msg, '', $req->get('reply_to'));
                    $req->delivery_info['channel']->basic_ack($req->delivery_info['delivery_tag']);
                    $now = new \DateTime();
                    echo '['.$now->format('d/m/Y H:i:s').'] '.$this->name.":: req => '{$req->body}' response=> '{$response}'\n";
                });
            while (count($channel->callbacks)) {
                $channel->wait();
            }
            $channel->close();
            $connection->close();
        }
    }
    
    var amqp = require('amqplib/callback_api');
    
    var RPC = function (name, conf) {
        var generateUuid = function () {
            return Math.random().toString() +
                Math.random().toString() +
                Math.random().toString();
        };
    
        return {
            call: function () {
                var params = [];
                for (i = 0; i < arguments.length - 1; i++) {
                    params.push(arguments[i]);
                }
                var callback = arguments[arguments.length - 1];
    
                amqp.connect(`amqp://${conf.server.user}:${conf.server.pass}@${conf.server.host}:${conf.server.port}`, function (err, conn) {
                    conn.createChannel(function (err, ch) {
                        ch.assertQueue('', conf.queue, function (err, q) {
                            var corr = generateUuid();
    
                            ch.consume(q.queue, function (msg) {
                                if (msg.properties.correlationId == corr) {
                                    callback(JSON.parse(msg.content.toString()));
                                    setTimeout(function () {
                                        conn.close();
                                        process.exit(0)
                                    }, 500);
                                }
                            }, conf.consumer);
                            ch.sendToQueue(name,
                                new Buffer(JSON.stringify(params)),
                                {correlationId: corr, replyTo: q.queue});
                        });
                    });
                });
            },
            server: function (callback) {
                amqp.connect(`amqp://${conf.server.user}:${conf.server.pass}@${conf.server.host}:${conf.server.port}`, function (err, conn) {
                    conn.createChannel(function (err, ch) {
                        ch.assertQueue(name, conf.queue);
                        console.log(new Date().toString() + ' RPC ' + name + ' initialized');
                        ch.prefetch(1);
                        ch.consume(name, function reply(msg) {
                            console.log(new Date().toString(), msg.fields.routingKey, " :: ", msg.content.toString());
                            var response = JSON.stringify(callback.apply(this, JSON.parse(msg.content.toString())));
                            ch.sendToQueue(msg.properties.replyTo,
                                new Buffer(response),
                                {correlationId: msg.properties.correlationId});
    
                            ch.ack(msg);
    
                        }, conf.consumer);
                    });
                });
            }
        };
    };
    
    module.exports = RPC;
    

    You can see whole projects at github: RabbitMQ-php, RabbitMQ-node

    Playing with Docker, Silex, Python, Node and WebSockets

    I’m learning Docker. In this post I want to share a little experiment that I have done. I know the code looks like over-engineering but it’s just an excuse to build something with docker and containers. Let me explain it a little bit.

    The idea is build a Time clock in the browser. Something like this:

    Clock

    Yes I know. We can do it only with js, css and html but we want to hack a little bit more. The idea is to create:

    • A Silex/PHP frontend
    • A WebSocket server with socket.io/node
    • A Python script to obtain the current time

    WebSocket server will open 2 ports: One port to serve webSockets (socket.io) and another one as a http server (express). Python script will get the current time and it’ll send it to the webSocket server. Finally one frontend(silex) will be listening to WebSocket’s event and it will render the current time.

    That’s the WebSocket server (with socket.io and express)

    var
        express = require('express'),
        expressApp = express(),
        server = require('http').Server(expressApp),
        io = require('socket.io')(server, {origins: 'localhost:*'})
        ;
    
    expressApp.get('/tic', function (req, res) {
        io.sockets.emit('time', req.query.time);
        res.json('OK');
    });
    
    expressApp.listen(6400, '0.0.0.0');
    
    server.listen(8080);
    

    That’s our Python script

    from time import gmtime, strftime, sleep
    import httplib2
    
    h = httplib2.Http()
    while True:
        (resp, content) = h.request("http://node:6400/tic?time=" + strftime("%H:%M:%S", gmtime()))
        sleep(1)
    

    And our Silex frontend

    use Silex\Application;
    use Silex\Provider\TwigServiceProvider;
    
    $app = new Application(['debug' => true]);
    $app->register(new TwigServiceProvider(), [
        'twig.path' => __DIR__ . '/../views',
    ]);
    
    $app->get("/", function (Application $app) {
        return $app['twig']->render('index.twig', []);
    });
    
    $app->run();
    

    using this twig template

    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="utf-8">
        <meta http-equiv="X-UA-Compatible" content="IE=edge">
        <meta name="viewport" content="width=device-width, initial-scale=1">
        <title>Docker example</title>
        <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">
        <link href="css/app.css" rel="stylesheet">
        <script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
        <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
    </head>
    <body>
    <div class="site-wrapper">
        <div class="site-wrapper-inner">
            <div class="cover-container">
                <div class="inner cover">
                    <h1 class="cover-heading">
                        <div id="display">
                            display
                        </div>
                    </h1>
                </div>
            </div>
        </div>
    </div>
    <script src="//localhost:8080/socket.io/socket.io.js"></script>
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
    <script>
    var socket = io.connect('//localhost:8080');
    
    $(function () {
        socket.on('time', function (data) {
            $('#display').html(data);
        });
    });
    </script>
    </body>
    </html>
    

    The idea is to use one Docker container for each process. I like to have all the code in one place so all containers will share the same volume with source code.

    First the node container (WebSocket server)

    FROM node:argon
    
    RUN mkdir -p /mnt/src
    WORKDIR /mnt/src/node
    
    EXPOSE 8080 6400
    

    Now the python container

    FROM python:2
    
    RUN pip install httplib2
    
    RUN mkdir -p /mnt/src
    WORKDIR /mnt/src/python
    

    And finally Frontend contailer (apache2 with Ubuntu 16.04)

    FROM ubuntu:16.04
    
    RUN locale-gen es_ES.UTF-8
    RUN update-locale LANG=es_ES.UTF-8
    ENV DEBIAN_FRONTEND=noninteractive
    
    RUN apt-get update -y
    RUN apt-get install --no-install-recommends -y apache2 php libapache2-mod-php
    RUN apt-get clean -y
    
    COPY ./apache2/sites-available/000-default.conf /etc/apache2/sites-available/000-default.conf
    
    RUN mkdir -p /mnt/src
    
    RUN a2enmod rewrite
    RUN a2enmod proxy
    RUN a2enmod mpm_prefork
    
    RUN chown -R www-data:www-data /mnt/src
    ENV APACHE_RUN_USER www-data
    ENV APACHE_RUN_GROUP www-data
    ENV APACHE_LOG_DIR /var/log/apache2
    ENV APACHE_LOCK_DIR /var/lock/apache2
    ENV APACHE_PID_FILE /var/run/apache2/apache2.pid
    ENV APACHE_SERVERADMIN admin@localhost
    ENV APACHE_SERVERNAME localhost
    
    EXPOSE 80
    

    Now we’ve got the three containers but we want to use all together. We’ll use a docker-compose.yml file. The web container will expose port 80 and node container 8080. Node container also opens 6400 but this port is an internal port. We don’t need to access to this port outside. Only Python container needs to access to this port. Because of that 6400 is not mapped to any port in docker-compose

    version: '2'
    
    services:
      web:
        image: gonzalo123/example_web
        container_name: example_web
        ports:
         - "80:80"
        restart: always
        depends_on:
          - node
        build:
          context: ./images/php
          dockerfile: Dockerfile
        entrypoint:
          - /usr/sbin/apache2
          - -D
          - FOREGROUND
        volumes:
         - ./src:/mnt/src
    
      node:
        image: gonzalo123/example_node
        container_name: example_node
        ports:
         - "8080:8080"
        restart: always
        build:
          context: ./images/node
          dockerfile: Dockerfile
        entrypoint:
          - npm
          - start
        volumes:
         - ./src:/mnt/src
    
      python:
          image: gonzalo123/example_python
          container_name: example_python
          restart: always
          depends_on:
            - node
          build:
            context: ./images/python
            dockerfile: Dockerfile
          entrypoint:
            - python
            - tic.py
          volumes:
           - ./src:/mnt/src
    

    And that’s all. We only need to start our containers

    docker-compose up --build -d
    

    and open our browser at: http://localhost to see our Time clock

    Full source code available within my github account

    Playing with arduino, IoT, crossbar and websockets

    Yes. Finally I’ve got an arduino board. It’s time to hack a little bit. Today I want to try different things. I want to display in a webpage one value from my arduino board. For example one analog data using a potentiometer. Let’s start.

    We are going to use one potentiometer. A potentiometer is a resistor with a rotating contact that forms an adjustable voltage divider. It has three pins. If we connect one pin to 5V power source of our arduino, another one to the ground and another to one A0 (analog input 0), we can read different values depending on the position of potentiometer’s rotating contact.

    arduino_analog

    Arduino has 10 bit analog resolution. That means 1024 possible values, from 0 to 1023. So when our potentiometer gives us 5 volts we’ll obtain 1024 and when our it gives us 0V we’ll read 0. Here we can see a simple arduino program to read this analog input and send data via serial port:

    int mem;
    
    void setup() {
      Serial.begin(9600);
    }
    
    void loop() {
      int value = analogRead(A0);
      if (value != mem) {
        Serial.println(value);
      }
      mem = value;
    
      delay(100);
    }
    

    This program is simple loop with a delay of 100 milliseconds that reads A0 and if value is different than previously read (to avoid sending the same value when nobody is touching the potentiometer) we send the value via serial port (with 9600 bauds)

    We can test our program using the serial monitor of our arduino IDE our using another serial monitor.

    Now we’re going to create one script to read this serial port data. We’re going to use Python. I’ll use my laptop and my serial port is /dev/tty.usbmodem14231

    import serial
    
    arduino = serial.Serial('/dev/tty.usbmodem14231', 9600)
    
    while 1:
      print arduino.readline().strip()
    

    Basically we’ve got our backend running. Now we can create a simple frontend.

    ...
    <div id='display'></div>
    ...
    

    We’ll need websockets. I normally use socket.io but today I’ll use Crossbar.io. Since I hear about it in a Ronny’s talk at deSymfony conference I wanted to use it.

    I’ll change a little bit our backend to emit one event

    import serial
    from os import environ
    from twisted.internet.defer import inlineCallbacks
    from twisted.internet.task import LoopingCall
    from autobahn.twisted.wamp import ApplicationSession, ApplicationRunner
    
    arduino = serial.Serial('/dev/tty.usbmodem14231', 9600)
    
    class SeriaReader(ApplicationSession):
        @inlineCallbacks
        def onJoin(self, details):
            def publish():
                return self.publish(u'iot.serial.reader', arduino.readline().strip())
    
            yield LoopingCall(publish).start(0.1)
    
    if __name__ == '__main__':
        runner = ApplicationRunner(environ.get("GONZALO_ROUTER", u"ws://127.0.0.1:8080/ws"), u"iot")
        runner.run(SeriaReader)
    

    Now I only need to create a crossbar.io server. I will use node to do it

    var autobahn = require('autobahn'),
        connection = new autobahn.Connection({
                url: 'ws://0.0.0.0:8080/ws',
                realm: 'iot'
            }
        );
    
    connection.open();
    

    And now we only need to connect our frontend to the websocket server

    $(function () {
        var connection = new autobahn.Connection({
            url: "ws://192.168.1.104:8080/ws",
            realm: "iot"
        });
    
        connection.onopen = function (session) {
            session.subscribe('iot.serial.reader', function (args) {
                $('#display').html(args[0]);
            });
        };
    
        connection.open();
    });
    

    It works but thre’s a problem. The first time we connect with our browser we won’t see the display value until we change the position of the potentiometer. That’s because ‘iot.serial.reader’ event is only emitted when potentiometer changes. No change means no new value. To solve this problem we only need to change a little bit our crossbar.io server. We’ll “memorize” the last value and we’ll expose one method ‘iot.serial.get’ to ask about this value

    var autobahn = require('autobahn'),
        connection = new autobahn.Connection({
                url: 'ws://0.0.0.0:8080/ws',
                realm: 'iot'
            }
        ),
        mem;
    
    connection.onopen = function (session) {
        session.register('iot.serial.get', function () {
            return mem;
        });
    
        session.subscribe('iot.serial.reader', function (args) {
            mem = args[0];
        });
    };
    
    connection.open();
    

    An now in the frontend we ask for ‘iot.serial.get’ when we connect to the socket

    $(function () {
        var connection = new autobahn.Connection({
            url: "ws://192.168.1.104:8080/ws",
            realm: "iot"
        });
    
        connection.onopen = function (session) {
            session.subscribe('iot.serial.reader', function (args) {
                $('#display').html(args[0]);
            }).then(function () {
                    session.call('iot.serial.get').then(
                        function (result) {
                            $('#display').htmlresult);
                        }
                    );
                }
            );
        };
        connection.open();
    });
    

    And thats all. The source code is available in my github account. You also can see a demo of the working prototype here