Blog Archives

Working with SAPUI5 locally (part 3). Adding more services in Docker

In the previous project we moved one project to docker. The idea was to move exactly the same functionality (even without touching anything within the source code). Now we’re going to add more services. Yes, I know, it looks like overenginering (it’s exactly overenginering, indeed), but I want to build something with different services working together. Let start.

We’re going to change a little bit our original project. Now our frontend will only have one button. This button will increment the number of clicks but we’re going to persists this information in a PostgreSQL database. Also, instead of incrementing the counter in the backend, our backend will emit one event to a RabbitMQ message broker. We’ll have one worker service listening to this event and this worker will persist the information. The communication between the worker and the frontend (to show the incremented value), will be via websockets.

With those premises we are going to need:

  • Frontend: UI5 application
  • Backend: PHP/lumen application
  • Worker: nodejs application which is listening to a RabbitMQ event and serving the websocket server (using socket.io)
  • Nginx server
  • PosgreSQL database.
  • RabbitMQ message broker.

As the previous examples, our PHP backend will be server via Nginx and PHP-FPM.

Here we can see to docker-compose file to set up all the services

version: '3.4'

services:
  nginx:
    image: gonzalo123.nginx
    restart: always
    ports:
    - "8080:80"
    build:
      context: ./src
      dockerfile: .docker/Dockerfile-nginx
    volumes:
    - ./src/backend:/code/src
    - ./src/.docker/web/site.conf:/etc/nginx/conf.d/default.conf
    networks:
    - app-network
  api:
    image: gonzalo123.api
    restart: always
    build:
      context: ./src
      dockerfile: .docker/Dockerfile-lumen-dev
    environment:
      XDEBUG_CONFIG: remote_host=${MY_IP}
    volumes:
    - ./src/backend:/code/src
    networks:
    - app-network
  ui5:
    image: gonzalo123.ui5
    ports:
    - "8000:8000"
    restart: always
    volumes:
    - ./src/frontend:/code/src
    build:
      context: ./src
      dockerfile: .docker/Dockerfile-ui5
    networks:
    - app-network
  io:
    image: gonzalo123.io
    ports:
    - "9999:9999"
    restart: always
    volumes:
    - ./src/io:/code/src
    build:
      context: ./src
      dockerfile: .docker/Dockerfile-io
    networks:
    - app-network
  pg:
    image: gonzalo123.pg
    restart: always
    ports:
    - "5432:5432"
    build:
      context: ./src
      dockerfile: .docker/Dockerfile-pg
    environment:
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_DB: ${POSTGRES_DB}
      PGDATA: /var/lib/postgresql/data/pgdata
    networks:
    - app-network
  rabbit:
    image: rabbitmq:3-management
    container_name: gonzalo123.rabbit
    restart: always
    ports:
    - "15672:15672"
    - "5672:5672"
    environment:
      RABBITMQ_ERLANG_COOKIE:
      RABBITMQ_DEFAULT_VHOST: /
      RABBITMQ_DEFAULT_USER: ${RABBITMQ_DEFAULT_USER}
      RABBITMQ_DEFAULT_PASS: ${RABBITMQ_DEFAULT_PASS}
    networks:
    - app-network
networks:
  app-network:
    driver: bridge

We’re going to use the same docker files than in the previous post but we also need new ones for worker, database server and message queue:

Worker:

FROM node:alpine

EXPOSE 8000

WORKDIR /code/src
COPY ./io .
RUN npm install
ENTRYPOINT ["npm", "run", "serve"]

The worker script is simple script that serves the socket.io server and emits a websocket within every message to the RabbitMQ queue.

var amqp = require('amqp'),
  httpServer = require('http').createServer(),
  io = require('socket.io')(httpServer, {
    origins: '*:*',
  }),
  pg = require('pg')
;

require('dotenv').config();
var pgClient = new pg.Client(process.env.DB_DSN);

rabbitMq = amqp.createConnection({
  host: process.env.RABBIT_HOST,
  port: process.env.RABBIT_PORT,
  login: process.env.RABBIT_USER,
  password: process.env.RABBIT_PASS,
});

var sql = 'SELECT clickCount FROM docker.clicks';

// Please don't do this. Use lazy connections
// I'm 'lazy' to do it in this POC 🙂
pgClient.connect(function(err) {
  io.on('connection', function() {
    pgClient.query(sql, function(err, result) {
      var count = result.rows[0]['clickcount'];
      io.emit('click', {count: count});
    });

  });

  rabbitMq.on('ready', function() {
    var queue = rabbitMq.queue('ui5');
    queue.bind('#');

    queue.subscribe(function(message) {
      pgClient.query(sql, function(err, result) {
        var count = parseInt(result.rows[0]['clickcount']);
        count = count + parseInt(message.data.toString('utf8'));
        pgClient.query('UPDATE docker.clicks SET clickCount = $1', [count],
          function(err) {
            io.emit('click', {count: count});
          });
      });
    });
  });
});

httpServer.listen(process.env.IO_PORT);

Database server:

FROM postgres:9.6-alpine
COPY pg/init.sql /docker-entrypoint-initdb.d/

As we can see we’re going to generate the database estructure in the first build

CREATE SCHEMA docker;

CREATE TABLE docker.clicks (
clickCount numeric(8) NOT NULL
);

ALTER TABLE docker.clicks
OWNER TO username;

INSERT INTO docker.clicks(clickCount) values (0);

With the RabbitMQ server we’re going to use the official docker image so we don’t need to create one Dockerfile

We also have changed a little bit our Nginx configuration. We want to use Nginx to serve backend and also socket.io server. That’s because we don’t want to expose different ports to internet.

server {
    listen 80;
    index index.php index.html;
    server_name localhost;
    error_log  /var/log/nginx/error.log;
    access_log /var/log/nginx/access.log;
    root /code/src/www;

    location /socket.io/ {
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_pass "http://io:9999";
    }

    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

    location ~ \.php$ {
        try_files $uri =404;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass api:9000;
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param PATH_INFO $fastcgi_path_info;
    }
}

To avoid CORS issues we can also use SCP destination (the localneo proxy in this example), to serve socket.io also. So we need to:

  • change our neo-app.json file
  • "routes": [
        ...
        {
          "path": "/socket.io",
          "target": {
            "type": "destination",
            "name": "SOCKETIO"
          },
          "description": "SOCKETIO"
        }
      ],
    

    And basically that’s all. Here also we can use a “production” docker-copose file without exposing all ports and mapping the filesystem to our local machine (useful when we’re developing)

    version: '3.4'
    
    services:
      nginx:
        image: gonzalo123.nginx
        restart: always
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-nginx
        networks:
        - app-network
      api:
        image: gonzalo123.api
        restart: always
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-lumen
        networks:
        - app-network
      ui5:
        image: gonzalo123.ui5
        ports:
        - "80:8000"
        restart: always
        volumes:
        - ./src/frontend:/code/src
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-ui5
        networks:
        - app-network
      io:
        image: gonzalo123.io
        restart: always
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-io
        networks:
        - app-network
      pg:
        image: gonzalo123.pg
        restart: always
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-pg
        environment:
          POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
          POSTGRES_USER: ${POSTGRES_USER}
          POSTGRES_DB: ${POSTGRES_DB}
          PGDATA: /var/lib/postgresql/data/pgdata
        networks:
        - app-network
      rabbit:
        image: rabbitmq:3-management
        restart: always
        environment:
          RABBITMQ_ERLANG_COOKIE:
          RABBITMQ_DEFAULT_VHOST: /
          RABBITMQ_DEFAULT_USER: ${RABBITMQ_DEFAULT_USER}
          RABBITMQ_DEFAULT_PASS: ${RABBITMQ_DEFAULT_PASS}
        networks:
        - app-network
    networks:
      app-network:
        driver: bridge
    

    And that’s all. The full project is available in my github account

    Advertisements

    Working with SAPUI5 locally (part 2). Now with docker

    In the first part I spoke about how to build our working environment to work with UI5 locally instead of using WebIDE. Now, in this second part of the post, we’ll see how to do it using docker to set up our environment.

    I’ll use docker-compose to set up the project. Basically, as I explain in the first part, the project has two parts. One backend and one frontned. We’re going to use exactly the same code for the frontend and for the backend.

    The frontend is build over a localneo. As it’s a node application we’ll use a node:alpine base host

    FROM node:alpine
    
    EXPOSE 8000
    
    WORKDIR /code/src
    COPY ./frontend .
    RUN npm install
    ENTRYPOINT ["npm", "run", "serve"]
    

    In docker-compose we only need to map the port that we´ll expose in our host and since we want this project in our depelopemet process, we also will map the volume to avoid to re-generate our container each time we change the code.

    ...
      ui5:
        image: gonzalo123.ui5
        ports:
        - "8000:8000"
        restart: always
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-ui5
        volumes:
        - ./src/frontend:/code/src
        networks:
        - api-network
    

    The backend is a PHP application. We can set up a PHP application using different architectures. In this project we’ll use nginx and PHP-FPM.

    for nginx we’ll use the following Dockerfile

    FROM  nginx:1.13-alpine
    
    EXPOSE 80
    
    COPY ./.docker/web/site.conf /etc/nginx/conf.d/default.conf
    COPY ./backend /code/src
    

    And for the PHP host the following one (with xdebug to enable debugging and breakpoints):

    FROM php:7.1-fpm
    
    ENV PHP_XDEBUG_REMOTE_ENABLE 1
    
    RUN apt-get update && apt-get install -my \
        git \
        libghc-zlib-dev && \
        apt-get clean
    
    RUN apt-get install -y libpq-dev \
        && docker-php-ext-configure pgsql -with-pgsql=/usr/local/pgsql \
        && docker-php-ext-install pdo pdo_pgsql pgsql opcache zip
    
    RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
    
    RUN composer global require "laravel/lumen-installer"
    ENV PATH ~/.composer/vendor/bin:$PATH
    
    COPY ./backend /code/src
    

    And basically that’s all. Here the full docker-compose file

    version: '3.4'
    
    services:
      nginx:
        image: gonzalo123.nginx
        restart: always
        ports:
        - "8080:80"
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-nginx
        volumes:
        - ./src/backend:/code/src
        - ./src/.docker/web/site.conf:/etc/nginx/conf.d/default.conf
        networks:
        - api-network
      api:
        image: gonzalo123.api
        restart: always
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-lumen-dev
        environment:
          XDEBUG_CONFIG: remote_host=${MY_IP}
        volumes:
        - ./src/backend:/code/src
        networks:
        - api-network
      ui5:
        image: gonzalo123.ui5
        ports:
        - "8000:8000"
        restart: always
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-ui5
        networks:
        - api-network
    
    networks:
      api-network:
        driver: bridge
    

    If we want to use this project you only need to:

    • clone the repo fron github
    • run ./ui5 up

    With this configuration we’re exposing two ports 8080 for the frontend and 8000 for the backend. We also are mapping our local filesystem to containers to avoid to regenerate our containers each time we change the code.

    We also can have a variation. A “production” version of our docker-compose file. I put production between quotation marks because normally we aren’t going to use localneo as a production server (please don’t do it). We’ll use SCP to host the frontend.

    This configuration is just an example without filesystem mapping, without xdebug in the backend and without exposing the backend externally (Only the frontend can use it)

    version: '3.4'
    
    services:
      nginx:
        image: gonzalo123.nginx
        restart: always
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-nginx
        networks:
        - api-network
      api:
        image: gonzalo123.api
        restart: always
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-lumen
        networks:
        - api-network
      ui5:
        image: gonzalo123.ui5
        ports:
        - "8000:8000"
        restart: always
        build:
          context: ./src
          dockerfile: .docker/Dockerfile-ui5
        networks:
        - api-network
    
    networks:
      api-network:
        driver: bridge
    

    And that’s all. You can see the all the source code in my github account

    Working with SAPUI5 locally and deploying in SCP

    When I work with SAPUI5 projects I normally use WebIDE. WebIDE is a great tool but I’m more confortable working locally with my local IDE.
    I’ve this idea in my mind but I never find the time slot to work on it. Finally, after finding this project from Holger Schäfer in github, I realized how easy it’s and I started to work with this project and adapt it to my needs.

    The base of this project is localneo. Localneo starts a http server based on neo-app.json file. That means we’re going to use the same configuration than we’ve in production (in SCP). Of course we’ll need destinations. We only need one extra file called destination.json where we’ll set up our destinations (it creates one http proxy, nothing else).

    In this project I’ll create a simple example application that works with one API server.

    The backend

    I’ll use in this example one PHP/Lumen application:

    $app->router->group(['prefix' => '/api', 'middleware' => Middleware\AuthMiddleware::NAME], function (Router $route) {
        $route->get('/', Handlers\HomeHandler::class);
        $route->post('/', Handlers\HomeHandler::class);
    });
    

    Basically it has two routes. In fact both routes are the same. One accept POST request and another one GET requests.
    They’ll answer with the current date in a json file

    namespace App\Http\Handlers;
    
    class HomeHandler
    {
        public function __invoke()
        {
            return ['date' => (new \DateTime())->format('c')];
        }
    }
    

    Both routes are under one middleware to provide the authentication.

    namespace App\Http\Middleware;
    
    use Closure;
    use Illuminate\Http\Request;
    
    class AuthMiddleware
    {
        public const NAME = 'auth';
    
        public function handle(Request $request, Closure $next)
        {
            $user = $request->getUser();
            $pass = $request->getPassword();
    
            if (!$this->validateDestinationCredentials($user, $pass)) {
                $headers = ['WWW-Authenticate' => 'Basic'];
    
                return response('Backend Login', 401, $headers);
            }
    
            $authorizationHeader = $request->header('Authorization2');
            if (!$this->validateApplicationToken($authorizationHeader)) {
                return response('Invalid token ', 403);
            }
    
            return $next($request);
    
        }
    
        private function validateApplicationToken($authorizationHeader)
        {
            $token = str_replace('Bearer ', null, $authorizationHeader);
    
            return $token === getenv('APP_TOKEN');
        }
    
        private function validateDestinationCredentials($user, $pass)
        {
            if (!($user === getenv('DESTINATION_USER') && $pass === getenv('DESTINATION_PASS'))) {
                return false;
            }
    
            return true;
        }
    }
    

    That means our service will need Basic Authentication and also one Token based authentication.

    The frontend

    Our ui5 application will use one destination called BACKEND. We’ll configure it in our neo-app.json file

        ...
        {
          "path": "/backend",
          "target": {
            "type": "destination",
            "name": "BACKEND"
          },
          "description": "BACKEND"
        }
        ...
    

    Now we’ll create our extra file called destinations.json. Localneo will use this file to create a web server to serve our frontend locally (using the destination).

    As I said before our backend will need a Basic Authentication. This Authentication will be set up in the destination configuration

    {
      "server": {
        "port": "8080",
        "path": "/webapp/index.html",
        "open": true
      },
      "service": {
        "sapui5": {
          "useSAPUI5": true,
          "version": "1.54.8"
        }
      },
      "destinations": {
        "BACKEND": {
          "url": "http://localhost:8888",
          "auth": "superSecretUser:superSecretPassword"
        }
      }
    }
    

    Our application will be a simple list of items

    <mvc:View controllerName="gonzalo123.controller.App" xmlns:html="http://www.w3.org/1999/xhtml" xmlns:mvc="sap.ui.core.mvc" displayBlock="true" xmlns="sap.m">
      <App id="idAppControl">
        <pages>
          <Page title="{i18n>appTitle}">
            <content>
              <List>
                <items>
                  <ObjectListItem id="GET" title="{i18n>get}"
                                  type="Active"
                                  press="getPressHandle">
                    <attributes>
                      <ObjectAttribute id="getCount" text="{/Data/get/count}"/>
                    </attributes>
                  </ObjectListItem>
                  <ObjectListItem id="POST" title="{i18n>post}"
                                  type="Active"
                                  press="postPressHandle">
                    <attributes>
                      <ObjectAttribute id="postCount" text="{/Data/post/count}"/>
                    </attributes>
                  </ObjectListItem>
                </items>
              </List>
            </content>
          </Page>
        </pages>
      </App>
    </mvc:View>
    

    When we click on GET we’ll perform a GET request to the backend and we’ll increment the counter. The same with POST.
    We’ll also show de date provided by the backend in a MessageToast.

    sap.ui.define([
      "sap/ui/core/mvc/Controller",
      "sap/ui/model/json/JSONModel",
      'sap/m/MessageToast',
      "gonzalo123/model/api"
    ], function (Controller, JSONModel, MessageToast, api) {
      "use strict";
    
      return Controller.extend("gonzalo123.controller.App", {
        model: new JSONModel({
          Data: {get: {count: 0}, post: {count: 0}}
        }),
    
        onInit: function () {
          this.getView().setModel(this.model);
        },
    
        getPressHandle: function () {
          api.get("/", {}).then(function (data) {
            var count = this.model.getProperty('/Data/get/count');
            MessageToast.show("Pressed : " + data.date);
            this.model.setProperty('/Data/get/count', ++count);
          }.bind(this));
        },
    
        postPressHandle: function () {
          var count = this.model.getProperty('/Data/post/count');
          api.post("/", {}).then(function (data) {
            MessageToast.show("Pressed : " + data.date);
            this.model.setProperty('/Data/post/count', ++count);
          }.bind(this));
        }
      });
    });
    

    Start our application locally

    Now we only need to start the backend

    php -S 0.0.0.0:8888 -t www

    And the frontend
    localneo

    Debugging locally

    As we’re working locally we can use local debugger in the backend and we can use breakpoints, inspect variables, etc.

    We also can debug the frontend using Chrome developer tools. We can also map our local filesystem in the browser and we can save files directly in Chrome.

    Testing

    We can test the backend using phpunit and run our tests with
    composer run test

    Here we can see a simple test of the backend

        public function testAuthorizedRequest()
        {
            $headers = [
                'Authorization2' => 'Bearer superSecretToken',
                'Content-Type'   => 'application/json',
                'Authorization'  => 'Basic ' . base64_encode('superSecretUser:superSecretPassword'),
            ];
    
            $this->json('GET', '/api', [], $headers)
                ->assertResponseStatus(200);
            $this->json('POST', '/api', [], $headers)
                ->assertResponseStatus(200);
        }
    
    
        public function testRequests()
        {
    
            $headers = [
                'Authorization2' => 'Bearer superSecretToken',
                'Content-Type'   => 'application/json',
                'Authorization'  => 'Basic ' . base64_encode('superSecretUser:superSecretPassword'),
            ];
    
            $this->json('GET', '/api', [], $headers)
                ->seeJsonStructure(['date']);
            $this->json('POST', '/api', [], $headers)
                ->seeJsonStructure(['date']);
        }
    

    We also can test the frontend using OPA5.

    As Backend is already tested we’ll mock the backend here using sinon (https://sinonjs.org/) server

    ...
        opaTest("When I click on GET the GET counter should increment by one", function (Given, When, Then) {
          Given.iStartMyApp("./integration/Test1/index.html");
          When.iClickOnGET();
          Then.getCounterShouldBeIncrementedByOne().and.iTeardownMyAppFrame();
        });
    
        opaTest("When I click on POST the POST counter should increment by one", function (Given, When, Then) {
          Given.iStartMyApp("./integration/Test1/index.html");
          When.iClickOnPOST();
          Then.postCounterShouldBeIncrementedByOne().and.iTeardownMyAppFrame();
        });
    ...
    

    The configuration of our sinon server:

    sap.ui.define(
      ["test/server"],
      function (server) {
        "use strict";
    
        return {
          init: function () {
            var oServer = server.initServer("/backend/api");
    
            oServer.respondWith("GET", /backend\/api/, [200, {
              "Content-Type": "application/json"
            }, JSON.stringify({
              "date": "2018-07-29T18:44:57+02:00"
            })]);
    
            oServer.respondWith("POST", /backend\/api/, [200, {
              "Content-Type": "application/json"
            }, JSON.stringify({
              "date": "2018-07-29T18:44:57+02:00"
            })]);
          }
        };
      }
    );
    

    The build process

    Before uploading the application to SCP we need to build it. The build process optimizes the files and creates Component-preload.js and sap-ui-cachebuster-info.json file (to ensure our users aren’t using a cached version of our application)
    We’ll use grunt to build our application. Here we can see our Gruntfile.js

    module.exports = function (grunt) {
      "use strict";
    
      require('load-grunt-tasks')(grunt);
      require('time-grunt')(grunt);
    
      grunt.config.merge({
        pkg: grunt.file.readJSON('package.json'),
        watch: {
          js: {
            files: ['Gruntfile.js', 'webapp/**/*.js', 'webapp/**/*.properties'],
            tasks: ['jshint'],
            options: {
              livereload: true
            }
          },
    
          livereload: {
            options: {
              livereload: true
            },
            files: [
              'webapp/**/*.html',
              'webapp/**/*.js',
              'webapp/**/*.css'
            ]
          }
        }
      });
    
      grunt.registerTask("default", [
        "clean",
        "lint",
        "build"
      ]);
    };
    

    In our Gruntfile I’ve also configure a watcher to build the application automatically and triggering the live reload (to reload my browser every time I change the frontend)

    Now I can build the dist folder with the command:

    grunt

    Deploy to SCP

    The deploy process is very well explained in the Holger’s repository
    Basically we need to download MTA Archive builder and extract it to ./ci/tools/mta.jar.
    Also we need SAP Cloud Platform Neo Environment SDK (./ci/tools/neo-java-web-sdk/)
    We can download those binaries from here

    Then we need to fulfill our scp credentials in ./ci/deploy-mta.properties and configure our application in ./ci/mta.yaml
    Finally we will run ./ci/deploy-mta.sh (here we can set up our scp password in order to input it within each deploy)

    Full code (frontend and backend) in my github account